Introduction to the blog LLM tool

As you can see to the right, we have a dropdown for the different chats. We have a dummy conversation loaded right now, and if you click here you will get to it's first message. If you click here you will get to it's sixth message, which is the first one including a code snippet. You can collapse individual code snippets, but you can also change from "showing code by default" to "hiding code by default" if you feel like the code takes up to much space. This will probably be useful in later chats when there are going to be large code snippets that are just being referenced back and forth, especially if you are more interested in the conversation than the code-specifics.

Now you know how this tool works, and we can move on to development!

Planning

Lets start by telling Claude about our idea and see if it has tips for features to add. Most (if not all) of the tips we get back are obvious, and not necessary to define this early. We move onto figuring out our tech stack.As you can see I have some preferences and we take that into consideration. We have a somewhat interesting discussion over the backend, and we arrive to the conclusion to use supabase auth and supabase db (which is postgres), but to still have a server layer in between that we control. Here's the summary of the conversation

Getting started

At this point we have made some tech stack decisions and we have also decided what our application architecture will look like from a basic point of view. We are now ready to set up the project, which we do by following the commands outlined by Claude. When the frontend application is up and running, we decide to implement some basic UI components. As you can see I'm not impressed by the design initially, so we iterate on that until we get to something better.

We then move over to authentication. We have already decided to not roll our own auth and to use supabase, so I just follow all the instructions outlined by Claude for setting up supabase. Since we want google sign-in, we also need to configure some stuff in the google cloud console. At this point, there have been zero issues. We have a some type errors that Claude fixes for us. We now have google authentication working for our product.

Setting up the server

Here we move on to setting up the server. I give some instructions for how we should set up or first endpoint and then we create our first table in the database, and we write our authentication middleware that integrates with the supabase client. We set up our authenticated routes and test them by locally by grabbing a JWT from the chrome console, and sends request with the vscode rest client.

After verifying that our endpoints work as expected, we move back to the frontend. Claude moves to the code way to fast here, and I tell it to chill, we first need to think about our code before writing it. I propose that we use react-query for server state because I'm familiar with it, and then we create the initial page for reviewing your cards. Always having the quality of our codebase in mind, I propose to Claude that we should create custom hooks with react-query. Here all of a sudden after pasting I see that we are getting errors! It appears that Claude is using an old version of react-query, and I also note that we should always use the library defaults until we notice a problem.

When starting to fetch from our actual server, we get the famous CORS error, which Claude fixes without issues.

Review functionality

Moving on to setting up the review functionality. One thing that you can see that I like to have in context in every conversation is the current database schema and the file structure. As I tell Claude here, I like to practice UI driven development where I first imagine (and maybe sketch out) what something should look like, implement it with dummy data, and only after that integrate real functionality.

I know that there is some science behind space repetition algorithms, so I ask Claude about it. I get a good answer, and we decide to use the SuperMemo SM-2 algorithm. I ask Claude for step by step instructions, and I get them. They are mostly good, but Claude doesn't follow the pattern when using react-query as we have used previously. Using different patterns like this is obviously bad, and I remind Claude and they come back with better code.

After a while we have the review functionality up and running, but there is some janky behaviour in the UI. I suggest to Claude that we should use optimistic updates here to fix it, and Claude agrees. We get some more issues with the card rendering, and Claude suggests a solution with useEffect that I don't like. After trying a few things, I get enough and take a step back and actually start using my brain. After outlining the problem, Claude finally realizes how to do it (using a key prop on the flashcard component).

We then fix some bugs in the UI and add react-router for routing functionality. I get mad at Claude for changing the UI when that had nothing do with what I ask for. This sometimes happens, and is pretty annoying.

Creating cards

We now move onto thinking about the functionality of creating cards. As usual I summarize what we have done so far along with the DB schema and project file structure. What's new here is that I provide a mockup (that I just created in excalidraw) which provides the structure of the page I'm imaging. We iterate on the UI here, and I think this workflow is kind of nice where I give Claude a list of changes I want to see and just get them. Feels like programming in english!

We change our schema to fit the UI, and start working on new routes on the backend. Claude gives me an ugly sql query that we fix. After all the new routes are confirmed to be working, we move back to the frontend where Claude once again uses the wrong react-query syntax.

We move on thinking about how we should display already created cards, and I suggest tanstack table which we decide to use. It basically works on the first try, and we move on to the delete card functionality. Claude messes up a react-query hook, and I point out the issue and we fix it. Finally, we add the functionality of creating a new deck, where we have some issues with whether we should generate UUIDs on the client or the server.

State of the project

Here you can see the UI for creating cards, with all the functionality working.

https://github.com/holma91/deepcards/tree/creating-cards

Adding sidebar

• I see a bug that Claude missed about the Auth
• We iterate on the design with pretty good results
• We have a UI issue in our tanstack table component
• Claude gives me a tooltip solution for showing text that doesn't work, we decide that we don't need it

Don't think anything here warranted a change in the counter.

UI improvements

Now we start a session where we just want some UI improvements. These sessions are generally very good. In the end it's not only UI improvements as I decide to add a dashboard page as well.

• Improving the UI for the review screen
• Issues with positioning
• Improving the Flashcard component by adding space bar navigation
• Create dashboard page
• Adding real data by providing step-by-step instructions
• Claude forgets the authentication middleware
• Issue with SQL query

Creating a home page

At this point we want a home page, what users that aren't signed in will see. Here I trust Claude to give a good design without even providing a mockup, and they do a good job!

• We add a login modal (suggested by Claude)
• Pretty cool how we can just add the Google SVG because Claude has it memorized

Adding more advanced functionality

Here I have this idea that I want the sidebar to show how many cards are due, and that it should be possible to review cards on a per deck basis. We iterate on the design here for a while, and there are some bugs that we squash.

• Claude gives us a bad SQL query
• Claude tries to access a field on a variable that doesn't exist, I have to come up with the solution
• Claude comes up with another bad SQL query
• I have to remind Claude that we sometimes should invalidate react-query queries
• I notice that we have snake_case in the javascript at some places, Claude fixes it
• I get an idea for improving the UI, which is to move the decks to the sidebar under "Cards".
• Claude makes a mistake with protected routes
• Claude adds a line for our react-query query that makes it not work

UI improvements II

Here we have three goals in mind, we want add the possibility to remove decks, we want to render cards in markdown and we want to make the loading state UI prettier.

• Adding a settings modal with delete and rename functionality
• I notice that we have two modals when we only need one
• We start working on better loading states
• Common theme, removing unnecessary useEffects that Claude adds
• Adding markdown support for rendering cards

Code review

This is a very important section. After copy pasting so much code, you get to a point where you don't have a good overview of the codebase. I felt that I needed to take the time to just read the whole codebase, ask clarifying questions and remove code that wasn't necessary. This also led to some improvements. Notable things:

• I try to be clever and remove some things and Claude agrees.

Turns out I was wrong after thinking a bit, so this is an example of Claude not really having strong opinions and just agreeing

• Refactoring messy react-query hooks into multiple hooks
• Claude adds an unnecessary useEffect (again)
• We look at the sidebar

Claude suggests that we use a modal for creating decks and I agree

• Removing loading states where we use optimistic updates

A common theme through the whole conversation, we have code where we update the UI optimistically but where we are also waiting for confirmation from the server

UI improvements III

As mentioned above, we will now try to write better prompts. I start by summarizing the application as usual, and I then go into detail of how I think we can improve the UI on the cards page along with a mockup from excalidraw. After iterating a bit on the design, we get to a good solution very quick, and we test it with some "semi-advanced" markdown.

Before:

After:

I think this looks pretty good! We also add the possibility to edit already created cards. This is an example of something that went perfect, I gave Claude all necessary context and we got a result that worked on the first try.

• We look for improvements in the review UI
• We once again remove a useEffect

Chat UI

We finally get to a more exciting part where we add the "deep dive feature". I provide a somewhat detailed description of what I'm imagining, along with two excalidraw mockups. We use our favorite prompt, just giving Claude a list of UI improvements. We do it again. And again.

• I point out a logic error that we fix
• I notice that we don't encapsulate functionality like we should

While fixing this, Claude messes up some styling that previously worked fine (a somewhat common theme)

State of the project

LLM integration

In the previous section we implemented the chat UI with dummy responses, now lets do the actual LLM integration! As you can see, my initial idea was to let users provide their own api key and interact with OpenAI directly from the client. However, after reading that OpenAI isn't really a fan of this pattern, we decide to let everything go through our server. Instead of allowing the user to provide their own key, I'll just utilize the fact that I have $500 in OpenAI credits and let everything go through my key.

• Step-by-step instructions for the OpenAI integration
• Bugs that we squash one by one

We go back and forth for a bit until everything works.

UI for LLM generated flashcards

We now want to look into the possibility of generating flashcards automatically from conversations. The idea here is that you should be able to have a conversation, and then ask for some flashcards that will help you remember what you learnt in the future. We explore this as a concept before diving into the code. After figuring out what the user experience should be like, we start implementing the UI with dummy data. After iterating on the design for a while (in our usual fashion by just providing bulletpoints of improvements), we get to a design that we are happy with:

We then move on to slowly integrating real functionality.

• We get some issues with a library we add "headlessui"
• We add a general "chat" choice in the sidebar
• Claude messes up the old styling when making an unrelated change
• We squash a few bugs one by one

Changing the layout

Here I get the idea that I want to change the layout a bit. Read it if you want, but it's not really that interesting.

• Making the sidebar dismissable
• Changing the font
• Adding a settings dropdown instead of logout button

Chat persistance

Currently the chats we have are just stored in memory, and are not persisted anywhere. We now want to add persistance! Here we start from the schema, and work our way up to to the UI. This all just a bunch of CRUD stuff, we basically just add routes, create react-query hooks and incorporate them into the UI.

• I remind Claude that we should utilize the powers of react-query
• Discussion about generating uuids on the client or the server
• Discussion about schema changes for automatic card generation

Generating flashcards

Now we finally get to the interesting part of the application!

• We think about the feature
• We use Zod and Instructor for getting structured output from the LLM
• I get a little mad because we are on the verge of counterproductive usage
• Claude gives me a crappy sql query

Code Review II

This was supposed to just be a code review session, but it ended up being much more.

• Adding flashcards as part of conversations
• Improve how we generate messages

This goes on for a while and it's not that interesting

• Improve flashcard generation

If you are interested in the generative AI part of the application, most of the interesting stuff will start somewhere around here.

• Start persisting rejected flashcards

Obviously it's useful for the LLM when generating new cards to have accepted AND rejected earlier cards in the prompt

• Realize that we can use types generated by supabase

Big mistake that we didn't do this earlier. Sometimes it's still good to read the docs! Kind of funny that I counted adding supabase as a 10x improvement in the beginnning because I did not have to read the docs, and now we probably lost more time here than we gained there. Anyway, we replace all manually created types on both the server and client, and this becomes a pretty annoying side task but leads to a better looking codebase in the end.

• Realize that it's better UX to generate flashcards inline than in a modal

This results in a big UX improvement in my opinion (I dislike modals generally). Before and after below.

Making it good

This was also a code review session, sort of. This is where I think the app actually becomes good.

• We add some basic error handling
• Claude gives me an overengineered abstraction

This happens once in a while. You really need to push for simplicity over complexity.

• Rendering code and math

We add react-syntax-highlighter and katex, and have to work through a few bugs until we get it to work. Rendering math is a little harder than expected because we just get back text from the LLM. Claude provides a "classical" solution, that made sense in the age before LLMs. However, now that we have LLMs that are so good at following instructions, we can just ask the LLM to gives us math in the format that katex can render directly by improving the prompt. This looks pretty good:


• Automatic naming of chats

Easily achievable with instructor!

Security review & improving auth

I start with the goal of adding password based authentication, but instead we end up doing a

• security review of the application

We discuss row-level security (RLS) and we agree that it's not necessary for our current setup.

• Implementing password based authentication

After some thinking I decide to do magic links and OTPs instead of passwords. We go back and forth a bit for designing the signup/signin UI.

UI improvements IV

This was a really cool session where we just go through the app page by page and make the design actually good. We tell Claude in the beginnning that we want a "linear-inspired" design, and this is basically the design you see on most modern SAAS products, inspired by linear.app. Instead of going through the conversation here by text, I'll just provide you with some before and after images. I think the general look and feel with font sizes and spacing across the application is the biggest improvement!

Before

After

At this point, none of our views are mobile friendly so this becomes our next job. Here are some of the mobile views:

Profile settings

Here I felt that it was time to create something that allowed users to change their settings. Like, default language model, spaced repetition algorithm and light/dark mode.

• Discussion about how to store user settings

We decide to use JSONB on the profiles table instead of individual fields

• Set up trigger for creating profile on user creation
• UI/UX for changing settings

Spaced repetition algorithm

This was an important session as I hadn't really dived into exactly our spaced repetition algorithm worked earlier.

• Start thinking about a design where users can choose basic or advanced

Claude starts throwing code at me directly, even though we are just in the thinking stage (I hate that). We find some unused stuff in our schema that we remove. Time zone discussion.

• Start looking at a modified SM-2 algorithm (like Anki)

Most interesting part here is that we like Anki now label cards as either being in a learning stage or review stage, with the review schedules being different. Claude continues to throw code at me when I don't want it.

• Our decided review function

We want the users to be able to choose "advanced" or "basic" spaced repetition algorithm, and we come up with a clever solution that avoids complexity.

UI Improvement V

Okay you are probably sick of these UI Improvement sections so I won't write much here. Most important UI changes here is that we make it possible to modify suggestions and cards wherever they appear.

• Realize that modified suggestions are important for the prompt

While chasing UI bugs we realize something more important, that we need to change our schema bit to allow for modified suggestions, and that we need to save the original suggestions because that's valuable info for the LLM.

Optimizations and Scalability

I thought this was going to a be an important session, but there wasn't that much to optimize. We analyze a few queries, but realize that they are all fast enough. We start discussing rate limiting, and Claude almost convinces me of using redis, but after some back and forth and realizing I have 0 daily active user (to be fair, 1 with me) I decide to put Redis in the bucket of overengineering for now, and just go with an in-memory rate-limiting solution (yes redis is also in-memory but you know what I mean).

Deployment

Finally, we get to an interesting session.

• Choosing our stack

I like Cloud Run on GCP since earlier, so I'm pretty set on using that for the server. After some discussion I get interested in Cloudflare Pages for our frontend, mainly because of all the good things I've heard about Cloudflare. I almost get tricked into using Cloudflare for the server aswell, but decide that Cloud run is better in the end. I then buy a domain on namecheap, and start signing up at Cloudflare.

• Cloudflare sucks

I immediately notice that the Cloudflare dashboard is very buggy and bad, but I'm like whatever, I've only heard good things about Cloudflare. What happens then is that I try to sign up for a paid plan. I give them my card info and click the button, and wait. It's taking a long time. After 2-3 minutes I get to an error screen. Okay so what I assume happened now is that I either got charged AND got upgraded to the paid plan, or that I didn't get charged AND also didn't get upgraded to the paid plan. But no, what happened is that I got charged and did NOT get upgraded to the paid plan. This is just embarrassing by Cloudflare on so many levels, so I realize there is no way I'm going to use Cloudflare for anything. (I opened a support ticked immediately, I have not gotten a response yet in 5 days, and I'm still on the free plan even though I payed $240). A complete joke, and by looking at their forums I see that this is a common problem.

• Drinking the Google Kool-Aid

We swap out Cloudflare Pages for Firebase Hosting, which is nice because now we will have both our server and frontend on GCP.

• Deploying the server on Cloud Run

We struggle a bit with docker of course but after a while we are up and running.

• Deploying the frontend on Firebase Hosting

Mobile UI issues

When deploying, I for the first time tested the UI from my actual phone, and there where some issues that didn't show up in the chrome console mobile view.

• Input zoom bug
• The 100vh problem
• Remove stupid manual header height calculation

Prompt engineering

This is an important session, where the focus is to improve the prompt given to the LLM for generating flashcards.

• What is a good flashcard?
• Improving the prompt

We include the characteristics of a good flashcard in the prompt, along with a few good and bad examples.

• Improving the math rendering instruction

We have some issues here, which requires some debugging. We finally fix it, and the bug was that the LLM gave us text with latex text mode inside of latex math mode, which apparently Katex doesn't like.

• Continue to struggle with math rendering

Everytime I think it's fixed, I notice another problem. This leads into a very annoying bug hunt that lasts for a while. Read if you want to. String processing is hard.

UI Improvements VI

Another UI improvements session? Yes. Why do we have so many of these? Because I use the app and notice things I'm either annoyed of or want added. The only interesting part here is probably when we design the homepage. Also, I ask Claude why they give me complex solutions when simple and better ones exist.

State of the project

The homepage:

Code Review III

The final code review before releasing this blog post, so I mainly just want to make sure that there isn't any stupid stuff left in the codebase.

• Modifying the client folder structure a bit
• Quick bug fix