Ragnhild Remembers Nothing

Introduction

#

In last week’s post, I settled on a Mistral agent with attached knowledge. Building a RAG from scratch was out of scope, because I thought it meant hosting a model myself and keeping the data in a PostgreSQL database. This week Ragnhild got built from scratch after all, with no agent and no database.

Monday’s Session, RAG II, went through what makes a RAG useful, from preparing the data to keeping it up to date.

Changing the Plan

#

Before writing any code, I grilled the plan against Mistral, letting the model question every decision until I could either defend it or had to drop it. What came out of it:

• As few dependencies as possible. Every extra service is something else to host and keep updated.
• Nothing stored about visitors. If I cannot tell a visitor exactly what I keep, I should not keep it.
• Room for opt-in logging later, done the GDPR-compliant way as a separate module.

With those rules, the agent had to go, and so did the PostgreSQL idea. The agent was one more dependency, and it came with the question I left open last week about data from visitor conversations. The database was meant to hold my content, and my content turned out to fit in two files.

I was also wrong about hosting a model. The models stay behind Mistral’s API, and the only data I host is the index.

How Ragnhild Works

#

Ragnhild is based on Mistral’s RAG quickstart and needs three Python packages: mistralai, faiss-cpu and numpy. There is no database. The whole knowledge base is two files:

The vectors come from mistral-embed, which turns a piece of text into numbers describing what it is about. FAISS searches through them. I use its simplest index, IndexFlatL2, which compares the question with every chunk and returns the closest. There is nothing to tune.

A visitor asks a question through one endpoint, POST /ask, and this happens:

flowchart TD
  Q["Visitor question"] --> E["mistral-embed"]
  E -->|"question vector"| F["FAISS IndexFlatL2"]
  F -->|"nearest chunk IDs"| M["metadata.json"]
  M -->|"chunk text"| C["Ministral 3 3B"]
  Q -->|"question"| C
  C --> A["Answer with citations"]

The question is embedded with the same model as the content, FAISS finds the nearest chunk IDs, and the metadata turns those back into text. The chat model gets the question and the chunks in one prompt and writes the answer. That grounds the answer in my content instead of whatever the model learned in training. The answer comes back with deduplicated citations, so a post that matched three times is only linked once.

Every request is single-turn. Nothing carries over from one question to the next, which is where the title for this post comes from.

Choosing the Chat Model

#

Last week I called it “Mistral 3B”. Its actual name is Ministral 3 3B, the smallest model in the Ministral 3 family. I compared it with the next sizes up:

Ragnhild does not have to reason much. The answer is in the chunks, and the model mostly has to turn them into readable sentences. That fits what I wrote last week about picking the smallest model that does the job. If the answers turn out too thin, the 8B model is the next step up.

Preparing the Data

#

One of the Session’s goals was converting data to Markdown. Hugo had already done that for me. Every post, project and the about page is a Markdown file in the site’s content/ folder, so Ragnhild reads those files directly instead of the sitemap I mentioned last week.

The files are split into chunks on their headings first, then by size with a bit of overlap if a section is still too long. I write my posts in sections anyway, so a chunk usually covers one idea. Drafts and future-dated posts are left out.

Keeping It Up to Date

#

Every chunk carries a hash of the content it came from, and on each sync every content item gets one of four operations:

• Add: new content, so embed it and insert it.
• Update: the hash changed, so replace its chunks.
• Delete: removed or turned back into a draft, so remove its chunks.
• Ignore: the hash is unchanged, so skip it without calling the embedding model.

The sync runs as a step in the site’s deploy pipeline, right after the rsync to the server. When I publish a post, only that post gets embedded, which is the kind of token saving I had in mind when planning.

Hosting and the Widget

#

Ragnhild runs in Docker on the same DigitalOcean droplet as this site, which has 1 GB of memory. The knowledge base is well under a megabyte.

On the site, a small Blowfish partial adds a chat widget that calls POST /ask. The widget says plainly that you are talking to an AI, and every answer ends with a footer that says the same.

Ethics and Law

#

This is the part I promised last week.

Ragnhild stores my public portfolio content and nothing else. It keeps no questions, IP addresses, timestamps, answers or sessions. Your question does pass through Mistral’s API to be embedded and answered, which is one of the reasons I wanted an EU provider for this, but Ragnhild forgets it once the answer is sent. The GDPR asks for data minimisation, and storing nothing is as minimal as it gets.

The downside is that I cannot see what people ask, or where Ragnhild gets it wrong. If I want that later, it will be a separate opt-in module: the visitor gives consent first, logs are deleted after a fixed period, and there is a way to have your own data erased. I would rather build that properly than collect data now and work out the rules afterwards.

Under the EU AI Act, an assistant like Ragnhild is limited risk. The main obligation is transparency (Article 50): people have to know they are talking to an AI system. The disclosure in the widget and the footer on each answer cover that.

What I Think of RAG Now

#

What I like about RAG is that the model does not need to know anything about me beforehand. It gets the relevant text at the moment of the question, and the citations let the reader check the source instead of taking the answer on trust. When I write something new, Ragnhild knows about it after the next deploy, without any training.

The limits come from the same place. Ragnhild is only as good as what it retrieves. If the right chunk is not among the nearest ones, the model answers from the wrong text. It cannot answer anything I have never written down, and without history it cannot handle a follow-up question.

Dify would have worked, but it is a generic tool, and I wanted to control where the data goes. Because I built Ragnhild myself, I can say exactly what it keeps.

A flat index is fine up to roughly 100,000 to a million chunks, far more than my portfolio will ever have. Beyond that comes a compressed index (IVF with product quantization), and a vector database only if operations demand it.

Conclusion

#

The plan I ended up with is smaller than the one I started with, and unlike with the agent, I understand every part of it.

Friday’s Session was about code agents, and we were asked to add a feature to our own sites with one. My attempt is the subject of the next post.

The code for Ragnhild is on Codeberg:

- Daniel