RAG Mode

By default, a Chat Node sends the full content of every connected node to the model. RAG Mode (Retrieval Augmented Generation) changes that: your connected content is indexed, and on each message only the chunks most relevant to your question are retrieved and sent.

---

When to use it

Use RAG Mode when the incoming context is large — long documents, many connected nodes, big web pages. Retrieval trims the input dramatically, which cuts cost and keeps you inside the model's context window.

For small context sizes (under roughly 10,000 tokens), RAG Mode is not recommended — sending everything is simpler and often gives better answers.

---

Enabling RAG Mode

1. Open the + tools dropdown in the chat input

2. Toggle the RAG Mode switch — or click the scissors icon in the chat input directly

3. The scissors icon highlights and shows RAG Mode Active on hover

RAG Mode requires a paid plan. On the free plan the toggle shows a lock icon, and clicking it opens a RAG Mode Requires a Paid Plan modal. See Account Management.

AI Search in the ⌘K command menu uses the same index and requires the cloud indexing opt-in under Settings → Account → Privacy Settings.

---

What happens when you send

While RAG Mode is active, the token pill in the chat input is masked with asterisks — the real input size isn't known until retrieval runs. When you send a message, indexing runs first, and status banners appear above the chat in this order:

1. Indexing documents… — connected content is being chunked and embedded

2. Documents indexed — the index is up to date

3. Retrieving relevant chunks… — your question is matched against the index

4. Reranking results… — the best chunks are ordered by relevance

5. A result banner reports the savings: RAG Mode reduced your input size by N% from X to Y tokens.

If any stage fails, a red banner appears with the error. You can turn RAG Mode off and retry without it.

---

Constraints

RAG with Rabbitholes managed models is temporarily unavailable in the desktop app — you'll see Use a BYOK provider for RAG. Select a model from one of your own providers (see Models & Providers).

File Nodes must finish processing before their content joins the RAG index. You can still chat with them immediately without RAG — only RAG Mode waits for processing.

Deleting a node removes its embeddings from the index, and undoing the delete does not restore them. The content is re-indexed on your next RAG message.

---

Use cases

• Chatting with a 300-page PDF: connect the File Node, enable RAG Mode, and ask targeted questions — only the relevant pages are sent.
• Research canvas with many sources: a chat node connected to a dozen Web Nodes and YouTube Nodes stays well under the context limit.
• Cutting token spend: on BYOK providers, a 90% input reduction is a 90% reduction in input cost for that message.
• Fitting oversized context: when the token pill warns your input exceeds the model's window, RAG Mode is usually the fix.