Work Buddy

Work Buddy is a macOS assistant I built for myself, living inside Raycast. I wanted two things without handing anything to a hosted provider: to chat with a model running on my own machine, and to ask questions about my own documents. It has two modes. Talk goes straight from the launcher to a local model. RAG Talk answers from files I have ingested, through a local retrieval service that owns the documents.

Context

The two things I wanted have very different appetites. Direct chat is one HTTP call to a model that is already running; it should cost nothing to start and nothing to keep around. Document question-answering needs file parsing, chunking, an embedding model, a vector store that survives a restart, and somewhere for the documents to actually live.

Building both into the same Raycast extension would have made the cheap thing pay for the expensive one. Every time I wanted a quick answer, I would have been standing up retrieval infrastructure to get it. And a Raycast command is a short-lived UI process, not a good owner of a database connection or an ingestion pipeline.

The constraint underneath all of it is that nothing is hosted. The chat model, the embedding model, and the vector database all run on the same laptop as the editor and the browser. There is no remote service to hide latency or lifecycle behind, so context size, retrieval breadth, and process startup are things the user pays for directly.

My role

Solo project. I made the architecture decisions on both sides of the client/service boundary and wrote the Raycast extension and the retrieval service.

Architecture

Raycast owns interaction: the commands, chat UX, conversation files on disk, and a local registry of which files the user has added. A separate local Express service owns documents: loading, chunking, embedding, vector storage, retrieval, and generation. A keyed Zustand store backs the chat experience across services, so Talk and RAG Talk share one set of interaction patterns instead of each reimplementing streaming, history, and error states.

The two modes deliberately do not share infrastructure. Talk calls local Ollama directly from the extension, so it works with nothing else running. RAG Talk goes through the Express service, which is packaged with PostgreSQL and pgvector in Docker Compose. Splitting them means the heavier lifecycle only exists when documents are involved, rather than being the price of entry for a one-line question.

Retrieval state lives outside the extension. An ingested corpus is long-lived and a Raycast command is not, so the vector store, the documents, and the pipeline that fills them belong to a process with its own lifetime. That leaves the extension responsible for what it is genuinely good at: selecting files, holding a conversation, and rendering a stream.

Stack

Assistant — Raycast · React · TypeScript · Zustand

Retrieval service — Node.js · Express · LangChain

Models and storage — Ollama · nomic-embed-text · PostgreSQL · pgvector · Docker Compose

Engineering challenges

A corpus that disappeared whenever the extension restarted

The problem. The first document mode kept vectors in an in-process memory store inside the extension. Ingesting a file was work you had to redo, because the corpus lived and died with a UI process.

Why it was hard. This was not a matter of swapping one store for another. The ephemeral store was the only reason ingestion could sit inside the extension at all. Persisting embeddings means a database, a database means a process to run and a connection to own, and neither belongs to a Raycast command. Fixing the storage problem pulled directly on the system boundary.

The decision. Move vectors into PostgreSQL with pgvector, then move ingestion and retrieval out of the extension entirely, into a local service packaged alongside the database.

Implementation. The Express service took over document loading, chunking, embedding through local nomic-embed-text into 768-dimensional vectors, writing content and metadata and vectors into pgvector, multipart upload, and metadata-based deletion. Docker Compose brings the service and the database up together. The chat model stays a host process, so the container reaches Ollama through the host-gateway mapping rather than pulling the model inside the container. The extension kept interaction, local conversation files, and the file registry, and talks to the service over HTTP.

Outcome. Documents ingested once stay ingested, and restarting a command no longer destroys work. Each side ended up with a single job, which is what made adding upload and deletion straightforward rather than another thing wired into the UI process.

Streaming an answer across three processes

The problem. Tokens are generated by Ollama, orchestrated by Express, and displayed by Raycast. Waiting for a complete answer before showing anything makes a local model feel broken, because local generation is not instant.

Why it was hard. Three boundaries, each with its own idea of a message. On top of that, Raycast re-renders on state change, so pushing every arriving token into state makes the UI the slowest part of the pipeline. And a stream is a poor thing to build persistence on: the conversation has to end up on disk correctly whether the response finishes cleanly or not.

The decision. Stream over server-sent events for the whole path, and decouple the rate tokens arrive from the rate the UI redraws.

Implementation. The service streams generated tokens back as SSE. The client parses the event framing, accumulates the response, and throttles updates into the Raycast view instead of rendering per token. The completed conversation is written to local storage once the stream ends. Before either mode runs, the extension checks that Ollama is actually alive, so an unavailable model surfaces as an error toast rather than a stream that never starts.

Outcome. Answers appear as they are produced, in both modes, and the display rate is independent of how fast the local machine happens to be generating.

Follow-up questions that made sense to a person and not to a vector search

The problem. People ask document questions conversationally. "What about the second one?" is perfectly clear in a chat and embeds into nothing useful.

Why it was hard. The obvious fix, sending the whole conversation into retrieval, makes the query worse by diluting it, and it competes for the resource that is already scarcest. A locally run model has a finite context window, and every token in it costs generation time on the user's own hardware. Retrieval breadth and conversation history are spending from the same budget.

The decision. Rewrite the question before retrieving anything, and keep both the history window and the retrieved set small on purpose.

Implementation. A history-aware retriever rephrases a follow-up into a standalone query using a bounded window of recent turns. That standalone query goes to pgvector, and the top three chunks are stuffed into a source-aware prompt so the answer can point at the file it came from. The history the client sends to the endpoint is limited rather than unbounded.

Outcome. Follow-ups resolve against the corpus rather than against their literal wording, and what reaches the model stays within what a local model can hold.

One ingestion path, many kinds of file

The problem. Files get picked from Raycast, and they are whatever the user happens to have: PDFs, Word and PowerPoint documents, Markdown, CSV, JSON and JSONL, plain text, and TypeScript or JavaScript source. Retrieval only works if all of that becomes the same thing on the way in.

Why it was hard. Each format has its own parser and its own failure modes, so a batch upload has many independent ways to go wrong. Aborting the whole batch because one PDF would not parse is the wrong behavior for a personal tool. Metadata compounds it: source and filename are what make a document removable later, so if they are not attached consistently at ingestion, a file can be added and then never cleanly deleted.

The decision. Dispatch on file type in one place, normalize everything into a single document shape with consistent metadata, and collect failures per file instead of per batch.

Implementation. The service selects a loader by file type, enriches each loaded document with filename and source metadata before splitting, then chunks, embeds, and stores it. Per-file load errors accumulate and come back to the extension as toasts naming what failed. Deletion works from that same metadata, and the extension's local file registry is cleaned up in step with it so the file picker and the corpus do not drift apart.

Outcome. Adding a document is one action regardless of format, one unreadable file does not cost the rest of the batch, and everything that gets ingested stays addressable for removal.

Outcome

Work Buddy runs as a complete local path: two Raycast commands, a Dockerized retrieval service, a persistent corpus of ingested personal documents, and answers streamed from a model on the same machine. Everything that happens during a conversation, from embedding the question to generating the response, happens locally.

The thing the project validated is the boundary. It would have been easier, at the start, to keep building into the extension, and the memory vector store is exactly what that path looks like. Being forced to persist embeddings is what exposed the real seam, and moving retrieval behind a service made the extension simpler rather than more complicated. The lightweight mode stayed lightweight, and the expensive mode got a lifecycle that matches what it actually does.

It is a single-machine tool by design. The service, the database, and the models are meant to sit together on one trusted laptop, which is the assumption that lets document chat happen without a document ever leaving it.