Jun 4, 2026 Omar Rizwan Undergraduate Intern, Spring 2026

Building a RAG Chatbot for the Utah Digital Newspaper Archive

When I joined the Digital Matters Lab last year, I was handed a problem I didn’t expect to find in a humanities space: how do you make roughly 300 million newspaper article chunks actually searchable by someone who doesn’t know the exact headline or date they’re looking for? The Utah Digital Newspapers collection is a remarkable archive, over a century of Utah history digitized and waiting, but keyword search only gets you so far.

My project under the guidance of Rebekah Cummings and Harish Mariganti, was to build a Retrieval-Augmented Generation (RAG) chatbot that lets users ask the archive questions in plain English and get synthesized, cited answers pulled from the primary sources themselves.

The technical side was the hard part. Scaling RAG to 300M vectors forced me to confront infrastructure problems I’d only read about. A flat FAISS index would have been around 450GB, which was unusable. I moved to an IVF+PQ compressed index, which got it down to roughly 15GB, paired with SQLite for metadata and on-demand CSV lookups to keep memory
reasonable. After several iterations on chunking and retrieval, I settled on ALL-MINI-LM-L6-V2 for embeddings and Groq’s Llama 3.3 70B for synthesis, with Ollama as a local fallback. The
first time the full index built end-to-end without crashing was one of the better moments of my semester.

The internship dovetailed tightly with my CS coursework. My classes cover systems, ML, and software engineering in isolation, but this project forced me to integrate all of it: vector
databases, embedding models, API design, frontend, and the systems thinking needed to keep a 300M-vector index from falling over. Applying for the internship was a chance to do applied AI
work on a dataset with real cultural weight, not just another benchmark.

What I took away: infrastructure matters more than model choice. The LLM is the flashy part, but the retrieval layer is what determines whether answers are grounded or fabricated. I also
came away with real respect for the librarians and archivists who understand these collections in ways no embedding model will capture soon.

For future applicants, pick a project that forces you to learn something you don’t already know. Scope small, but design for scale from day one. And talk to your manager early and often. The
humanities side of Digital Matters sharpened my engineering instincts in ways pure CS never would have.