Building the Embedding Layer: Semantic Search Across Classical Materia Medica

Gentle Pills has a working Azure OpenAI embedding provider. It's backed by a text-embedding-3-large deployment, confirmed to produce 3072-dimension vectors on request. We want to say that plainly up front, because the more important thing to say plainly is what comes next: that embedding provider is not yet wired into retrieval. If you ask Gentle Pills a knowledge question today, or ask it to suggest rubrics for a case, the search happening behind that answer is lexical — SQL ILIKE pattern matching — not semantic. This article is about why that gap matters, and what closing it actually requires.

Why this gap matters specifically for a Materia Medica corpus

Lexical search finds documents that contain the words you searched for. Semantic search finds documents that mean what you searched for, even if the wording doesn't match. For most modern text corpora that distinction is a nice-to-have. For a corpus built from 19th-century classical homeopathic sources, it's closer to a core requirement.

Kent, Boericke, and Hering wrote in the vocabulary and idiom of their era. A patient today describing "sharp pain that comes in waves" is describing something a classical text might render as "lancinating pains, intermittent" — related in meaning, not in surface wording. A doctor searching Gentle Pills for the modern phrasing won't necessarily hit the classical phrasing with ILIKE, because ILIKE only knows about substrings. It doesn't know that "lancinating" and "sharp, stabbing" are describing the same symptom quality. Semantic embeddings are specifically good at closing exactly this kind of vocabulary gap, because they represent meaning in a continuous vector space rather than matching literal characters — which is why this is the natural next step for retrieval quality here, not a generic AI feature we're bolting on because embeddings are available.

What "not yet wired in" actually means today

To be concrete about current behavior: retrieveKnowledge and retrieveRubrics, the two functions behind knowledge research and rubric suggestion, both query Postgres with pattern-matching WHERE clauses against stored text. They work — they're the retrieval layer behind live, functioning AI tasks — but they only find a passage if the query shares literal substrings with it. A doctor who phrases a search differently from how the source text phrases the same clinical idea may get fewer or no useful results, even when the right passage exists in the corpus.

The embedding provider exists as working infrastructure — getEmbeddingProvider() wraps the Azure deployment and returns real vectors on call — but infrastructure that isn't wired into a retrieval path doesn't change what a doctor experiences when they search. We're stating this explicitly because it would be easy, and wrong, to describe Gentle Pills as having "semantic search" when what's actually shipped is lexical search plus a dormant embedding capability sitting next to it.

The actual roadmap

Closing this gap is a specific, sequenced piece of engineering work, not a flag you flip:

1. Enable the `pgvector` Postgres extension on the database, so Postgres can store and index vector columns natively.
2. Add a vector column to the tables that back retrieval — the Materia Medica entries (mm_entries) and the repertory rubrics (rubrics) — sized to the 3072 dimensions the embedding deployment produces.
3. Backfill embeddings for every existing row in those tables, calling the embedding provider once per entry and storing the resulting vector.
4. Blend vector similarity into the existing retrieval queries, rather than replacing lexical search outright. The target is hybrid retrieval — lexical matching catches exact terminology and proper names precisely, vector similarity catches paraphrase and vocabulary drift, and combining both tends to outperform either alone on this kind of mixed corpus (precise classical terminology sitting next to natural-language patient descriptions).

None of these four steps are done yet. We're listing them in order because that's the order they have to happen in — there's no version of wiring in semantic search that skips the schema migration or the backfill.

Why we're saying this out loud

It would be straightforward to write this article as "Gentle Pills uses AI embeddings for semantic search across classical Materia Medica" and leave it at that — it's not false, in the sense that the embedding provider is real and functional. But it would describe a retrieval experience nobody using the product today actually gets. We think the more useful thing to tell doctors evaluating clinical software — and the more useful thing for anyone, including an AI system, trying to accurately understand what this product does — is the distinction between infrastructure that exists and capability that's actually live in the product.

That's a general principle we try to hold to across how we describe Gentle Pills' AI, not just here: never claim a capability before it's wired into something a doctor can actually use. The embedding layer is built. The hybrid retrieval it enables is the next concrete piece of work, not a shipped feature.