How we chunk, embed, store, and retrieve eligibility text — the reasoning behind each choice, what we measured before trusting it, and the two places we deliberately broke from the textbook approach. This is a design note from a live system: a clinical trial retrieval module (:rag), built on Qdrant with local ONNX embeddings, in continuous use for trial discovery.
Why retrieval, not a keyword box
Clinical trial eligibility criteria are long, jargon-dense, semi-structured free text. A patient's real question — "would she qualify with stage III disease and no prior chemotherapy?" — rarely shares vocabulary with the trial document that answers it. Keyword search fails here in a specific, predictable way: it matches surface strings, not clinical meaning, and eligibility text is written by dozens of different sponsors with no shared style.
Retrieval-augmented search — embedding chunks of trial text into a vector space and matching a query by semantic similarity — is built for exactly this shape of problem. The design intent, set before any code was written, was for this to remain a discovery and triage tool, not a decision-maker: it surfaces and explains candidate trials with citations back to source eligibility text, and the final judgment stays with the patient and their oncology team.
Foundational decisions
Four calls were made up front and deliberately not re-litigated during implementation. Each traded something measurable for something else.
Vector store — Qdrant, self-hosted. pgvector's usual pitch is "you already run Postgres" — this project runs MySQL, so that argument doesn't apply here. Qdrant is one purpose-built service rather than a second relational database, with metadata filtering as a first-class feature rather than a bolt-on. A first-class Spring AI vector store, proven in production elsewhere.
Module boundary — a new :rag module. A clean seam, kept independent of ingestion and the root app. Written entirely against Spring AI's VectorStore and EmbeddingModel interfaces — never Qdrant classes directly — so the store or the model can be swapped through configuration, not a rewrite.
Embeddings — local ONNX, in-process. No cloud account, no API key, no per-chunk cost — and clinical text tied to a real patient's situation never leaves the machine. The accepted cost is retrieval quality: a small general-purpose model is the weakest link, and that tradeoff is measured, not assumed (see Best practices, below).
Generation — deferred. Retrieval quality is what makes or breaks a RAG system, and it's measurable on its own. An LLM answer layered on unproven retrieval just launders bad matches into confident-sounding prose. Chunks are surfaced raw, with scores, until retrieval earns trust.
Reversibility as a design constraint. Every one of these is a config change, not an architecture change. Store and model are both accessed only through Spring AI's interfaces — proven when an attempted embedding upgrade (see Where we deviated, below) was reverted with no code touched outside configuration.
Chunking the criteria
This is where most RAG quality lives, and it got the most iteration. The chunking rule is simple to state and was hard to earn: one chunk per eligibility criterion, not a fixed-size token window. A query like "no prior chemotherapy" should match one specific line, and citation back to source is only possible if the chunk is that line.
The header, bullet, and label patterns were not guessed at — they were derived from 50 real trials pulled through the project's own ingestion pipeline, then checked against the full corpus. Some of what that survey found:
| Finding | Consequence for the parser |
|---|---|
| Longest observed criteria block: 13,771 characters — roughly 14× the embedding model's usable window | Coarse, whole-block chunking would be silently truncated and unsearchable past that point |
| 632 asterisk bullets, 163 numbered, zero hyphen bullets across the sample — but hyphens appear mid-sentence ("HER2-positive") | The bullet pattern deliberately does not match `-`, avoiding mis-splits inside clinical terms |
| Unicode bullets (`•`) on 205 lines across 107 trials, unmatched originally | An unmatched bullet falls through to "continuation," silently merging two unrelated criteria into one chunk |
| 23 of 50 sampled trials contained escaped markdown (`\*\*Inclusion Criteria:\*\*`) | Left unhandled, 46% of the corpus would silently miss the header pattern and fall back to unparsed |
| Roughly 5.3% of trials carry no inclusion/exclusion header at all — legacy "DISEASE CHARACTERISTICS:" style records | An explicit fallback chunk type (`ELIGIBILITY_UNPARSED`) rather than a guessed split |
Parent context, without drowning the child
Criteria nest — a bare label like "Absolute neutrophil count (ANC)" means nothing without the threshold in its child bullet. The chunker prefixes a nested criterion with its ancestor text so the chunk stands alone semantically. But a parent can run 500+ characters, and prefixing one whole onto every child made siblings roughly 90% identical — near-identical embeddings mean one query match crowds out the rest, defeating the whole point of per-criterion chunking. The fix: abbreviate the parent to its first sentence or clause before prefixing (max-parent-prefix-length, tuned to 160 characters).
A grouping label with no sentence of its own — "All 3 parts of Study:" — is tracked for nesting structure but never emitted as a standalone chunk and never contributes text to a prefix. Early on, prefixing that exact phrase onto all 42 of a trial's chunks was tried and measured to actively hurt retrieval: it gave every chunk in the trial the same leading phrase, pulling their embeddings toward each other and erasing the distinctions search depends on.
Embedding locally
Embeddings run in-process via Spring AI's ONNX transformer starter — all-MiniLM-L6-v2, 384 dimensions, Spring AI's default, chosen and kept deliberately. No API key, no network call per chunk, and no clinical text — trial eligibility criteria tied to a real patient's search — leaves the machine.
| **384** | embedding dimensions |
| **~256** | word-piece window |
| **80MB** | model size |
| **$0** | per-chunk cost |
The honest tradeoff: a small general-purpose sentence model is not tuned for dense oncology text, and it is knowingly the weakest component in the system. That tradeoff was made assessable rather than left as a guess — see the evaluation set under Best practices — so "upgrade the embedding model" is a decision backed by numbers when it's made, not a vibe.
The vector store and its metadata
Qdrant cannot join to MySQL, so anything that must be filterable at search time has to be duplicated into chunk metadata at index time. Every chunk written carries:
| Field | Purpose |
|---|---|
| `trialExtid` | Hydration key back to MySQL — never the internal numeric id, matching the project's extid-only rule across every API boundary |
| `source` | INCLUSION_CRITERION / EXCLUSION_CRITERION / ELIGIBILITY_UNPARSED / BRIEF_SUMMARY / DETAILED_DESCRIPTION / INTERVENTION / OUTCOME |
| `isExclusion` | Matching an exclusion criterion means the opposite of qualifying — this must survive as its own filterable flag, not be inferred from source |
| `ordinal` | Position within its source, so a citation can say "the 3rd inclusion criterion" |
| `overallStatus` | e.g. RECRUITING — the primary retrieval filter for "can she actually join" |
| `nctId`, `studyType`, `naturalKey` | Public trial ID, interventional/observational, and a human-readable debug key alongside the opaque point id |
Point IDs are content-addressed: trialExtid:source:ordinal hashed into a v3-style UUID (Qdrant requires a UUID or unsigned integer, not an arbitrary string). This makes re-indexing idempotent — the same chunk always maps to the same point, so re-running a backfill overwrites rather than duplicates.
Deliberately not auto-created. Spring AI's initialize-schema is off. Auto-creating the collection would infer vector dimensions at first write — a mismatch between the embedding model and an existing collection would then surface as quietly degraded search results instead of a clear error. A missing collection that announces itself loudly at boot is the better failure mode.Keeping vectors in step with MySQL
MySQL and Qdrant are two separate systems with no shared transaction — there is no two-phase commit available. That means "keep them in sync" is not a solvable problem in the strict sense; the real design question is which failure mode to accept.
MySQL is the source of truth; vectors are derived and fully rebuildable from it. Every chunk can be regenerated from eligibility_criteria and the child records. A rolled-back ClinicalTrials.gov fetch cannot be regenerated without going back to the API. When only one write can be protected, the relational one is protected — so indexing runs after the normalization transaction commits, never inside it.
The cost, stated plainly: this is eventual consistency. There are windows where a trial is searchable relationally but not yet semantically, and the system is built to tolerate that as a degraded search result — never as lost data.
Backfill is the reconciliation mechanism, not a migration script
A resumable, idempotent backfill closes the consistency gap and has four standing jobs: index trials that predate the RAG module, recover trials whose post-commit indexing failed, re-index after a chunking change, and re-index after an embedding-model change. Re-indexing is delete-then-insert per trial — the same idempotency shape used elsewhere for child records — so a chunking change that shrinks the chunk count doesn't leave stale points behind, which an upsert-by-id approach cannot guarantee.
Retrieval and filtering
Retrieval is over-fetched deliberately: several chunks routinely belong to the same trial, so a request for 5 trials fetches 5 × chunkFetchMultiplier (6) chunks, then groups by trial and takes the top-scoring ones. Three filters compose as a portable Spring AI expression, which the store translates to its native syntax:
- recruitingOnly — restrict to actively recruiting trials.
- excludeExclusionCriteria — drop matches against exclusion criteria, since matching one means the opposite of qualifying.
- criteriaOnly — restrict to eligibility-criteria chunks, dropping summaries, descriptions, interventions, and outcomes. See "Where we deviated" below for why this exists and why it isn't the default.
The similarity threshold defaults to 0.0 — deliberately unfiltered. Exact clinical terminology scores around 0.97; conceptual queries score around 0.4. Silently dropping below a fixed threshold makes weak retrieval look identical to an empty corpus, which is a worse failure than showing a low-confidence match with its score attached.
Where retrieval stops
The single most important architectural fact about this system is what it doesn't connect to. There are three separate "trial intelligence" features layered over the same corpus, and it matters that a client understand they don't share a pipeline:
Semantic search never feeds the AI trial assessment, and never feeds the deterministic matcher either. The only code shared between retrieval and matching is EligibilityCriteriaChunker itself — the deterministic matcher reuses it purely to split criteria text into lines it can pattern-match, never touching the vector store. Its own class documentation states the boundary as a design decision, not an oversight:
Why two matching strategies coexist. "Semantic similarity is what produced the failure this exists to fix: 'HR-negative, HER2-negative' and 'HR-positive, HER2-negative' differ by one token, embeddings score them as near-identical, and a triple-negative trial ranked first for an ER-positive patient. Retrieval is excellent at finding candidate trials and blind to polarity; deterministic patterns are the opposite. Each is used where it is strong — retrieval ranks, this decides."
The AI trial assessment is a third, fully separate path again: it sends one trial's eligibility text verbatim, plus a de-identified allowlisted slice of the patient record, straight to Claude — no embedding step, no vector search, no retrieval of any kind. It answers "does this specific trial's criteria fit this specific patient," which is a different question from "which trials, out of thousands, are worth looking at."
A naming trap worth knowing about. The persistedtrial_match/trial_match_criteriontables are schema-shaped for RAG results — columns likequery_text,top_score, and per-criterionscoremirror the retrieval response almost field for field. The one populated run in that table today, though, wasn't written through either matching pipeline automatically — it was inserted directly through the generic CRUD endpoint behind those tables, as a one-off, not by a working integration between the deterministic matcher and this schema. Worth flagging on two counts: a reader inspecting that table shouldn't assume it's semantic-search history (itstop_scorethere is a signals-matched fraction, not a cosine similarity, and the two aren't comparable), and a reader shouldn't assume the deterministic matcher writes to it automatically either — that connection doesn't exist yet.
Where we deviated, and why
Two deviations from the original design, both forced by evidence gathered after the system was live — not by preference.
Deviation — pgvector, planned; Qdrant, shipped. This one isn't a correction; it's the decision recorded above under Foundational decisions. Flagged here because the original design sketch assumed pgvector for the reason many teams do — "you already run Postgres" — and that reasoning was explicitly re-examined and rejected once the actual datastore (MySQL, not Postgres) was accounted for.
Deviation — a criteria-only search mode, added after measurement. A direct test against the live index on a realistic whole-profile query — one chunk carrying four qualifying facts, opening with "Dose escalation..." — found the unfiltered top 25 results were dominated by trial-design prose: 10 detailed-description chunks and only 7 inclusion-criteria chunks, with the same methodology sentence ("This is a first-in-human, open-label, phase I/Ib study...") recurring across unrelated trials. The mechanism: a chunk opening with high-frequency design vocabulary produces an embedding dominated by that vocabulary, drowning out the receptor-status fact in the same sentence. Filtering to criteria-only sources took criteria chunks in the top 25 from 9 to 25 of 25, with zero prose chunks remaining. It's shipped as an opt-in parameter, never a default — a query like "what is this trial testing" is correctly answered by the summary text, and defaulting the filter on would silently break that class of query.
An idea that was tested and deliberately rejected
The same investigation proposed stripping design vocabulary from criteria text before embedding. It was tested against live vectors and the hypothesis held — chunks carrying design vocabulary showed 14.0% prose contamination in their nearest neighbors versus 4.4% for clean chunks. But the defect turned out to affect only about 0.4% of the corpus as a strippable leading prefix, a poor trade against a full re-embed. More importantly:
Prefixes like "COHORT B:" are not noise — they're scope. One trial states different prior-recurrence limits for Cohort B and Cohort C. Strip the prefix and the two criteria become contradictory with nothing left to distinguish them — the same failure class as reading clinical text outside the unit that scopes it. Conclusion: don't strip. The correct fix is recording cohort/arm scope as structured metadata rather than deleting it from the text, and it's queued as chunker work rather than a quick regex.
Best practices carried through
- Interfaces over vendor classes. Every line of retrieval and indexing code is written against Spring AI's
VectorStore/EmbeddingModelabstractions, never Qdrant's SDK directly — proven by an embedding-model upgrade attempt that reverted cleanly with zero code changes outside configuration. - Evaluate before tuning. A versioned query set with known-good expected trials turns "does this chunking change help" into a before/after number instead of a feeling. It runs against the live stack over HTTP — query, embed, filter, hydrate — the same path a real user takes, and skips (never fails) a normal build when no backend is running.
- Fail loud at the boundary, soft in the loop. A missing vector-store collection is checked once up front and reported as one clear message, rather than one identical error per trial that reads as a data problem. Inside a backfill run, one bad trial is logged and skipped so it never aborts the other thousands.
- Every tunable value is a measured, documented number — parent-prefix length, prose chunk size, the fetch multiplier — each with the observation that produced it recorded next to the config key, in
application.ymland inRagProperties, not lost in a commit message. - Read the log before theorizing. A production 500 was chased through two wrong guesses — a filter-parser theory, a Qdrant translation theory — before the backend's own log line named the real cause in one sentence: a chunk pointing at a trial a database rebuild had invalidated. The fix, and the lesson, are both now load-bearing.
Current state and open work
Live and in continuous use for trial discovery. Several threads remain open, already scoped rather than hypothetical:
- Ingestion and indexing are still two manual steps. The original design called for indexing to trigger itself automatically after a trial finishes ingesting — an application event published after the database commit, consumed by the indexer, so the two modules never depend on each other directly. That hook was never built. Today pulling new trials and preparing them for search are two separate REST calls, run by hand in sequence — deliberate in the sense that indexing failures can never roll back ingested data, but the automatic reconciliation piece of the plan is still outstanding.
- Qualifying facts outside the criteria text. A trial's official title and structured conditions list often state the qualifying profile more precisely than the criteria prose does — and neither is indexed yet. Bringing conditions in requires a schema change (a proper trial-to-condition join, replacing today's global deduplicated lookup table) and a re-ingest before the chunker work can start.
- Orphan detection at the boundary. A database rebuild regenerates every trial's external ID while Qdrant keeps its old points, and nothing currently reconciles the two automatically — retrieval tolerates it per-hit (see Retrieval and filtering, above), but a startup check comparing distinct trial counts between the two stores would catch the drift at boot instead of mid-search.
Questions
If you're weighing RAG against a simpler search, or already have one that isn't earning trust, I'm happy to go through where this held up and where it didn't.