aka Feature-Training-Inference Split, FTI Architecture for LLMs
Decompose an LLM/RAG system into three independently-deployable pipelines — feature, training, inference — communicating only via a feature store and a model registry. A RAG-and-fine-tuned-model product splits into three pipelines. The data team owns the feature pipeline that ingests Confluence and Salesforce, embeds, and writes to Pinecone. The ML team owns the training pipeline that periodically pulls eval-curated feature subsets and produces DPO-tuned models registered in MLflow. The platform team owns the inference service that reads Pinecone at request time and loads the current registered model. Each team deploys without coordination.
An LLM application team owns data ingestion (cleaning raw documents into RAG features), model adaptation (SFT / DPO over the resulting datasets), and serving (retrieval + generation). Each axis has different cadence, hardware, and team ownership. Bundling them into one repository and deploy cycle couples otherwise independent work.
A monolithic LLM application makes every change touch every team. Re-embedding the corpus requires a deploy that the inference path inherits. Bumping the SFT recipe forces retraining tied to the inference release cycle. Serving SLOs are held hostage by data-pipeline failures. Without a clean decomposition along the F/T/I axes, teams step on each other and the system drifts toward incoherent versioning.
Competing forces:
flowchart LR Raw[Raw data] --> Feat[Feature pipeline] Feat --> Store[(Feature store)] Store --> Train[Training pipeline] Train --> Reg[(Model registry)] Store --> Inf[Inference pipeline] Reg --> Inf Inf --> User
Therefore: decompose into three pipelines communicating only via a feature store and a model registry, so each pipeline iterates on its own cadence with its own ownership and no direct code coupling.
schema FeatureRecord:
doc_id: str
chunk_id: str
text: str
embedding: list[float]
meta: dict
schema ModelArtifact:
name: str
version: str
weights_uri: str
eval_score: float
# --- Feature pipeline: ingest -> clean -> chunk -> embed -> store. -
function feature_pipeline(raw_docs):
for doc in raw_docs:
cleaned = normalise(doc)
for chunk in split(cleaned):
rec = FeatureRecord(
doc_id=doc.id, chunk_id=uuid(),
text=chunk.text, embedding=embed(chunk.text),
meta=chunk.meta,
)
validate(rec, FeatureRecord)
feature_store.upsert(rec)
# --- Training pipeline: features -> SFT/DPO -> model registry. ----
function training_pipeline():
examples = feature_store.read_training_split()
model = supervised_finetune(base=BASE_MODEL, data=examples)
score = eval_suite.score(model)
artifact = ModelArtifact(name="qa-tuned", version=git_sha(),
weights_uri=upload(model), eval_score=score)
model_registry.publish(artifact)
# --- Inference pipeline: feature store + registry, at request time.
function answer(query):
schema = StructuredSchema({"answer": str, "citations": list[str]})
model = model_registry.load_latest("qa-tuned")
docs = feature_store.search(query, k=5)
reply = model.generate(prompt=render(query, docs),
schema_hint=schema.to_json_schema())
parsed = parse_and_validate(reply.text, schema)
return parsed
# Why this works: the only contracts that cross pipeline boundaries
# are the two typed integration surfaces, so each pipeline iterates
# on its own cadence with its own ownership and no direct code
# coupling crosses team lines.
# Related: see `business-llm-microservice-split`, `llm-as-periphery`.schema FeatureRecord:
schema_version: str; doc_id: str; chunk_id: str; text: str
embedding: list[float]; embed_model: str
schema ModelArtifact:
name: str; version: str; weights_uri: str
expected_feat_ver: str; expected_embed_model: str
function feature_pipeline(raw_docs):
for d in raw_docs:
for ch in split(normalise(d)):
feature_store.upsert(FeatureRecord(
schema_version="feat.v4", doc_id=d.id, chunk_id=uuid(),
text=ch.text, embed_model=EMBED_MODEL,
embedding=embed(ch.text, model=EMBED_MODEL)))
function answer(query):
schema = StructuredSchema(RagAnswer)
artifact = model_registry.load_latest("qa-tuned")
docs = feature_store.search(query, k=5)
for d in docs:
if d.schema_version != artifact.expected_feat_ver: raise FeatureDrift()
if d.embed_model != artifact.expected_embed_model: raise EmbedDrift()
reply = artifact.generate(prompt=render(query, docs),
schema_hint=schema.to_json_schema())
return parse_and_validate(reply.text, schema)
# Why this works: version metadata on both integration surfaces makes drift
# fail loudly at request time, not silently in answers.
# Related: see `business-llm-microservice-split`, `cdc-vector-sync`.schema FeatureRecord:
doc_id: str; chunk_id: str; text: str
embedding: list[float]; op: str # "upsert" | "delete"
schema RagAnswer:
answer: str; citations: list[str]
schema = StructuredSchema(RagAnswer)
function on_cdc_event(event):
if event.op == "delete":
feature_store.delete(doc_id=event.doc_id); return
for ch in split(normalise(event.doc)):
rec = FeatureRecord(doc_id=event.doc.id, chunk_id=uuid(),
text=ch.text, embedding=embed(ch.text), op="upsert")
validate(rec, FeatureRecord); feature_store.upsert(rec)
function answer(query):
model = model_registry.load_latest("qa-tuned")
docs = feature_store.search(query, k=5)
prompt = render(query, docs)
for attempt in range(3):
reply = model.generate(prompt=prompt, schema_hint=schema.to_json_schema())
try: return parse_and_validate(reply.text, schema)
except ParseError as e: prompt += "\nPrev reply invalid: " + str(e)
raise ExhaustedRetries()
# Why this works: CDC keeps the index near-real-time with source-of-truth;
# validation+retry stops single bad completions from failing the request.
# Related: see `cdc-vector-sync`, `structured-output`.composes-with → business-llm-microservice-splitcomposes-with → cdc-vector-synccomposes-with → streaming-feature-pipelinecomplements → naive-raguses → vector-memorycomplements → augmented-llmcomposes-with → crawler-dispatcher