Fixed scope · written delivery · no meetings

A RAG system that refuses to guess — and the numbers to prove it.

TriageOps is a working demo: a staged agent over eight procedure documents that checks its evidence at every step, computes the decision in code, and escalates when the evidence is missing. The same pipeline is available as fixed-price work on your own documents.

See the three packages Read the source ↗

20synthetic evaluation cases, fixed seed

25%escalated to a human on purpose

6checkpoints that refused to pass

36tests over rules, tools, citations

What is actually measured

The interesting number is not accuracy.

A demo that says “100% correct” is measuring itself against its own rule engine. The numbers worth reading are the ones where the pipeline stopped: the cases it sent to a human, and the checkpoints that failed because a required record did not exist.

Metric	Value	Read it as
Cases	20	fixed synthetic set, re-runnable
Decision match vs. rule engine	100%	a consistency check, not an accuracy claim
Escalation rate	25%	cases routed to a human instead of guessing
Checkpoint failures	6	all: no capability record available
Average tool calls / case	5.55	bounded at 6, then escalate
Tests	36	rules, tool arguments, checkpoints, citations

Provider: mock, no network, so latency and token counts describe the harness rather than a hosted model. Run python -m app.cli --eval in the repository to reproduce the table.

TriageOps pipeline: intake, evidence tools, checkpoints, rule-based decision, cited justification — Pipeline: the model writes the justification; the decision is rule code.

Running a single warranty claim through the Streamlit interface — One claim end to end, with the evidence set visible.

Guardrail view showing an unsupported citation being rejected — A citation that cannot be resolved is rejected before it reaches the user.

Evaluation tab with per-case verdicts — The evaluation tab: per-case verdict, confidence, tool calls.

Three fixed-scope packages

Start with the measurement, not the rebuild.

All three are delivered as documents, code and a report. Nothing is scoped by the hour, and nothing depends on a call.

Diagnose

RAG failure report from 20 of your own questions

$1503 days · 1 revision

Your current assistant run against 20 questions you choose
A verdict per question: answered, answered wrongly, or refused
Failure modes ranked by user impact
The eval script, so you can re-run it after every change

This package measures. It does not change your system.

Order on Upwork ↗

Build

A document chatbot that cites its sources

$6007 days · 2 revisions

Answers only from your files, with the source inline
Every citation validated before it is shown
Refuses and says what is missing when evidence is absent
Source code, Dockerfile, 20-question baseline

Single corpus, one interface. Multi-team rollout is out of scope.

Order on Upwork ↗

Full build

Citation-checked RAG assistant over your documents

$90010 days · 2 revisions

Multi-document corpus with retrieval tuning
Staged checkpoints: evidence, decision, justification
Evaluation harness and a written failure analysis
Handover notes your team can maintain from

Scoped to one corpus and one deployment target. Not a managed service.

Order on Upwork ↗

How it runs

Four steps, all in writing.

You send the materialA folder of documents and, for the $150 report, the 20 questions that matter to your users.

I run the baselineThe current behaviour is recorded case by case before anything is changed.

You get the verdictA ranked failure list with the evidence for each entry, plus the script that produced it.

Fixing is a separate scopePriced after the diagnosis, so the estimate is based on what actually breaks.

Everything is delivered in writing. No calls are required — which also means every decision has a paper trail you can hand to whoever signs off.
The $150 report measures; it does not fix. It says what the fix should be and ranks it. Implementation is a separate scope, priced after the diagnosis.
No accuracy number is promised before the evaluation runs. Any figure quoted up front would be invented. The 20-question baseline exists so the number comes from your data.

Questions people actually ask

Before you order.

Is the demo a real product I can use?

No. TriageOps is a portfolio demonstration over synthetic manufacturing documents generated by a fixed seed. It exists to show the architecture and the failure behaviour, and the repository says so at the top.

Why is there no accuracy percentage?

Because no evaluation has been run against your data yet. The demo’s 100% decision match is a consistency check against the same rule engine that produced the expected answers — it is not evidence of accuracy on unseen cases.

Do we need a video call?

No. Everything is written: scope, baseline, findings, handover. If your procurement process requires a call, say so in the job post and it can be arranged — but the default is asynchronous.

What do you need from me to start?

For the $150 report: the documents your assistant currently uses, and 20 questions with the answers you would accept. For the builds: the same corpus, plus one sentence on who reads the output.

What happens if the documents are confidential?

Work can run on a de-identified sample, on your machine via a script you run, or on a subset you approve. No client data is published — the demo screenshots use synthetic records only.

Which providers does this work with?

The LLM layer is an interface with three adapters: a deterministic mock, OpenAI, and Anthropic. Retrieval is local BM25 + TF-IDF, so the stack does not depend on a hosted vector database.