Cost & the business case
What a conversation costs, and how the number is built
A text session with the assistant costs about 430× less than a human agent handles the same conversation. Every figure here is generated by a transparent cost model (mlops/cost_model.py) and regenerates from one command:
PYTHONPATH=. uv run python -m mlops.cost_modelThe assumptions, the per-stage price of a turn, the session totals, and the division all follow.
The assumptions
Provider list prices (2026), each tied to the stage it pays for:
| Stage | Provider / unit | Price |
|---|---|---|
| Generation (workhorse) | Groq Llama 3.3 70B | $0.59 / $0.79 per 1M in/out tokens |
| Routing tie-break | Groq Llama 3.1 8B | $0.05 / $0.08 per 1M in/out tokens |
| Embedding | Cohere embed-v4 | $0.12 per 1M input tokens |
| Rerank | Cohere rerank v3.5 | $0.002 per search |
| Human agent | fully-loaded rate | $20.00 per hour |
Usage per turn and per session:
| Assumption | Value | Why |
|---|---|---|
| Prompt tokens | 1,500 | retrieved context + history + system |
| Completion tokens | 250 | a concise, grounded answer |
| Turns per session | 8 | a typical assistant conversation |
| 8B tie-break share | ~15% | only ambiguous turns pay for the model call |
| Human handle time | 4 min / turn | the lever the whole comparison turns on |
The math, stage by stage
One text turn is the sum of the stages it actually runs:
| Stage | Calculation | Cost |
|---|---|---|
| Routing | (1500 × $0.05 + 250 × $0.08) / 1M × 15% | $0.000014 |
| Retrieval | 25 tok × $0.12/1M + $0.002 rerank | $0.002003 |
| Generation | 1500 × $0.59/1M + 250 × $0.79/1M | $0.001082 |
| Text turn total | $0.0031 |
One session is eight of those turns. The human baseline is a fully-loaded agent’s time:
| Per turn | × 8 turns = per session | |
|---|---|---|
| This assistant (text) | $0.0031 | $0.0248 |
| A human agent | $20/hr × 4 min = $1.3333 | $10.6664 |
\[\frac{\$10.6664}{\$0.0248} \;\approx\; 430\times\]
That is the whole derivation. Change the handle time to three minutes, or the model to a frontier one, and the ratio moves, the point is that it moves in the open, from constants you can see.
The full comparison, one model
| Option | Per session | Versus a human | What you trade |
|---|---|---|---|
| This assistant (text) | $0.025 | ~430× cheaper | handles the routine majority; escalates the hard cases |
| This assistant (voice) | $0.28 | ~38× cheaper | the premium voice, not the model, is most of the cost |
| A frontier model in its place | $0.08 – $0.13 | ~80–130× cheaper | 3.3–5.1× the text cost, for no measured quality gain |
| A human agent | $10.67 | 1× (baseline) | the gold standard for the hard cases, too costly for the routine ones |
What it means for the business
The case is not “replace the humans.” It is deflection: the assistant resolves the routine majority itself, so the human team spends its expensive minutes on the cases that actually need them. The live business dashboard measures the real containment rate from traffic (/api/admin/business), so the saving is tracked, not assumed. The decision to skip a frontier model is the same logic in miniature: it would 3–5× the per-turn cost, and the evaluation showed no lift, so the money stays where it buys something.
The honest caveats
- These are list prices and a bottom-up estimate, not a metered production invoice. Volume discounts and busier sessions move them.
- The 4-minute handle time and the $20/hour loaded rate are the two levers the whole ratio turns on. Both are stated as constants so you can substitute your own numbers and rerun.
- Every figure regenerates from
mlops/cost_model.py, so the economics stay honest as prices and models change, and this page can never drift from the code.