LLM cost modeling methodology

beforeyouship is a pre-deployment cost model for LLM apps. You describe the architecture and usage pattern; the tool returns a monthly cost forecast across model tiers with honest multipliers for retries, caching, batching, and growth. It is a planning tool for the design phase — before you commit to a stack.

1. Pick an app archetype. Seven presets cover the common cost profiles (chatbot, RAG, multi-step agent, doc processor, etc.). Each one pre-fills sensible token defaults and retry behaviour.
2. Tune the usage pattern. Calls per day, LLM calls per request, average input and output token counts. Paste your prompt instead of guessing — the built-in tokenizer estimates the count locally.
3. Adjust cost multipliers. Retry rate, cache hit rate, batch eligibility, infrastructure overhead. Each one has a defined formula; see How costs are modeled below.
4. Read the cost table. Three columns: Naive (raw tokens × prices), Realistic (your multipliers applied), Worst Case (elevated retry rate and, for agent/RAG, cascading retries).
5. Read the recommendation. The engine picks the best-fit model for your archetype — not just the cheapest. See Recommendation engine for the per-archetype scoring rules.

Every model in the catalog produces three monthly cost numbers. They all use the same input and output token counts; only the multipliers differ.

Base cost per call

base_cost = (in_tokens × in_price + out_tokens × out_price) / 1,000,000

The bare token math — no retries, no caching, no batch. This is what feeds the Naive $/mo column when multiplied by calls/day × 30 × calls/request.

Retry overhead

retry_cost_per_call = base_cost_per_call × retry_rate × 0.3

Default retry rate is 8% (15% for RAG and multi-step agents, which retry more often because failures compound across the chain). The 0.3 factor reflects that most retries are short — they regenerate only the failing call, not the full output.

Cache savings

savings_per_hit  = (in_tokens / 1M) × (in_price - cached_in_price)
write_overhead   = (in_tokens / 1M) × (cache_write_price - in_price) × (1 - hit_rate)
cache_savings    = savings_per_hit × hit_rate - write_overhead

Anthropic charges cache writes at 1.25× the base input price; OpenAI and Google have no write surcharge. Cache reads run 90% cheaper on OpenAI (gpt-5.x), Anthropic, and Gemini, and 98% cheaper on DeepSeek V4 Flash. The formula above captures both sides.

Batch savings

in_save  = (in_tokens / 1M) × (in_price - batch_in_price)
out_save = (out_tokens / 1M) × (out_price - batch_out_price)
batch_savings = (in_save + out_save) × batch_pct

The batch API is roughly 50% cheaper than real-time pricing on both input and output, at the cost of higher latency (minutes to hours). Only useful for offline workloads — the Document processor archetype enables it by default at 80% eligibility.

Infrastructure overhead

A flat 15% line item covers hosting, vector DB, orchestration, and observability tooling that scale with API spend. Adjust under settings; set to 0 if you self-host the infrastructure side.

Putting it together

effective_cost_per_call = base + retry - cache_savings - batch_savings
monthly_cost = effective_cost_per_call
             × calls_per_day × 30 × calls_per_request
             × (1 + infra_overhead_pct / 100)

The three output columns

• Naive $/mo — base cost × call count. No retries, no caching, no batch. The number you’d compute on the back of a napkin. Intentionally muted in the UI; never shown in isolation.
• Realistic $/mo — your multipliers applied as defined above. The primary number — this is the one to plan against.
• Worst Case $/mo — uses an elevated retry rate (default 20%). For RAG and multi-step agent archetypes, retries compound across the chain via a cascading formula. Stress-tests how exposed you are to bad-traffic days.

Token counts drive the entire cost calculation, so accuracy matters. The challenge is that every provider uses a different tokenizer.

The baseline

Paste mode runs the OpenAI native tokenizer (o200k_base) locally in your browser via WebAssembly. Your prompt text never leaves the page.

Per-model adjustment

For non-OpenAI models, the cost engine multiplies the o200k_base count by a calibrated ratio. Approximate values for English text:

• OpenAI, Azure OpenAI, Llama 3.x, Amazon Nova: 1.00 (baseline)
• Gemini 2.5 Flash: ≈ 0.97 (SentencePiece, slightly more efficient on English)
• Mistral Large 3 / Small 4: ≈ 1.03
• Cohere Command R+/R, DeepSeek V4 Flash: ≈ 1.05
• Claude Sonnet 4.6 / Haiku 4.5: ≈ 1.10 (verbose on English)

When to verify

Calibrated ratios are accurate to roughly ±5% for English. They’re less accurate for heavy code, multilingual content, or unusual formatting. If your workload is one of those, paste a representative sample through your provider’s own count_tokens endpoint and compare to the number shown in paste mode — divide the provider’s count by the o200k_base count to get your real ratio.

Seven presets cover the meaningfully different LLM cost profiles. Each preset fills in default token counts, calls-per-request, retry rate, and (where appropriate) batch eligibility. All values stay editable after selection.

Each archetype, in one line plus a typical example

If your app doesn’t fit any preset cleanly, pick the closest one and overwrite the usage inputs. The cost engine doesn’t know which preset you picked — it just uses the numbers you supply. The archetype affects only the recommendation logic and a few UI hints.

The engine picks one model as the strongest fit — not necessarily the cheapest. The scoring rule depends on the archetype.

Step 1 — eliminate gross-outliers

Sort by realistic monthly cost ascending. Drop any model that costs more than 2× the cheapest, with one exception: a premium-tier model within 20% of the cheapest gets a pass on quality grounds.

Step 2 — archetype-specific scoring

Access-aware recommendation

Free users see a recommendation drawn only from the free-tier pool (the model card on the main page). When a cheaper alternative exists in the extended catalog, a FOMO banner below the recommendation surfaces the alternative’s price (without naming the model) and an upgrade CTA. Pro users see a recommendation drawn from the full catalog — consistent across the main card and the model-catalog modal.

Three Pro-only refinements that sit on top of the base cost engine. None of them changes what shows up in the Naive / Realistic / Worst Case columns — they refine how you reason about the inputs.

Input token breakdown

Splits the avg input tokens number into four named layers — system prompt, retrieval context (RAG), conversation history, and the user’s actual message. The total feeds the cost engine identically; the breakdown surfaces which layer is dominating the bill and which layer is the best caching candidate (system prompt and history are the most cache-friendly).

Cost per user / MAU

Divides the realistic monthly cost by your monthly active user count. Useful for pricing-page sanity checks (“we’re at $X / MAU at 1k users, $Y / MAU at 10k”) and board reports. Assumes calls per day scale linearly with users — adjust calls per day for your actual usage curve if it’s super-linear or sub-linear.

Global retry budget

Replaces the 8% / 15% retry-rate heuristic with a deterministic cap: max retries per call, with an optional max retries per hour. If your code has exponential backoff with a hard limit, this is more realistic than the rate-based model. The Worst Case column respects the cap when set.

beforeyouship runs a remote MCP (Model Context Protocol) server, so you can query cost models without leaving your editor — from Claude Code, Cursor, or any MCP client that supports the Streamable HTTP transport. Same cost engine, same pricing data, same Naive / Realistic / Worst Case output as the website. Also listed on Smithery.

Tools

• list_archetypes — the seven app archetypes with their default usage parameters.
• get_model_prices — current per-1M-token pricing for the full catalog, with staleness metadata.
• estimate_cost — full cost model for an archetype at a given usage pattern: Naive / Realistic / Worst Case monthly cost per model, growth scenarios, and the recommendation.

Setup

1. Generate an API key. Account menu → API keys → Generate API key. The key is shown once — store it in your secrets manager. Up to five active keys; revoke any time.
2. Add the server. In Claude Code:

claude mcp add --transport http beforeyouship \
  https://beforeyouship.dev/api/mcp \
  --header "Authorization: Bearer bys_..."

In Cursor or other MCP clients, add a remote server with URL https://beforeyouship.dev/api/mcp and the same Authorization header.

3. Try it. Paste this into your editor:

Estimate the monthly cost of a RAG pipeline at 10,000 requests/day

Demo mode

No key yet? Skip the --header flag and the server runs in demo mode — all three tools work against the six free-tier models. A Pro key unlocks the full catalog across all providers.

Prices come from the official rate cards of each provider. They’re stored in /data/prices.json and reviewed monthly. A weekly GitHub Action checks staleness and pings the maintainer if any entry hasn’t been re-verified in 30 days.

The "!" warning

Any model whose pricing hasn’t been re-verified within the last 30 days gets a small amber "!" next to its name. Hover the icon for the last-verified date and the source URL. It’s a heads-up, not a defect — prices rarely change between checks, but the warning prevents stale numbers from being trusted blindly.

Free vs Pro models

Free tier covers six models that span the four largest providers and the full quality spectrum: GPT-5.4, GPT-5.4 mini (OpenAI), Claude Sonnet 4.6, Claude Haiku 4.5 (Anthropic), Gemini 2.5 Flash (Google), DeepSeek V4 Flash. Pro adds twelve more across Mistral, Cohere, AWS Bedrock (Nova), Azure OpenAI, Together AI, and Groq.

Caching support

Not every provider exposes prompt caching at the API tier. OpenAI, Anthropic, Google, DeepSeek, and Azure OpenAI do; Mistral, Cohere, AWS Nova, Together, and Groq don’t. If you set a cache hit rate > 0, models without caching simply show zero cache savings — their realistic monthly cost is unaffected.

Provider deprecations

DeepSeek’s legacy endpoint names (deepseek-chat and deepseek-reasoner) retire on 2026-07-24 and already route to V4 Flash transparently. Use deepseek-v4-flash going forward — $0.14 / $0.28 per 1M with a 98% cache read discount.

These are real costs that you’ll incur in production but the tool deliberately leaves out. Each one is too project-specific to model honestly with usage inputs alone.

• GPU rental and self-hosted inference. Running Llama or Mistral on your own GPUs replaces per-token billing with hourly GPU rental — a completely different cost structure that depends on QPS, batch size, and GPU class.
• Embedding models. Embeddings are usually a separate budget line (vector indexing, semantic search). Worth modelling, but not in this tool.
• Fine-tuning. Fine-tune training costs and the per-token premium on the resulting model are highly variable; not in scope.
• Vector database. The 15% infrastructure overhead is a rough placeholder. Pinecone, Weaviate, or pgvector at your scale should be modelled separately if they’re a meaningful budget line.
• Human-in-the-loop labeling and evals. Quality assurance, eval datasets, and human review time can dwarf the API bill in regulated domains. Out of scope here.
• Observability tooling. LangSmith, Helicone, PromptLayer, etc. — modelled implicitly through the 15% overhead, but if your observability stack is significant, account for it separately.

Naive estimate

Base token math only — no retries, caching, batch, or infra overhead.

Realistic estimate

Naive + your retry, cache, batch, and overhead multipliers applied.

Worst Case

Realistic with an elevated retry rate (default 20%) and, for RAG/agent archetypes, cascading retries across the chain.

Retry rate

Fraction of calls that fail and are retried once. Default 8% (15% for RAG/agent). Retries are billed at 30% of base call cost (regenerating only the failing call, not the full output).

Cache hit rate

Fraction of input tokens served from prompt cache rather than full input price. Default 40%; realistic range 30–60% on workloads with reusable system prompts.

Cache write

First-time tokens written to cache. Billed at 1.25× the base input price on Anthropic; free on OpenAI and Google.

Cached input

Tokens read from cache on subsequent calls. Billed at 0.1× of the base input price on OpenAI (gpt-5.x), Anthropic, and Gemini; 0.02× on DeepSeek V4 Flash.

Batch API

Asynchronous API tier ~50% cheaper than real-time on both input and output. Minutes-to-hours latency. Available on OpenAI, Anthropic, Google, AWS.

Infra overhead

Flat percentage added to monthly cost to cover hosting, vector DB, orchestration, and observability. Default 15%; adjustable under Settings.

Cascading retries

On agent/RAG archetypes, a single retry can compound across N chained calls. Worst Case uses this formula instead of a flat retry rate.

Tokenizer ratio

Per-model multiplier applied to the o200k_base count to approximate that model's native tokenizer. See Tokenization section.

Premium / mid / budget tier

Internal quality classification used by the recommendation engine. Roughly: frontier models, capable mid-range, and small/cheap models.