LLM Provider Comparison: OpenAI vs Anthropic vs Google

OpenAI, Anthropic, Google, and hosts of open-weight models can all be candidates for a business AI system. A provider-wide claim such as “Claude is best for contracts” or “GPT is best at tools” needs a defined model version, task, and test. Without that evidence, it is an opinion rather than a reliable comparison.

Provider documentation checked on 15 September 2026. This article describes how to compare candidates; it does not report a Fernside benchmark or customer deployment.

Compare Models and Deployment Routes

OpenAI offers models and API features with different supported inputs, tools, costs, and lifecycle dates. Select the specific model and endpoint you intend to use rather than treating the whole GPT family as one configuration.

Anthropic documents the Claude models’ capabilities and context limits by model. Availability also varies by deployment platform. Check the Claude model catalogue against the features your application needs.

Google documents Gemini models with different input modalities, context limits, and lifecycle states. Confirm whether a candidate is stable or preview and whether you are buying through the Gemini API or a Google Cloud service. See Gemini models.

Open-weight models expose model weights under a licence. That is not automatically the same as unrestricted open-source software. Hosted inference adds the host’s terms, capacity, and data handling; self-hosting adds infrastructure and operational work. Check the specific model licence before commercial deployment or redistribution.

A direct API, a cloud marketplace deployment, and a consumer chat subscription are different products. Compare the contracts and features of the route you will actually buy.

Build a Task-Based Comparison

Use a fixed set of inputs from the intended workflow. Keep sensitive data within approved processing arrangements.

Task	What to measure
Answers over company documents	Supported facts, citations, missing-answer handling, access isolation
Contract or document analysis	Relevant passages found, omissions, false assertions, input limits
Structured extraction	Field accuracy and schema validity separately
Tool use	Correct tool, valid arguments, authorisation, duplicate side effects
Classification	Errors by class, especially costly false positives and negatives
Customer-facing responses	Correctness, tone, escalation, latency, and cost

A valid JSON response can still contain an incorrect invoice amount. A large context window does not prove that the model will find every relevant clause. Treat feature support as an entry requirement and measured task success as the selection evidence.

Include failure cases and have a knowledgeable person review the results. OpenAI’s evaluation guidance recommends task-specific evaluations, representative data, and continued evaluation as systems change. Apply the same discipline across providers.

Calculate Cost from Usage

Do not assign a generic price range to “flagship” or “mid-tier”: those labels do not define a model, billing mode, or rate. Record the candidate’s input, output, cached-input, and any tool or storage charges from its current price list.

The text-token component is:

monthly input tokens ÷ 1,000,000 × input rate + monthly output tokens ÷ 1,000,000 × output rate

For an explicitly hypothetical workload of 10,000 requests a day for 30 days, with 2,000 input tokens and 300 output tokens per request, monthly usage is 600 million input tokens and 90 million output tokens. At hypothetical rates of $1 input and $5 output per million, that component would cost $1,050. Those rates are arithmetic assumptions, not a vendor quote.

Add retries, fallback calls, reasoning or tool charges where applicable, embeddings, retrieval, hosting, and human review. Measure tokens with the selected model’s accounting rather than assuming that an identical document yields identical token counts everywhere.

Compare cost per successful business outcome. A cheaper call that requires repeated corrections can cost more overall.

Reliability, Data Handling, and Support

Lifecycle. Pin a supported version where available and monitor retirement notices. Pinning does not prevent a vendor from withdrawing a model. OpenAI publishes deprecation dates and migration information; include lifecycle review in your operational process.

Service behaviour. Test timeouts, rate limits, and recovery. Historical status pages and contractual SLAs answer different questions. Neither supports an unsupported claim that one provider has more spare capacity or better support for your business.

Data terms. Check training use, retention, subprocessors, processing locations, deletion, and contractual commitments. For example, OpenAI states that API data is not used for training by default unless the customer opts in, while retention and regional controls vary by feature and eligibility. See OpenAI API data controls. Do not infer another provider’s terms from that policy.

Support. Confirm the purchased support level and escalation route. A named account contact is a procurement choice, not a general UK GDPR requirement, and buying through a hyperscaler is not the only possible compliant arrangement.

Routing and Fallback

Keep provider-specific request and response handling in an adapter when that makes future changes practical. Preserve prompts, evaluation cases, and source documents outside a provider playground.

A fallback should meet the same permission, privacy, quality, and safety requirements as the primary. Use it for defined availability failures; do not automatically send a safety refusal to another model in an attempt to obtain the refused output.

Select routing rules using measured task performance. A model’s statement that it is “confident” is not a calibrated probability of correctness. Check that the routing decision actually predicts failures before relying on it.

Make the Decision Reviewable

Record the model identifiers, deployment route, test date, evaluation results, cost assumptions, data terms, and remaining limitations. Re-run the comparison when a model changes or the workload materially changes. That gives the next reviewer evidence for the choice and a starting point for migration planning.

Where Fernside Fits

Advisory is the right starting point when the question is “which provider, and how many.” We then implement the interface, evals, and routing as AI systems so the answer is not “we hardcoded GPT because the tutorial did.”

Want an LLM architecture recommendation against your workload and risk tolerance? Book a discovery call and we will model quality, spend, and fallback before a volume commitment.