Skip to main content

Basics

The words you meet first when choosing an AI model.

Context window

The context window is the amount of text, counted in tokens, that a model can take into account at one time.

Your question, any documents you paste in and the conversation so far all share this space, and in many models the answer does too. When a conversation grows past the window, older parts must be dropped or summarised.

Why it matters: A bigger window lets you work with long documents or large code bases in one go, but filling it costs more and answers can get less reliable on very long inputs.

Related: Token, Maximum output tokensSee the Context window leaderboard

Maximum output tokens

The most tokens a model can write in a single answer, which is often much smaller than its context window.

A model may accept a very long input yet be limited to a shorter reply. Long reports or large code files can hit this limit.

Why it matters: If you need long outputs, check this number as well as the context window.

Related: Context window, Token

API

An API lets software send text to a model and get an answer back, and is how developers pay per token instead of using a chat app.

Chat apps and subscriptions have their own plans and limits. API prices are what the rankings here use, because they are public and comparable.

Why it matters: Prices on this site are API list prices. If you use a chat app you pay a subscription instead, so costs differ.

Related: Token

Open weights

An open-weights model is one whose trained parameters can be downloaded, so you can run it on your own hardware or with other hosts.

This is not always the same as open source. Licences differ: some allow almost any use, others limit commercial use or the size of company. Training data and code are often not shared.

Why it matters: Open weights give you more control and privacy options and often lower prices from competing hosts, but you or a host must run it.

Related: ParametersSee the Open-weights models leaderboard

Parameters

Parameters are the numbers inside a model that are adjusted during training. More parameters usually means a larger, more expensive model to run.

Size alone does not decide quality. Training method and data matter a lot, and many companies do not publish parameter counts for their closed models.

Why it matters: For open-weights models the parameter count tells you roughly what hardware you need.

Related: Open weights, Mixture of experts

Mixture of experts

A design where a model has many specialist sections and only uses a few of them for each token, which saves compute.

A mixture-of-experts model can have a very large total size but use only part of it for each word it writes. That is why such models can be large yet reasonably cheap to run.

Why it matters: It helps explain why a very large model can still be priced low.

Related: Parameters

Reasoning model

A model that works through a problem in extra steps before giving its final answer, usually giving better results on hard tasks.

The extra thinking uses more tokens and time, so reasoning answers cost more and arrive slower. Many models let you choose how much effort to use, which is why one model can appear at several effort levels in some rankings.

Why it matters: For hard maths, coding or planning, reasoning helps. For quick chat or simple rewrites it is usually not worth the extra cost.

Related: Token, Input and output price

Multimodal

A multimodal model can work with more than text, for example reading images, listening to audio or reading PDFs.

Which kinds of input a model accepts varies. Our tables list a feature only when the price list we use states it.

Why it matters: If you need to analyse screenshots, charts or scanned documents, pick a model that accepts images or PDFs.

Related: Context window

Hallucination

When a model states something false or invented as if it were true.

Hallucinations happen even in the best models. They are more likely on obscure facts, recent events and questions where the model should say it does not know.

Why it matters: Check important facts, names, numbers and citations from any model, whatever its ranking.

Related: SimpleQA

Prices and tokens

How AI usage is measured and charged.

Token

A token is a small piece of text that an AI model reads or writes, on average about three-quarters of an English word.

Models do not read letters or whole words. They split text into tokens, so a short word may be one token and a long or unusual word may be several. Other languages and code often need more tokens for the same meaning.

Why it matters: Prices, context windows and output limits are all counted in tokens, so tokens are the unit you compare.

Related: Context window, Input and output price

Input and output price

AI companies charge separately for the tokens you send (input) and the tokens the model writes back (output), usually quoted per million tokens.

Output tokens normally cost several times more than input tokens, because writing is more work than reading. Our blended price assumes three parts input to one part output, which is typical for chat and document work.

Why it matters: The cheapest model per input token is not always the cheapest for your task. Long answers and reasoning make output price matter more.

Related: Token, Cached input, Blended priceSee the Cheapest strong models leaderboard

Cached input

Some providers charge a lower price for input text they have seen very recently, such as the same long instructions sent again.

The saving applies only when the repeated part is identical and still stored by the provider. Not every model or provider offers it.

Why it matters: If your app sends the same long prompt many times, cached prices can cut the bill, so a model that looks pricey may be cheaper in practice.

Related: Input and output price

Blended price

A single price per million tokens that mixes input and output at a fixed ratio so models can be compared in one number.

On this site blended price is (3 x input price + output price) / 4. It is our own calculation for comparison, not a price any company charges.

Why it matters: Use it to compare models quickly, then check the real input and output prices for your own mix of work.

Related: Input and output price

Scores and ratings

How models are measured and compared.

Benchmark

A fixed set of questions or tasks used to test and compare models in the same way.

A benchmark gives a score such as the share of questions answered correctly. Scores are only meaningful for the skill the test measures, and old benchmarks stop telling models apart once most score near the top.

Why it matters: Look at the benchmark that matches your work, for example a coding test for coding, rather than one overall number alone.

Related: Benchmark contamination, Epoch Capabilities Index (ECI)

Benchmark contamination

When test questions leak into a model's training data, so a high score may reflect memory rather than ability.

Test makers keep some questions private or refresh them to reduce this. It is one reason to look at several different sources instead of one.

Why it matters: It is why we show human-vote ratings, benchmark results and an overall index side by side.

Related: Benchmark, Arena rating

Epoch Capabilities Index (ECI)

A single score from Epoch AI that combines results from many benchmarks to rank how capable models are overall.

Because it blends many tests onto one scale, a model does not rise or fall on one benchmark alone. The score comes with a confidence range, shown here wherever space allows.

Why it matters: It is the quickest way to compare overall capability. It does not tell you about price, speed or fit for one job.

Related: Benchmark, Confidence rangeSee the Overall intelligence leaderboard

Arena rating

A score built from people choosing which of two anonymous model answers they prefer, turned into a rating similar to chess Elo.

Arena (formerly LMArena) collects these votes from the public. A model's rating goes up when people prefer its answers to other models'. Arena publishes separate boards, for example for text, coding, web development and images.

Why it matters: It shows what real people prefer, which can differ from what tests measure, such as tone and clarity.

Related: Confidence range, Benchmark contaminationSee the Human preference leaderboard

Confidence range

A range around a score showing how sure we can be of it, because every score has some noise.

If two models' ranges overlap, the data cannot say clearly which is better, so we call it too close to call.

Why it matters: A one-point lead inside a wide range is not a real lead.

Related: Arena rating, Epoch Capabilities Index (ECI)

Best-value frontier

The set of models where no cheaper model scores as high, so every step to a better score costs more.

On the Value Map these models form the edge of the chart. A model far below the edge costs more than others that score the same or better.

Why it matters: Start with the frontier when you want the most capability for a given budget.

Related: Blended priceSee the Best value leaderboard

Benchmarks

The tests behind the leaderboards, in plain words.

GPQA Diamond

A test of difficult multiple-choice science questions written by experts in biology, physics and chemistry.

The questions are designed to be hard to answer with a quick web search. Diamond is the highest-quality, hardest subset of the full GPQA set.

Why it matters: It indicates scientific reasoning, not everyday writing or coding.

Related: BenchmarkSee the GPQA Diamond leaderboard

Humanity's Last Exam (HLE)

A very hard set of expert-written questions across many subjects, built to test the limits of today's models.

It was created because older exams became too easy for top models. Scores are usually much lower than on older benchmarks.

Why it matters: It helps tell the very strongest models apart on difficult knowledge and reasoning.

Related: BenchmarkSee the Humanity's Last Exam leaderboard

SWE-bench Verified

A coding test where a model must fix real bugs from open-source software projects, checked by running the project's own tests.

Each task comes from a real issue reported on GitHub. People checked that the tasks are fair and solvable, which is what Verified means. The score is the share of tasks fixed.

Why it matters: It is closer to real software work than short coding puzzles, which makes it a useful signal for coding.

Related: Benchmark, Aider Polyglot, Terminal-BenchSee the SWE-bench Verified leaderboard

Aider Polyglot

A coding test of hard programming exercises in several languages, where the model must edit code files correctly.

It is run with the open-source Aider coding assistant. The score is the share of exercises solved, and a model must also follow the required edit format.

Why it matters: It checks both coding skill and the ability to make clean edits to existing files.

Related: SWE-bench Verified, BenchmarkSee the Aider Polyglot leaderboard

Terminal-Bench

A test where an AI agent completes real tasks in a command-line terminal, such as setting up software or fixing a broken build.

Success is checked automatically at the end of each task. It measures whether a model can act on a computer, not just write code.

Why it matters: It matters if you want an AI agent to run commands for you.

Related: SWE-bench Verified, BenchmarkSee the Terminal-Bench leaderboard

ARC-AGI

A set of visual puzzles that test whether a model can work out a new rule from a few examples.

Each puzzle is different, so memorising answers does not help. ARC-AGI-2 is a harder version created after the first became easier for top models.

Why it matters: It points to flexible problem solving rather than recall.

Related: BenchmarkSee the ARC-AGI-2 leaderboard

FrontierMath

A set of original, research-level maths problems written by mathematicians and kept private to avoid leaks.

Epoch AI runs the tests. Problems have exact answers that can be checked automatically.

Why it matters: It separates the strongest maths reasoning from models that only do well on school-level maths.

Related: Benchmark, Benchmark contaminationSee the FrontierMath (Tiers 1-3) leaderboard

OSWorld

A test where an AI agent uses a real computer, clicking and typing in desktop programs to finish everyday tasks.

Tasks span office apps, web browsing and file management. Results are checked by inspecting the final state of the computer.

Why it matters: It matters if you want an AI to operate software for you.

Related: Terminal-Bench, Benchmark

SimpleQA

A test of short factual questions with a single correct answer, used to see how often a model gets facts right.

It is hard because models must either know the answer or admit uncertainty. The Verified version is a cleaned-up set of the questions.

Why it matters: It is one way to compare how reliable a model is on facts.

Related: Hallucination, BenchmarkSee the SimpleQA Verified leaderboard

MMLU

A broad multiple-choice test covering many school and university subjects, from history to law to medicine.

It was once a main way to compare models. Top models now score very high, so it no longer separates them well.

Why it matters: It still helps compare smaller and older models.

Related: Benchmark

Want to see these ideas in action? Browse the LLM leaderboards, compare AI models, or read how the numbers are made.

We use optional cookies for analytics and referral tracking. Learn more about our privacy policy