Compare AI Model Claims Against Your Actual Task

This URL previously contained a named-model comparison with exact scores and dates but inadequate primary citations. It no longer makes those assertions or declares a winner. This correction does not claim that every named product is nonexistent; it means this page did not establish the claims.

Choose a fictional brief comparison, a small code fixture or a source-grounded question. Specify the same permitted input and acceptance criteria for both systems. Record model identity, account limits, settings and date from the actual run.

For brief comparison, change one deadline and keep the other facts fixed. For code, execute the relevant tests. For research, open each source and check the passage. A fluent answer or a vendor score is not proof it fits your workflow.

Include unsuccessful attempts, sample size, reviewer judgment, time/cost measured in the run and any tool-access difference. Do not turn a tiny task test into a universal model ranking.

Select based on your constraints and observed results. If no matched experiment is available, label the discussion documentation-based and leave performance superiority unknown.

• Prompting: clear instructions and examples

Use the models currently selectable in your accounts and record their exact labels. The historical names in this URL are not a recommendation that those versions are available today. Do not infer a release date, score or price from an article title.

Give both assistants this fictional input:

Brief A: Launch Friday. Budget 500. Owner Mina. Dependencies: artwork.
Brief B: Launch Monday. Budget 500. Owner not assigned. Dependencies: artwork, legal review.
Task: Return a table of changed fields with the exact supporting phrases.
Keep unchanged fields separate. Identify missing facts; do not assign an owner.

The reference answer has three material changes: Friday to Monday, Mina to unassigned, and legal review added. Budget stays 500 and artwork remains a dependency. A response that says "Mina owns both plans" fails even if its formatting is excellent.

Run the same input without changing one assistant's prompt halfway through the comparison. Record account surface, selected model, date, any browsing or tools, the complete output and your check. If you revise the prompt, rerun both or treat it as a separate trial. One successful response is not a broad reliability score.

Evaluate factual fidelity first, then review effort, clarity and account limits. Measure elapsed time only after defining whether it includes setup and verification. A preference for one writing style does not prove superiority in coding or research. This page supplies the evaluation fixture; no matched model trial is claimed.

Work through the project-brief lesson · ChatGPT listing · Claude listing