🔍 Read the full analysis: I Tested 71 AI Setups. Here’s Why My News Pipeline Uses Another on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
Thorsten Meyer says he tested 71 AI model configurations on news-writing tasks from September 8 to October 13, 2026. GPT-6 Astra scored highest, but he chose GPT-6 Luna for production because it was much cheaper; the test also led him to remove two fallback models that produced fabricated quotes or unusable output.
Meyer tested models using the production prompt from his news pipeline and two source texts: a 6,340-character product filing and a 619-character political news item. He generated each article in English and German and had two AI judges score the work across source fidelity, fabrication, clarity about uncertainty, news value, structure and polish. German articles also received a language-quality score. Meyer said the short source better reflects much of his usual news input.
In the initial test, GPT-6 Astra scored 4.00 out of 5 in both languages and received perfect source-fidelity marks. In a further 10-article robustness run, it scored 3.93 across 12 articles. Luna scored 3.67 in English and 3.68 in German in the initial comparison, and 3.67 in the robustness run, with source fidelity of 4.71. Meyer reported that all Luna articles in that run produced valid output under the pipeline’s normal repair step.
The reported per-article costs were about $0.12 for Astra and $0.0012 for Luna; Meyer said the models took around 50 seconds and 17 seconds per article, respectively. He also tested Luna at six reasoning-effort settings. He found no reliable quality gain from higher settings, while the maximum setting cost about 12 times as much and took about 12 times as long as the no-effort production setting.
Infographic 01 · Who writes the best news article?
The best writer is not the one in production
71 model configurations, one real job: turn a news source into an article for an English and a German edition. GPT-6 Astra writes the best articles. GPT-6 Luna writes them for about a hundredth of the price.
Robustness run: 12 real news sources, English edition
Dot = mean score (1–5), bar = 95% bootstrap interval. German edition mean below each score. Overlapping bars are not a real difference.
Two judges, two caveats: GPT-6.1 Sol and Claude Opus 5.5 grade every article and were also tested as writers, so they partly grade themselves. Claude models ran with effort medium unless noted.
Source: Thorsten Meyer AI model-scout benchmark, 8 Sep – 13 Oct 2026. Task: write a news article from a real source with the production prompt, English and German editions. Judges: GPT-6.1 Sol (reasoning high) and Claude Opus 5.5, both with the source in context; score 1–5, mean of both judges. Robustness run: 2 test articles plus 10 more real sources, mostly short like production; mean with 95% bootstrap interval.
Powered by Thorsten Meyer AI
Cost And Quality In Production
The results illustrate why a model leaderboard may not identify the right tool for a specific publishing job. Meyer’s test measured performance against his own prompt, sources and output requirements, rather than general intelligence or coding ability. He concluded that paying more for the highest-scoring model did not make sense for his routine news production, where the lower-cost model cleared his quality bar.
The test also points to a risk beyond the main model’s average score: what happens when it fails. Meyer said two fallback models generated invented quotes or unusable output. For a newsroom or publisher, a backup that produces plausible but unsupported quotations can create a source-fidelity problem even if the primary model performs well. The findings are a report about one pipeline, not proof that the same rankings apply to other publishers or tasks.
As an affiliate, we earn on qualifying purchases.
How Meyer Tested The Models
The test covered API, subscription and free-tier models, as well as open-weight models running on Meyer’s hardware. Every configuration worked from real news sources and the same production prompt. The two AI judges—GPT-6.1 Sol and Claude Opus 5.5—both received the original source material when scoring articles.
Meyer said he used 95% bootstrap intervals and treated differences below roughly 0.3 points in the two-article standard test as noise. He regarded the additional robustness run, which used 10 more real sources and mostly short ones, as a more dependable basis for comparing leading configurations. These are the author’s stated methods and judgments; the source does not provide an independent replication of the test.
““For a high-volume task, the right model is the cheapest one that clears your quality bar reliably, not the one at the top of the chart.””
— Thorsten Meyer, writing on ThorstenMeyerAI.com
Limits Of The Reported Results
The findings come from one publisher’s pipeline and a limited set of source material, including just two texts in the standard comparison. Meyer says the additional run used 10 more sources, mostly short ones, but the article does not establish how the results would change with other topics, prompts, languages, editing standards or model versions. The scores are evaluations by two AI judges, not a reported human-editor panel.
It is also not clear from the published material whether every model received identical access conditions or how the reported prices may vary for other users. Meyer’s cost and timing figures describe his test and production setup. The article does not provide independent verification of the scores, the fabricated quotations he describes, or the performance of the models in other workflows.
Luna Remains The Production Choice
Meyer said GPT-6 Luna is now the production model for his news route and that GPT-4.1 nano and gpt-oss-120b were removed as fallbacks on October 7. He reported keeping Luna at the no-reasoning-effort setting, where his production articles cost about $0.0013 each over the preceding week and took a median of 17 seconds.
The source does not give a date for another test or say whether Meyer will add replacement fallback models. The next relevant evidence would be further performance in production and repeat tests as models, prices and the pipeline change.
Key Questions
Which model scored highest in Meyer’s test?
GPT-6 Astra scored 4.00 out of 5 in both English and German in the standard test and 3.93 across 12 articles in the robustness run, according to Meyer.
Why did Meyer choose GPT-6 Luna instead?
Meyer said Luna offered reliable results at about $0.0012 per article, compared with about $0.12 for Astra. He selected Luna for routine production because of that cost difference.
Did more reasoning effort improve Luna’s articles?
Meyer reported that differences among Luna’s six reasoning-effort settings were within the test’s noise. The maximum setting took and cost about 12 times as much as the no-effort setting without a reliable quality gain.
What happened with the fallback models?
Meyer said GPT-4.1 nano generated invented quotes, while locally run gpt-oss-120b produced no valid articles in four attempts and also generated named quotes he described as invented. He said both were removed from the news route on October 7.
Do these results show Luna is the best model for every newsroom?
No. The findings reflect Meyer’s prompt, sources, languages and evaluation method. The report does not establish that the same model would perform best for other publishers or tasks.
Source: ThorstenMeyerAI.com
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
