📊 Full opportunity report: Engineering Is Automated. Research Is the Residual. on ThorstenMeyerAI.com — validation score, market gap, and execution plan.
TL;DR
Recent developments indicate AI systems are now capable of automating the majority of engineering tasks in AI research, with research itself remaining less automated. This shift could reshape the landscape of AI development and research practices.
Recent benchmarks and research indicate that AI systems have achieved near-complete automation of core engineering tasks in AI research, while the automation of research itself remains uncertain.
Multiple independent benchmarks, including CORE-Bench and MLE-Bench, show AI systems are reaching or surpassing saturation levels in tasks crucial to AI engineering, such as reproducing research, optimizing models, and designing hardware kernels. For example, CORE-Bench, which measures the ability to reproduce research papers, reached a 95.5% success rate in December 2025, with some authors declaring it ‘solved.’ Similarly, MLE-Bench, assessing performance in Kaggle competitions, hit 64.4% in February 2026, equating to mid-tier human performance.
These advances suggest that the bottleneck for AI research shifts from engineering to the research process itself, which involves hypothesis generation, experimental design, and interpretive insights. The evidence indicates that AI’s capacity to handle engineering at scale is approaching or exceeding human-level performance, raising questions about the future role of human researchers in technical development.
Engineering is automated.
Research is the residual.
Six skill benchmarks. Edison’s framing. The question Clark leaves open is whether research is just engineering at scale.
Jack Clark’s Import AI #455 catalogs six benchmarks measuring AI capability on AI R&D tasks and concludes “AI can today automate vast swatches, perhaps the entirety, of AI engineering.” The residual question is research. The structural read on the residual: it may not be a permanent moat.
Six skills. One trajectory.
Clark catalogs six benchmarks measuring AI capability on AI R&D-relevant tasks. Each individual benchmark could be noise. Six benchmarks moving together is a curve. The pattern is the cascade observed across the broader Clark series — visible here in the specific R&D-skill domain.
AI research automation software
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Three data points. Mixed signal.
Clark provides three data points on the creative-spark question. Yes-evidence: Erdős-1051, centaur math discovery, sporadic Move-37-style moments. No-evidence: low yield, framing dependence, absence of acceleration. The mixed signal is the honest read.
The data supports two readings. Pessimistic: rare moments suggest creative insight is qualitatively distinct from engineering work. Optimistic: rare moments are an artifact of low-volume exploration; more shots on goal yields more discoveries. Both readings are consistent with Clark’s “vast swatches, perhaps the entirety” claim. They differ on the residual.
Practical Python AI Projects: Mathematical Models of Optimization Problems with Google OR-Tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Five dimensions Clark gestures at but leaves underdeveloped.
Clark’s section is rigorous on the empirical evidence. Five strategic dimensions matter for the institutional response that the Clark series synthesis argues is structurally inadequate.
hardware kernels for AI development
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Two readings. Different equilibria.
The structural question Clark leaves open: is research a permanent moat that bounds automated AI R&D, or is it engineering at scale that dissolves with more shots on goal? Both readings are consistent with the current data. They differ by orders of magnitude in consequences.
Productivity multiplier years
Recursive loop operational
research automation platforms
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Five audiences. Asymmetric cost of being wrong.
The institutional response should not bet on inspiration being a permanent moat. If the distinction holds, capacity built is still useful. If it closes, capacity is necessary. Asymmetric cost-of-being-wrong points toward building now.
IN INDUSTRY
IN ACADEMIA
POLICYMAKERS
INVESTORS
EVERYONE ELSE
Engineering is automated. The residual is the question. The institutional response should not bet on inspiration being a permanent moat.
Implications of Automated AI Engineering on Research Processes
The automation of engineering tasks in AI research could dramatically accelerate development cycles, reduce costs, and shift the role of human researchers toward higher-level hypothesis and theory work. However, it also raises concerns about the future structure of research teams, the need for new oversight mechanisms, and potential shifts in innovation dynamics. The institutional response should consider how to adapt to a landscape where engineering is largely automated, and research remains the primary frontier.
Recent Benchmark Progress and AI Capability Trajectory
Over the past 18 months, multiple benchmarks measuring AI capabilities relevant to research and engineering have shown rapid progress. CORE-Bench, focused on research reproduction, improved from 21.5% to 95.5%. MLE-Bench, testing Kaggle competition performance, advanced from 16.9% to 64.4%. These trends follow a pattern of saturation, indicating that AI systems are approaching or surpassing human-level proficiency in core engineering tasks essential for AI R&D. Prior to these developments, AI’s role was primarily supportive; now, it appears capable of automating substantial portions of the engineering process.
“The pattern across multiple benchmarks suggests that AI can now automate vast swaths, perhaps the entirety, of AI engineering.”
— Thorsten Meyer
Uncertain Extent of AI’s Research Automation Capabilities
While engineering tasks are increasingly automated, it remains unclear how much of the research process—such as hypothesis formulation, experimental interpretation, and theory development—can be automated. The structural question posed by Clark suggests that research may itself be a form of large-scale engineering, which could mean that research automation is closer than currently understood, but definitive evidence is lacking.
Next Steps in Monitoring AI’s Research and Engineering Progress
Researchers and institutions will need to observe ongoing benchmark developments, particularly in research-centric tasks. Further advances could lead to a redefinition of research workflows, with AI systems taking on more creative and interpretive roles. The next 12-24 months will be critical for assessing whether the residual research tasks can be automated, and how institutions adapt to this shift.
Key Questions
What does the automation of engineering mean for AI research teams?
If engineering tasks are largely automated, research teams may shift focus from technical implementation to higher-level hypothesis generation, strategy, and interpretation, potentially accelerating innovation.
Are there limits to AI automation in research?
Yes, it is still uncertain how much of the creative, interpretive, and hypothesis-driven aspects of research can be automated. Current benchmarks mainly measure engineering and implementation skills.
How soon could AI fully automate research processes?
While engineering automation is advancing rapidly, full automation of research tasks remains uncertain. Experts estimate this could take several years, depending on breakthroughs in AI’s creative and reasoning capabilities.
What are the risks of relying on AI for engineering and research?
Dependence on AI could lead to reduced human oversight, potential biases, and challenges in ensuring scientific rigor. Careful governance and validation will be necessary as capabilities expand.
Source: ThorstenMeyerAI.com
