AIThis post was created with the assistance of artificial intelligence (AI).

🔍 Read the full analysis: Inkling 2 Mini: What To Know About This Local AI Option on ThorstenMeyerAI.com

Before you orderOffer from Amazon

Get the latest gadgets delivered free with Prime

  • Fast, free delivery on millions of items
  • Prime Video, Amazon Music and more included
  • Member-only deals all year
Start your free Prime trial Free trial for eligible customers · Cancel anytime
As an affiliate, we earn on qualifying purchases.

TL;DR

Thinking Machines Lab has released Inkling 2 Mini, a multimodal mixture-of-experts model that the company describes as having about 26 billion total parameters and activating about 3 billion per token. It supports several inference runtimes and includes a research version with adjustable layer passes, but the supplied release material does not independently verify its benchmark claims or establish practical hardware needs.

Thinking Machines Lab has released Inkling 2 Mini, a multimodal model aimed at local AI setups that the company says has about 26 billion total parameters and activates about 3 billion per token. The release includes support for several inference tools and a research checkpoint with adjustable repeated-layer passes; the company reports strong benchmark results, but independent validation and practical hardware requirements are not established in the material provided. See the original analysis for additional context.

The company describes Inkling 2 Mini as accepting text and image input and offering a 1 million-token context window. Its architecture has 24 physical decoder layers. The 12 middle layers run three times, while the first and last six layers run once, resulting in 48 logical layer visits per token. The repeated layers reuse weights rather than storing separate weights for each visit.

Inkling 2 Mini is a mixture-of-experts model: at each layer visit, a routing system selects eight experts from a pool of 128, according to the release. The company says the fixed-pass model is supported by Transformers, llama.cpp and MLX, with SGLang support for datacenter inference. It is also distributing expert weights in NVFP4 and MXFP8 formats and including a DSpark drafter, which it says is intended to speed generation with a small memory cost.

A separate research artifact, TML-d24-varloop-research, allows users to change the number of passes through the repeated layers. Thinking Machines says fewer passes can improve inference speed, potentially at a cost to quality. The supplied material does not give a complete model file size, minimum hardware configuration or independent measurements of speed and memory use.

At a glance
announcementWhen: Released; the supplied material is date…
The developmentThinking Machines Lab released Inkling 2 Mini, a local-oriented multimodal model with multiple runtime options and a research checkpoint that can vary its repeated-layer passes.
AI Models · Briefing

Inkling 2 Mini for Local AI

Thinking Machines Lab’s new multimodal model combines reused layers with expert routing. Its practical speed, hardware needs and benchmark results still need outside validation.

26B
Total parameters, about
3B
Parameters active per token, about
1M
Token context window

A loop reuses model layers

The design reuses middle-layer weights to build capacity, while routing selects part of the expert pool at each layer visit.

1
Start with 24 layers

The first six and last six physical decoder layers run once.

2
Repeat the middle

The same 12 middle layers run in three passes, for 48 logical layer visits per token.

3
Route to experts

At each visit, the model selects eight experts from a pool of 128.

Options for inference

The release offers several runtimes and a research checkpoint with adjustable loop passes.

Fixed model

Multiple runtimes

Transformers, llama.cpp and MLX support the fixed-pass model; SGLang supports datacenter inference.

Research variant

Fewer passes

The research checkpoint allows one or two passes. The company says this can be faster, with a possible quality trade-off.

Claims need outside checks

The company reports strong benchmark comparisons, but the supplied material does not include independent replication or a full score table.

Reported results
  • Leads in 32 of 48 benchmarks against named models.
  • Reported tie with Qwen3.8-27B on Terminal-Bench 4.0.
  • Trails on AutomationBench-AA v1.0.6.
Still unclear
  • Complete memory requirements and device-specific speed.
  • Independent image and long-context performance.
  • Quality at reduced loop counts.

What local users should assess

Runtime availability is a starting point; actual usefulness depends on hardware and workload.

Memory useGeneration speedImage handlingLong-context behaviorOutput quality
CaveatThe release does not specify a complete file size or minimum hardware configuration.
The takeaway
Inkling 2 Mini brings an interesting local inference design, but independent tests and hardware-specific measurements will show whether its reported promise holds in practice.

What Local AI Users Can Gain

The release adds another multimodal option for local-model users, with support for image input and several commonly used inference runtimes. Runtime availability may make it easier for developers and hardware owners to test the model in different environments. Its repeated-layer design also offers a different way to reuse weights while retaining a comparatively large number of layer computations.

Those features do not establish that the model will run comfortably on a particular desktop or workstation. The company’s stated parameter count and weight formats alone do not answer questions about total memory use, generation speed or image quality. Performance will also depend on runtime implementation, hardware, settings and the size and nature of the prompt. For buyers choosing local AI software or equipment, hands-on tests are needed before treating the release as a practical fit.

The adjustable-pass research version could help users examine a speed-quality trade-off, but the source material does not report results for one- or two-pass configurations. Until those tests and independent benchmark checks are available, the release’s performance case remains a company-reported claim, not a confirmed third-party result.

Products Worth Considering

LG 27GS60QC-B Ultragear 27-inch Curved Gaming Monitor (QHD, 180Hz, 1ms, 1000R)

LG 27GS60QC-B Ultragear 27-inch Curved Gaming Monitor (QHD, 180Hz, 1ms, 1000R)

As an affiliate, we earn on qualifying purchases.

How the Repeated Layers Work

Inkling 2 Mini’s central architectural choice is to run the same 12 middle physical layers in three passes. The first six decoder layers run before that repeated section, and the final six run afterward. Learned messages carry information between passes, according to the company’s technical description. The design therefore reuses some weights, but it still executes 48 logical layer visits per token; the release also says each visit retains its own key-value and short-convolution state.

The company compares Mini with its Inkling 2 Small model. Both are described as making 48 layer visits per token, but Mini has a 2,304 hidden width compared with 4,608 for Small, and routes among 128 experts per layer rather than 384. The release says the smaller configuration is substantially smaller, but the supplied material does not specify its full size on disk or a minimum amount of usable memory.

Thinking Machines calls Inkling 2 Mini state of the art in its class on Artificial Analysis Intelligence Index v5 and says it leads in 32 of 48 comparisons against Gemma 4 31B, Muse Glimmer, Qwen3.8-27B and Nemotron 3.5 Super VL. It also reports a lead on FrontierCode v2.0, a tie with Qwen3.8-27B on Terminal-Bench 4.0 and a trailing result on AutomationBench-AA v1.0.6. These are vendor-reported comparisons; the supplied material does not include the full score table or independent replication.

“Inkling 2 Mini is a mixture-of-experts model with 24 decoder layers, but the 12 middle layers run three times in a loop.”

— Thinking Machines Lab, in the supplied release material

Hardware and Benchmark Questions

The supplied material does not establish independent benchmark results, and it does not provide a full score table for the company’s comparisons. It is also unclear how the model performs on consumer hardware across supported runtimes, including its actual memory requirements and measured generation speeds.

Other questions remain open: whether the claimed 1 million-token context is useful in real workloads, how well image input performs, and how output quality changes when the research checkpoint uses fewer passes. The material does not include independent long-context or multimodal evaluations, nor does it confirm a minimum hardware setup. These details should not be inferred from parameter counts or the list of supported software alone.

Evidence Needed From Testing

The next useful evidence would include the full model card, reproducible benchmark scores and reports from users testing Inkling 2 Mini on specified hardware and runtime versions. Tests that report memory use, generation speed, image-task results and long-context behavior would help readers compare the model with alternatives on a consistent basis.

Testing of the variable-loop research artifact could also clarify how one-, two- and three-pass settings affect speed and answer quality. Until those results are available, users can treat the release as an option to evaluate rather than a verified performance recommendation. The supplied material does not state when additional model-card information or independent evaluations will appear.

Key Questions

What is Inkling 2 Mini?

It is a multimodal mixture-of-experts model released by Thinking Machines Lab. The company says it accepts text and image input and has about 26 billion total parameters, with about 3 billion activated per token.

Can Inkling 2 Mini run locally?

The release lists support for Transformers, llama.cpp and MLX, with SGLang support for datacenter inference. However, the supplied material does not specify a minimum hardware configuration or measured performance on consumer devices.

What does the variable-loop research version change?

The research artifact allows the number of passes through the repeated middle layers to be adjusted. Thinking Machines says fewer passes may speed inference, but could reduce quality; the supplied material does not give independent measurements of that trade-off.

Are the reported benchmark results independently verified?

Not in the material provided. The results are reported by Thinking Machines Lab, and the source does not include the complete score table or independent replication.

Primary source: Hugging Face · via ThorstenMeyerAI.com

You May Also Like

Enhancing AI Performance Through Fine-Tuning Multi-Vector Sentence Embeddings

Sentence Transformers v6.0 introduces MultiVectorEncoder for domain-specific retrieval, showing promising results in medical search, pending independent validation.

What Does Anthropic’s Arrival Mean For phData Funding In 2026?

A 2026 headline links phData and Anthropic, but the available Google News listing and linked page do not confirm funding, a partnership, deal terms or either company’s role.

From Selling To Investing: TSMC, Nvidia And Google Back AI Labs

TSMC approved up to $1 billion for Anthropic and OpenAI, while seven companies agreed to buy $26 billion of Anthropic shares after its IPO.

AI Keys On The Black Market: How Stolen Tokens Become Cash

Railway says an attacker may have accessed some customer environment variables and links the incident to attacks targeting AI inference providers.