Models·
Inkling 2 Mini: A Looped MoE Model Built for Local AI Inference
Thinking Machines Lab releases Inkling 2 Mini, a 26B-parameter multimodal mixture-of-experts model that uses layer looping to achieve 48 logical layers with only 24 physical ones. It supports 1M-token context, mixed-precision formats (NVFP4/MXFP8), and runs locally via llama.cpp, MLX, and SGLang.

Thinking Machines Lab has introduced Inkling 2 Mini, a compact multimodal mixture-of-experts (MoE) model tailored for local AI deployments. With roughly 26 billion physical parameters but only 3 billion active per token, the model delivers performance comparable to larger 27B-class models while fitting comfortably on consumer-grade GPUs. Its standout feature is a looping architecture: the middle 12 of 24 decoder layers are executed three times, effectively creating 48 logical layer visits without increasing the weight footprint.
Looping Architecture and Cross-Pass Messaging
Each pass through the looped core starts from the same intermediate representation, and information flows between passes via learned cross-pass messages. A gating controller blends the current input with messages from the previous pass, allowing iterative refinement. The model also employs sliding-window attention (1,024 tokens) with dynamic convolutions for full-attention layers, and routes each token to 8 of 128 SwiGLU experts per layer.
Inkling 2 Mini ships in mixed-precision formats — NVFP4 and MXFP8 for expert weights — alongside a DSpark drafter for speculative decoding. Native support exists for Hugging Face Transformers, llama.cpp, MLX, and SGLang, enabling day-zero deployment on both workstation and datacenter GPUs. The 1-million-token context window and image-text multimodality further expand its utility for long-document and vision-language tasks.
Why It Matters for GPU and AI Infrastructure
This release highlights how architectural efficiency — layer reuse, sparse MoE routing, and low-precision quantization — can bring high-quality multimodal reasoning to local hardware. For infrastructure teams, the model's compatibility with SGLang and llama.cpp means existing GPU clusters can serve it with minimal engineering overhead. The NVFP4/MXFP8 formats align with Hopper and Blackwell tensor-core capabilities, reducing memory bandwidth pressure during inference.
Inkling 2 Mini signals a broader shift: state-of-the-art capabilities no longer require massive active parameter counts. Organizations evaluating on-premise or edge AI solutions should test this model on their target hardware to gauge real-world throughput and latency.
- aigpu
- ai gpu
- ai gpu cloud
- aigpu dubai
- inkling-2-mini
- mixture-of-experts
- local-ai
- gpu-inference
By AiGpu Editorial · Editorial rewrite based on public reporting (Hugging Face Blog)
← All articles