Cursor Research, the engineering arm behind the widely utilized AI code editor Cursor, has officially open-sourced Mixture-of-Kittens (MoK), a highly optimized training megakernel designed for Mixture-of-Experts (MoE) models. This release marks a significant milestone in the infrastructure layer of large-scale artificial intelligence, specifically targeting the newest generation of NVIDIA hardware. MoK is the underlying technology that powers Cursor’s "Composer" models, and its public release under the Apache-2.0 license provides the broader research community with a blueprint for achieving massive throughput gains on high-end hardware.
The primary innovation of MoK lies in its architectural approach to the MoE layer, which has historically been a significant performance bottleneck in distributed training. By fusing every communication and computation step into a single, deterministic megakernel, Cursor reports a throughput increase of up to 2.37 times compared to the most efficient public baselines. This development comes at a time when the AI industry is shifting toward MoE architectures, such as DeepSeek-V3 and Qwen-3.5, which utilize sparse activation to scale model capacity without a linear increase in computational cost.
The Evolution of MoE Training and the Communication Bottleneck
To understand the significance of Mixture-of-Kittens, one must look at the trajectory of model architecture over the past two years. Mixture-of-Experts models operate by routing input tokens to a subset of specialized "expert" layers rather than processing them through every available parameter. While this reduces the total FLOPs (floating-point operations) required per token, it introduces a massive overhead in data movement. In a distributed environment, tokens must be "routed" or sent across different GPUs to reach the specific expert assigned to them.
Previously, the Cursor Research team focused on optimizing the compute-heavy portions of this process. They developed custom kernels for MXFP8 and NVFP4 precision training and a specialized "warp decode" path for MoE inference. However, as models scaled to tens of thousands of GPUs, the team identified that the MoE layer was consuming more than 50% of the end-to-end training time. The bottleneck had shifted from raw mathematical computation to the latency and synchronization issues inherent in inter-GPU communication.
The transition to NVIDIA’s Blackwell architecture, specifically the GB300 NVL72 racks, further complicated this dynamic. These racks consist of 72 GPUs within a single NVLink domain, allowing for extremely fine-grained data overlapping. However, the integrated Grace CPUs in these systems are relatively slow compared to the massive power of the Blackwell GPUs. Traditional training scripts that rely on frequent CPU-GPU synchronization suffer from "bubble" periods where the GPU sits idle, waiting for the CPU to manage the next step of the communication protocol. MoK was engineered specifically to solve this by minimizing CPU intervention and leveraging the Blackwell hardware’s native scheduling capabilities.
Technical Architecture and the Megakernel Approach
MoK is categorized as a "megakernel," a design philosophy that consolidates multiple operations—such as token routing, expert computation, and result gathering—into a single execution block on the GPU. This approach eliminates the overhead of launching multiple smaller kernels and ensures that the GPU remains fully utilized throughout the MoE processing step.
A critical feature of MoK is its support for both BF16 (Brain Floating Point 16) and MXFP8 (Microscaling Formats). MXFP8 is particularly relevant for next-generation training, as it allows for lower-precision arithmetic that maintains the accuracy of higher-precision formats while significantly reducing memory bandwidth requirements. By fusing the MXFP8 scaling and de-scaling logic directly into the kernel, MoK avoids the memory-roundtrip penalties usually associated with precision conversion.
Furthermore, MoK implements a fully deterministic execution path. In many distributed training setups, the order in which data is received from different GPUs can vary slightly, leading to non-deterministic results. While this is often acceptable for standard pretraining, it becomes a major hurdle for Reinforcement Learning (RL) and internal ablations where reproducibility is paramount. MoK ensures that every training run with the same seed and hardware configuration yields the exact same numerical output.
The scheduling within MoK is handled via Blackwell’s Cluster Launch Control. This hardware feature allows for the coordination of tasks across a cluster without serializing behind inter-rack RDMA (Remote Direct Memory Access). Additionally, MoK incorporates a SonicMoE-style calculation for router weight gradients, which is fused directly into the SwiGLU (Swish-Gated Linear Unit) backward pass. This integration reduces the number of passes over the data and optimizes the backpropagation process.
Hardware Prerequisites and Deployment Constraints
While the performance gains offered by Mixture-of-Kittens are substantial, the barrier to entry is high. The software is specifically tuned for the latest NVIDIA Blackwell architectures, specifically the SM100 or SM103 GPUs. This effectively limits its use to GB200 NVL72 or GB300 NVL72 rack configurations.
The software stack requirements are equally modern, requiring Python 3.12 or higher, PyTorch 2.10 or higher, and the CUDA toolkit 13.0+. A notable technical requirement is the use of PyTorch symmetric memory for inter-GPU buffers. Symmetric memory allows for more efficient peer-to-peer data transfers within an NVLink domain, which is essential for the "all-to-all" communication patterns used in MoE routing.
Because of these requirements, MoK is not intended for small-scale developers or shops operating with standard 8-GPU nodes (such as H100 HGX systems). Instead, it is targeted at:
- Frontier Labs: Organizations developing the next generation of trillion-parameter models.
- Funded Model Startups: Well-capitalized entities with access to Blackwell-level compute.
- GPU Neoclouds: Infrastructure providers looking to offer optimized training environments to their tenants.
- National Computing Centers: Large-scale research facilities focused on high-performance computing (HPC) and AI.
Benchmarking Results and Performance Metrics
Cursor Research provided extensive benchmarking data comparing MoK against several established baselines, including NCCL+PyTorch, DeepEP+PyTorch, DeepEP+TransformerEngine, and HybridEP+Megatron. The benchmarks were conducted on a single NVL72 rack with an Expert Parallelism (EP) degree of 64. Each GPU was tasked with holding 2,048 tokens before routing, simulating the high-density workloads typical of modern LLM training.
The results demonstrate a clear lead for MoK across multiple precision modes and model architectures:
- MXFP8 Forward Pass: MoK achieved up to 2.37x higher throughput than the strongest baseline.
- MXFP8 Backward Pass: A 1.78x improvement was recorded.
- BF16 Forward Pass: Throughput increased by 1.92x.
- BF16 Backward Pass: A 1.58x gain was observed.
When scaled to end-to-end training across 512 GPUs (spanning multiple GB300 NVL72 racks), the impact on total system efficiency was significant. The tokens-per-second per GPU rose from 760.9 in the baseline configuration to 1,070.2 with MoK. This represents a 1.41x overall gain in training efficiency. In the context of models that cost tens of millions of dollars to train, a 41% increase in throughput translates to millions of dollars in saved compute costs or a significantly faster time-to-market.
The benchmarks covered several prominent model shapes, including Kimi K2.7 Code, GLM-5.2, Qwen3.5-397B-A17B, and DeepSeek-V4-Pro. The consistency of the gains across these varied architectures suggests that MoK is a robust solution for the current trend of "DeepSeek-style" MoE models.
Industry Implications and the Move Toward Open-Source Infrastructure
The decision by Cursor to open-source MoK is seen by industry analysts as a strategic move to solidify its position as a leader in AI systems engineering. While Cursor is primarily known for its consumer-facing AI editor, the development of MoK demonstrates a deep vertical integration, where the company builds the very kernels used to train the models that power its software.
By releasing MoK under the Apache-2.0 license, Cursor is effectively setting a new standard for MoE training efficiency. This move may pressure other infrastructure providers and research labs to release similar optimizations or risk falling behind the performance curve. It also benefits the broader ecosystem by providing a high-performance alternative to proprietary or closed-source training stacks.
The focus on determinism is particularly relevant for the growing field of "on-policy" Reinforcement Learning. In RL, models are often updated based on data they generate themselves in real-time. Without a deterministic training path, it becomes nearly impossible to isolate the effects of algorithmic changes from hardware-induced noise. MoK’s determinism provides the "clean" environment necessary for advanced RL research.
Conclusion and Future Outlook
Mixture-of-Kittens represents a shift in how AI researchers approach the "systems" side of machine learning. As models grow larger and hardware becomes more complex, the days of relying on generic communication libraries like NCCL for maximum performance appear to be fading. Instead, the future belongs to highly specialized, hardware-aware megakernels that can extract every last bit of performance from specific architectures like Blackwell.
For organizations with the hardware capacity to support it, MoK offers a path to significantly faster and more cost-effective model development. For the rest of the industry, it serves as a technical benchmark and a clear indicator of the direction in which high-performance AI training is headed. As Blackwell racks become more widely available in data centers globally throughout 2025 and 2026, the adoption of specialized kernels like MoK is expected to become a standard requirement for competitive model training.
Cursor Research has indicated that it will continue to update the GitHub repository as new hardware iterations and training techniques emerge. The AI community now has a powerful new tool in its arsenal, one that bridges the gap between raw silicon power and the sophisticated demands of Mixture-of-Experts architectures.
