Mirendil debuts Fern, a model with a 10 million token context

Mirendil introduced Fern, its first model, which it says it pretrained from scratch and built to support context windows of up to 10 million tokens. The first version, Fern 0.1, went through a short round of post-training on top of the Mirendil-Base model, which has between 0.1 and 0.5 trillion parameters. Mirendil says it plans to ship new versions frequently, and two larger base models, Mirendil-Base-Large and Mirendil-Base-XL, are still in pretraining.

The company frames Fern around the idea that a model's usable context matters as much as its parameter count, since more context lets a model hold more of a codebase, more agent history, or more documents at once. Mirendil says 10 million tokens is roughly equivalent to 23 megabytes of code, or about half a million lines, comparable in scale to a library like Megatron-LM. The company says it ran early tests showing that more context helps even basic next-token prediction, and ran a baseline long-context test using the RULER benchmark, with more comprehensive long-context results still to come. These performance claims come from Mirendil itself.

Fern is pitched at high-volume agentic work such as debugging from logs, auditing training data for duplicates or contamination, and reading full agent traces end to end. Access to Fern is not yet open; Mirendil is taking requests for early access, and says availability is coming soon.

Why it matters

A 10-million-token context window, if it holds up outside company benchmarks, would let models reason over entire large codebases at once.

Source: Mirendil