Tencent's Hy4 Preview: A 770B-Parameter Open-Weight Flagship with 1M Context
Tencent has released Hy4 Preview, a 770-billion-parameter open-weight language model featuring a 1-million-token context window and a sparse activation architecture.
Tencent announced the immediate availability of Hy4 Preview, a new open-weight large language model restricted to text input. The model totals 770 billion parameters but utilizes a sparse mixture-of-experts architecture that activates only 49 billion parameters per inference step. It ships with a 1-million-token context window and occupies 1.56TB of storage on Hugging Face. This release represents a substantial scale-up from Tencent's previous iteration, Hy3, which launched in July with 295 billion total parameters, 21 billion active parameters, and a 256,000-token context limit.
Inspection of the model's `chat_template.jinja` file on Hugging Face reveals a hardcoded reasoning control mechanism. The template enforces a binary `reasoning_effort` parameter, defaulting to "high" or accepting "no_think" to disable extended reasoning traces. Attempts to pass undefined or invalid values trigger explicit exceptions within the template logic. Early testing via OpenRouter using the default high-effort setting produces visible reasoning traces characterized by truncated English grammar. This stylistic choice appears deliberate, prioritizing token efficiency over linguistic perfection for hidden intermediate steps.
The jump from Hy3 to Hy4 marks one of the most aggressive parameter scaling moves in the open-weight sector this quarter. While Hy3 offered a capable 295B dense or sparse structure depending on interpretation, Hy4 pushes the total parameter count well beyond the half-trillion mark while maintaining a manageable active parameter count for inference. The 1M context window places it in direct competition with other long-context flagships, though the absence of vision capabilities limits its modality scope compared to some contemporaries. The model is available now for download and integration via standard tooling that supports the specified chat templates.