Mistral released the weights of Mistral Large 4 and published a safety number with them: an average refusal rate of 96.6 percent on malicious cyber requests, which the company calls "the highest we measure among open models".

The figure is an average over the cyber-related prompts of three public benchmarks: JailbreakBench, StrongREJECT and AgentHarm. The model scores 51 percent on the Artificial Analysis Cyber Index. The weights are released under what Mistral calls its Ecosystem Open-Weight licence, described on the page as a modified MIT licence.

Two other sentences sit on the same page. After monitoring live traffic, Mistral writes that it saw "a potential for uplift to conduct real attacks" and is modifying its guardrails accordingly. And a specialised variant optimised for advanced cybersecurity workflows is in preview with vetted enterprise partners such as Orange Cyberdefense, under contract, with testing restricted to owned or explicitly authorised systems.

So: the open model refuses 96.6 percent of malicious cyber requests, the company has seen real-world signs of uplift, and the version intended for offensive work is behind a contract.

What a refusal rate measures

It measures a checkpoint. That is the whole of it.

A refusal rate is a property of a specific set of weights at a specific moment, measured against a specific prompt set. It is a real measurement and 96.6 percent is a real result. It describes the model as shipped, on day one, before anybody has done anything to it.

Open weights are weights you can modify. Fine-tuning is cheap, well documented and widely practised, and refusal behaviour is one of the easier things to move because it is a thin layer of trained behaviour over capability that remains underneath. The capability is what the 51 percent on the Cyber Index measures, and the capability does not go anywhere when the refusals do.

This is not a criticism of Mistral's number. It is a point about what the number can and cannot tell a reader. Published alongside an API, a refusal rate is a meaningful operating statistic, because the company controls the weights and can keep measuring them. Published alongside a download, it describes the first state of an object that is now out of the publisher's hands.

The arithmetic nobody puts in the headline

96.6 percent refusal is 3.4 percent compliance.

That is the floor. Before modification, before fine-tuning, before anybody tries a second phrasing, a model scoring 51 percent on a cyber capability index answers roughly one malicious request in thirty.

Whether that is high or low depends entirely on what you compare it to, and Mistral's own framing invites the comparison: highest among open models. Which means every other open model of comparable capability answers more often than one in thirty, and all of them are also downloads.

The week's other release

The comparison writes itself, because the two happened within a fortnight.

OpenAI put its cyber model behind Daybreak Red: approved customers only, IP allowlisting on by default, a Critical classification under its Preparedness Framework, misuse monitoring, and a separate programme for critical infrastructure defenders. Access is a decision the company makes per customer, and it can be withdrawn.

Mistral measured a refusal rate and published a file.

Both companies gave reasons, and both sets of reasons are coherent. Gating keeps capability away from people who should not have it and concentrates it in the hands of whoever does the gating. Open weights put defensive capability into the hands of organisations that will never be approved for anybody's vetted programme, and remove a single company's veto over who gets to do security research.

The asymmetry is not about which argument is better. It is that only one of the two decisions can be reversed. OpenAI can revoke an API key tomorrow. Mistral cannot unpublish a checkpoint that has already been downloaded, and the sentence about potential uplift to conduct real attacks appears on the same page as the download link.

What is not established

  • Whether anybody has independently reproduced the 96.6 percent.
  • How much of the refusal behaviour survives a modest fine-tune. We have not tested it and will not.
  • What the restricted partner variant can do that the open model cannot. Mistral describes it only as optimised for advanced cybersecurity workflows.
  • Whether the Cyber Index score is comparable with the benchmarks OpenAI reports. They are different evaluations and should not be read against each other.
  • What the uplift observation actually was. The page says live traffic showed a potential for uplift to conduct real attacks and does not elaborate.
  • Why the page's chart shows a 95 percent preview figure and a 97 percent release figure when the text says 96.6. The three do not obviously reconcile and we did not resolve it.