A Mathematician Puts ChatGPT 5.5 Pro Through Its Paces

Evaluations from domain experts carry more signal than benchmark tables. When a working mathematician sits down with a frontier model and writes up what happened, the result is worth reading carefully.

Gowers ran ChatGPT 5.5 Pro through problems in his area and recorded the interaction on his blog. The post is notable not because it confirms the model is capable — that is expected at this tier — but because a specialist can locate the exact seam where fluency ends and actual reasoning breaks down. Generic users rarely find that seam. Experts do.

For engineers building on top of OpenAI's API, this kind of qualitative audit matters. Benchmark scores on MATH or AIME tell you aggregate pass rates. A mathematician working through a problem in real time tells you something different: whether the model tracks a multi-step argument, whether it catches its own errors when pushed, and whether it confabulates intermediate steps in a way that looks correct but isn't.

The practical implication for teams using LLMs in technical workflows is straightforward. If your application requires formal reasoning — proof steps, symbolic manipulation, structured derivations — do not rely on leaderboard position alone. Run your own domain-specific eval. What a model does on a curated test set and what it does under expert interrogation are not the same thing.

ChatGPT 5.5 Pro sits at the high end of OpenAI's current lineup. Whether the session reveals meaningful improvement over prior versions on hard mathematical reasoning, or exposes familiar failure modes dressed in more fluent prose, depends on reading Gowers's account directly. The distinction matters for anyone scoping what frontier models can actually do in production today, as opposed to what release positioning implies they can do.

Source