Paper Instruments reports that Ultramarine improves on its GLM-5.3 Flash base across all six knowledge-work benchmarks it tested, by 4.1 points on average, while generating 6.5% fewer tokens[^1]. On its internal PI Knowledge Work suite it scores 59.3 at $0.13 per task, ahead of Claude Fable 5 at 57.8 ($8.19) and GPT-5.6 Sol at 55.1 ($2.09), and within three points of Claude Opus 5.5 and GPT-6 Sol[^1]. The company says those internal scores correlate with the Artificial Analysis Intelligence Index at r = 0.81[^1].
The public benchmarks are more mixed. Ultramarine tops JobBench at 62.9 and FrontierFinance at 53.3 among the models Paper Instruments compares, but trails Gemini 3.8 Flash and Claude Fable 5 on DiligenceBench and APEX-Agents 1.1[^2]. All of these are the company's own runs.