The Library Media — Science, Technology & Health · AI News & Intelligence Desk
The Benchmark That Makes Agents Read the Blueprint: Google’s 200-Task Test of Visual Knowledge Work
By Catalina Clarke
Follow The Library Media
Google DeepMind published results on Tuesday, November 10, for a new benchmark built to test something most AI evaluations never touch: whether a model can look at professional visual work — a CAD drawing, a 3D scene, a set of design documents — and actually do the job. The Massive Knowledge Agent (MKA) benchmark covers 200 human-designed tasks across 25 sub-domains of computer-aided design and 3D work, scored through roughly 9,000 human ratings.
In Google’s own published results, its Gemini 4 Carbon model reached a win rate of almost 70 percent overall, and Google says the model ranked second in both the CAD and 3D-design categories — at roughly one-sixth of the operating cost of the domain leaders it was measured against. Those are Google’s numbers, published by Google, and they have not been independently recomputed; what follows treats them as the company’s claims, because that is what they are.
The design of the test is the genuinely new part. MKA runs in two directions. The first is visual question-answering: the model is shown screenshots and design documents and must answer questions about them — reading the blueprint, in effect. The second is a long-horizon agentic path: the model is put to work inside real professional software, operating Blender through its MCP interface to complete design tasks that unfold over many steps. Most benchmarks ask a model to predict the next answer. This one asks it to finish the work and show its output to human judges.
Why the shift matters is worth stating plainly. The jobs AI is now being sold into — drafting, design review, 3D production — do not fail at arithmetic. They fail at looking: at noticing the dimension that contradicts the drawing, the part that cannot be machined, the scene that does not match the brief. A benchmark that tests looking, and that tests sustained work inside the actual tools of the trade, is a different instrument from a quiz, and the field has needed one.
The honest caveat rides with the headline numbers. A vendor-published benchmark, with the vendor’s model placing near the top of it at a fraction of rivals’ cost, is a claim awaiting an audit. The next signal that matters is not another launch post — it is independent evaluators running the same 200 tasks and publishing their own win rates. If the ranking survives outside hands, the cost figure becomes the story: near-frontier visual judgment at one-sixth the price changes what small studios and independent designers can afford to automate. If it does not survive, MKA still leaves something behind — a harder test than the one the industry has been grading itself with.
Either way, the direction is set. Evaluation is moving from what a model can say to what a model can see, build, and hand back finished. The desks that cover this industry will be reading benchmarks the way they read balance sheets: for what is claimed, for what is sourced, and for what is still missing.
— Catalina Clarke, AI News & Intelligence Desk
From The Library — the estate of our family.