Attempts to automate LEED v4.1 BD+C certification using general-purpose cloud models are hitting a harsh reality: neural networks are catastrophically bad at arithmetic. In a recent study, Arithro Dey and Juliana Felkner from the University of Texas at Austin confirm that relying on pure LLMs for high-stakes tasks is a risk, not a solution. The researchers integrated deterministic numerical verification into the pipeline, creating a neurosymbolic architecture where rigid logic corrects neural "hallucinations." The result is striking: accuracy in the critical EA-p2 category jumped from a dismal 50% to a perfect 100%.
This case clearly demonstrates the death of the "bigger is better" dogma. Tests on actual documentation from four university buildings showed that gemma3, a small 4-billion parameter model, outperforms the 8-billion parameter llama3.1 in specific LEED screening tasks. The secret lies not in the number of weights, but in the quality of prompt engineering and agentic retrieval-augmented generation (RAG) tailored to the structure of specific document sections. For a privacy-obsessed industry, this is excellent news: local Small Language Models (SLMs) are not only more accurate but also eliminate the risk of data leaks to Big Tech clouds.
Key Research Takeaways
Pure neural networks fail arithmetic checks in 50% of cases, making them unsuitable for compliance without external logic. Small local models (SLMs) outperform larger counterparts when RAG and data structures are properly configured. Multimodality degrades system performance when handling low-resolution technical drawings.
"Attempting to force the system to 'look' at drawings made it dumber than if it had simply read the dry text."
However, multimodality—the year’s biggest marketing hook—turned out to be a trap in practice. Analysis revealed that adding low-resolution drawings (150–300 dpi) to the model consistently reduced overall accuracy. Instead of "seeing" the essence, the models drown in incorrectly extracted figures and visual noise. Even with the neurosymbolic method's overall accuracy at 61.6%, it occasionally lagged behind text-only approaches specifically due to failures in image processing.
Industry leaders have promised for years that AI would effortlessly "read" complex blueprints and PDF reports to automate ESG audits. The reality is more ironic: businesses should stop chasing "all-purpose" models and focus on data hygiene. Without a clean dataset and a hard mathematical layer sitting atop the neural network, no "intelligence" will pass even basic compliance.