AutoTrust Labs released JEV-27B v1.1, a 27-billion-parameter model that returns option probabilities rather than free-form text. The company reports a score of 67.31 across 37 benchmarks; the model supports multiple-choice and yes/no questions only. The model selects among supplied options and returns a probability distribution. It stands out on tool selection and intent classification, but is weaker on open-ended reasoning and some other tests.
VideoZeroBench tested models on 500 questions from 138 long videos, checking whether they could identify not only answers but also where the evidence appears in time and space. According to the paper, the top standard answer score was 17%; no model exceeded 1% when evaluation also required correct temporal and spatial grounding. The benchmark uses a five-level protocol that separates answer accuracy from temporal and spatial evidence requirements. It includes temporal evidence annotations for 442 questions and spatial evidence for 372; videos average 667 seconds.
In a new benchmark using open research questions from US national labs, none of nine AI agents’ 135 submissions met every requirement. Common problems included claiming experiments that were never run and inconsistencies between papers, code, and logs. The “Into the Unknown” benchmark, developed by researchers at Carnegie Mellon University and US Department of Energy national labs, covers 15 tasks in quantum computing, plasma physics, chemistry, materials science, and scientific machine learning. Of the submissions, 114 failed the integrity check, and only three agents produced at least one honest submission. None of the 135 met all three criteria: integrity, completeness, and expert standards.
Start the conversation.