Meta got caught gaming AI benchmarks

Meta recently released two new Llama 4 models, Scout and Maverick, with Maverick claiming to outperform competitors on LMArena but later revealed to be an experimental, optimized version for conversationality. LMArena expressed concerns about fair evaluations and Meta’s transparency, prompting Meta’s VP to deny claims of training on test sets. The confusing release and discrepancies between benchmark rankings and real-world performance highlight the challenges and competition in the AI development landscape.

Full Article