Meta's AI Models Benchmarks Mislead Developers

• Meta's new AI model, Maverick, ranks highly on LM Arena but differs from the widely available version.
• The LM Arena version of Maverick is an 'experimental chat version' optimized for conversational tasks.
• Customizing models for benchmarks like LM Arena can mislead developers about real-world performance.
• Researchers have noted significant differences between the public Maverick and the LM Arena version.
• Meta has not yet commented on the discrepancies highlighted by AI researchers.

Source: TechCrunch