Activision Blizzard has seen a 43% decrease in toxic voice chat in Call of Duty thanks to AI-based moderation by ToxMod, which analyzes voice chat transcripts and flags violations for human review, resulting in a 67% decrease in repeat offenders. The system monitors keywords, player emotions, and game conduct to distinguish between trash talk and harassment, with plans to expand language support in Black Ops 6. AI moderation has also been successful in text chat, with Community Sift blocking 45 million messages since its introduction in 2019.
Full Article
Anthropic details Constitutional Classifiers, a system that aims to guard AI models against jailbreaks, monitoring both inputs and outputs for harmful content (Cristina Criddle/Financial Times)
Cristina Criddle reports on Constitutional Classifiers, a system developed to protect AI models from jailbreaks by monitoring inputs and outputs for harmful content. Leading tech companies like Microsoft and Meta are also investing in similar safety measures. This new technology aims to enhance the security and reliability of artificial intelligence systems. Full Article
Read more