As large language models (LLMs) become more prevalent in providing responses to complex inquiries, state governments are increasingly concerned about the potential for these models to disseminate foreign propaganda, particularly from adversaries like Russia. In response, the Estonian Language Institute (ELI) has launched a “Propaganda Resistance” benchmark that assesses various LLMs on their capacity to avoid endorsing topics that align with Russian strategic narratives. Estonia, having regained independence from the Soviet Union only a few decades ago, remains vigilant against perceived disinformation from Russia. The ELI, in collaboration with the volunteer-run defense group Propastop, identified 14 key categories of Russian influence, such as narratives surrounding Crimea and the war in Ukraine. Researchers crafted a series of questions designed to test the LLMs’ responses, which were evaluated by an AI model calibrated in line with expert opinions from Propastop.
Why It Matters
This initiative highlights the ongoing concern among nations, particularly those with historical tensions with Russia, about the spread of misinformation through advanced technologies. By developing benchmarks to gauge LLMs’ resistance to propaganda, Estonia aims to protect public discourse from foreign manipulation. The focus on specific narratives tied to geopolitical events, such as the war in Ukraine and historical justifications for territorial claims, underscores the significant role of information warfare in modern conflicts. This benchmark may serve as a model for other countries seeking to mitigate the influence of foreign propaganda on digital platforms.
Want More Context? 🔎
