Tech News
← Home  ·  All topics

Ai Benchmark

2 GoKawiil briefs on this topic · RSS for ai benchmark

Benchmark finds leading AI models omit Women, Peace and Security considerations

Our Secure Future, using the Collective Intelligence Project's Weval platform, built a public benchmark testing whether large language models apply Women, Peace and Security principles in peacekeeping contexts. The results show models can recognize WPS concepts when prompted but fail to raise them unprompted, a pattern the group calls the WPS Competence Gap. OSF says this gap means AI systems largely ignore the experiences of over half the global population, particularly in the Global South.

Researchers use IKEA furniture assembly to test AI reasoning limits

A new benchmark evaluates AI systems on their ability to interpret instructions and reason through the multi-step process of assembling IKEA furniture, a task long considered notoriously difficult even for humans. Early results reportedly show that AI models still struggle to fully complete such assembly tasks without errors.