02/10/2026
We've just launched the IEEE CIS Technical Challenge on Domain-Specific Question Answering with LLMs!
Today's LLMs are amazing… until you ask them something truly technical about a niche field. And in many real settings, sending your data to the cloud isn't an option, whether because resources are limited or because the data simply can't leave the building.
So we built a challenge around exactly that:
🔬 A hidden, expert-curated benchmark on IFMIF-DONES, the particle accelerator under construction in Granada, and on specialized AI research literature
💻 One fixed, small open model (OLMo) on a single GPU, with no internet access
🧠 The goal is to make it answer like an expert, using computational intelligence rather than brute-force compute
💰 Up to US$20,000 in prizes, funded by IEEE CIS
📅 Register by 1 Nov · Final submissions by 30 Nov
📄 Top teams move on to expert review and write a 4-page IEEE paper
This is the result of almost a year of work with the experts at IFMIF-DONES and DaSCI, within the DONES-AXIA project (CDTI INNTERCONECTA) and the UGR DONITOS programme. Huge thanks to the team: Jaime Uclés, Carlos Peláez-González, Ignacio Aguilera-Martos, Xingyu Wu, Christian Wagner and Katherine Malan.
Please share it with your students and colleagues, and come join us! 👇
https://cis.taskforce.ieee.org/gpais/tc-domain-qa-llms/
Consider a thought experiment. People routinely ask large language models (LLMs) to draft an email or summarize a report, tasks that almost anyone could do unaided. Imagine instead putting one through a job interview, as a candidate for a position as a highly specialized in-house expert. The questio...
