L3S Best Publication of the Quarter (Q3+Q4/2025)
Category: AutoML
Authors: Difan Deng, Marius Lindauer
Presented at NeurIPS 2025
Current AI models are incredibly powerful, but their operational costs are also huge because they struggle to prioritise information. They try to remember every single word, no matter how irrelevant. Humans don’t work that way; we naturally focus on what matters.
This paper proposes Neural Attention Search (NAtS), a framework that allows AI to mimic human focus. Instead of following a fixed rule to determine, NAtS learns to only focus on the important information and ignore the other uncorrelated words. The result is a ‘leaner’ AI that reduces energy consumption and hardware costs by up to 85% while maintaining peak performance. It makes high-end AI both economically viable and environmentally sustainable.
Link to the full paper: https://arxiv.org/abs/2502.13251
