EPFL researchers have discovered key “units” in large AI models that seem to be important for language, mirroring the brain’s language system. When these specific units were turned off, the models got much worse at language tasks.
Large language models (LLMs) are not just good at understanding and using language, they can also reason or think logically, solve problems and some can even predict the thoughts, beliefs or emotions of people they interact with.
Despite these impressive feats, we still don’t fully understand how LLMs work “under the hood,” particularly when it comes to how different units or modules perform different tasks. So, researchers in the NeuroAI Laboratory, part of both the School of Computer and Communication Sciences (IC) and the School of Life Sciences (SV), and the Natural Language Processing Laboratory (IC), wanted to find out whether LLMs have specialized units or modules that do specific jobs. This is inspired by networks that have been discovered in human brains, such as the Language Network, Multiple Demand Network and Theory of Mind network.
