Neural network language models as models of human language processing and cognition more generally
Only published papers are included; for preprints, see Papers.
Last Updated: May 2024
In recent years, availability of massive language corpora, advances in machine learning, and increases in computing power have led to engineering breakthroughs in artificial intelligence, including prominently in language. As a result, efforts have sprouted across labs and domains to leverage models from AI as models of human perception, cognition, and motor control, by directly relating model representations to human behavior and neural responses. We are actively working in this space, so more to come soon.
Language models capture human neural responses to linguistic input
This paper uses early word-embedding models (like GloVe) to decode linguistic meaning from neural activity. We show that a decoder that is trained on imaging (fMRI) data collected while participants process individual word meanings can decode semantic vector representations from imaging data collected while participants read sentences about a variety of topics.
This paper evaluates >40 neural network language models in a model-to-brain encoding framework and shows that representations from unidirectional transformer models, like GPT-2, capture a non-trivial amount of variance in neural responses to sentences. Further, models that perform better on the next-word prediction task fare better in predicting human responses, which suggests that optimizing for predictive representations may be a shared objective of in silico and biological language systems.
This paper builds on the Schrimpf et al. (2021) paper and attempts to isolate the features of the model representations that matter the most for the model-to-brain match. It finds that word meanings and compositional meaning matter more than syntactic structure.
Language models capture human neural responses to computer code
This paper explores what properties of computer code are represented by code models and the brain during simulated execution. We find a correspondence between encodings of critical features including iteration and conditional evaluation, allowing for successful decoding of program structure from the brain using code model proxy representations.
Methodological considerations in relating model representations to human neural responses
This paper formulates guidelines for when different mapping models might be most appropriate in research that attempts to relate model representations to human brain responses.
Model interpretability in AI versus neuroscience
This opinion piece discusses some differences in the construal of model interpretability in AI versus neuroscience and talks about ways to leverage the synergies between the fields.
