Cortical Readout

The cortex (“rind”, as in the rind of an orange) is the outermost part of mammalian brains. Recordings from cortical neurons reveal that their activity is correlated with as many different aspects of external stimuli and internal states as you care to analyze. My favorite hypothesis about cortex is that it is a giant echo state (or liquid state) network. Cortex is a rich representational space: it is a reservoir for just about everything that is computable from environmental inputs and internal states of the animal.

This leads to the question of readout: how do these representations influence information processing in the rest of the brain and what role do they play in animal behavior?

Another interesting characteristic of cortex is that it is, largely, a two dimensional structure, consisting of a thin sheet of neurons (Which in some mammals, like humans, has been crumpled up to form many convolutions). Is there a deep computational reason for having a sheet of neurons, or is it an epiphenomenon of development?

In this cartoon neural network, each circle represents a node in a cortical network. Some nodes are input nodes (red) and some are output nodes (yellow). In mammalian brains, input and output nodes are not neatly divided. It is clear, however, that some parts of the cortex represent sensory events and some parts represent motor events.

The chain of neural activation that leads from sensory input signals to output behavioral decisions of animals is mysterious and poorly understood. My adviser John Maunsell and I were interested in the question of how signals from spatially different parts of cortex are pooled together for further processing, ultimately resulting in behavioral decisions. Specifically, we were interested in the question of what kind of model best described how signals at two different nodes of the cortical network (say as indicated by the blue circles) acted on the rest of the network and influenced downstream processing, and whether the mode of signal processing depended at all on the physical distance between the nodes.

We found that when pairs of nodes are within about 1mm of each other the signals from the nodes interact in a linear fashion (i.e. the brain is capable of linearly adding signals that are within 1mm of each other on cortex). When the pairs of signals were inserted into nodes further than that, however, a strange thing happened. We found that the brain could only perform comparative operations on the signals. The brain was only capable of testing whether one signal was larger than the other. It appears that linear processing, such as adding the signals, is not possible.

It is not clear why there should be such a restriction on cortical readout and what kind of benefit this would provide to information processing in the brain. On the face of it, this would indicate that the brain is crippled in how it can process most cortical information, since rich linear processing seems to only occur within local 1mm patches of the cortex. Cortical locations that are further apart physically even by a little bit, seem to be incredibly far apart computationally. In fact, we found that signals from cortex 1mm apart are just as ‘isolated’ from each other as signals from the two different hemispheres of the brain!

It is possible that this property of readout from cortex has some benefit in terms of how the rest of the brain learns to interpret cortical signals, but I have not thought about this deeply enough.