Bayesian stats (githubbed, iPython notebooked)

Someone has put up on github a very interesting experiment. An experiment I hope will succeed well. They have started a tutorial on Bayesian statistics in the form of an iPython notebook (which as you may recall allows you to embed python code, plots, text and math, into a notebook format, much like Maple or Mathematica). They have put this notebook collection up on github. This is awesome, because you can play with the examples as you learn about the theory and you can put in corrections/enhancements to the text/code and share it with everyone else.

The other great analog-digital debate

Some of you are old enough to remember the great analog-v-digital debate: Vinyl or CD? This post is about the OTHER great (but slightly less well known) analog-v-digital debate: do we simulate neurons on digital computers or on custom designed analog VLSI chips?

When I was at the Univ. of Maryland I was hooked on Neuromorphic engineering by Timothy Horiuchi. The central tenet of Neuromorphic engineering is that transistors operating in the subthreshold (analog) zone are great mimics of the computations done by neurons, and the way to intelligent machines is through building networks of such neuro-mimetic neurons on analog Very Large Scale Integration (aVLSI) chips. This press release of some work being done at INI at Zurich reminded me of this field.

What the writeup also reminded me about, was the great debate between digital and analog implementations of neural circuits. Proponents of Neuromorphic VLSI base their work on the idea that transistors working in the sub-threshold zone give, for “free”, a nice non-linearity between input and output that is at the heart of neural circuits. When applying for funds from DARPA they also remind the grant reviewers that aVLSI circuits have very low power consumption compared to digital VLSI circuits.

A well designed and debugged aVLSI Neuromorphic chip is a great feat of engineering (often taking several fabrication rounds to get all the design problems weeded out) which makes iterating over designs very time consuming and unwieldy.

The proponents of old school digital computation, where neural behavior is encoded in an algorithm (implementing differential equation models of neurons) point to the ease of implementation (you can use your favorite programing language) the ease of debugging (you just recompile while you have a drink) and the ease of modifying and elaborating the design (comment your code!!).

There are some specific issues with aVLSI too. When you make giant neural networks hooking up digital neurons is usually done using a connection matrix. This matrix simply tells the simulating program which neuron gets inputs from which other neurons and which neurons it projects to. In aVLSI you need to physically wire up neurons on the chip layout. This means you can no longer modify the network organization to test out ideas on the fly – you need to design a new chip layout, send it for fabrication wait, debug and so on. (And the moment you start changing connections you have to start moving the whole design around because the exact routing of the wires affects the behavior of chip because everything is so close and the voltages so low that the capacitance between wires matters. As I said, it is a true feat of engineering).

People have come up with non-analog solutions to this ‘routing’ problem, by creating hardware versions of the connection matrix: separate circuits, often on a separate chip, that are dedicated to hooking up neurons to other neurons, somewhat like a telephone switchboard. These lose the low power advantage of aVLSI and increase the complexity of the circuits.

You know that I’m going to give my two cents. I think, not being very qualified to comment on either analog and digital implementations of Neural circuits, that aVLSI might have some niche applications in tiny devices tailored to a specific task where small size and low power consumption are important. However, for the vast majority of machine intelligence applications, I think simple simulations of neural circuits, performed by ever more powerful and power efficient digital circuits will prevail.

Optogenetics vs Electrics

Since very early on, we have known that electrical currents can affect the nervous system. The most well known and successful use of electrical stimulation in the nervous system is the cochlear implant. A tiny and clever signal processor converts sounds recorded in a microphone into electrical pulses that drive neurons in the patient’s auditory nerve, passing enough information for the patient to interpret speech. A second well known use of electrical stimulation in the nervous system is deep brain stimulation, where electrodes are placed in the basal ganglia and change the activity of certain motor circuits that have been damaged by Parkinson’s disease.

In the 1960s many experiments were performed that demonstrated how electrically activating neurons can elicit movements, sensations and even trigger memory retrieval and strange emotional states. A routine procedure in neurosurgery is electrical mapping, where a sedated patient is electrically stimulated to map out the functions of a part of the brain that is going to be operated on, to ensure that important cognitive and motor functions are not damaged.

Optogenetics is the emerging field of using lasers, instead of electric currents, to activate or deactivate specially treated neurons. In basic science it is used to perturb the activity of neurons to observe effects on the neural circuit and/or behavior. In translational science the goal is to supersede electrical-microstimulation as a means of activating neurons to deliver information or fix broken neural circuits. Here, instead of pulsing electricity to activate neurons in a particular way, we pulse laser light to switch the neurons on and off to deliver our message.

The neurons are treated by infecting them with a virus carrying genetic data that forces the infected neuron to produce special channels. The channels are activated by laser light of particular colors. When laser light is directed at the neurons bearing these channels, the current from the opened/closed channel changes the firing of the neuron, either activating it or silencing it. The viruses are designed not to destroy the neuron or hijack its machinery for replication but simply to encourage the neuron to produce the special light activated channels.

In order to use this method in the living subject one must first deliver the virus to the appropriate location in the nervous system. One direct way is to simply take a syringe with a tiny bit of virus and inject it into a brain region. Another way, which is currently not possible in humans or other primates but is possible in mice, is to develop a genetically altered organism that has special ‘markers’ on neurons of interest and when virus is injected into the brain the virus only infects those specially marked neurons.

The next step is to deliver laser light to the infected neurons. This requires either a fiber optic cable that reaches the target or small lasers (probably solid state lasers – small microchip LEDs) that can be implanted into the target. This still requires a harness that delivers power and signal to the laser for a remotely located signal processor.

This section is simply a crass opinion from a person who has done some research in electrical micro-stimulation and has only seen optogenetics second hand.

As a basic research tool optogenetics is the bees knees right now. I think the usefulness of the tool is rather oversold. In research preparations where direct access to the neurons is practical, such as slice or culture preparations, people have used intracellular and patch recordings to directly manipulate neural activity using electricity and chemicals to uncover the hidden mechanisms of neural machinery. In preparations where behaving subjects are needed (to see what the role of a particular brain region is in shaping behavior) clever experiments have already uncovered how the activation or silencing of neurons affects processing. Such experiments have been done using electrical stimulation to activate neurons, cooling to deactivate neurons and pharmacology (drugs) to do both, often targeting specific neuron types. Optogenetics merely replicates such findings (which isn’t a bad thing) or refines them by incremental amounts.

As a translational tool – a tool that will be used regularly in surgeries and therapies in humans – I think the probability of optogenetics replacing electrical microstimulation is very, very low. First, the viruses need to be cleared for human use. We need extensive trials, eventually in humans, that indicate that the viruses will not harm the patient. Secondly the infrastructure for optogenetics is cumbersome. Not only do we need to inject the virus into the brain, we then need to lower fiber optics or the tether for a solid state laser, into the target area. This is about the same disruption we inflict when we lower electrodes for electrical stimulation in the brain.

The big sell for optogenetics is that we can activate or deactivate neurons based on the channel type. However, in terms of translational tools, the final effect of activation/deactivation is the result of complex interactions within the local circuit. It has not yet been shown that adding the ability to deactivate a group of neurons is going to offer us any added benefit over our existing crude excitation only electrical stimulation techniques.

Another sell for optogenetics is that we can activate neurons of only a given type. Again, if we had detailed knowledge of local brain circuitry (and indeed if we were convinced that local circuitry is simply not randomly wired up) we might be able to extract some extra mileage out of stimulating just one type of neuron in a local circuit. Again, this is yet to be shown to have a therapeutic benefit.

I believe that in terms of brain interfaces and therapies optogenetics will have minimal impact in both improving our knowledge of brain function and in being used as a way to deliver information into the brain. Electrical micro-stimulation is unsexy because it has been used for a long time. This does not make it an in-effective tool as many believe.