Last I heard (and I believe this could be wrong) my professor said that we basically understand how a single neuron works. That like basically if we do X input we get Y output, up to some accuracy. He used this to discuss the idea behind neural networks -- that each neuron is simple enough to model, all we need to worry about is the weights and the dynamics of the network as a whole.
How much of a simplification is that? And how much does the accuracy of such a model matter, in the grand scheme of things?
oh finally something that I learned a lot about :)
Such research is the area of computational neuroscience - one thing that such people do is try to model parts of the brain (or just a single neuron) with computers.
A Neuron (=nerve cell in the brain) is a very complex beast. In rough terms they work like this: They collect signals (electrical impulses) via their small appendages called dendrites. when the sum of the signals reaches a certain threshold a large electrical impulse is generated at the cell body that will travel trough its "output" appendage (called axon) that connected to another neuron's cell body or to its dendrite.
Neurons display a dazzling variety in all these parameters:
- In morphology, e.g. they can look like a pine tree http://www.scholarpedia.org/article/Pyramidal_neuron (I really recommend scholarpedia, also this article has a nice animation on how electrical impulses propagate) or like a sea urchin.
- it really matters where the cell gets its impulse from: A neuron stimulated near its cell body will be much more sensitive to the input than being stimulated far away.
- Their response characteristics are wildly varied too. Some give off one large impulse, some a quick burst of impulses. Some are preventing others from giving out impulses from stimulation (inhibitor neurons)
- This whole mess can be modulated with chemical compounds that are released by the body -- some make some neurons more sensitive, some less.
- Also we still discover every year some new mechanism that modulates how they function.
The issue is that this results in such a complex system that a modern PC cant even simulate 1 detailed neuron model realtime (these tools are open source, try them out! for example https://neuron.yale.edu/ ). Now we know that we're simulating things that likely do not matter (e.g. we don't need a neuron model that consist of 10.000+ segments), but we do not know which parts we need to remove to have a faithful simulation. Also we might simply simulate some parts wrong because our knowledge of the subject is not enough.
But on the upside we've reached some great things already, for example we know how our brain calculates from our head and eye position the orientation of the things we're looking at
> All we need to worry about is the weights and the dynamics of the network as a whole. How much of a simplification is that?
A lot. Parallel optimization is an art form. These models are trained on static datasets, they can't intervene in the environment to infer causal relations, so they need legs and hands.
I would say quite a bit. Adding even a third body makes it impossible to calculate physics with certainty. A complex system with any number of individual components is hard to understand with certainty and/or calculations can become exponentially more complex .
The intent was to illustrate that complexities of a system of simple components can be pretty difficult. Automata theory has more appropriate examples perhaps.
That’s correct understanding as of 1943 when the “artificial neural network“ model your professor is teaching was developed.
There is a whole lot of new knowledge on how live neurons and networks of neurons work that had been collected in the last 75 years in the neuroscience domain but it’s mostly ignored by computer scientists.
How much of a simplification is that? And how much does the accuracy of such a model matter, in the grand scheme of things?