You do not have permission to edit this page, for the following reasons:

This IP address has been blocked from editing Wikipedia.
This does not affect your ability to read Wikipedia pages.
Most people who see this message have done nothing wrong. Some kinds of blocks restrict editing from specific service providers or telecom companies in response to recent abuse or vandalism, and can sometimes affect other users who are unrelated to that abuse. Review the information below for assistance if you do not believe that you have done anything wrong.

The IP address or range 75.101.128.0/17 has been blocked (disabled) by ‪JJMC89‬ for the following reason(s):

The IP address that you are currently using has been blocked because it is believed to be a web host provider or colocation provider. To prevent abuse, web hosts and colocation providers may be blocked from editing Wikipedia.
You will not be able to edit Wikipedia using a web host or colocation provider because it hides your IP address, much like a proxy or VPN.
We recommend that you attempt to use another connection to edit. For example, if you use a proxy or VPN to connect to the internet, turn it off when editing Wikipedia. If you edit using a mobile connection, try using a Wi-Fi connection, and vice versa. If you are using a corporate internet connection, switch to a different Wi-Fi network. If you have a Wikipedia account, please log in.
If you do not have any other way to edit Wikipedia, you will need to request an IP block exemption.

How to appeal if you are confident that your connection does not use a colocation provider's IP address:
If you are confident that you are not using a web host, you may appeal this block by adding the following text on your talk page: ((unblock|reason=Caught by a colocation web host block but this host or IP is not a web host. My IP address is _______. Place any further information here. ~~~~)). You must fill in the blank with your IP address for this block to be investigated. Your IP address can be determined here. Alternatively, if you wish to keep your IP address private you can use the unblock ticket request system. There are several reasons you might be editing using the IP address of a web host or colocation provider (such as if you are using VPN software or a business network); please use this method of appeal only if you think your IP address is in fact not a web host or colocation provider.

Administrators: The IP block exemption user right should only be applied to allow users to edit using web host in exceptional circumstances, and requests should usually be directed to the functionaries team via email. If you intend to give the IPBE user right, a CheckUser needs to take a look at the account. This can be requested most easily at SPI Quick Checkuser Requests. Unblocking an IP or IP range with this template is highly discouraged without at least contacting the blocking administrator.

This block will expire on 05:39, 2 October 2024.

Even when blocked, you will usually still be able to edit your user talk page, as well as email administrators and other editors.

For information on how to proceed, please read the FAQ for blocked users and the guideline on block appeals. The guide to appealing blocks may also be helpful.

Other useful links: Blocking policy · Help:I have been blocked
Your IP address is in a range that has been blocked on all Wikimedia Foundation wikis. The block was made by ‪Jon Kolbert‬. The reason given is Open proxy/Webhost: See the help page if you are affected .
- Start of block: 05:40, 21 February 2024
- Expiry of block: 05:40, 21 February 2029
Your current IP address is 75.101.254.2. The blocked range is 75.101.128.0/17. Please include all above details in any queries you make. If you believe you were blocked by mistake, you can find additional information and instructions in the No open proxies global policy. Otherwise, to discuss the block please post a request for review on Meta-Wiki. You could also send an email to the stewards VRT queue at stewards@wikimedia.org including all above details.

You can view and copy the source of this page:

((multiple issues|
((tone|date=February 2024))
((duplication|date=February 2024))
((unbalanced|date=February 2024))
))
The network of the human nervous system is composed of nodes (for example, neurons) that are connected by links (for example, synapses). The connectivity may be viewed anatomically, functionally, or electrophysiologically. These are presented in several Wikipedia articles that include [[Connectionism]] (a.k.a. [[Parallel Distributed Processing]] (PDP)), [[Biological neural network]], [[Artificial neural network]] (a.k.a. [[Neural network]]), [[Computational neuroscience]], as well as in several books by Ascoli, G. A. (2002),<ref name="Ascoli, G.A. (2002)">Ascoli, G.A. (Ed). (2002). Computational Neuroanatomy: Principles and Methods. Totowa, New Jersey: Humana Press.</ref> Sterratt, D., Graham, B., Gillies, A., & Willshaw, D. (2011),<ref name="Sterratt, D., Graham, B., Gillies, A., & Willshaw, D. (2011)">Sterratt, D., Graham, B., Gillies, A., & Willshaw, D. Ch 9 (2011). Principles of Computational Modelling in Neuroscience, Chapter 9. Cambridge, U.K.: Cambridge University Press.</ref> [[Wulfram Gerstner|Gerstner, W.]], & Kistler, W. (2002),<ref name="Gerstner, W., & Kistler. W, (2002)">Gerstner, W. and Kistler, W. (2002). Spiking Neuron Models, Chapter 9. Cambridge, U. K.: Cambridge University Press.</ref> and [[David Rumelhart|Rumelhart, J. L.]], [[James McClelland (psychologist)|McClelland, J. L.]], and PDP Research Group (1986)<ref name="Rumelhart, J. L., McClelland, J. L., and PDP Research Group (1986)">Rumelrhart, D., McClelland, J. L., & the PDP Research Group (1986). Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Volume 1: Foundations. Cambridge: The MIT Press.</ref> among others. The focus of this article is a comprehensive view of modeling a neural network (technically neuronal network based on neuron model). Once an approach based on the perspective and connectivity is chosen, the models are developed at microscopic (ion and neuron), mesoscopic (functional or population), or macroscopic (system) levels. Computational modeling refers to models that are developed using computing tools.

==Introduction==
The nervous system consists networks made up of neurons and synapses connected to and controlling tissues as well as impacting human thoughts and behavior. In modeling neural networks of the nervous system one has to consider many factors. The brain and the neural network should be considered as an integrated and self-contained firmware system that includes hardware (organs), software (programs), memory (short term and long term), database (centralized and distributed), and a complex network of active elements (such as neurons, synapses, and tissues) and passive elements (such as parts of visual and auditory system) that carry information within and in-and-out of the body.

Why does one want to model the brain and neural network? Although highly sophisticated computer systems have been developed and used in all walks of life, they are nowhere close to the human system in hardware and software capabilities. So, scientists have been at work to understand the human operation system and try to simulate its functionalities. In order to accomplish this, one needs to model its components and functions and validate its performance with real life. Computational models of a well simulated nervous system enable learning the nervous system and apply it to real life problem solutions.

What is brain and what is neural network? Section 2.1 addresses the former question from an evolutionary perspective. The answer to the second question is based on the neural doctrine proposed by Ramon y Cajal (1894).<ref>Ramon y Cajol, S. (1894). Textura del Sistema Nervioso del Hombre u los Vertebrados. English translation by Swanson, N. & L. W. (1994). New York: Oxford University Press.</ref> He hypothesized that the elementary biological unit is an active cell, called neuron, and the human machine is run by a vast network that connects these neurons, called neural (or neuronal) network. The neural network is integrated with the human organs to form the human machine comprising the nervous system.

Innumerable number of models of various aspects of the nervous system has been developed and there are several Wikipedia articles identified above that have been generated on the subject. The purpose of this article is to present a comprehensive view of all the models and provide the reader, especially a novice, to the neuroscience, with reference to the various sources.

==Network characteristics==
((See also|Biological neural networks))

The basic structural unit of the neural network is connectivity of one neuron to another via an active junction, called synapse. Neurons of widely divergent characteristics are connected to each other via synapses, whose characteristics are also of diverse chemical and electrical properties. In presenting a comprehensive view of all possible modeling of the brain and neural network, an approach is to organize the material based on the characteristics of the networks and the goals that need to be accomplished. The latter could be either for understanding the brain and the nervous system better or to apply the knowledge gained from the total or partial nervous system to real world applications such as [[artificial intelligence]], [[Neuroethics]] or improvements in medical science for society.

===Network connectivity and models===

On a high level representation, the neurons can be viewed as connected to other neurons to form a neural network in one of three ways. A specific network can be represented as a physiologically (or anatomically) connected network and modeled that way. There are several approaches to this (see Ascoli, G.A. (2002)<ref name="Ascoli, G.A. (2002)"/> Sporns, O. (2007),<ref name="Sporns, O. (2007)">((Cite journal | doi=10.4249/scholarpedia.4695| bibcode=2007SchpJ...2.4695S| title=Brain connectivity| year=2007| last1=Sporns| first1=Olaf| journal=Scholarpedia| volume=2| issue=10| pages=4695| doi-access=free))</ref> [[Connectionism]], Rumelhart, J. L., McClelland, J. L., and PDP Research Group (1986),<ref name="Rumelhart, J. L., McClelland, J. L., and PDP Research Group (1986)"/> Arbib, M. A. (2007)<ref name="Arbib, M. A. (2007)">((Cite journal |doi = 10.4249/scholarpedia.1869|bibcode = 2007SchpJ...2.1869A|title = Modular models of brain function|year = 2007|last1 = Arbib|first1 = Michael|journal = Scholarpedia|volume = 2|issue = 3|pages = 1869|doi-access = free))</ref>). Or, it can form a functional network that serves a certain function and modeled accordingly (Honey, C. J., Kotter, R., Breakspear, R., & Sporns, O. (2007),<ref>((cite journal | last1=Honey | first1=C. J. | last2=Kotter | first2=R. | last3=Breakspear | first3=M. | last4=Sporns | first4=O. | title=Network structure of cerebral cortex shapes functional connectivity on multiple time scales | journal=Proceedings of the National Academy of Sciences USA| publisher=Proceedings of the National Academy of Sciences | volume=104 | issue=24 | date=2007-06-04 | issn=0027-8424 | doi=10.1073/pnas.0701519104 | pmid=17548818 | pmc=1891224 | bibcode=2007PNAS..10410240H | pages=10240–10245|doi-access=free))</ref> Arbib, M. A. (2007)<ref name="Arbib, M. A. (2007)"/>). A third way is to hypothesize a theory of the functioning of the biological components of the neural system by a mathematical model, in the form of a set of mathematical equations. The variables of the equation are some or all of the neurobiological properties of the entity being modeled, such as the dimensions of the dendrite or the stimulation rate of action potential along the axon in a neuron. The mathematical equations are solved using computational techniques and the results are validated with either simulation or experimental processes.  This approach to modeling is called computational neuroscience. This methodology is used to model components from the ionic level to system level of the brain. This method is applicable for modeling integrated system of biological components that carry information signal from one neuron to another via intermediate active neurons that can pass the signal through or create new or additional signals. The computational neuroscience approach is extensively used and is based on two generic models, one of cell membrane potential Goldman (1943)<ref>((cite journal | last=Goldman | first=David E. | title=Potential, impedance, and rectification in membrane | journal=The Journal of General Physiology | publisher=Rockefeller University Press | volume=27 | issue=1 | date=1943-09-20 | issn=1540-7748 | doi=10.1085/jgp.27.1.37 | pmid=19873371 | pmc=2142582 | pages=37–60))</ref> and Hodgkin and Katz (1949),<ref>((cite journal | last1=Hodgkin | first1=A. L. | last2=Katz | first2=B. | title=The effect of sodium ions on the electrical activity of the giant axon of the squid | journal=The Journal of Physiology | publisher=Wiley | volume=108 | issue=1 | year=1949 | issn=0022-3751 | doi=10.1113/jphysiol.1949.sp004310 | pmid=18128147 | pages=37–77| pmc=1392331 |doi-access=free))</ref> and the other based on Hodgkin-Huxley model of action potential (information signal).<ref>((cite journal | last1=Hodgkin | first1=A. L. | last2=Huxley | first2=A. F. | title=Currents carried by sodium and potassium ions through the membrane of the giant axon of Loligo | journal=The Journal of Physiology | publisher=Wiley | volume=116 | issue=4 | date=1952-04-28 | issn=0022-3751 | doi=10.1113/jphysiol.1952.sp004717 | pmid=14946713 | pmc=1392213 | pages=449–472|doi-access=free))</ref><ref>((cite journal | last1=Hodgkin | first1=A. L. | last2=Huxley | first2=A. F. | title=The components of membrane conductance in the giant axon of Loligo | journal=The Journal of Physiology | publisher=Wiley | volume=116 | issue=4 | date=1952-04-28 | issn=0022-3751 | doi=10.1113/jphysiol.1952.sp004718 | pmid=14946714 | pmc=1392209 | pages=473–496|doi-access=free))</ref><ref>((cite journal | last1=Hodgkin | first1=A. L. | last2=Huxley | first2=A. F. | title=The dual effect of membrane potential on sodium conductance in the giant axon of Loligo | journal=The Journal of Physiology | publisher=Wiley | volume=116 | issue=4 | date=1952-04-28 | issn=0022-3751 | doi=10.1113/jphysiol.1952.sp004719 | pmid=14946715 | pmc=1392212 | pages=497–506|doi-access=free))</ref><ref>((cite journal | last1=Hodgkin | first1=A. L. | last2=Huxley | first2=A. F. | title=A quantitative description of membrane current and its application to conduction and excitation in nerve | journal=The Journal of Physiology | publisher=Wiley | volume=117 | issue=4 | date=1952-08-28 | issn=0022-3751 | doi=10.1113/jphysiol.1952.sp004764 | pmid=12991237 | pmc=1392413 | pages=500–544|doi-access=free))</ref>

===Modeling levels===

Sterratt, D., Graham, B., Gillies, A., & Willshaw, D. (2011)<ref name="Sterratt, D., Graham, B., Gillies, A., & Willshaw, D. (2011)"/> classify the biological model of neuroscience into nine levels from ion channels to nervous system level based on size and function. Table 1 is based on this for neuronal networks.

{| class="wikitable"
|-
! Level !! Size !! Description and Functions
|-
| Nervous system|| >&nbsp;1&nbsp;m|| Total system controlling thought, behavior, and sensory & motor functions
|-
| Subsystem|| 10&nbsp;cm|| Subsystem associated with one or more functions
|-
| Neural network|| 1&nbsp;cm|| Neural networks for system, subsystem, and functions
|-
| Microcircuit|| 1&nbsp;mm|| Networks of multilevel neurons, e.g., visual subsystem
|-
| Neuron|| 100&nbsp;μm|| Elementary biological unit of neuronal network
|-
| Dendric subunit|| 10&nbsp;μm|| Arbor of receptors in neuron
|-
| Synapse|| 1&nbsp;μm|| Junction or connectivity between neurons
|-
| Signalling pathway|| 1&nbsp;nm|| Link between connecting neurons
|-
| Ion channels|| 1&nbsp;pm|| Channels that act as gateway causing voltage change
|}

Sporns, O. (2007)<ref name="Sporns, O. (2007)"/> presents in his article on brain connectivity, modeling based on structural and functional types. A network that connects at neuron and synaptic level falls into the microscale. If the neurons are grouped into population of columns and minicolumns, the level is defined as mesoscale. The macroscale representation considers the network as regions of the brain connected by inter-regional pathways.

Arbib, M. A. (2007)<ref name="Arbib, M. A. (2007)"/> considers in the modular model, a hierarchical formulation of the system into modules and submodules.

===Signaling modes===

The neuronal signal comprises a stream of short electrical pulses of about 100 millivolt amplitude and about 1 to 2 millisecond duration (Gerstner, W., & Kistler, W. (2002)<ref name="Gerstner, W., & Kistler. W, (2002)"/> Chapter 1). The individual pulses are action potentials or spikes and the chain of pulses is called spike train. The action potential does not contain any information. A combination of the timing of the start of the spike train, the rate or frequency of the spikes, and the number and pattern of spikes in the spike train determine the coding of the information content or the signal message.

The neuron cell has three components&nbsp;– dendrites, soma, and axon as shown in Figure 1. Dendrites, which have the shape of a tree with branches, called arbor, receive the message from other neurons with which the neuron is connected via synapses. The action potential received by each dendrite from the synapse is called the postsynaptic potential. The cumulative sum of the postsynaptic potentials is fed to the soma. The ionic components of the fluid inside and outside maintain the cell membrane at a resting potential of about 65 millivolts. When the cumulative postsynaptic potential exceeds the resting potential, an action potential is generated by the cell body or soma and propagated along the axon. The axon may have one or more terminals and these terminals transmit neurotransmitters to the synapses with which the neuron is connected. Depending on the stimulus received by the dendrites, soma may generate one or more well-separated action potentials or spike train. If the stimulus drives the membrane to a positive potential, it is an excitatory neuron; and if it drives the resting potential further in the negative direction, it is an inhibitory neuron.
[[File:Caption 1.jpg|thumb|Figure 1. Neuron anatomy for network model]]

The generation of the action potential is called the "firing". The firing neuron described above is called a spiking neuron. We will model the electrical circuit of the neuron in Section 3.6. There are two types of spiking neurons. If the stimulus remains above the threshold level and the output is a spike train, it is called the Integrate-and-Fire (IF) neuron model. If output is modeled as dependent on the impulse response of the circuit, then it is called the Spike Response Model (SRM) (Gestner, W. (1995)<ref name="Gerstner, W. 1995">((cite journal | last=Gerstner | first=Wulfram | title=Time structure of the activity in neural network models | journal=Physical Review E | publisher=American Physical Society (APS) | volume=51 | issue=1 | date=1995-01-01 | issn=1063-651X | doi=10.1103/physreve.51.738 | pmid=9962697 | bibcode=1995PhRvE..51..738G | pages=738–758| url=http://infoscience.epfl.ch/record/97769 ))</ref>).

The spiking neuron model assumes that frequency (inverse of the rate at which spikes are generated) of spiking train starts at 0 and increases with the stimulus current. There is another hypothetical model that formulates the firing to happen at the threshold, but there is a quantum jump in frequency in contrast to smooth rise in frequency as in the spiking neuron model. This model is called the rate model. Gerstner, W., & Kistler, W. (2002),<ref name="Gerstner, W., & Kistler. W, (2002)"/> and Sterratt, D., Graham, B., Gillies, A., & Willshaw, D. (2011)<ref name="Sterratt, D., Graham, B., Gillies, A., & Willshaw, D. (2011)"/> are good sources for a detailed treatment of spiking neuron models and rate neuron models.

===Biological vs. artificial neural network===
The concept of artificial neural network (ANN) was introduced by McCulloch, W. S. & Pitts, W. (1943)<ref name="McCulloch, W. S., & Pitts W. (1943)">((cite journal | last1=McCulloch | first1=Warren S. | last2=Pitts | first2=Walter | title=A logical calculus of the ideas immanent in nervous activity | journal=Bulletin of Mathematical Biophysics | publisher=Springer Science and Business Media LLC | volume=5 | issue=4 | year=1943 | issn=0007-4985 | doi=10.1007/bf02478259 | pages=115–133))</ref> for models based on behavior of biological neurons. [[Norbert Wiener]]  (1961) <ref name=W1948>((cite book |last=Wiener |first=Norbert |title=Cybernetics, or Control and Communication in the Animal and the Machine |year=1948 |publisher=MIT Press |location=Cambridge))</ref> gave this new field the popular name of [[cybernetics]], whose principle is the interdisciplinary relationship among engineering, biology, control systems, brain functions, and computer science. With the computer science field advancing, the von Neumann-type computer was introduced early in the neuroscience study. But it was not suitable for symbolic processing, nondeterministic computations, dynamic executions, parallel distributed processing, and management of extensive knowledge bases, which are needed for biological neural network applications; and the direction of mind-like machine development changed to a learning machine. Computing technology has since advanced extensively and computational neuroscience is now able to handle mathematical models developed for biological neural network. Research and development are progressing in both artificial and biological neural networks including efforts to merge the two.

==Nervous system models==
===Evolutionary brain model===
((See also|Triune brain))

The "triune theory of the brain" McLean, P. (2003)<ref>Mclean, P. (2003). Triune Brain. http://malankazlev.com/kheper/topics/intelligence/MacLean.htm</ref> is one of several models used to theorize the organizational structure of the brain. The most ancient neural structure of the brain is the brain stem or "lizard brain". The second phase is limbic or paleo-mammalian brain and performs the four functions needed for animal survival&nbsp;– fighting, feeding, fleeing, and fornicating.  The third phase is the neocortex or the neo-mammalian brain. The higher cognitive functions which distinguish humans from other animals are primarily in the cortex. The reptilian brain controls muscles, balance, and autonomic functions, such as breathing and heartbeat. This part of the brain is active, even in deep sleep. The limbic system includes the hypothalamus, hippocampus, and amygdala. The neocortex includes the cortex and the cerebrum. It corresponds to the brain of primates and, specifically, the human species. Each of the three brains is connected by nerves to the other two, but each seems to operate as its own brain system with distinct capacities. (See illustration in [[Triune brain]].)

===PDP / connectionist model===
((See also|Connectionism))

The connectionist model evolved out of Parallel Distributed Processing framework that formulates a metatheory from which specific models can be generated for specific applications. PDP approach (Rumelhart, J. L., McClelland, J. L., and PDP Research Group (1986)<ref name="Rumelhart, J. L., McClelland, J. L., and PDP Research Group (1986)"/>) is a distributed parallel processing of many inter-related operations, somewhat similar to what's happening in the human nervous system. The individual entities are defined as units and the units are connected to form a network. Thus, in the application to nervous system, one representation could be such that the units are the neurons and the links are the synapses.

===Brain connectivity model===
There are three types of brain connectivity models of a network (Sporns, O. (2007)<ref name="Sporns, O. (2007)"/>). "Anatomical (or structural) connectivity" describes a network with anatomical links having specified relationship between connected "units". If the dependent properties are stochastic, it is defined as "functional connectivity". "Effective connectivity" has causal interactions between distinct units in the system. As stated earlier, brain connectivity can be described at three levels. At microlevel, it connects neurons through electrical or chemical synapses. A column of neurons can be considered as a unit in the mesolevel and regions of the brain comprising a large number of neurons and neuron populations as units in the macrolevel. The links in the latter case are the inter-regional pathways, forming large-scale connectivity.
[[File:Figure 2-Types of brain connectivity.jpg|thumb|Figure 2 Types of brain connectivity]]

Figure 2 shows the three types of connectivity. The analysis is done using the directed graphs (see Sporns, O. (2007)<ref name="Sporns, O. (2007)"/> and Hilgetag, C. C. (2002)<ref name="Hilgetag, C. C. (2002)">Hilgetag, C. C. (2002). Ascoli, G. A. (Ed), Computational Methods for the Analysis of Brain Connectivity. Computational Neuroanatomy: Principles and Methods, Chapter 14. Totowa, New Jersey: Humana Press.</ref>). In the structural brain connectivity type, the connectivity is a sparse and directed graph. The functional brain connectivity has bidirectional graphs. The effective brain connectivity is bidirectional with interactive cause and effect relationships. Another representation of the connectivity is by matrix representation (See Sporns, O. (2007)<ref name="Sporns, O. (2007)"/>). Hilgetag, C. C. (2002)<ref name="Hilgetag, C. C. (2002)"/> describes the computational analysis of brain connectivity.

===Modular models of brain function===

Arbib, M. A. (2007)<ref name="Arbib, M. A. (2007)"/> describes the modular models as follows. "Modular models of the brain aid the understanding of a complex system by decomposing it into structural modules (e.g., brain regions, layers, columns) or functional modules (schemas) and exploring the patterns of competition and cooperation that yield the overall function." This definition is not the same as that defined in functional connectivity. The modular approach is intended to build cognitive models and is, in complexity, between the anatomically defined brain regions (defined as macrolevel in brain connectivity) and the computational model at the neuron level.

There are three views of modules for modeling. They are (1) modules for brain structures, (2) modules as schemas, and (3) modules as interfaces. Figure 3 presents the modular design of a model for reflex control of saccades (Arbib, M. A. (2007)<ref name="Arbib, M. A. (2007)"/>). It involves two main modules, one for superior colliculus (SC), and one for brainstem. Each of these is decomposed into submodules, with each submodule defining an array of physiologically defined neurons. In Figure 3(b) the model of Figure 3(a) is embedded into a far larger model which embraces various regions of cerebral cortex (represented by the modules Pre-LIP Vis, Ctx., LIP, PFC, and FEF), thalamus, and basal ganglia. While the model may indeed be analyzed at this top level of modular decomposition, we need to further decompose basal ganglia, BG, as shown in Figure 3(c) if we are to tease apart the role of dopamine in differentially modulating (the 2 arrows shown arising from SNc) the direct and indirect pathways within the basal ganglia (Crowley, M. (1997)<ref>Crowley, M. (1997). Modeling Saccadic Motor Control: Normal Function. Sensory Remapping and Basal Ganglia Dysfunction. Unpublished Ph.D. thesis, University of Southern California.</ref>).
Neural Simulation Language (NSL) has been developed to provide a simulation system for large-scale general neural networks. It provides an environment to develop an object-oriented approach to brain modeling. NSL supports neural models having as basic data structure neural layers with similar properties and similar connection patterns. Models developed using NSL are documented in Brain Operation Database (BODB) as hierarchically organized modules that can be decomposed into lower levels.

===Artificial neural networks===
((main article|Neural network|Artificial neural network))

As mentioned in Section 2.4, development of artificial neural network (ANN), or neural network as it is now called, started as simulation of biological neuron network and ended up using artificial neurons. Major development work has gone into industrial applications with learning process. Complex problems were addressed by simplifying the assumptions. Algorithms were developed to achieve a neurological related performance, such as learning from experience. Since the background and overview have been covered in the other internal references, the discussion here is limited to the types of models. The models are at the system or network level.

The four main features of an ANN are topology, data flow, types of input values, and forms of activation (Meireles, M. R. G. (2003),<ref name="Meireles, M. R. G. (2003)">((cite journal | last1=Meireles | first1=M.R.G. | last2=Almeida | first2=P.E.M. | last3=Simoes | first3=M.G. | title=A comprehensive review for industrial applicability of artificial neural networks | journal=IEEE Transactions on Industrial Electronics | publisher=Institute of Electrical and Electronics Engineers (IEEE) | volume=50 | issue=3 | year=2003 | issn=0278-0046 | doi=10.1109/tie.2003.812470 | pages=585–601| s2cid=11278899 ))</ref> Munakata, T. (1998)<ref>Munakata, T. (1998). Fundamentals of the New Artificial Intelligence&nbsp;— Beyond Traditional Paradigms. Berlin, Germany: Springer-Verlag.</ref>). Topology can be multilayered, single-layered, or recurrent. Data flow can be recurrent with feedback or non-recurrent with feedforward model. The inputs are binary, bipolar, or continuous. The activation is linear, step, or sigmoid. Multilayer Perceptron (MLP) is the most popular of all the types, which is generally trained with back-propagation of error algorithm. Each neuron output is connected to every neuron in subsequent layers connected in cascade and with no connections between neurons in the same layer. Figure 4 shows a basic MLP topology (Meireles, M. R. G. (2003)<ref name="Meireles, M. R. G. (2003)"/>), and a basic telecommunication network (Subramanian, M. (2010)<ref>Subramanian, M.(2010): Network Management: Principles and Practice, Chapter 1, Pearson, Chennai</ref>) that most are familiar with. We can equate the routers at the nodes in telecommunication network to neurons in MLP technology and the links to synapses.

[[File:Figure 4(a).jpg|thumb|Figure 4(a)Basic telecommunication network]]
[[File:Figure 4 (b) Basic MLP technology model.jpg|thumb|Figure 4(b)Basic MLP technology model
 Figure 4. Telecommunication network and neural network topologies]]

===Computational neuron models===
((See also|Computational neuroscience))

Computational neuroscience is an interdisciplinary field that combines engineering, biology, control systems, brain functions, physical sciences, and computer science. It has fundamental development models done at the lower levels of ions, neurons, and synapses, as well as information propagation between neurons. These models have established the enabling technology for higher-level models to be developed. They are based on chemical and electrical activities in the neurons for which electrical equivalent circuits are generated. A simple model for the neuron with predominantly potassium ions inside the cell and sodium ions outside establishes an electric potential on the membrane under equilibrium, i.e., no external activity, condition. This is called the resting membrane potential, which can be determined by Nernst Equation (Nernst, W. (1888)<ref>((cite journal | last=Nernst | first=W. | title=Zur Kinetik der in Lösung befindlichen Körper | journal=Zeitschrift für Physikalische Chemie | publisher=Walter de Gruyter GmbH | volume=2U | issue=1 | date=1888-01-01 | issn=2196-7156 | doi=10.1515/zpch-1888-0274 | pages=613–637| s2cid=202552528 | url=http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:hebis:30-1123540 |language=de))</ref>). An equivalent electrical circuit for a patch of membrane, for example an axon or dendrite, is shown in Figure 5. E<sub>K</sub> and E<sub>Na</sub> are the potentials associated with the potassium and sodium channels respectively and R<sub>K</sub> and R<sub>Na</sub> are the resistances associated with them. C is the capacitance of the membrane and I is the source current, which could be the test source or the signal source (action potential). The resting potential for potassium-sodium channels in a neuron is about -65 millivolts.
[[File:Figure 5 Membrane model.jpg|thumb|Figure 5 Membrane model]]

The membrane model is for a small section of the cell membrane; for larger sections it can be extended by adding similar sections, called compartments, with the parameter values being the same or different. The compartments are cascaded by a resistance, called axial resistance. Figure 6 shows a compartmental model of a neuron that is developed over the membrane model. Dendrites are the postsynaptic receptors receiving inputs from other neurons; and the axon with one or more axon terminals transmits neurotransmitters to other neurons.
[[File:Figure 6 Neuron model.jpg|thumb|Figure 6 Neuron model]]

The second building block is the Hodgkin-Huxley (HH) model of the action potential. When the membrane potential from the dendrites exceeds the resting membrane potential, a pulse is generated by the neuron cell and propagated along the axon. This pulse is called the action potential and HH model is a set of equations that is made to fit the experimental data by the design of the model and the choice of the parameter values.

Models for more complex neurons containing other types of ions can be derived by adding to the equivalent circuit additional battery and resistance pairs for each ionic channel. The ionic channel could be passive or active as they could be gated by voltage or be ligands. The extended HH model has been developed to handle the active channel situation.

Although there are neurons that are physiologically connected to each other, information is transmitted at most of the synapses by chemical process across a cleft. Synapses are also computationally modeled. The next level of complexity is that of stream of action potentials, which are generated, whose pattern contains the coding information of the signal being transmitted. There are basically two types of action potentials, or spikes as they are called, that are generated. One is "integrate-and-fire" (the one we have so far addressed) and the other which is rate based. The latter is a stream whose rate varies. The signal going across the synapses could be modeled either as a deterministic or a stochastic process based on the application (See Section 3.7). Another anatomical complication is when a population of neurons, such as a column of neurons in visionary system, needs to be handled. This is done by considering the collective behavior of the group (Kotter, R., Nielson, P., Dyhrfjeld-Johnson, J., Sommer, F. T., & Northoff, G. (2002)<ref name="Kotter, R. 2002">Kotter, R., Nielson, P., Dyhrfjeld-Johnson, J., Sommer, F. T., & Northoff, G. (2002). Multi-level Neuron and Network Modeling in Computational Neuroanatomy.  Ascoli, G. A (Ed). Computational Neuroanatomy: Principles and Methods, Chapter 16 363-364. Totawa, New Jersey: Humana Press.</ref>).

===Spiking neuron models===
((See also|Spiking neural network))

The action potential or the spike does not itself carry any information. It is the stream of spikes, called spike train, that carry the information in its number and pattern of spikes and timing of spikes. The postsynaptic potential can be either positive, the excitatory synapse or negative, inhibitory synapse. In modeling, the postsynaptic potentials received by the dendrites in the postsynaptic neuron are integrated and when the integrated potential exceeds the resting potential, the neuron fires an action potential along its axon. This model is the Integrate-and-Fire (IF) model that was mentioned in Section 2.3. Closely related to IF model is a model called Spike Response Model (SRM) (Gerstner, W. (1995)<ref name="Gerstner, W. 1995"/> Pages 738-758) that is dependent on impulse function response convoluted with the input stimulus signal. This forms a base for a large number of models developed for spiking neural networks.

The IF and SR model of spike train occurs in Type I neurons, in which the spike rate or spike frequency of the occurrence increases smoothly with the increase in stimulus current starting from zero. Another phenomenon of spike train generation happens in Type II neurons, where firing occurs at the resting potential threshold, but with a quantum jump to a non-zero frequency. Models have been developed using the rate (frequency) of the spike train and are called rate-based models.

What is important for understanding the functions of the nervous system is how the message is coded and transported by the action potential in the neuron. There are two theories on how the signal that is being propagated is coded in the spikes as to whether it is pulse code or rate code. In the former, it is the time delay of the first spike from the time of stimulus as seen by the postsynaptic receiver that determines the coding. In the rate code, it is average rate of the spike that influences the coding. It is not certain as to which is really the actual physiological phenomenon in each case. However, both cases can be modeled computationally and the parameters varied to match the experimental result. The pulse mode is more complex to model and numerous detailed neuron models and population models are described by Gerstner and Kistler in Parts I and II of Gerstner, W., & Kistler, W. (2002)<ref name="Gerstner, W., & Kistler. W, (2002)"/> and Chapter 8 of Sterratt, D., Graham, B., Gillies, A., & Willshaw, D. (2011).<ref name="Sterratt, D., Graham, B., Gillies, A., & Willshaw, D. (2011)"/> Another important characteristic associated with SR model is the spike-time-dependent-plasticity. It is based on Hebb's postulate on plasticity of synapse, which states that "the neurons that fire together wire together". This causes the synapse to be a long-term potentiation (LTP) or long-term depression (LTD). The former is the strengthening of the synapse between two neurons if the postsynaptic spike temporally follows immediately after the presynaptic spike. Latter is the case if it is reverse, i.e., the presynaptic spike occurs after the postsynaptic spike. Gerstner, W. & Kistler, W. (2002)<ref name="Gerstner, W., & Kistler. W, (2002)"/> in Chapter 10 and Sterratt, D., Graham, B., Gillies, A., & Willshaw, D. (2011)<ref name="Sterratt, D., Graham, B., Gillies, A., & Willshaw, D. (2011)"/> in Chapter 7 discuss the various models related to Hebbian models on plasticity and coding.

===Nervous system network models===

The challenge involved in developing models for small, medium, and large networks is one of reducing the complexity by making valid simplifying assumptions in and extending the Hodgkin-Huxley neuronal model appropriately to design those models ( see Chapter 9 of Sterratt, D., Graham, B., Gillies, A., & Willshaw, D. (2011),<ref name="Sterratt, D., Graham, B., Gillies, A., & Willshaw, D. (2011)"/> Kotter, R., Nielson, P., Dyhrfjeld-Johnson, J., Sommer, F. T., & Northoff, G. (2002),<ref name="Kotter, R. 2002"/> and Chapter 9 of Gerstner, W., & Kistler, W. (2002)<ref name="Gerstner, W., & Kistler. W, (2002)"/>). Network models can be classified as either network of neurons propagating through different levels of cortex or neuron populations interconnected as multilevel neurons. The spatial positioning of neuron could be 1-, 2- or 3-dimensional; the latter ones are called [[small-world network]]s as they are related to local region. The neuron could be either excitatory or inhibitory, but not both. Modeling design depends on whether it is artificial neuron or biological neuron of neuronal model. Type I or Type II choice needs to be made for the firing mode. Signaling in neurons could be rate-based neurons, spiking response neurons, or deep-brain stimulated. The network can be designed as feedforward or recurrent type. The network needs to be scaled for the computational resource capabilities. Large-scale thalamocortical systems are handled in the manner of the Blue Brain project (Markam, H. (2006)<ref>((cite journal | last=Markram | first=Henry | title=The Blue Brain Project | journal=Nature Reviews Neuroscience | publisher=Springer Science and Business Media LLC | volume=7 | issue=2 | year=2006 | issn=1471-003X | doi=10.1038/nrn1848 | pmid=16429124 | pages=153–160| s2cid=15752137 ))</ref>).

===Nervous system development models===

No generalized modeling concepts exist for modeling the development of anatomical physiology and morphology similar to the one of behavior of neuronal network, which is based on HH model. Shankle, W. R., Hara, J., Fallon, J. H., and Landing, B. H. (2002)<ref>Shankle, W. R., Hara, J., Fallon, J. H., and Landing, B. H. (2002). How the Brain Develops and How It functions. Ascoli, G A. (Ed) Computational Neuroanatomy: Principles and Methods, Chapter 18. Totawa, New Jersey: Humana Press.</ref> describe the application of neuroanatomical data of the developing human cerebral cortex to computational models. Sterratt, D., Graham, B., Gillies, A., & Willshaw, D. (2011)<ref name="Sterratt, D., Graham, B., Gillies, A., & Willshaw, D. (2011)"/> discuss aspects of the nervous system of computational modeling in the development of nerve cell morphology, cell physiology, cell patterning, patterns of ocular dominance, and connection between nerve cell and muscle, and retinotopic maps. Carreira-Perpinan, M. A. & Goodhill, G. J. (2002)<ref>Carreira-Perpinan, M. A., & Goodhill, G. J. (2002). Development of columnar structures in Visual Cortex. In Ascoli, G. A. (Ed), Computational Neuroanatomy: Principles and Methods, Chapter 15. Totowa, New Jersey: Humana Press.</ref> deal with the optimization of the computerized models of the visual cortex.

===Modeling tools===

With the enormous number of models that have been created, tools have been developed for dissemination of the information, as well as platforms to develop models. Several generalized tools, such as GENESIS, NEURON, XPP, and NEOSIM are available and are discussed by Hucka, M. (2002).<ref>Hucka, M., Shankar, K., Beeman, D., and Bower, J. M. (2002). Ascoli, G. A (Ed), The Modeler's Workspace. Computational Neuroanatomy: Principles and Methods, Chapter 5. Totowa, New Jersey: Humana Press.</ref>

==See also==
* [[Connectionism]]
* [[Computational neuroscience]]
* [[Neuroethics]]
* [[Non-spiking neuron]]
* [[Triune Brain]]

==References==
((reflist|30em))

[[Category:Computational neuroscience]]
[[Category:Neural circuitry]]
[[Category:Cognitive modeling]]

Pages transcluded onto the current version of this page (help):

Nervous system network models (edit)
Template:Ambox (view source) (template editor protected)
Template:Cite book (view source) (protected)
Template:Cite journal (view source) (protected)
Template:DMCA (view source) (template editor protected)
Template:Dated maintenance category (view source) (template editor protected)
Template:Dated maintenance category (articles) (view source) (template editor protected)
Template:Duplication (view source) (semi-protected)
Template:FULLROOTPAGENAME (view source) (template editor protected)
Template:Main (view source) (template editor protected)
Template:Main article (view source) (template editor protected)
Template:Main other (view source) (protected)
Template:Multiple issues (view source) (template editor protected)
Template:Multiple issues/styles.css (view source) (template editor protected)
Template:Ns has subpages (view source) (protected)
Template:Reflist (view source) (protected)
Template:Reflist/styles.css (view source) (protected)
Template:See also (view source) (template editor protected)
Template:Tone (view source) (template editor protected)
Template:Unbalanced (view source) (semi-protected)
Module:Arguments (view source) (protected)
Module:Category handler (view source) (protected)
Module:Category handler/blacklist (view source) (protected)
Module:Category handler/config (view source) (protected)
Module:Category handler/data (view source) (protected)
Module:Category handler/shared (view source) (protected)
Module:Check for unknown parameters (view source) (protected)
Module:Citation/CS1 (view source) (protected)
Module:Citation/CS1/COinS (view source) (protected)
Module:Citation/CS1/Configuration (view source) (protected)
Module:Citation/CS1/Date validation (view source) (protected)
Module:Citation/CS1/Identifiers (view source) (protected)
Module:Citation/CS1/Utilities (view source) (protected)
Module:Citation/CS1/Whitelist (view source) (protected)
Module:Citation/CS1/styles.css (view source) (protected)
Module:Format link (view source) (template editor protected)
Module:Hatnote (view source) (template editor protected)
Module:Hatnote/styles.css (view source) (template editor protected)
Module:Hatnote list (view source) (template editor protected)
Module:Labelled list hatnote (view source) (template editor protected)
Module:Message box (view source) (protected)
Module:Message box/ambox.css (view source) (protected)
Module:Message box/configuration (view source) (protected)
Module:Namespace detect/config (view source) (protected)
Module:Namespace detect/data (view source) (protected)
Module:Ns has subpages (view source) (protected)
Module:String (view source) (protected)
Module:Unsubst (view source) (protected)
Module:Yesno (view source) (protected)

Return to Nervous system network models.

Retrieved from "https://en.wikipedia.org/wiki/Nervous_system_network_models"