Human Resource Accounting For Implementation In Actual Organizations

Given the dynamical electrochemical properties of neurons and their interconnections (synapses), we readily understand schemes that use a few neurons to obtain elementary useful biological behavior (1-3). Our understanding of such simple circuits in electronics allows us to plan larger and more complex circuits which are essential to large computers. Because evolution has no such plan, it becomes relevant to ask whether the ability of large collections of neurons to perform "computational" tasks may in part be a spontaneous collective consequence of having a large number of interacting simple neurons. In physical systems made from a large number of simple elements, interactions among large numbers of elementary components yield collective phenomena such as the stable magnetic orientations and domains in a magnetic system or the vortex patterns in fluid flow. Do analogous collective phenomena in a system of simple interacting neurons have useful "computational" correlates? For example, are the stability of memories, the construction of categories of generalization, or time-sequential memory also emergent properties and collective in origin? This paper examines a new modeling of this old and fundamental question (4-8) and shows that important computational properties spontaneously arise.

All modeling is based on details, and the details of neuroanatomy and neural function are both myriad and incompletely known (9). In many physical systems, the nature of the emergent collective properties is insensitive to the details inserted in the model (e.g., collisions are essential to generate sound waves, but any reasonable interatomic force law will yield appropriate collisions). In the same spirit, I will seek collective properties that are robust against change in the model details. The model could be readily implemented by integrated circuit hardware. The conclusions suggest the design of a delocalized content-addressable memory or categorizer using extensive asynchronous parallel processing.

“A general content addressable memory would be capable of retrieving this entire memory item on the basis of sufficient partial information. The input might suffice. An ideal memory could deal with errors and retrieve this reference even from the input. In computers, only relatively simple forms of content-addressable memory have been made in hardware (10, 11). Sophisticated ideas like error correction in accessing information are usually introduced as software (10). There are classes of physical systems whose spontaneous behavior can be used as a form of general (and error-correcting) content-addressable memory. Consider the time evolution of a physical system that can be described by a set of general coordinates. A point in state space then represents the instantaneous condition of the system. This state of flow patterns are possible, but the systems of use for memory particularly include those that flow toward locally stable points from anywhere within regions around those points. A particle with frictional damping moving in a potential well with two minima exemplifies such a dynamics. If the flow is not completely deterministic, the description is more complicated. In the two-well problems above, if the frictional force is characterized by a temperature, it must also produce a random driving force. The limit points become small limiting regions, and the stability becomes not absolute. But as long as the stochastic effects are small, the essence of local stable points remains.

Most neurons are capable of generating a train of action potentials-propagating pulses of electrochemical activity-when the average potential across their membrane is held well above its normal resting value. The mean rate at which action potentials are generated is a smooth function of the mean membrane potential, having the general form shown in Fig. 1. The biological information sent to other neurons often lies in a short-time average of the firing rate (12). When this is so, one can neglect the details of individual action potentials and regard as a smooth input-output relationship. [Parallel pathways carrying the same information would enhance the ability of the system to extract a short-term average firing rate (13, 14).] A study of emergent collective effects and spontaneous computation must necessarily focus on the nonlinearity of the input-output relationship. The essence of computation is nonlinear logical operations. The particle interactions that produce true collective effects in particle dynamics come from a nonlinear dependence of forces on positions of the particles. Whereas linear associative networks have emphasized the linear central region (15) we will replace the input-output relationship by the dot-dash step. Those neurons whose operation is dominantly linear merely provide a pathway of communication between nonlinear neurons. Thus, we consider a network of "on or off" neurons, granting that some of the interconnections may be by way of neurons operating in the linear regime. Delays in synaptic transmission (of partially stochastic character) and in the transmission of impulses along axons and dendrites produce a delay between the input of a neuron and the generation of an effective output. All such delays have been modeled by a single parameter, the stochastic mean processing time 1/W.

In the model network each "neuron" has elementary properties, and the network has little structure. Nonetheless, collective computational properties spontaneously arose. Memories are retained as stable entities or Gestalts and can be correctly recalled from memories can also be encoded. These properties follow from the nature of the flow in phase space produced by the processing algorithm, which does not appear to be strongly dependent on precise details of the modeling. This robustness suggests that similar effects will obtain even when more neurobiological details are added. Much of the architecture of regions of the brains of higher animals must be made from a proliferation of simple local circuits with well-defined functions. The bridge between simple circuits and the complex computational properties of higher nervous systems may be the spontaneous emergence of new computational capabilities from the collective behavior of large numbers of simple processing elements. Implementation of a similar model by using integrated circuits would lead to chips which are much less sensitive to element failure and soft-failure than are normal circuits. Such chips would be wasteful of gates but could be made many times larger than standard designs at a given yield. Their asynchronous parallel processing capability would provide rapid solutions to some special classes of computational problems.

1. Willows, A. 0. D., Dorsett, D. A. & Hoyle, G. (1973) J. Neurobiol 4, 207-237, 255-285. 2. Kristan, W. B. (1980) in Information Processing in the Nervous System, eds. Pinsker, H. M. & Willis, W. D. (Raven, New York), 241-261. 3. Knight, B. W. (1975) Lect. Math. Life Sci. 5, 111-144. 4. Smith, D. R. & Davidson, C. H. (1962)J. Assoc. Comput. Mach. 9, 268-279. 5. Harmon, L. D. (1964) in Neural Theory and Modeling, ed. Reiss, R. F. (Stanford Univ. Press, Stanford, CA), pp. 23-24. 6. Amari, S.-I. (1977) Bio. Cybern. 26, 175-185. 7. Amari, S.-I. & Akikazu, T. (1978) Biol Cybern. 29, 127-136. 8. Little, W. A. (1974) Math. Biosci. 19, 101-120. 9. Marr, J. (1969) J. Physiol 202, 437-470. 10. Kohonen, T. (1980) Content Addressable Memories (Springer, New York). 11. Palm, G. (1980) Biol Cybern. 36, 19-31. 12. McCulloch, W. S. & Pitts, W. (1943) BulL Math Biophys. 5, 115-133.

14. Rosenblatt, F. (1962) Principles of Perceptrons (Spartan, Washington, DC). 15. Cooper, L. N. (1973) in Proceedings of the Nobel Symposium on Collective Properties of Physical Systems, eds. Lundqvist, B. & Lundqvist, S. (Academic, New York), 252-264.