Solving Optimal Neural Layout by Gibbs Sampling

T. Jian, P. Xu, J. S. Baras

8th Joint Conference on Information Sciences 2005 (JCIS 2005 in conjunction with the 7th International Conference on Computational Intelligence and Natural Computation 2005, CINC), Salt Lake City, Utah, July 21-16, 2005. 

 

Abstract

The functions of neural systems of organisms are relying on the massive connections between individual components. Computation and communication in neural networks are costly and under the evolution pressure. Evolution perfected neural systems by maximizing their functionality while minimizing the associated cost. This tradeoff can be formulated as a constrained optimization problem. In another physical process, annealing, the system approaches minimal energy configuration with decreasing temperature. In this paper, by exploring the similarities of these two natural phenomena, we use simulated annealing implemented via Gibbs sampler to investigate the minimal cost placement of individual components in neural systems. We show that given the constraints and the cost function associated with the wiring, we can find the neural network configuration corresponding to the minimal cost. By adjusting the cost function and comparing with the actual configuration in neural systems, we can estimate the actual cost function associated with the wires used by nature. This provides a powerful tool to biologists for investigating the configurations of neural systems. Furthermore, we propose that local interaction based updating could be one way by which neural systems approach optimal layout.