Learning Behaviors for Coordination in Networked Systems and a Generalized Consensus Protocol

P. Hovareshti and J.S. Baras

Proceedings of the 50th IEEE Conference on Decision and Control and European Control Conference (CDC-ECC 2011), pp. 2447-2452, Orlando, USA, December 12-15, 2011.

Abstract

We consider the problem of coordination for efficient joint decision making in networks of autonomous agents. When making a decision, an agent is influenced by its knowledge about others’ behaviors. Agents’ understanding of others’ behaviors is shaped through observing their actions over a long time. We have modeled the decision making on whether to cooperate in a group effort as a result of a series of two-person games between agents, where the payoff of each agent is computed as the sum of its payoffs from each of these games. The agents initially have different behaviors. In order to maximize their pay-off, they need to learn the others’ behavior and coordinate with them. We consider a behavior learning algorithm for a class of behavior functions and study its effects on the emergence of coordination in the network. The conditions under which the learning algorithm converges are studied. We show that for a class of linear functions the learning algorithm results in an extension of non-homogeneous consensus protocol to the more general case of block-stochastic matrices.