A multi-cast algorithm for robust average consensus over internet of things environments

https://doi.org/10.1016/j.comcom.2019.04.007Get rights and content

Abstract

The initial cloud-managed-network conceptualization of the IoT has mutated into a sparsely coupled, distributed system of interacting smart objects, or things. In this context, distributed control techniques have emerged as a tool for performing tasks locally without the support of a back-end infrastructure. Consensus algorithms are the flagship of distributed control and find application in solving problems as varied as distributed task assignment, distributed estimation, and distributed optimization. Unfortunately, typical consensus algorithms deteriorate their performance when faced with realistic communication phenomena as asynchronous communications, packet losses and channel delays. In this work, an algorithm for achieving average consensus over an IoT environment is introduced and evaluated extensively in a large-scale IoT testbed. Evaluation results show that the proposed algorithm achieves average consensus in all evaluated scenarios despite heterogeneity in the network in terms of processing and networking capabilities, packet losses and delays.

Introduction

The IoT was conceived as a large-scale network of heterogeneous objects with Internet connectivity, uniquely addressable using standardized protocols, mainly IPv6 [1], that promotes the appearance of data-driven services [1] in what is known as a cloud centric view [2], with an associated service-oriented layered architecture [3]. As communication technologies have enlarged the set of devices with networking capabilities, the original definition of the IoT needed to be updated to include new classes of objects [4]. Consequently, the initial cloud-managed-network conceptualization of the IoT has mutated into a sparsely coupled, distributed system of interacting smart objects, or things, defined as autonomous physical devices with sensing/actuating, processing, storing, and networking capabilities [4]. In this context, tackling tasks locally without the support of a back-end infrastructure has emerged as an appealing strategy to be considered, in particular, solving control-oriented tasks using the resources of the IoT has arisen as a viable option, closer to the control point of view of networked cyber–physical systems [1]. In this setting, distributed control techniques, i.e., control strategies that rely only on local information, appear as the natural tool to use in IoT environments.

Consensus algorithms [5] are the flagship of distributed control and have been used as the basis for solving problems such as distributed estimation [6], and distributed optimization [7]. Consensus algorithms are simple distributed protocols, suitable for low-performance “things”, that require only minimal computation and communication to reach a network-wide common value for locally observed variables, the most popular formulation being the one ensuring convergence to the average of the initially observed values: the average consensus algorithm. Although consensus algorithms are mature well studied strategies that naturally adjust to networks, their theoretical properties rely heavily on strong assumptions such as perfect synchronization, instantaneous transmissions, concurrent updates, and nominally identical agent dynamics. Hence, evaluating their performance in real environments is a must to appreciate their real usefulness in practical applications, particularly in IoT environments, where average consensus algorithms can help in solving operational problems as task assignment [8] and time synchronization [9], and to provide services to the nodes, as distributed localization [9].

From a communications point of view, consensus algorithms can be implemented either using a uni-cast, or gossip-based, strategy, or a multi-cast, or broadcast-based, strategy. The classical broadcast-based formulation originates from the theoretical work [10], which requires a network-wide synchronous update that is not feasible to achieve in a real environment. Consequently, efforts like the broadcast gossip [11], [12] fail to converge to the average under realistic conditions. An improved version are the so called push-sum algorithms [13] which use a double consensus scheme to achieve the average under a broadcast-based scheme despite asynchronous transmissions in the network. Recently, a push-sum inspired algorithm that is robust to packet losses and asynchronous transmissions was proposed in [14]. However, its evaluation in a real testbed has not been presented yet. On the gossips side, [15] presents a series of protocols for achieving average consensus in ideal conditions, yet performance in real environments is not presented. In the same line, [16] presents a modified version that includes an extra variable known as surplus, that allows achieving consensus in directed networks not necessarily symmetrical; however, performance over realistic communication networks is not guaranteed. The work [17] provides a theoretical analysis for a broader class of algorithms (both gossips and broadcast-based) under asynchronous communications and packet drops, which indicates that for classical formulation convergence to the average fails under these phenomena and, in particular, extra variables are required to deal with packet drops. The recent work [18] explores further this idea and proposes the use of a vector of auxiliary variables to design an updating rule which guarantees almost sure convergence to the average for both the gossip and broadcast cases; a real implementation of these algorithms is not presented. In a slightly different line, an improvement over the classical gossip formulation is presented in [19] to deal with an undesired phenomenon pervasive in processor-based implementations known as deadlocks, however, the proposed protocol is not evaluated in real communication channels and its theoretical properties do not include robustness to delays or packet losses, hence its performance in a real implementation is uncertain. Recently, the work [20] introduced and evaluated in a real testbed a gossip-based protocol designed for IoT environments that is able to compensate for packet losses and is robust to delays. The objective of the present manuscript is to complement the work in [20] by presenting and evaluating a similar algorithm that uses multi-cast communications instead of gossips. Since recent experiments in IoT testbeds [21] have shown that classical multi-cast average consensus efforts with strong theoretical guaranties deteriorate their performance when faced with phenomena found in IoT environments, a second goal of this manuscript is to provide a deep evaluation in a real testbed of the proposed algorithm, and the algorithm in [14] to provide a meaningful performance comparison. Consequently, the contributions of this manuscript are twofold, namely,

  • The formulation of a new distributed multi-cast algorithm to achieve average consensus over IoT environments.

  • An extensive evaluation of the proposed algorithm over a real large-scale IoT testbed, including a comparison against an state-of-the-art algorithm.

The rest of the manuscript is organized as follows. Section 2 summarizes preliminaries regarding graph theory, networking, system theory, and the consensus problem. In Section 3 a fully distributed multi-cast-based algorithm for achieving average consensus over IoT environments is proposed. In Section 4 an exhaustive evaluation of the protocol is reported. Section 5 concludes the paper.

Section snippets

Notation and basic definitions

In this work, N denotes the natural numbers, R the real numbers, R0 the set of nonnegative real numbers, and Z0 the set of nonnegative integers. Rn denotes the Euclidean space of dimension n. For a countable set χ, |χ| denotes its cardinality.

For time dependent variables, the superscript + denotes the value after an impulsive update has taken place. In most sections of this manuscript, with slight abuse of notation, the time index is omitted to keep the expressions as clean as possible. Thus,

Proposed protocol

We propose a multi-cast-based protocol, in the sense that at each transmission instant information is sent to a group of agents rather than to an individual neighbor, that solves the average consensus problem over IoT environments. The general flow of the protocol is as follows: if a timer event occurs at agent i, then it generates a transmission to every neighbor kNi. Whenever agent k detects the event of data reception it updates its state with the received information.

Remark 2

Note that the system

Protocol implementation

To implement the proposed protocol in real hardware, a complete networking stack, where addressing is possible at the network layer, must be used. In particular, we propose to use the public Internet protocol version 6 (IPv6) at the network layer, in line with the current trends for the IoT. Moreover, we propose the use of the UDP protocol at the transport layer to ensure speed over reliablity. Since the protocol does not use functionalities from the lower layers, in principle any IP-compatible

Conclusions

In this paper, a multi-cast-based algorithm designed to solve the average consensus problem over IoT environments, was implemented and evaluated over a large-scale IoT testbed. The proposed algorithm is suitable for low-capable IoT devices communicating using UDP over IPv6, which makes it ideal for low-energy deployments.

Performance was evaluated in a large-scale network with 280 agents over the public FiT IoT-LAB using nodes from the cities of Paris, Strasbourg, Saclay and Grenoble.

References (23)

  • R. Olfati-Saber, Distributed Kalman filtering for sensor networks, in: 2007 46th IEEE Conference on Decision and...
  • This work was supported in part by grant CONICYT/FONDECYT, Chile 1161039: “Distributed Multi-Agent Control for the Internet of Things”.

    View full text