A congestion control framework for delay- and disruption tolerant networks

Delay and Disruption Tolerant Networks (DTNs) are networks that experience frequent and long-lived connectivity disruptions. Unlike traditional networks, such as TCP/IP Internet, DTNs are often subject to high latency caused by very long propagation delays (e.g., interplanetary communication) and/or intermittent connectivity. In DTNs there is no guarantee of end-to-end connectivity between source and destination. Such distinct features pose a number of technical challenges in designing core network functions such as routing and congestion control mechanisms. Detecting and dealing with congestion in DTNs is an important problem since congestion can significantly deteriorate DTN performance. Most existing DTN congestion control mechanisms have been designed for a specific DTN application domain and have been shown to exhibit inadequate performance when used in different DTN scenarios and conditions. In this paper, we introduce Smart-DTN-CC, a novel DTN congestion control framework that adjusts its operation automatically based on the dynamics of the underlying network and its nodes. Smart-DTN-CC is an adaptive and distributed congestion aware framework that mitigates congestion using reinforcement learning, a machine learning technique known to be well suited to problems where: (1) the environment, in this case the network, plays a crucial role; and (2) yet, no prior knowledge about the target environment can be assumed, i.e., the only way to acquire information about the environment is to interact with it through continuous online learning. Smart-DTN-CC nodes receive input from the environment (e.g., buffer occupancy, neighborhood membership, etc), and, based on that information, choose an action to take from a set of possible actions. Depending on the selected action's effectiveness in controlling congestion, a reward will be given. Smart-DTN-CC's goal is to maximize the overall reward which translates to minimizing congestion. To our knowledge, Smart-DTN-CC is the first DTN congestion control framework that has the ability to automatically and continuously adapt to the dynamics of the target environment. As demonstrated by our experimental evaluation, Smart-DTN-CC is able to consistently outperform existing DTN congestion control mechanisms under a wide range of network conditions and characteristics.