Optimal multi-layered congestion based pricing schemes for enhanced QoS

Abstract

Pricing is an effective tool to control congestion and achieve quality of service (QoS) provisioning for multiple differentiated levels of service. In this paper, we consider the problem of pricing for congestion control in the case of a network of nodes with multiple queues and multiple grades of service.

We present a closed-loop multi-layered pricing scheme and propose an algorithm for finding the optimal state dependent price levels for individual queues, at each node. This is different from most adaptive pricing schemes in the literature that do not obtain a closed-loop state dependent pricing policy. The method that we propose finds optimal price levels that are functions of the queue lengths at individual queues. Further, we also propose a variant of the above scheme that assigns prices to incoming packets at each node according to a weighted average queue length at that node. This is done to reduce frequent price variations and is in the spirit of the random early detection (RED) mechanism used in TCP/IP networks.

We observe in our numerical results a considerable improvement in performance using both of our schemes over that of a recently proposed related scheme in terms of both throughput and delay performance. In particular, our first scheme exhibits a throughput improvement in the range of 67–82% among all routes over the above scheme.

Introduction

Pricing of network resources has been proposed as an effective means for managing congestion in the Internet. The overall quality of service that the network provides can be considerably improved through appropriate usage and congestion based pricing. Most of the literature on pricing of network resources considers a model for rate control by users where the network adjusts the prices according to the demand, and the users respond by adjusting their transmission rates to optimize a (possibly local) utility function. Some works [1], [2] describe a family of window-based allocation schemes that can be used to achieve a proportionally fair allocation or approximate a max–min fair allocation arbitrarily closely. These schemes were motivated by TCP, the Internet congestion control protocol. Gibbens and Kelly [3] describes another marking scheme for TCP packets to achieve proportional fairness. This paper does not consider the rate control of sources. Rather, it assumes that each link in the network can accommodate multiple classes of service by defining the desired average per hop behavior for each of these classes on the link.

In the differentiated service model the basic idea is to divide the available bandwidth among the various competing classes of traffic in a static or dynamic manner. In [4], the work conserving Tirupati Pricing (TP) scheme is proposed as an alternative to the Paris Metro Pricing (PMP) scheme. In [5], an adaptive stochastic approximation based pricing scheme for the case of a single-node system that is based on actual congestion levels in queues is analyzed and its performance studied numerically. In [6], the performance of TP is compared with the PMP scheme for a single-node model and it is observed that TP performs better than PMP, the latter being non-work conserving. Join the minimum cost queue (JMCQ) policy has been proposed in [6] for evaluating the performance (in terms of price, revenue rate and disutility) of TP and PMP. A stochastic approximation based adaptive pricing methodology is considered in [7] in order to bring the congestion along any route (from source to destination) on the network to a prescribed nominal level. The objective function there depends on the price values and not actual congestion levels as with [5]. Moreover, prices for the entire routes are considered and not of individual queues on each link along the route. In [8], we adapt the algorithm of [7] to the case when prices for individual queues on each link along a route are separately considered. This allows for much greater flexibility in implementation as packets from one service grade at a link can shift to another service grade on another link – a scenario not allowed in [7]. The scheme in [8] is seen to exhibit much better performance over the one in [7].

In this paper, we consider pricing in a system with multiple queues and multiple grades of service, and adopt the TP pricing scheme. We consider prices for individual queues on each link as with [8] and present a state-dependent multi-layered pricing methodology whereby states in each queue are clustered together into various levels and prices are assigned to individual state or queue length levels. Thus each queue ‘manager’ charges a price to an incoming packet joining that queue on the basis of the level of congestion in the queue.

Our method of differentiated pricing depends on various levels of congestion. As pointed in several references such as [9], a congestion-based scheme is preferable over a price discriminated one. Odlyzko [10] pointed out that price differentiation is a popular economic control instrument for managing communication networks such as the Internet and is justifiable as long as it is based on performance measures like QoS levels as opposed to content or application based mechanisms. It has also been argued that as Internet usage becomes more bandwidth intensive, the best-effort model of network management can be prohibitively expensive for the consumers [11]. It can also be inadequate to meet QoS-sensitive applications. Yuksel et al. [12], show analytically and through careful simulations that in the presence of bursty traffic, the required extra capacity (REC) for a single class model of network to meet the same delay and loss assurances of a two-class diff-serv based network can be as high as 60%. In view of such observations one can view a price differentiated scheme of network management, such as the one we propose, as an interesting alternative to best-effort single price model. In [5], [7], [8], variable rate pricing that is based on the actual congestion levels in queues has been proposed. The textbook [13] summarizes many mathematical models for pricing based on designing contracts and auctions, that are relevant in communication networks.

We adopt variable rate pricing in our work and propose a stochastic approximation based gradient search methodology for finding the optimal price levels viz., the optimal multi-level pricing policy at each queue on each link in the network. This is unlike many other schemes in the literature that compute a single price for the whole queue. In fact, the scheme in [7] computes one price for an entire route comprising of a sequence of link-queue tuples. We present two schemes that differ from each other in the form of the pricing policy used. In the first scheme, the price charged to an incoming packet at a given queue on a link is a function of the instantaneous queue length at that queue, while in the second, the above price is considered to be a function of the weighted average queue length. The latter policy is in the spirit of the random early detection (RED) mechanism in TCP/IP based networks and helps in reducing rapid price variations due to instantaneous queue fluctuations. The idea in the second scheme is to avoid changing prices too often in order to prevent rapid fluctuations or volatility in traffic flows. By charging different prices for different congestion levels, our proposed algorithms are able to control congestion better as they make better use of the available state information.

For finding the optimal pricing policy within the given class of policies in either scheme, we adapt a simultaneous perturbation stochastic approximation (SPSA) algorithm that uses two-timescales and deterministic perturbation sequences [14]. SPSA, originally developed in [15] has been known to be a powerful technique for parameter optimization. The original SPSA algorithm in [15] requires two parallel simulations and uses random perturbations. In [14], a one-simulation analog of SPSA based on deterministic perturbations that only requires one-simulation has been developed.

We adapt the algorithm of [14] to find optimal policies in both the schemes that we propose in this paper. We extensively compare the performance of both our schemes with that of [8]. We observe that both our schemes uniformly show across all configurations, more than an order of magnitude better performance over the algorithm in [8]. As already stated the algorithm of [8] shows considerable performance improvements over the scheme in [7]. We refer the reader to [8] for details of the extensive numerical experiments describing performance comparisons of [8] with the scheme in [7]. As can be expected, our state-dependent pricing methodology reduces congestion considerably, increases utilization and minimizes delays in comparison to the methodology in [8]. In particular one of our schemes shows an improvement in throughput performance in the range of 67–82% over the scheme in [8], while the other shows a similar improvement in the range of 34–69%. The above improvement in performance is due to the fact that our scheme utilizes resources in a better manner.

In our model, as with [8], each node in the network maintains a separate logical queue for each service grade on each outgoing link and services packets according to the rules of the pricing scheme. The amount of computational effort required using our approach is marginally higher as compared to [8]. Also, in our experiments, we use the JMCQ policy at each node in the network for evaluating performance in terms of price and disutility as with [6]. (Recall however that the proposed scheme in [6] is only for a single node. We extend it to the case of a network of nodes for our purposes.) We observe from our experiments that disutility using both our schemes is significantly less as compared to the scheme in [8]. This is a highly desirable feature exhibited by our schemes. Both our schemes provide (a) well defined guarantees on the various grades of service, (b) support congestion control and traffic management and (c) are easy to implement and require minimal measurements.

We now comment on the relation of our work to our other related work. The second scheme that we present in this paper has been presented previously in a conference paper [16]. The first scheme that we observe shows the best results overall, has however not been presented before in the literature. We prove here the convergence of both our algorithms. Unlike here, a proof of convergence of even the second scheme has not been shown in [16]. Further, the simulation results that we present here are much more detailed than those presented in [16]. For instance, we also analyze join the minimum cost queue (JMCQ) policy with a single class of packets and present a disutility analysis as well. This has not been done in [16]. Further, as explained before, in [8], the algorithm of [7] has been adapted to the case of link-route pricing. SPSA based algorithms under multi-class graded feedback policies as we develop in this paper have not been studied in [8]. As a result, our algorithms are seen to exhibit an order of magnitude better performance than the algorithm in [8] (see Section 5 for results of detailed experiments). Finally, in a recent work [17], a game theoretic approach is used for the problem of multiclass internet pricing. This approach is fundamentally different from ours as it models nodes as players (in a game theoretic setting) that are competing against one another in order to maximize their individual payoffs. On the other hand, our aim is to minimize the queue length at each queue on any link by tuning the vector of price parameters using two different SPSA simulation optimization schemes. Our model does not use game theory at all.

The rest of the paper is organized as follows: In Section 2, we briefly discuss the link route pricing methodology and present the network and user model. In Section 3, we describe both of our closed loop feedback pricing schemes and the SPSA based algorithm for finding optimal policies within each class of these. In Section 4, we present the convergence analysis. Section 5 presents our detailed numerical experiments. Finally, Section 6 presents the concluding remarks.