-
Singular Control of (Reflected) Brownian Motion: A Computational Method Suitable for Queueing Applications
Authors:
Baris Ata,
J. Michael Harrison,
Nian Si
Abstract:
Motivated by applications in queueing theory, we consider a class of singular stochastic control problems whose state space is the d-dimensional positive orthant. The original problem is approximated by a drift control problem, to which we apply a recently developed computational method that is feasible for dimensions up to d=30 or more. To show that nearly optimal solutions are obtainable using t…
▽ More
Motivated by applications in queueing theory, we consider a class of singular stochastic control problems whose state space is the d-dimensional positive orthant. The original problem is approximated by a drift control problem, to which we apply a recently developed computational method that is feasible for dimensions up to d=30 or more. To show that nearly optimal solutions are obtainable using this method, we present computational results for a variety of examples, including queueing network examples that have appeared previously in the literature.
△ Less
Submitted 16 April, 2024; v1 submitted 18 December, 2023;
originally announced December 2023.
-
A Diffusion Model of Dynamic Participant Inflow Management
Authors:
Baris Ata,
Deishin Lee,
Mustafa H. Tongarlak
Abstract:
This paper studies a diffusion control problem motivated by challenges faced by public health agencies who run clinics to serve the public. A key challenge for these agencies is to motivate individuals to participate in the services provided. They must manage the flow of (voluntary) participants so that the clinic capacity is highly utilized, but not overwhelmed. The organization can deploy costly…
▽ More
This paper studies a diffusion control problem motivated by challenges faced by public health agencies who run clinics to serve the public. A key challenge for these agencies is to motivate individuals to participate in the services provided. They must manage the flow of (voluntary) participants so that the clinic capacity is highly utilized, but not overwhelmed. The organization can deploy costly promotion activities to increase the inflow of participants. Ideally, the system manager would like to have enough participants waiting in a queue to serve as many individuals as possible and efficiently use clinic capacity. However, if too many participants sign up, resulting in a long wait, participants may become irritated and hesitate to participate again in the future. We develop a diffusion model of managing participant inflow mechanisms. Each mechanism corresponds to choosing a particular drift rate parameter for the diffusion model. The system manager seeks to balance three different costs optimally: i) a linear holding cost that captures the congestion concerns; ii) an idleness penalty corresponding to wasted clinic capacity and negative impact on public health, and iii) costs of promotion activities. We show that a nested-threshold policy for deployment of participant inflow mechanisms is optimal under the long-run average cost criterion. In this policy, the system manager progressively deploys mechanisms in increasing order of cost, as the number of participants in the queue decreases. We derive explicit formulas for the queue length thresholds that trigger each promotion activity, providing the system manager with guidance on when to use each mechanism.
△ Less
Submitted 5 December, 2023;
originally announced December 2023.
-
A Dynamic Model for Managing Volunteer Engagement
Authors:
Baris Ata,
Mustafa H. Tongarlak,
Deishin Lee,
Joy Field
Abstract:
Non-profit organizations that provide food, shelter, and other services to people in need, rely on volunteers to deliver their services. Unlike paid labor, non-profit organizations have less control over unpaid volunteers' schedules, efforts, and reliability. However, these organizations can invest in volunteer engagement activities to ensure a steady and adequate supply of volunteer labor. We stu…
▽ More
Non-profit organizations that provide food, shelter, and other services to people in need, rely on volunteers to deliver their services. Unlike paid labor, non-profit organizations have less control over unpaid volunteers' schedules, efforts, and reliability. However, these organizations can invest in volunteer engagement activities to ensure a steady and adequate supply of volunteer labor. We study a key operational question of how a non-profit organization can manage its volunteer workforce capacity to ensure consistent provision of services. In particular, we formulate a multiclass queueing network model to characterize the optimal engagement activities for the non-profit organization to minimize the costs of enhancing volunteer engagement, while maximizing productive work done by volunteers. Because this problem appears intractable, we formulate an approximating Brownian control problem in the heavy traffic limit and study the dynamic control of that system. Our solution is a nested threshold policy with explicit congestion thresholds that indicate when the non-profit should optimally pursue various types of volunteer engagement activities. A numerical example calibrated using data from a large food bank shows that our dynamic policy for deploying engagement activities can significantly reduce the food bank's total annual cost of its volunteer operations while still maintaining almost the same level of social impact. This improvement in performance does not require any additional resources - it only requires that the food bank strategically deploy its engagement activities based on the number of volunteers signed up to work volunteer shifts.
△ Less
Submitted 7 December, 2023; v1 submitted 5 December, 2023;
originally announced December 2023.
-
Dynamic Scheduling of a Multiclass Queue in the Halfin-Whitt Regime: A Computational Approach for High-Dimensional Problems
Authors:
Barış Ata,
Ebru Kaşıkaralar
Abstract:
We consider a multi-class queueing model of a telephone call center, in which a system manager dynamically allocates available servers to customer calls. Calls can terminate through either service completion or customer abandonment, and the manager strives to minimize the expected total of holding costs plus abandonment costs over a finite horizon. Focusing on the Halfin-Whitt heavy traffic regime…
▽ More
We consider a multi-class queueing model of a telephone call center, in which a system manager dynamically allocates available servers to customer calls. Calls can terminate through either service completion or customer abandonment, and the manager strives to minimize the expected total of holding costs plus abandonment costs over a finite horizon. Focusing on the Halfin-Whitt heavy traffic regime, we derive an approximating diffusion control problem, and building on earlier work by Han et al. (2018), develop a simulation-based computational method for solution of such problems, one that relies heavily on deep neural network technology. Using this computational method, we propose a policy for the original (pre-limit) call center scheduling problem. Finally, the performance of this policy is assessed using test problems based on publicly available call center data. For the test problems considered so far, our policy does as well as the best benchmark we could find. Moreover, our method is computationally feasible at least up to dimension 100, that is, for call centers with 100 or more distinct customer classes.
△ Less
Submitted 29 November, 2023;
originally announced November 2023.
-
Drift Control of High-Dimensional RBM: A Computational Method Based on Neural Networks
Authors:
Baris Ata,
J. Michael Harrison,
Nian Si
Abstract:
Motivated by applications in queueing theory, we consider a stochastic control problem whose state space is the $d$-dimensional positive orthant. The controlled process $Z$ evolves as a reflected Brownian motion whose covariance matrix is exogenously specified, as are its directions of reflection from the orthant's boundary surfaces. A system manager chooses a drift vector $θ(t)$ at each time $t$…
▽ More
Motivated by applications in queueing theory, we consider a stochastic control problem whose state space is the $d$-dimensional positive orthant. The controlled process $Z$ evolves as a reflected Brownian motion whose covariance matrix is exogenously specified, as are its directions of reflection from the orthant's boundary surfaces. A system manager chooses a drift vector $θ(t)$ at each time $t$ based on the history of $Z$, and the cost rate at time $t$ depends on both $Z(t)$ and $θ(t)$. In our initial problem formulation, the objective is to minimize expected discounted cost over an infinite planning horizon, after which we treat the corresponding ergodic control problem. Extending earlier work by Han et al. (Proceedings of the National Academy of Sciences, 2018, 8505-8510), we develop and illustrate a simulation-based computational method that relies heavily on deep neural network technology. For test problems studied thus far, our method is accurate to within a fraction of one percent, and is computationally feasible in dimensions up to at least $d=30$.
△ Less
Submitted 7 August, 2024; v1 submitted 20 September, 2023;
originally announced September 2023.
-
A Queueing Model of Dynamic Pricing and Dispatch Control for Ride-Hailing Systems Incorporating Travel Times
Authors:
Amir Anastasios Alwan,
Baris Ata,
Yuwei Zhou
Abstract:
A system manager makes dynamic pricing and dispatch control decisions in a queueing network model motivated by ride-hailing applications. A novel feature of the model is that it incorporates travel times. Unfortunately, this renders the exact analysis of the problem intractable. Therefore, we study this problem in the heavy traffic regime. Under the assumptions of complete resource pooling and com…
▽ More
A system manager makes dynamic pricing and dispatch control decisions in a queueing network model motivated by ride-hailing applications. A novel feature of the model is that it incorporates travel times. Unfortunately, this renders the exact analysis of the problem intractable. Therefore, we study this problem in the heavy traffic regime. Under the assumptions of complete resource pooling and common travel distribution, we solve the problem in closed form by analyzing the corresponding Bellman equation. Using this solution, we propose a policy for the queueing system and illustrate its effectiveness in a simulation study.
△ Less
Submitted 4 February, 2023;
originally announced February 2023.
-
Latent Agents in Networks: Estimation and Targeting
Authors:
Baris Ata,
Alexandre Belloni,
Ozan Candogan
Abstract:
We consider a network of agents. Associated with each agent are her covariate and outcome. Agents influence each other's outcomes according to a certain connection/influence structure. A subset of the agents participate on a platform, and hence, are observable to it. The rest are not observable to the platform and are called the latent agents. The platform does not know the influence structure of…
▽ More
We consider a network of agents. Associated with each agent are her covariate and outcome. Agents influence each other's outcomes according to a certain connection/influence structure. A subset of the agents participate on a platform, and hence, are observable to it. The rest are not observable to the platform and are called the latent agents. The platform does not know the influence structure of the observable or the latent parts of the network. It only observes the data on past covariates and decisions of the observable agents. Observable agents influence each other both directly and indirectly through the influence they exert on the latent agents.
We investigate how the platform can estimate the dependence of the observable agents' outcomes on their covariates, taking the latent agents into account. First, we show that this relationship can be succinctly captured by a matrix and provide an algorithm for estimating it under a suitable approximate sparsity condition using historical data of covariates and outcomes for the observable agents. We also obtain convergence rates for the proposed estimator despite the high dimensionality that allows more agents than observations. Second, we show that the approximate sparsity condition holds under the standard conditions used in the literature. Hence, our results apply to a large class of networks. Finally, we apply our results to two practical settings: targeted advertising and promotional pricing. We show that by using the available historical data with our estimator, it is possible to obtain asymptotically optimal advertising/pricing decisions, despite the presence of latent agents.
△ Less
Submitted 26 January, 2022; v1 submitted 14 August, 2018;
originally announced August 2018.
-
On optimality gaps in the Halfin--Whitt regime
Authors:
Baris Ata,
Itai Gurvich
Abstract:
We consider optimal control of a multi-class queue in the Halfin--Whitt regime, and revisit the notion of asymptotic optimality and the associated optimality gaps. The existing results in the literature for such systems provide asymptotically optimal controls with optimality gaps of $o(\sqrt{n})$ where $n$ is the system size, for example, the number of servers. We construct a sequence of asymptoti…
▽ More
We consider optimal control of a multi-class queue in the Halfin--Whitt regime, and revisit the notion of asymptotic optimality and the associated optimality gaps. The existing results in the literature for such systems provide asymptotically optimal controls with optimality gaps of $o(\sqrt{n})$ where $n$ is the system size, for example, the number of servers. We construct a sequence of asymptotically optimal controls where the optimality gap grows logarithmically with the system size. Our analysis relies on a sequence of Brownian control problems, whose refined structure helps us achieve the improved optimality gaps.
△ Less
Submitted 8 March, 2012;
originally announced March 2012.
-
Adaptive Training for Correlated Fading Channels with Feedback
Authors:
Manish Agarwal,
Michael Honig,
Baris Ata
Abstract:
We consider data transmission through a time-selective, correlated (first-order Markov) Rayleigh fading channel subject to an average power constraint. The channel is estimated at the receiver with a pilot signal, and the estimate is fed back to the transmitter. The estimate is used for coherent demodulation, and to adapt the data and pilot powers. We explicitly determine the optimal pilot and d…
▽ More
We consider data transmission through a time-selective, correlated (first-order Markov) Rayleigh fading channel subject to an average power constraint. The channel is estimated at the receiver with a pilot signal, and the estimate is fed back to the transmitter. The estimate is used for coherent demodulation, and to adapt the data and pilot powers. We explicitly determine the optimal pilot and data power control policies in a continuous-time limit where the channel state evolves as an Ornstein-Uhlenbeck diffusion process, and is estimated by a Kalman filter at the receiver. The optimal pilot policy switches between zero and the maximum (peak-constrained) value (``bang-bang'' control), and approximates the optimal discrete-time policy at low Signal-to-Noise Ratios (equivalently, large bandwidths). The switching boundary is defined in terms of the system state (estimated channel mean and associated error variance), and can be explicitly computed. Under the optimal policy, the transmitter conserves power by decreasing the training power when the channel is faded, thereby increasing the data rate. Numerical results show a significant increase in achievable rate due to the adaptive training scheme with feedback, relative to constant (non-adaptive) training, which does not require feedback. The gain is more pronounced at relatively low SNRs and with fast fading. Results are further verified through Monte Carlo simulations.
△ Less
Submitted 9 August, 2009;
originally announced August 2009.
-
Drift rate control of a Brownian processing system
Authors:
Bar Ata,
J. M. Harrison,
L. A. Shepp
Abstract:
A system manager dynamically controls a diffusion process Z that lives in a finite interval [0,b]. Control takes the form of a negative drift rate θthat is chosen from a fixed set A of available values. The controlled process evolves according to the differential relationship dZ=dX-θ(Z) dt+dL-dU, where X is a (0,σ) Brownian motion, and L and U are increasing processes that enforce a lower reflec…
▽ More
A system manager dynamically controls a diffusion process Z that lives in a finite interval [0,b]. Control takes the form of a negative drift rate θthat is chosen from a fixed set A of available values. The controlled process evolves according to the differential relationship dZ=dX-θ(Z) dt+dL-dU, where X is a (0,σ) Brownian motion, and L and U are increasing processes that enforce a lower reflecting barrier at Z=0 and an upper reflecting barrier at Z=b, respectively. The cumulative cost process increases according to the differential relationship dξ=c(θ(Z)) dt+p dU, where c(\cdot) is a nondecreasing cost of control and p>0 is a penalty rate associated with displacement at the upper boundary. The objective is to minimize long-run average cost. This problem is solved explicitly, which allows one to also solve the following, essentially equivalent formulation: minimize the long-run average cost of control subject to an upper bound constraint on the average rate at which U increases. The two special problem features that allow an explicit solution are the use of a long-run average cost criterion, as opposed to a discounted cost criterion, and the lack of state-related costs other than boundary displacement penalties. The application of this theory to power control in wireless communication is discussed.
△ Less
Submitted 11 May, 2005;
originally announced May 2005.
-
Heavy traffic analysis of open processing networks with complete resource pooling: asymptotic optimality of discrete review policies
Authors:
Baris Ata,
Sunil Kumar
Abstract:
We consider a class of open stochastic processing networks, with feedback routing and overlapping server capabilities, in heavy traffic. The networks we consider satisfy the so-called complete resource pooling condition and therefore have one-dimensional approximating Brownian control problems.
We propose a simple discrete review policy for controlling such networks.
Assuming 2+εmoments on t…
▽ More
We consider a class of open stochastic processing networks, with feedback routing and overlapping server capabilities, in heavy traffic. The networks we consider satisfy the so-called complete resource pooling condition and therefore have one-dimensional approximating Brownian control problems.
We propose a simple discrete review policy for controlling such networks.
Assuming 2+εmoments on the interarrival times and processing times, we provide a conceptually simple proof of asymptotic optimality of the proposed policy.
△ Less
Submitted 23 March, 2005;
originally announced March 2005.
