D.T. Nguyen, A. Kumar and H.C. Lau. Policy Gradient With Value Function Approximation For Collective Multiagent Planning. International Conference on Neural Information Processing Systems (NIPS), Long Beach, CA, USA, Dec 2017.
Decentralized (PO)MDPs provide an expressive framework for sequential decision making in a multiagent system. Given their computational complexity, recent research has focused on tractable yet practical subclasses of Dec-POMDPs. We address such a subclass called CDec-POMDP where the collective behavior of a population of agents affects the joint-reward and environment dynamics. Our main contribution is an actor-critic (AC) reinforcement learning method for optimizing CDec-POMDP policies. Vanilla AC has slow convergence for larger problems. To address this, we show how a particular decomposition of the approximate action-value function over agents leads to effective updates, and also derive a new way to train the critic based on local reward signals. Comparisons on a synthetic benchmark and a real world taxi fleet optimization problem show that our new AC approach provides better quality solutions than previous best approaches.
Ketki Kulkarni, Khiem Trong Tran, Hai Wang, Hoong Chuin Lau Efficient Gate System Operations for a Multi-Purpose Port Using Simulation-Optimization. Winter Simulation Conference 2017 (WSC17), Las Vegas, USA, Dec 2017
Port capacity is determined by three major infrastructural resources namely, berths, yards and gates. The advertised capacity is constrained by the least of the capacities of the three resources. While a lot of attention has been paid to optimizing berth and yard capacities, not much attention has been given to analyzing the gate capacity. The gates are a key node between the land-side and sea-side operations in an ocean-to-cities value chain. The gate system under consideration, located at an important port in an Asian city, is a multi-class parallel queuing system with non-homogeneous Poisson arrivals. It is hard to obtain a closed form analytical approach for such a system. In this paper, we describe an application of simulation techniques in analyzing the performance of gate operations. Further, we develop an optimization model that is integrated with simulation techniques to suggest efficient lane management policies for an outbound gate system.
Shashi Shekhar Jha, Shih-Fen Cheng, Meghna Lowalekar, Nicholas Wong Wai Hin, Rishikeshan Rajendram, Tran Trong Khiem, Pradeep Varakantham, Troung Troung Nghia, and Firmansyah Bin Abd Rahman. A Real-time Big Data Framework for Taxi Drivers. International Conference on Information and Knowledge Management (CIKM 2017), Singapore, Nov 2017
Traditional taxi fleets need to embrace technology in order to maintain their market share against ride-hailing services. However, very few past studies focus on addressing the practical difficulties faced by the traditional taxi fleet operators. By designing and implementing a big-data platform that is capable of processing real-time location updates from tens of thousands of taxis, we demonstrate how we can use such processing power to infer demands accurately and to generate driving recommendations at different levels for individual drivers. Using real-world dataset fed into our platform, we demonstrate that we can significantly increase average daily trips per driver aiding to the increase in taxi drivers’ revenues.
Baoxiang Li and Hoong Chuin Lau. Combinatorial Auction for Transportation Matching Service: Formulation and Adaptive Large Neighborhood Search Heuristic. International Conference on Computational Logistics (ICCL 2017), Southampton, United Kingdom, Oct 2017
This paper considers the problem of matching multiple shippers and multi-transporters for pickups and drop-offs, where the goal is to select a subset of group jobs (shipper bids) that maximizes profit. This is the underlying winner determination problem in an online auction-based vehicle sharing platform that matches transportation demand and supply, particularly in a B2B last-mile setting. Each shipper bid contains multiple jobs, and each job has a weight, volume, pickup location, delivery location and time window. On the other hand, each transporter bid specifies the vehicle capacity, available time periods, and a cost structure. This double-sided auction will be cleared by the platform to find a profit-maximizing match and corresponding routes while respecting shipper and transporter constraints. Compared to the classical Pickup-and-Delivery Problem, a key challenge is the dependency among jobs, more precisely, all jobs within a bid must either be accepted or rejected together and jobs within a bid may be assigned to different transporters. We formulate the mathematical model and propose an Adaptive Large Neighborhood Search approach to solve the problem heuristically. We also derive management insights obtained from our computational experiments.
T. Verma, P. Varakantham, S. Kraus and H.C. Lau. Augmenting Decisions of Taxi Drivers through Reinforcement Learning for Improving Revenues. 27th International Conference on Automated Planning and Scheduling (ICAPS 2017), Pittsburgh, USA, Jun 2017
Taxis (which include cars working with car aggregation systems such as Uber, Grab, Lyft etc.) have become a critical component in the urban transportation. While most research and applications in the context of taxis have focused on improving performance from a customer perspective, in this paper, we focus on improving performance from a taxi driver perspective. Higher revenues for taxi drivers can help bring more drivers into the system thereby improving availability for customers in dense urban cities.
T.H. Teng, H.C. Lau and A. Kumar. A Multi-Agent System for Coordinating Vessel Traffic (Demonstration). 16th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2017), Sao Paulo, Brazil, May 2017
Environmental, regulatory and resource constraints affects the safety and efficiency of vessels navigating in and out of the ports. Movement of vessels under such constraints must be coordinated for improving safety and efficiency. Thus, we frame the vessel coordination problem as a multi-agent path-finding (MAPF) problem. We solve this MAPF problem using a Coordinated Path-Finding (CPF) algorithm. Based on the local search paradigm, the CPF algorithm improves on the aggregated path quality of the vessels iteratively. Outputs of the CPF algorithm are the coordinated trajectories. The Vessel Coordination Module (VCM) described here is the module encapsulating our MAPF-based approach for coordinating vessel traffic. Our demonstration of VCM is conducted using two maritime scenarios of vessel traffic at two geographical regions of Singapore Waters.
T.H. Teng, H.C. Lau and A. Kumar. Coordinating Vessel Traffic to Improve Safety and Efficiency. 16th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2017), Sao Paulo, Brazil, May 2017
Global increase in trade leads to congestion of maritime traffic at the ports. This often leads to increased maritime incidents or near-miss situations. To improve maritime safety while maintaining efficiency, movement of vessels needs to be better coordinated. Our work formulates this problem of coordinating the paths of vessels as a multi-agent path-finding(MAPF) problem. To address this problem, we introduce an innovative application of MAPF in the maritime domain known as Vessel Coordination Module (VCM). Based on the local search paradigm, VCM plans on a joint state space updated using the Electronic Navigation Charts (ENC) and the paths of vessels. We introduce the notion of path quality that measures the number of positions on a vessel path that is too close to some other vessels spatially and temporally. VCM aims to improve the overall path quality of vessels by improving path quality of selected vessels. Experiments are conducted on the Singapore Straits to evaluate and compare performance of our proposed approach in heterogeneous maritime scenario. Our experiment results show that VCM can improve the overall path quality of the vessels.
T.H. Teng, H.C. Lau and A. Kumar. Prescribing routes to improve safety and efficiency of vessel traffic. 5th International Maritime and Port Technology and Development Conference (MTEC 2017), Singapore, April 2017
Larger vessels are bringing more cargoes into Singapore. Being one of the busiest ports, the safety of navigation in Singapore waters is becoming a real and urgent concern. Collision and near-miss incidents accounted for the majority of maritime incidents between 2011 and 2013. Incident investigations revealed human factor accounted for majority of the causal factors. Thus, it is evident that the existing approaches are inadequate. To improve the safety of navigation and boost the shipping capacity of Singapore waters, this work introduces a prescriptive system known as VTM-Prescribe capable of discovering coordinated paths that get vessels to move in coordinated manner. VTM-Prescribe discovers the set of coordinated paths using a search algorithm based on a local search paradigm. Treating vessels with circular ship domain, VTM-Prescribe prescribes appropriate paths to the vessels, possibly modifying their navigation routes and schedules. Meant for the real world, VTM-Prescribe learns coordination strategies that account for real world parameters such as speed-over-ground, maneuverability and turnaround time of vessels. Rigorous experiments performed using recent AIS data show VTM-Prescribe can reduce risk of navigation and increase the shipping capacity of Singapore waters. This is to be followed up with pilot tests of VTM-Prescribe in near-real world scenarios.
Lim-Wavde, K., Kauffman, R. J., Kam, T. S., and Dawson, G.S. Location matters: geospatial policy analytics over time for household hazardous waste collection in California. iConference 2017, Wuhan, China, March 2017. (Nominated for the Lee Dirks Award for Best Paper)
By integrating mapping and geospatial data into a county-level dataset for exploratory analysis, we will demonstrate how to provide useful insights for waste managers and local governments regarding spatial patterns of household hazardous waste (HHW) collection and how it changes over time. We use map-based visualization to display patterns of spatial intensity and county locations for HHW collection in California from 2004 to 2015. We use exploratory spatial data analytics methods to characterize the spatial distribution of HHW collected per person. When we considered the spatial relationships, we were able to develop and estimate a geographically-weighted regression to explain how different regional factors influence the amount of HHW collected. These factors include demographic characteristics, HHW management policy instruments, and environmental quality enforcement and consideration of these factors are necessary to create a successful recycling program.
X. Wu, A. Kumar, D. Sheldon, and S. Zilberstein. Robust Optimization for Tree-Structured Stochastic Network Design. 31st AAAI Conference on Artificial Intelligence (AAAI 2017), San Francisco, California, USA, Feb 2017
Stochastic network design is a general framework for optimizing network connectivity. It has several applications in computational sustainability including spatial conservation planning, pre-disaster network preparation, and river network optimization. A common assumption in previous work has been made that network parameters (e.g., probability of species colonization) are precisely known, which is unrealistic in real-world settings. We therefore address the robust river network design problem where the goal is to optimize river connectivity for fish movement by removing barriers. We assume that fish passability probabilities are known only imprecisely, but are within some interval bounds. We then develop a planning approach that computes the policies with either high robust ratio or low regret. Empirically, our approach scales well to large river networks. We also provide insights into the solutions generated by our robust approach, which has significantly higher robust ratio than the baseline solution with mean parameter estimates.
D.T. Nguyen, A. Kumar and H.C. Lau. Collective Multiagent Sequential Decision Making Under Uncertainty. 31st AAAI Conference on Artificial Intelligence (AAAI 2017), San Francisco, California, USA, Feb 2017
Multiagent sequential decision making has seen rapid progress with formal models such as decentralized MDPs and POMDPs. However, scalability to large multiagent systems and applicability to real world problems remain limited. To address these challenges, we study multiagent planning problems where the collective behavior of a population of agents affects the joint-reward and environment dynamics. Our work exploits recent advances in graphical models for modeling and inference with a population of individuals such as collective graphical models and the notion of finite partial exchangeability in lifted inference. We develop a collective decentralized MDP model where policies can be computed based on counts of agents in different states. As the policy search space over counts is combinatorial, we develop a sampling based framework that can compute open and closed loop policies. Comparisons with previous best approaches on synthetic instances and a real world taxi dataset modeling supply-demand matching show that our approach significantly outperforms them w.r.t.solution quality.
Last updated on 09 Feb 2018 .