Jianbin Huang
Abstract
Dynamic pricing plays an important role in solving the problems such as traffic load reduction, congestion control, and revenue improvement. Efficient dynamic pricing strategies can increase capacity utilization, total revenue of service providers, and the satisfaction of both passengers and drivers. Many proposed dynamic pricing technologies focus on short-term optimization and face poor scalability in modeling long-term goals for the limitations of solution optimality and prohibitive computation. In this article, a deep reinforcement learning framework is proposed to tackle the dynamic pricing problem for ride-hailing platforms. A soft actor-critic (SAC) algorithm is adopted in the reinforcement learning framework. First, the dynamic pricing problem is translated into a Markov Decision Process (MDP) and is set up in continuous action spaces, which is no need for the discretization of action space. Then, a new reward function is obtained by the order response rate and the KL-divergence between supply distribution and demand distribution. Experiments and case studies demonstrate that the proposed method outperforms the baselines in terms of order response rate and total revenue.
- [1] Didi Chuxing. Retrieved from http://www.didichuxing.com/en/.Google Scholar
- [2] New York City Taxi and Limousine Commission Dataset. Retrieved from https://www1.nyc.gov/site/.Google Scholar
- [3] New York City Trip Data. Retrieved from https://www1.nyc.gov/site/tlc/about/tlc-trip-record-data.page.Google Scholar
- [4] Uber. Retrieved from https://www.uber.com.Google Scholar
- [5] . 2018. ADAPT-pricing: A dynamic and predictive technique for pricing to maximize revenue in ridesharing platforms. In Proceedings of the 26th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. ACM, 189–198.Google ScholarDigital Library
- [6] . 2015. Pricing in ride-sharing platforms: A queueing-theoretic approach. In Proceedings of the 16th ACM Conference on Economics and Computation. ACM, 639–639.Google ScholarDigital Library
- [7] . 2019. Predicting real-time surge pricing of ride-sourcing companies. Transport. Res. Part C: Emerg. Technol. 107 (2019), 444–462.Google ScholarCross Ref
- [8] . 2019. Spatial pricing in ride-sharing networks. Oper. Res. 67, 3 (2019), 744–769.Google ScholarDigital Library
- [9] . 2017. Surge pricing solves the wild goose chase. In Proceedings of the ACM Conference on Economics and Computation. 241–242.Google ScholarDigital Library
- [10] . 2016. Dynamic pricing in a labor market: Surge pricing and flexible work on the Uber platform. In Proceedings of the 2016 ACM Conference on Economics and Computation. 455.Google ScholarDigital Library
- [11] . 2018. Pricing and matching with forward-looking buyers and sellers. Rotman School Manag. Work. Pap.2859864 (2018).Google Scholar
- [12] . 2019. Your Uber is arriving: Managing on-demand workers through surge pricing, forecast communication, and worker incentives. Manag. Sci. 65, 5 (2019),1995–2014.Google ScholarDigital Library
- [13] . 2018. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. arXiv preprint arXiv:1801.01290 (2018).Google Scholar
- [14] . 2015. Dynamic pricing and energy consumption scheduling with reinforcement learning. IEEE Trans. Smart Grid 7, 5 (2015), 2187–2198.Google ScholarCross Ref
- [15] . 2015. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971 (2015).Google Scholar
- [16] . 2019. Dynamic pricing on e-commerce platform with deep reinforcement learning.
arxiv:1912.02572 [cs.LG]Google Scholar - [17] . 2018. A dynamic pricing demand response algorithm for smart grid: Reinforcement learning approach. Appl. Energ. 220 (2018), 220–230.Google ScholarCross Ref
- [18] . 2018. Reinforcement learning for fair dynamic pricing. In Proceedings of SAI Intelligent Systems Conference. Springer, 120–135.Google Scholar
- [19] . 2013. Playing Atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602 (2013).Google Scholar
- [20] . 2013. A reinforcement learning approach to autonomous decision-making in smart electricity markets. Mach. Learn. 92, 1 (2013), 5–39.Google ScholarDigital Library
- [21] . 2017. Time-of-day pricing in taxi markets. IEEE Trans. Intell. Transport. Syst. (
01 2017).DOI: Google ScholarDigital Library - [22] . 2014. Real-time dynamic pricing in a non-stationary environment using model-free reinforcement learning. Omega 47 (2014), 116–126.Google ScholarCross Ref
- [23] . 2014. A review of Uber, the growing alternative to traditional taxi service. AFB AccessWorld® Mag. 51, 6 (2014).Google Scholar
- [24] . 1995. Problem Solving with Reinforcement Learning. Ph.D. Dissertation. University of Cambridge.Google Scholar
- [25] . 2015. Deep learning in neural networks: An overview. Neural Netw. 61 (2015), 85–117.Google ScholarDigital Library
- [26] . 2015. Trust region policy optimization. In Proceedings of the International Conference on Machine Learning. 1889–1897.Google Scholar
- [27] . 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017).Google Scholar
- [28] . 2020. Reinforcement mechanism design: With applications to dynamic pricing in sponsored search auctions. In Proceedings of the AAAI Conference on Artificial Intelligence. 2236–2243.Google ScholarCross Ref
- [29] . 1998. Introduction to Reinforcement Learning. Vol. 135. The MIT Press, Cambridge, MA.Google ScholarDigital Library
- [30] . 2019. Dynamic pricing and management for electric autonomous mobility on demand systems using reinforcement learning. arXiv preprint arXiv:1909.06962 (2019).Google Scholar
- [31] . 2019. Reinforcement learning for real-time pricing and scheduling control in EV charging stations. IEEE Trans. Industr. Inform. 30, 4 (2019), 2149–2159.Google Scholar
- [32] . 1989. Learning from delayed rewards. PhD Thesis. University of Cambridge, England.Google Scholar
- [33] . 2019. Dynamic pricing and matching in ride-hailing platforms. Nav. Res. Logist. 67, 8 (2019), 705–724.Google Scholar
- [34] . 2019. Multi-agent reinforcement learning for order-dispatching via order-vehicle distribution matching. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 2645–2653.Google ScholarDigital Library
Index Terms
- Deep Reinforcement Learning-based Trajectory Pricing on Ride-hailing Platforms
Recommendations
A Deep Reinforcement Learning Based Dynamic Pricing Algorithm in Ride-Hailing
Database Systems for Advanced ApplicationsAbstractOnline ride-hailing has become one of the most important transportation ways in the modern city. In the ride-hailing system, the vehicle supply and riding demand is different in different regions, and thus the passengers’ willingness to take a ...
Pricing in Ride-Sharing Platforms: A Queueing-Theoretic Approach
EC '15: Proceedings of the Sixteenth ACM Conference on Economics and ComputationWe study optimal pricing strategies for ride-sharing platforms, using a queueing-theoretic economic model. Analysis of pricing in such settings is complex: On one hand these platforms are two-sided - this requires economic models that capture the ...
A gradient-based reinforcement learning approach to dynamic pricing in partially-observable environments
As more companies are beginning to adopt the e-business model, it becomes easier for buyers to compare prices at multiple sellers and choose the one that charges the best price for the same item or service. As a result, the demand for the goods of a ...
Comments