Machine learning frameworks for urban logistics optimisation problems

Abstract

Vehicle Routing Problem(VRP), a challenging topic in Urban Logistics Optimization, is a combinatorial optimization problem with many exact and heuristic algorithm. VRP has many variants, for example, VRPTW describes a classic VRP with time window constraint. In this project, an end-to-end reinforcement learning(RL) framework is proposed to solve Vehicle Routing Problem with Time Window(VRPTW), which is an attempt to improve an existing RL framework for VRP. Applying Proximal Policy Optimization(PPO) and Random Network Distillation(RND), we attempt to improve the performance of proposed model. By observing the reward signals, a single policy model is trained to figure out the near-optimal solutions; PPO is applied to improve the policy gradient algorithm and optimize the parameters of a parameterized stochastic policy; RND introduces an exploration bonus into RL model to improve on hard exploration task. Instead of retraining for every instance, the proposed approach is able to generate a solution right away for any VRPTW instance with the same customer node(location, time window), vehicle capacity, and demand distributions as those for training. And a reasonable improvement on the performance can be seen due to the application of PPO and RND. Furthermore, the proposed framework has the potential to be improved for more complicated Urban Logistics Optimization Problems.