Reinforcement learning based predictive maintenance for a machine with multiple deteriorating yield levels

Abstract

This paper proposes a predictive maintenance methodology for a machine in manufacturing with deterior- ating quality states represented by multiple deteriorating yield levels. Imperfect minor maintenance and perfect major repair are considered. The underlying yield level cannot be directly obtained. Instead, product quality inspection information is used as the observed system state. The optimal maintenance policy associated with each possible observed system state is learnt by modeling the problem as hidden semi-Markov decision processes and solving it using policy iteration based Q-P learning. Then the future maintenance time can be estimated by re-simulating the system model using the learned maintenance policy. A set of experimental studies is conducted to testify the effectiveness of the proposed methodology and to investigate the impacts of involved system parameters.