Due to the impact of the global COVID-19, numerous industries have suffered from the disruption propagating along the supply chain, i.e. the ripple effect. To reduce adverse impact of the ripple effect, supply chain (SC) risk management under it is becoming an increasingly hot topic in both practice and research. In our former research, a robust dynamic bayesian network (DBN) approach has been developed for disruption risk assessment, whereas the solution methods adopted before (commercial solvers and simulated annealling algorithm) are not efficient enough, especially for large-size instances. For this reason, a new reinforcement learning variable neighborhood search (QVNS) is developed for solving the robust DBN optimization model, where the Q-learning algorithm is implemented to select the most efficient neighborhood structure in different stages of the search process. We conduct computational experiments on randomly generated instances, which indicates that Q-learning algorithm can improve significantly the performance of the VNS on large-size instances of the robust DBN optimization problem.