Reinforcement learning approach for robustness analysis of complex networks with incomplete information

Network robustness against sequential attacks is significant for complex networks. However, it is generally assumed that complete information of complex networks is obtained and arbitrary nodes can be removed in previous researches. In this paper, a sequential attack in complex networks is modeled as a partial observable Markov decision process (POMDP). Then a reinforcement learning (RL) approach for POMDP is proposed to analyze dynamical robustness of complex networks under sequential attacks, when information of networks is incomplete. According to this approach, an agent can learn to take action by exploiting experiences. To solve the problem of large state space in complex networks, deep $Q$-network algorithm is used to identify most damaging sequential attacks, as deep neural networks can build up progressively abstract representations of state space of complex networks. The performances of proposed approach are analyzed on scale-free networks and small-world networks. According to the numerical simulations, it is found that the RL-based sequential attacks perform better when load distributions are more heterogeneous and local connections are more significant. Furthermore, it is shown that increasing the proportions of observed and attacked nodes improves the performance of RL-based sequential attacks. Finally, the results are verified on the IEEE 300-bus system and the simulation results highlight the damages caused by RL-based sequential attacks.