Q-Learning Adaptive Security Framework using Digital Twin Artificial Intelligence for Self-Healing Network Defence

Abstract

Contemporary enterprise and critical-infrastructure networks face a widening adversarial gap between increasingly sophisticated multi-vector cyber threats and the static, rule-based intrusion detection and prevention systems deployed to counter them. Static systems cannot adapt their defensive posture in real time, leaving networks exposed during the interval between attack onset and human-analyst response. This paper proposes DT-AISHN (Digital Twin Arti-ficial Intelligence for Self-Healing Network Defence), an integrated framework combining three complementary components: (1) a Random Forest attack-risk model (n = 150 trees, max depth = 18) trained on the NSL-KDD benchmark to assign per-flow attack probability P (attack|x), achieving 97.12% accuracy and 99.23% AUROC; (2) a parametric Digital Twin of an eight-node, eleven-link enterprise network (NetworkX DiGraph with per-link latency, capacity, and routing-cost attributes) that mirrors real-time network state and simulates the effect of defensive actions before execution; and (3) a tabular Q-learning Adap-tive Defense Controller (α = 0.15, γ = 0.95, ε: 1.0=0.05) optimising a five-action policy—do-nothing, reroute, throttle, firewall-block, and segment-isolation—over a discretised (attack probability, load, service risk, route, previous action) state space. Evaluated across 24 training episodes on 20,000 NSL-KDD flows and compared against two competitive baselines (traditional IDS and SDN rule-based defence), DT-AISHN achieves a cumulative reward of −4,123 versus 8,956 (SDN) and −12,234 (IDS), with a 75.6% attack block rate, 26% latency reduction, 47% packet-loss reduction, and 48% attack-surface reduction relative to the IDS baseline. These results demonstrate that digital twin-guided, learning-based adaptive defence substantially outperforms both static and rule-based approaches, establishing DT-AISHN as a practical architecture for autonomous network self-healing.