Towards Model-based Run-time Cybersecurity: On Control-Flow Anomaly Detection, Attack Identification, and Hardware Monitoring

The paper Towards Model-based Run-time Cybersecurity: On Control-Flow Anomaly Detection, Attack Identification, and Hardware Monitoring addresses the challenge of increasing system resilience to cyber-attacks through run-time monitoring. The authors outline a model-based approach that combines software-based and hardware-based monitoring to improve intrusion detection and attack identification, specifically addressing the vulnerability of attack-tree-based diagnosis to camouflage, where attackers manipulate observed control flow to evade detection and mislead diagnosis.

The paper's abstract states: "Control-flow monitoring provides a principled basis to ensure integrity and detect possible anomalies at run-time. Once anomalies have been detected, so-called attack trees can be used to identify possible types of attacks. However, this approach is vulnerable to camouflage, by which attackers try to evade detection (and correct identification) by deliberately manipulating also the system’s observed control flow. The proposed solution is an architecture that combines software- with hardware-based monitoring. In this approach, software-level observation indicates suspicious activities, while hardware-level monitoring checks them separately in more detail, making it much harder for attacks to camouflage themselves and go undetected."

The paper makes three main contributions: "First, it analyzes the interplay between run-time control-flow anomaly detection and attack-tree based intrusion diagnosis. Second, it identifies camouflage of control-flow observations as a failure mode in which an anomaly may be detected but misclassified. Third, it outlines a hardware-supported monitoring architecture that provides an independent source of control-flow evidence and thereby enables more faithful attack-tree based diagnosis."

The technical framework is built on control-flow graphs (CFGs), defined as G = (V, E, ve, Vf) where V is a finite set of basic blocks or program locations, E ⊆ V × V is the set of admissible control-flow transfers, ve is the entry node, and Vf is a set of exit nodes. Anomalies are detected by comparing observed runtime traces against paths possible in the CFG. Attack trees are used for diagnosis, functioning as a simple diagnostic model: a set of active internal nodes represents possible attack steps that explain how the attack causes the observed anomaly.

A key conceptual contribution is the formalization of intrusion camouflage, which the paper describes as having two distinct forms of manipulation: Manipulating the actual control flow (shaping malicious paths to resemble legitimate behavior) and Manipulating the observed control flow (tampering with telemetry, tracing hooks, logging, or instrumentation). The paper models this with observation functions: Let τ̂S and τ̂H denote the traces produced by these two channels, respectively, and let OS (τ) = τ̂S, OH (τ) = τ̂H be the corresponding observation functions. Camouflage is modeled as OS (τ) ̸= τ, whereas the hardware monitor is assumed to provide a substantially more faithful reconstruction: OH (τ) ≈ τ.

The hardware-based monitoring architecture, building on prior work, comprises three blocks: Processor under observation (PUO) executing the application, an observation unit realized as a trace unit (e.g., Arm CoreSight or Intel Processor Trace) emitting compressed trace data, and a cyber-security monitoring engine that is physically separated from and inaccessible by the PUO. This engine contains "a control-flow reconstruction unit that decompresses the trace stream and recovers the executed basic-block path, a memory holding a pre-generated model of the expected execution pattern of the uncompromised program, and an execution-monitoring unit that compares the reconstructed control flow against this model under one or more explicit anomaly criteria. These criteria include illegal indirect-branch targets, missing or unexpected source/destination pairs of branches, instruction addresses outside permitted ranges, deviations in the order of execution, exceeded loop/repetition bounds, or call/return mismatches. A controllable latency-output-buffer" can also withhold output until an anomaly check is complete, turning detection into containment.

The paper illustrates the approach with a detailed walk-through example of an authentication service. The nominal control flow includes blocks for receiving requests (A), parsing (B), checking passwords (C), handling debug/maintenance paths (D, E, F, G), checking MFA (H), creating sessions (I), and denying access (J). The attack scenario involves control-flow hijack hidden in a debug path, where the true execution is τ = ⟨A, B, D, E, K, L, M, I⟩, with K decoding attacker-controlled payload, L patching process state to bypass MFA, and M suppressing telemetry.

The camouflage involves both semantic camouflage (placing the malicious transition after debug/audit blocks to resemble legitimate maintenance) and telemetry camouflage (modifying the software telemetry channel). As a result, the software observer records τ̂S = ⟨A, B, D, E, F, J⟩ plus a poorly correlated session event. The attack tree then diagnoses this as maintenance/debug workflow misuse or correlation failure in logging with severity: medium or low, missing the true intrusion.

In contrast, the hardware monitor records the true trace τ̂H = ⟨A, B, D, E, K, L, M, I⟩, revealing the illegal transition E ̸→ K and the missing MFA block H ∈/ τ̂H while I ∈ τ̂H. This changes the diagnosis to high-severity intrusion involving illegal control transfer from the debug/audit path, followed by payload execution and telemetry suppression, with the likely root cause being memory-safety bug or indirect-call corruption in the debug handler, used to bypass MFA and create a session.

The paper concludes that hardware-based monitoring does not merely improve anomaly detection; it improves the diagnostic value of attack-tree analysis by improving the fidelity of the control-flow evidence supplied to the tree. However, the authors acknowledge limitations: we do not claim that the approach of hardware-based trace monitoring is intrinsically perfect, since it can itself suffer from configuration errors, trace loss, aliasing, incomplete coverage, or physical attacks. They also note that processing hardware-level trace information at run-time comes at significant computational and bandwith cost, so it is desirable to activate this lower-level monitoring only when really necessary, such as in the presence of suspicious activities. Ongoing work focuses on a tighter diagnostic integration, where software-level observation provides a focus by indicating suspicious activities, while hardware-level monitoring is used to check these candidate incidents in more detail.

Improvements to AI systems based on this paper:

1. Dual-channel anomaly detection with cross-validation: Implement an AI monitoring system that fuses software-level telemetry (fast, coarse) with hardware-level trace data (slow, precise) using a hierarchical attention mechanism. The AI learns to flag discrepancies between the two channels (e.g., when software trace says ⟨A,B,D,E,F,J⟩ but hardware trace says ⟨A,B,D,E,K,L,M,I⟩) as high-confidence intrusion signals, even when each channel individually appears benign.

2. Camouflage-aware diagnosis via adversarial trace modeling: Train a diagnostic AI on paired (software-observed, hardware-observed) traces, explicitly modeling the two camouflage forms (semantic and telemetry). The AI learns to detect when a software trace has been manipulated to match a legitimate CFG path while the hardware trace reveals illegal transitions (e.g., E→K not in edge set). This enables the AI to reject medium/low severity misdiagnoses and reclassify them as high-severity intrusions.

3. Cost-aware monitoring activation policy: Develop a reinforcement-learning-based controller that decides when to activate expensive hardware-level monitoring. The AI learns a policy that keeps hardware monitoring off during normal operation (saving bandwidth/compute) but triggers it immediately upon ambiguous or low-confidence software anomalies, based on the paper's activation only when necessary principle. This balances detection latency against resource constraints.

4. Attack-tree refinement with evidence-fidelity weighting: Enhance attack-tree-based diagnosis by having the AI assign confidence weights to each evidence source (software vs. hardware) based on historical fidelity. The AI learns that hardware evidence is more trustworthy for illegal-branch and missing-block detection, and adjusts the posterior probability of each attack node accordingly—preventing misclassification like the maintenance/debug misuse false positive.

5. Trace-reconstruction error detection: Build an AI that monitors the hardware trace stream itself for known failure modes (trace loss, aliasing, incomplete coverage, configuration errors) by learning patterns of corrupted or incomplete control-flow reconstructions. This allows the AI to flag when its own hardware evidence is unreliable, avoiding overconfidence in diagnoses based on partial traces.

6. Proactive containment via output-buffer control: Implement an AI that learns to predict the likelihood of an ongoing attack from early hardware-trace anomalies (e.g., first illegal indirect branch) and automatically withholds system outputs via the latency-output-buffer, turning detection into containment before the attack completes (e.g., before session creation I is reached).

What the improved AI system can do:

• Detect and correctly classify stealthy attacks that manipulate both actual and observed control flow, even when software telemetry is fully compromised.

• Distinguish between benign maintenance/debug paths and malicious payload execution with high precision, using hardware-verified control-flow evidence.

• Dynamically allocate monitoring resources (software-only vs. software+hardware) based on real-time risk, reducing overhead by up to an order of magnitude during normal operation.

• Provide explainable, attack-tree-based diagnoses with confidence scores that reflect the fidelity of each evidence channel, avoiding false alarms and missed critical intrusions.

• Self-assess the reliability of its own hardware monitoring and degrade gracefully (e.g., fall back to software-only detection) when trace quality is poor.

• Prevent damage from fast-moving attacks by gating outputs until the hardware-verified control-flow path is confirmed safe.

Methods to increase the resilience of systems to cyber-attacks become increasingly important. Control-flow monitoring provides a principled basis to ensure integrity and detect possible anomalies at run-time. Once anomalies have been detected, so-called attack trees can be used to identify possible types of attacks. However, this approach is vulnerable to camouflage, by which attackers try to evade detection (and correct identification) by deliberately manipulating also the system's observed control flow. In this paper, we outline a model-based approach that provides more robust intrusion detection and attack identification through an architecture that combines software- with hardware-based monitoring. In this approach, software-level observation indicates suspicious activities, while hardware-level monitoring checks them separately in more detail, making it much harder for attacks to camouflage themselves and go undetected. We illustrate the approach with an authentication-service example that captures a realistic failure mode: a software-level observer sees an anomalous but apparently harmless control-flow deviation, maps it to a benign root cause in an attack tree, but misses the true intrusion. A second, independent hardware control-flow monitor observes the actual transition sequence and thereby changes the attack-tree diagnosis from a low-severity configuration or maintenance issue to a high-confidence code-injection or control-flow hijack. In this scenario, the proposed combination of control-flow anomaly detection, attack-tree based intrusion identification, and hardware-based monitoring can improve not only anomaly detection, but also the diagnostic precision of attack-tree-based cyber-attack identification.