Trident: Improving Malware Detection with LLMs and Behavioral Features

arXiv:2605.00297 · cs.CR, cs.LG · Submitted 2026-08-24 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Next we'll be talking about the paper "Trident: Improving Malware Detection with LLMs and Behavioral Features".

Jane: The paper was written by the authors from.

Tom: Stay tuned as we take you through the paper and discuss its implications.

Jane: We also have Lu with us today — senior AI researcher at Tsinghua.

Tom: We also have Meng with us today — lead engineer at a mysterious AI startup.

Jane: We also have Lalam with us today — the in-house Large Language Model.

Tom: Alright, let's get started.

Summary: Tom: We've got "Trident: Improving Malware Detection with LLMs and Behavioral Features," so let's talk about what the authors actually summarize regarding their methodology, because this is where the core of the innovation lies.

Jane: The paper summarizes that it starts by taking those massive sandbox behavior reports—the JSON files showing everything a sample does—and then feeding them into a frontier LLM like Gemini-three-Pro-Preview.

Tom: It's essentially using the LLM to read this huge, semi-structured report and deduce what malicious activity is happening, which is quite an advanced way of processing data.

Lu: The LLM acts as a highly sophisticated feature extractor here, identifying patterns that go far beyond simple counting or aggregation of features.

Jane: Exactly; it's not just that the LLM sees 'file written,' but it identifies *which* files and *why*, giving us a clear understanding of the context.

Meng: My take is that this process allows the us to automate what would otherwise be a lengthy, manual analysis by turning those observed behaviors into defined, executable rules.

Tom: It’s not just flagging an action; it's mapping out the whole sequence of events, which elevates the detection capability tremendously.

Lu: The paper highlights how this process helps us disambiguate benign software that might accidentally perform a few suspicious actions from malicious intent.

Jane: I like thinking of it as context—if an action alone is normal, but five actions happen in sequence that only make sense if you're trying to exfiltrate data, the LLM can spot that narrative.

Meng: From my side, the summary implies a much more robust feature engineering process upfront because we are leveraging the LLM to build a cohesive profile of the attack flow.

Tom: It means we don't have to rely on just one type of telemetry; we can use everything—network, memory, disk—and let the LLM knit it all together into actionable rules.

Lu: The paper shows how this approach helps us capture malicious behaviors that are shared across different malware families, making the resulting rules very generalizable.

Jane: That's a huge step toward consistency in how we analyze these different types of threats.

Meng: It means we’ can start thinking about the operational benefits of using LLM-generated rules as a viable defense mechanism instead of just using them for report writing.

Lalam: And that ability to synthesize disparate pieces of evidence into a single, actionable understanding is exactly how we move toward truly intelligent, holistic security platforms.

Improvements: Tom: We’ve seen the methodology; now let's talk about the actual improvements this paper suggests, because "Trident: Improving Malware Detection with LLMs and Behavioral Features" isn't just a model—it promises a new way of operating in cybersecurity.

Jane: The most significant improvement, as far as I can tell, is that it handles zero-day threats; since the rules are based on suspicious *behavior* rather than known signatures, it should catch things never seen before.

Lu: Exactly! It moves the detection paradigm from reactive—waiting for a sample to be cataloged—to proactive, identifying deviations from expected system normalcy by using those behavior clusters.

Meng: But does this mean we need to completely overhaul our existing security stacks to leverage these improvements? That's the operational hurdle I see right away when moving from theory to practical implementation.

Jane: The paper suggests that the LLM acts as an intelligent overlay, meaning it can potentially integrate with existing feature extraction systems without requiring a total replacement of all legacy tools.

Tom: It’s not just about catching more malware; it’s about catching it in a way that makes sense for the analysts who have to manage these systems daily.

Lu: I think that's because the behavioral rules are designed to be far more robust against concept drift, which is what plagues traditional static ML approaches over time.

Meng: From an engineering standpoint, having reduced maintenance costs due to concept drift management is a massive operational win that cannot be overstated for deployment at scale.

Jane: The paper explains that by focusing on core malicious techniques, the system maintains performance even as the specific tools or variants change.

Tom: This stability is what impresses me; it's not just about being good today, but ensuring longevity in how we maintain high detection rates.

Lu: I think that robustness is a direct result of the LLM identifying core malicious techniques that are slow to evolve, regardless of how specific the tools become.

Meng: This allows us to build deployment pipelines that can rely on consistent performance rather than needing constant retraining cycles just to stay current with threats.

Jane: The paper provides a framework where we can have confidence in the system’s output without having to manage a massive, constantly shifting training dataset.

Tom: It's not just about the performance metrics, but about building a reliable structure that is designed for longevity and consistency in operation.

Lu: I'm particularly excited about how this opens doors for integrating dynamic and static analysis in a way that was previously considered too complex to combine successfully at scale.

Meng: We need to figure out the infrastructure requirements to handle this combined voting system efficiently, but the theoretical gains are undeniable for practical deployment.

Jane: The paper shows that we can achieve very low false positive rates while using a behavioral approach, which is a huge step forward in reliability.

Lalam: And from my perspective, it’s about creating an operational culture where we trust the automated system because its clear and consistent performance allows us to manage security with confidence.

Paper discussion segment 3: Tom: So we’ve seen how "Trident: Improving Malware Detection with LLMs and Behavioral Features" pulls together static data, behavioral rules, and LLM analysis, but what does all that actually mean for the cybersecurity world?

Jane: It means we are fundamentally shifting our perspective from looking for specific attack signatures to understanding the malicious narrative of an attack.

Lu: That narrative concept is huge because it allows us to generalize much better than simple pattern matching; we’re capturing intent, not just isolated actions.

Meng: From a practical standpoint, this means security teams won't need to constantly scramble and retrain complex models when malware evolves—the system just keeps pace with the threat landscape.

Lalam: The most impactful vision here is that it allows us to move toward a truly proactive security culture, where we don't just react to threats but anticipate them based on consistent malicious behavior patterns.

Tom: That stability is what impresses me; the GBDT models often degrade over time, but this approach seems built for longevity and sustained high performance.

Jane: It’s because of those behavioral rules, which are designed to be far more robust against concept drift than standard ML features allow them to be.

Lu: I think that robustness is a direct result of the the LLM identifying core malicious techniques that haven' over time, even if the specific tools used change.

Meng: That’s a massive engineering win because maintaining constant high performance without manual retraining saves immense operational overhead and time for anyone managing the system.

Lalam: And because it gives us clear explanations—the rules or the LLM's reasoning—it drastically improves our ability to trust and audit security decisions, which is a huge cultural leap forward in transparency.

Tom: It’s not just about catching more malware; it’s about catching it in a way that makes sense for the analysts who have to manage these systems daily.

Jane: Exactly, so we are moving from a black box of classifications to an auditable explanation of what’s going on inside the file.

Lu: I'm particularly excited about how this opens doors for integrating dynamic and static analysis in a way that was previously considered too complex to combine successfully at scale.

Meng: We need to figure out how to make sure our infrastructure can handle this combined voting system efficiently, but the theoretical gains are undeniable for real-world deployment.

Lalam: This allows us to build a future where security is not just reactive, but truly predictive and understandable for everyone involved in the decision-making process.

Conclusion: Tom: We’ve spent a lot of time exploring how "Trident: Improving Malware Detection with LLMs and Behavioral Features" tackles this complex problem, so let's wrap up our discussion on its overall significance.

Jane: It really boils down to the fact that it successfully combines the strengths of different methods, creating a system that is both highly effective and remarkably stable over time in operation.

Lu: I think the sheer versatility of using LLMs—seeing how they process unstructured behavior reports—opens up possibilities for future AI applications across so many domains.

Meng: The practical implication is that we are moving toward maintenance-free, highly reliable security systems that operational efficiency is a major win.

Lalam: My final thought is that this allows us to foster a more trustworthy digital environment where automated security tools can be clearly understood and trusted by the public as well.

Tom: Trustworthy systems, indeed; it’s not just about the performance metrics but the long-term reliability of the entire infrastructure.

Jane: I agree, and we should definitely highlight that because of how it's designed to handle concept drift, which is such a persistent headache for most other security tools.

Lu: The way they manage concept drift by focusing on core malicious techniques is something that could inspire other researchers in completely different fields, too.

Meng: I hope we can start seeing this in production soon, making the transition from "academic success" to real-world deployment is the next critical step for us.

Lalam: We should all be optimistic about the future, knowing that tools like Trident enable a much more robust and transparent digital culture for everyone involved.

Tom: It seems like a huge leap forward in malware detection; it’s not just another incremental improvement, it's a fundamental shift in how we approach security analysis.

Jane: And I think that is the most important thing to take away from "Trident: Improving Malware Detection with LLMs and Behavioral Features" before we move on to our next topic.

cs.CR, cs.LG

Submitted: 2026-08-24

Updated: 2026-08-25

Code: https://github.com/brightertiger/pygarble

Importance score: 92/100

The gist: * Malware detection traditionally relies on static features—such as "byte histograms, string information, and PE header contents." However, this approach faces several challenges: static signatures

Key concepts

Large Language Model (LLM) as a Feature Extractor
The LLM reads large, semi-structured sandbox behavior reports (JSON files). It goes beyond simple counting or aggregation of features. It identifies the context—such as identifying *which* files are affected and *why*—to deduce complex malicious activity.
Malicious Narrative / Behavioral Rules
The system maps out the entire sequence of events, not just isolated actions. This allows it to capture malicious intent, such as a series of actions designed to exfiltrate data. These rules are highly generalizable across different malware families.
Concept Drift
This is the degradation of traditional machine learning models over time due to changes in the threat landscape. Trident's behavioral rules are designed to be robust against this drift, ensuring consistent detection performance even as specific tools or malware variants change.

Terminology

Summary

Malware detection traditionally relies on static features—such as byte histograms, string information, and PE header contents. However, this approach faces several challenges: static signatures are easy to evade, machine learning models relying on these features are brittle and easily manipulated by adversaries, and the entire field suffers from vulnerability to concept drift. Furthermore, traditional ML approaches struggle with dynamic features because of limitations in datasets or the inherent constraints of neural models regarding vocabulary size and context windows.

The paper proposes a novel approach that utilizes Large Language Models (LLMs) to overcome these limitations. The methodology involves two phases:

Phase I (Initialization):

Researchers utilize a frontier LLM, specifically Google’s Gemini-3-Pro-Preview [17] large language model with high reasoning, to analyze behavior reports generated from running malware samples in a sandbox. The LLM performs several critical tasks:

  1. Identify Malicious Behaviors: It analyzes the semi-structured JSON behavior report, identifying actions highly indicative of malicious activity.

  2. ** Craft Detection Rules:** It formalizes these identified behaviors into executable detection rules using the JQ language [14]. The goal is to create rules that are general enough to catch this malware sample and similar variants, but specific enough to avoid false positives on benign software.

Validation and Refinement:

Initial testing of LLM classification showed high recall (96.2%) but an unacceptable false positive rate (8.8%). To address this, the paper introduces a verification procedure:

  • Good Rule Criteria: Rules are kept only if they meet stringent quality standards, including FPR = 0 on the validation set and matching the sample they were generated from.

  • De-duplication: The resulting rules are clustered using HDBScan clustering [32] to manage redundancy, allowing researchers to count the number of rule clusters rather than individual rules.

Phase II (Deployment):

The system classifies unknown samples based on these behavioral rules and the LLM's analysis.

The classification process is governed by a threshold (tau):

  1. If no rules match, the sample is labeled as benign.

  2. If more than tau rule cluster(s) match a sample, label it as malicious.

  3. Otherwise, ask the LLM to label the sample based on the behavior report.

The paper finds that these behavioral rules are significantly more robust to concept drift than static methods. This is because core malicious techniques are slower to evolve, meaning the rules can detect new malware families even if they were not trained on any samples from the 'fynloski' family. The results show that detection from behavior-based rules generalizes better to unseen malware families than static analysis.

The ultimate contribution is Trident, a system that combines three distinct methods into a final decision-making process:

  1. Static Features: A classic decision tree model (GBDT).

  2. Behavioral Rules: The LLM-generated rules (with tau=6 clusters).

  3. Direct LLM Analysis: The raw verdict from the LLM.

Trident employs a majority voting system, with a specific tiebreak algorithm (Algorithm 1) used when the rules are uncertain and the GBDT/LLM models disagree.

Performance and Conclusion:

Trident outperforms its components: Trident outperforms standard methods using static features, outperforms behavior-based rules alone, and is highly stable. The system achieves a significantly lower false positive rate (FPR) than the LLM alone: When using the rules alone, our false positive rate never exceeds 3.5%... With strategic deferral, we are able to reduce the FPR to 1% with threshold tau=1 and 0.5% with threshold tau=6.

Furthermore, Trident achieves resilience to concept drift without requiring costly retraining: Trident is as resilient to concept drift as active learning methods without requiring retraining, providing superior performance and interpretable results without the cost of frequent retraining. Every malicious label is supported by either a set of rules or an LLM explanation, offering high interpretability.

Improvements for AI systems

Based on a thorough analysis of this research paper (Trident), I have identified several critical architectural and methodological improvements that can be applied to enhance existing AI systems in cybersecurity, specifically for malware detection.

The following improvements focus on integrating unstructured dynamic data into verifiable, interpretable detection pipelines while mitigating the inherent weaknesses of traditional static classifiers.

The core improvement is shifting from relying solely on a static feature vector to implementing a multi-stage, hybrid decision framework that fuses three distinct data sources: Static Features, LLM-Generated Behavioral Rules, and Direct LLM Reasoning.

The Improvement: Instead of relying on human analysts to manually define threat signatures or relying solely on a complex neural network to learn patterns, we implement a structured pipeline where Large Language Models (LLMs) act as automated security rule compilers.

  • Mechanism: A specialized LLM is prompted (using the provided prompt structure) to transform unstructured JSON behavior reports (from sandboxes) into highly specific, executable detection rules written in a filtering language like JQ.

  • Validation Layer: Crucial for stability, these generated rules must pass through an automated validation step. Any rule that triggers a high False Positive Rate (FPR > 0%) on a pre-defined set of benign samples is discarded.

  • Resulting Capability: The system can identify and formalize novel malicious behaviors (e.g., AppInit DLL persistence or Registry key deletion for proxy bypass) into quantifiable, executable patterns instantly, achieving a low FPR (approaching 0.2%) without human intervention.

The Improvement: Moving away from brittle static feature reliance to leveraging dynamic behaviors to ensure long-term detection efficacy.

  • Mechanism: The system utilizes Rule Clustering (e.g., HDBScan) on the generated ruleset. Instead of counting individual rule hits, we cluster rules by their semantic similarity (using character and token n-grams). A sample is flagged if it matches a specific number of these clusters (tau).

  • Resulting Capability: This approach allows the system to detect new, unseen malware families that share core malicious actions with previously seen families—a phenomenon static classifiers fail at. The system maintains stable recall over time without requiring costly, monthly retraining cycles inherent to traditional Active Learning methods.

The Improvement: Replacing a single, monolithic classifier with a Majority Voting/Tiebreak Mechanism that handles classification uncertainty across three distinct models.

  • Mechanism:

  1. Static Verdict (V GBDT): The traditional GBDT classifier (using static features) provides the initial prediction.

  2. Behavioral Verdict (V Rules): The rule-based system determines if a sample matches tau clusters (Malicious), no rules match (Benign), or falls into an uncertain middle ground (about LLM Deferral).

  3. LLM Verdict (V LLM): The LLM provides a final, high-level contextual verdict.

  • Tiebreak Logic: A dedicated algorithm (Algorithm 1) resolves conflicts: if V Rules is uncertain, the decision is deferred to the LLM; if both static and dynamic conflict, the tiebreaker uses confidence levels (e.g., requiring high GBDT probability for a Benign label to be trusted).

  • Resulting Capability: The system achieves significantly higher accuracy and reliability than any single component. It handles evasion attempts (when behavior is sparse) by falling back on the static features, and handles novel behaviors that evade static learning by leveraging dynamic rules.

The Improvement: Transforming black box classification decisions into transparent, actionable insights.

  • Mechanism: Every Malicious verdict is inherently supported by a traceable path: either the specific rule cluster matched, or the LLM's detailed explanation of key behavioral indicators. For some Benign classifications, the LLM provides an affirmative explanation of why it is benign (e.g., recognizing an installer based on its printed output).

  • Resulting Capability: Security analysts can audit and validate every Malicious label against the exact evidence (the matched rules or the LLM's reasoning), drastically improving trust and accelerating threat investigation, unlike traditional ML models where features are often correlated but not directly explained.

Feature Traditional Static ML System Improved Behavioral Fusion Engine

:---:---:---

Data Handling Byte histograms, fixed features. (Static) Unstructured JSON Behavior Reports. (Dynamic)

New Malware Detection Poor; requires explicit training on seen families. (Brittle) Excellent; detects shared malicious behaviors across previously unseen families via rule clustering. (Robust)

Concept Drift Requires costly, monthly retraining/Active Learning. Stable and self-corrective through behavioral generalization without retraining. (Efficient) Automated Rule Generation Manual signature creation or complex feature engineering. LLM-driven JQ Code Generation. (Scalable)

Explainability Low; relies on feature weights/importance. (Black Box) High; provides specific rules and LLM justifications for every verdict. (Transparent)

Sources

Related papers