Signature-based detection uses known patterns to identify threats, such as file hashes, byte sequences, malicious domains, network protocol and port number indicators, etc. This approach efficiently detects known malware and attack patterns, with generally low false positive rates. Automated systems excel at signature-based detection, processing data quickly to identify matches.

The primary benefit of signature-based detection lies in its efficiency and accuracy for known threats. Signatures provide high-confidence indicators that an attack has occurred, enabling rapid automated responses such as quarantining files, blocking network connections, or alerting analysts. The rapid processing speed of signature matching allows organizations to handle large volumes of data across many potential threats.

The primary limitation of signature-based detection is its reactive nature. New malware variants, zero-day exploits, and novel attack techniques evade signature detection until researchers analyze samples and create matching patterns. Attackers who reverse-engineer detection signatures can modify their tools to avoid matches, rendering signature-based systems ineffective against sophisticated adversaries.

Behavioral detection monitors actions rather than static indicators, identifying threats by what they do rather than what they look like. This approach establishes baselines of normal activity and flags deviations that warrant investigation.

Behavioral detection excels at identifying threats that signature-based approaches miss:

Behavioral detection requires careful tuning to balance detection coverage with false-positive rates. Analysts should work with their teams to establish accurate baselines that reflect normal organizational operations.

Machine learning enhances behavioral detection by scoring events and correlating patterns at scale. Unsupervised learning algorithms establish baselines automatically, while supervised models classify events as malicious or benign based on training data. These capabilities prove particularly valuable for high-volume telemetry analysis and detecting subtle patterns across distributed systems.

Organizations implementing ML-based detection should understand the limitations of these systems, including model drift, concerns about training data quality, and evasion techniques.

Model drift occurs as attacker techniques evolve and organizational environments change, gradually degrading detection accuracy. A model trained on last year’s attack patterns may fail to recognize this year’s techniques, particularly as adversaries adapt their methods to evade detection. Environmental drift contributes to this problem: as organizations adopt new technologies, change business processes, or shift workforce patterns, the baseline of "normal" activity shifts.

Training data quality impacts detection accuracy and directly influences what ML-based detection can achieve. Models trained on incomplete datasets will fail to characterize attack techniques absent from training data. Further, training data contaminated with mislabeled examples teaches models to make incorrect classifications. Supervised models struggle to identify attack patterns they have never encountered.

In model evasion attacks, adversaries craft inputs designed to bypass ML classification. Attackers who understand how detection models work can modify their techniques to fall just below detection thresholds or exploit deficiencies in model training. Model evasion attacks have been applied to evade a wide range of classifiers, including malware detection, spam filtering, and image recognition systems.

Human-in-the-loop (HITL) approaches address these limitations by combining machine efficiency with human judgment. Rather than treating ML outputs as definitive determinations, effective detection programs use machine learning to surface high-priority signals for human review. Analysts bring contextual understanding that models lack: knowledge of recent organizational changes, awareness of ongoing projects that might explain unusual activity, and intuition developed through experience investigating similar alerts.

HITL workflows also create feedback loops that improve model performance over time, as analyst decisions on flagged events provide labeled data for model retraining. Where possible, organizations should design their detection workflows to leverage ML for initial triage and prioritization while preserving human decision-making authority for final classification and response actions. This approach allows human expertise to contribute to ongoing improvements in models and classifiers.

Effective detection programs combine multiple methodologies to leverage their complementary strengths. By integrating signature-based, behavioral, and machine learning techniques, organizations can leverage the strengths of each approach to build robust detection capabilities while minimizing false positives and false negatives. Many modern threat detection tools integrate signature matching, behavioral analysis, and machine learning within a single platform.

Detection logic increasingly maps attacker tactics, techniques, and procedures to frameworks like MITRE ATT&CK. ^[1] By focusing on adversary behaviors rather than specific indicators, organizations can create detection rules that remain effective even as attackers modify their tools. Further, the detection method, whether signature-based, ML-based, or behavioral, becomes less important than the underlying technique being detected. When combined, analysts can use multiple detection methods to identify threats, increasing overall detection confidence and reducing the likelihood of false positives.

This layered approach provides defense-in-depth for detection: signatures catch known threats efficiently, behavioral analysis identifies novel techniques, and machine learning surfaces subtle patterns that might escape rule-based detection.

Detection depends on the technical infrastructure deployed during preparation activities. In Prepare Activity, we examined the deployment and configuration of detection infrastructure in detail. This section focuses on how analysts use these capabilities for threat detection.

Endpoint Detection and Response (EDR) provides host-level visibility, which is essential for detecting post-compromise activity. EDR telemetry reveals process injection attempts, credential dumping activity, persistence mechanism deployment, and suspicious parent-child process relationships. Analysts can query EDR platforms to hunt for indicators across managed endpoints and correlate endpoint activity with network-based detections. While no single EDR solution provides perfect protection, these platforms generate valuable data for incident detection and investigation.

Where endpoint systems provide focused insight into individual hosts, network monitoring provides broad visibility across the environment. Network Detection and Response (NDR) and flow data enable threat detection by analyzing network traffic attributes and communication patterns. Detection use cases include identifying C2 beaconing through connection timing analysis, detecting lateral movement through unusual internal traffic patterns, and flagging potential data exfiltration through anomalous outbound data volumes.

Security Information and Event Management (SIEM) platforms allow analysts to perform simple searches and more complex correlation-based detection across diverse log sources. Detection rules can identify attack patterns spanning multiple systems, such as authentication failures followed by successful login, privilege escalation sequences, or coordinated access to sensitive resources. SIEM tools like Splunk, Elastic Security, and Microsoft Sentinel provide powerful search capabilities that analysts can leverage for both alerting and threat hunting.

Security Orchestration, Automation, and Response (SOAR) platforms reduce Mean Time to Respond (MTTR) and risk by automating response to high-confidence alerts. When detection systems identify clear threats, SOAR tools can immediately quarantine endpoints, block malicious IP addresses, disable compromised accounts, and capture data for forensic analysis. This automation reduces attacker dwell time while freeing analysts to focus on complex investigations requiring human judgment. Analysts can leverage SOAR case management capabilities to track detection events through investigation, link related alerts, and document findings for future reference.

This is not an exhaustive list of technical detection sources. Email gateways, web proxies, cloud-native logging, Cloud Workload Protection Platforms (CWPP), and other specialized monitoring tools all contribute valuable data for detection activities. Organizations should use the platforms that provide the best visibility for threat detection in their specific environments.

Human observations remain invaluable for detection, often identifying issues that automated systems miss. In Prepare Activity, we examined security awareness training programs that develop this detection capability, allowing analysts to use employee observations as a source of incident detection.

Employees frequently serve as the first line of defense for threats, reporting suspicious emails, unusual system behavior, or social engineering attempts. Effective training and a culture that encourages reporting suspicious events help staff recognize indicators of compromise and understand reporting procedures. Tying in pop culture references can enhance engagement, like the Dall-E-generated poster in Figure 5 encouraging employees to report suspicious activity.

System administrators and help desk personnel often identify anomalies during routine operations: a server that keeps crashing, unexpected network traffic patterns, or unauthorized changes to systems they manage. These observations, when properly escalated, can reveal ongoing compromises. Establishing clear escalation paths during preparation ensures these observations reach the incident response team promptly.

Third-party notifications from customers, partners, or security researchers sometimes provide the first indication of a breach. While not ideal from a detection timeline perspective, these external reports remain an important source of detection. Organizations should establish procedures for receiving and validating external notifications as part of preparation activities.

Cloud environments require detection approaches adapted to their unique characteristics: ephemeral workloads, API-driven infrastructure, shared responsibility models, and distributed architectures. Detection operations should use cloud-native logging, cloud provider APIs, and specialized detection tools designed for cloud workloads. Specialized detection solutions vary depending on the cloud deployment architecture: containerized, serverless, or hybrid environments.

Container and Kubernetes Detection

Container and Kubernetes detection employs multiple approaches to identify threats in containerized environments. Kernel-level monitoring using extended Berkeley Packet Filter (eBPF) enables systems to observe system calls from containerized workloads, identifying unexpected processes, file system modifications, or anomalous network connections. Container runtime socket monitoring provides an alternative detection method by observing the Docker or containerd API for container lifecycle events, image operations, and configuration changes that might indicate compromise or policy violations.

Logging systems, including Kubernetes audit events, provide visibility into API server activity, enabling detection of suspicious actions such as unauthorized pod creation, privilege escalation attempts, or access to sensitive secrets. Service mesh architectures offer additional network-level visibility between services, capturing traffic patterns that kernel-based or runtime monitoring might miss. For traffic entering or leaving the cluster, NDR tools at ingress points provide detection capabilities that complement mesh-level monitoring.

Serverless and Microservices Detection

Detection in agentless environments, such as serverless functions and microservices, requires alternative approaches, as traditional endpoint agents cannot run in these execution environments.

Cloud provider logs, function telemetry, and event anomaly analysis serve as the primary sources of detection for serverless and corresponding microservices workloads. Analysts should watch for abnormal invocation patterns, unexpected outbound connection attempts, unusual access to secrets or environment variables, and deviations from baseline execution characteristics (such as timeouts, throttling, and unusual memory and CPU consumption). The ephemeral nature of these environments makes real-time detection important since evidence exists only briefly.

Hybrid Cloud

Detection across distributed infrastructure, including hybrid and multi-cloud environments, requires unifying telemetry and detection logic across on-premises systems and multiple cloud providers. Organizations deploy a mix of agentless approaches, including API access and snapshot-based analysis alongside agent-based runtime monitoring for workloads that support it. Cloud-aware network detection and control plane analytics provide visibility into cross-environment activity that traditional tools may miss.

The volume and velocity of security data generated in modern environments can overwhelm analysis capabilities and delay threat identification. Larger networks can easily generate terabytes of log data daily from endpoints, network devices, cloud services, and applications, making comprehensive analysis difficult even with sophisticated tools and automation.

The quantity of data from multiple, disparate sources requires careful prioritization and filtering to ensure analysts focus on the most relevant signals. Even with SIEM platforms and machine learning models, the volume of alerts can exceed analyst capacity, leading to alert fatigue and missed detections. Data retention costs add to this challenge, forcing organizations to make difficult decisions about which data to keep for historical analysis and which to discard after short periods.

Organizations can address data volume challenges through several practical strategies:

The goal is not to collect less data, but to collect the useful data and process it efficiently so analysts can focus on signals that matter.

Encryption limits network-based detection visibility into communications content. As more communications use TLS encryption for web traffic, encrypted DNS, and VPN connections, network monitoring tools increasingly rely on metadata analysis rather than deep packet inspection. While encryption provides important privacy and security benefits, it also creates areas of reduced visibility for threat hunting, where attackers can hide command-and-control communications or data exfiltration activity.

Behavioral context analysis provides additional insight for analysts when working with encrypted traffic. This approach focuses on detecting unusual patterns indicative of EOIs in encrypted network activity. Indicators include abnormal outbound data volumes, connections to unusual destinations, and new encrypted channels from systems that do not normally generate such traffic.

Organizations should focus network detection efforts on connection metadata, including destination addresses, connection timing and duration, data volume patterns, and certificate information that remains visible even when content is encrypted. TLS inspection capabilities for internal systems where policy permits can provide a valuable source of detection data for threat hunting and subsequent investigation. Combined, these metadata and behavioral approaches allow organizations to maintain meaningful detection coverage even as encryption adoption continues to grow.

Sophisticated attackers actively work to evade detection, treating security tools as obstacles to overcome rather than insurmountable barriers. Understanding common evasion techniques helps analysts recognize when attackers are attempting to hide their activities.

Attackers frequently target logging infrastructure to remove evidence of their activities. Clearing Windows Event Logs, disabling audit policies, or timestomping files to obscure activity timelines are common techniques. Analysts should monitor for gaps in log data and unexpected changes to logging configurations as potential indicators of compromise.

Rather than deploying custom malware that signature-based detection might catch, attackers increasingly use legitimate system tools for malicious purposes. PowerShell, Windows Management Instrumentation (WMI), PsExec, and other built-in utilities provide powerful capabilities that attackers exploit. These tools generate activity that blends with normal administrative operations, making behavioral detection essential.

Attackers who understand detection thresholds can operate just below them. Data exfiltration spread across many small transfers rather than one large transfer, authentication attempts spaced to avoid lockout policies, and lateral movement timed to coincide with normal business hours all exploit threshold-based detection limitations.

As attackers adapt their techniques, detection teams cannot solely rely on endpoint signatures or static rules. Threat intelligence on emerging evasion techniques, regular updates to detection rules, and purple-team exercises that test detection coverage all contribute to maintaining effective detection capabilities.

The skills gap in security operations means many organizations struggle to effectively use their detection tools and interpret the alerts they generate. The shortage of experienced analysts who can tune detection systems, investigate alerts, perform threat hunting, and develop new detection rules limits detection effectiveness across the industry.

Entry-level analysts often lack the deep technical knowledge needed to distinguish sophisticated attacks from benign anomalies, while experienced analysts are in high demand and difficult to retain. Organizations should invest in training programs that develop detection skills within their teams, leverage managed security service providers (MSSPs) to supplement internal capabilities, and implement knowledge management systems that capture detection expertise for future reference.

Automation and AI-assisted detection capabilities can help bridge some of the skills gap by providing analysts with context and recommendations (see Accelerating Incident Response with AI). While valuable, these tools also require oversight from experienced personnel to avoid missing threats or creating excessive false positives.

External breach notifications are an important passive detection source through which security researchers, customers, partners, or other third parties discover and report compromises that internal detection systems missed.

Ethan, a help desk analyst, receives an email on Tuesday morning with the subject line "Security Issue - Exposed Administrative Systems." He skims the message from someone named Tom Liston describing exposed administrative portals and suspicious accounts on the organization’s patient management system. The email includes several screenshots of administrative login pages accessible without authentication, along with evidence of accounts named admin_backup and svc_temp that don’t follow the organization’s naming conventions. Ethan forwards the email to the security team with a note: "Not sure if this is legit, but seems serious if true."

Yoshihiro, the security analyst on duty, opens the forwarded email with skepticism. He’s seen plenty of false alarms from well-meaning but mistaken researchers, and occasionally encounters social engineering attempts disguised as security notifications. He starts reviewing the technical details and examining the screenshots of the exposed administrative portals. The URLs appear legitimate and point to actual organizational infrastructure. The suspicious account names admin_backup and svc_temp are concerning because they don’t match the standard adm_firstname.lastname format used by IT. The researcher even provides authentication logs showing logins from IP addresses in Eastern Europe over the past forty-five days.

The technical details seem credible, but Yoshihiro remains cautious until he checks the sender’s name: Tom Liston. He recognizes the name immediately as a respected threat hunter known for responsibly disclosing security issues to affected organizations. Yoshihiro has followed Liston’s work, which documents the discovery and reporting of compromised systems found through simple internet searches. This pedigree elevates Yoshihiro’s assessment from potential security issue requiring validation to credible breach notification requiring immediate action.

Yoshihiro escalates the report to the incident response team lead, noting both the technical findings and the credible source. This detection event transitions to the verify and triage phase, where the team will validate the specific claims and determine the appropriate response.

Active threat hunting using SIEM tools enables analysts to proactively identify threats that automated alerting systems might miss, particularly when searching for suspicious patterns rather than known malicious indicators.

Every Wednesday morning, Lucía conducts her weekly threat-hunting session, dedicating two hours to searching for anomalies in the organization’s security data. This week, she decides to hunt for beaconing behavior in proxy server logs, a common indicator of command and control communication where compromised systems regularly check in with attacker infrastructure. Beaconing is particularly suspicious because legitimate applications rarely communicate with external servers at perfectly consistent intervals, while malware often implements regular callback schedules to receive commands or exfiltrate data.

Lucía opens Kibana and crafts an Elasticsearch aggregation query to identify URLs with consistent access patterns over the past twenty-four hours, shown in Listing 5. Her query groups all proxy requests by destination URL and analyzes the distribution of requests over time, looking for patterns that suggest automated, scheduled communication rather than human-driven web browsing.

The query returns hundreds of URLs with varying access patterns. Lucía reviews the results, examining the time distribution histograms for each URL. Most show the irregular patterns typical of human access: bursts of activity during business hours, gaps during lunch, quiet periods overnight. Others exhibit consistent patterns from legitimate automated systems like software update checkers and monitoring tools, which Lucía recognizes from previous hunting sessions.

One URL immediately catches her attention: api.cloudserv-cdn.com shows exactly 288 requests over the past twenty-four hours, with the histogram displaying perfectly spaced intervals. Lucía calculates the timing: twenty-four hours divided by 288 requests equals exactly five minutes between each request. This precision in timing is unusual.

Lucía digs deeper into this suspicious URL, crafting a follow-up query to identify which client systems are accessing it and examining the specific timestamps. She finds that all requests originate from a single workstation in the finance department, and the timestamps confirm the pattern: requests occur at exactly :05, :10, and every five minutes thereafter with no variation in timing. The domain itself raises additional concerns when she checks threat intelligence feeds. It was registered only three weeks ago using privacy protection services, and the hosting provider is commonly associated with malicious infrastructure.

Lucía documents her findings, noting the suspicious beaconing pattern, the recently registered domain, the consistent five-minute interval, and the affected workstation details. She escalates this detection event to the incident response team for verification and triage, where they will investigate whether the workstation is genuinely compromised or if there’s a legitimate explanation for this highly regular communication pattern.