Threat Hunting: How to Stop Collecting Logs and Start Catching Hackers

The average breakout time (how long it takes attackers to move from initial compromise to spreading across the network, known as lateral movement) is often under two hours. Some adversaries do it in minutes.

If your SOC (Security Operations Center) depends on analysts manually reviewing alerts from static rules, you've lost the race before you even get out of your chair. We need to build a system that makes life hard for the attacker, not for the analyst.

Here is an expanded architecture for proactive threat hunting.

Phase 1: Data Hygiene (The Work Nobody Wants to Do but Everyone Has To)

Most SIEM projects don't fail for lack of expensive tools. They fail because the data is a mess. Picture trying to track down a “suspicious man” when one witness describes him in French, another in Japanese, and a third gives you only his shoe size.

1. Normalization Is Your Esperanto

I mentioned ECS (Elastic Common Schema) and OCSF. Let's go further. Normalization isn't just about mapping fields. It's about standardizing values.

• Example: Is a login event “Logon,” “Login,” “SessionStart” or event ID 4624? Your SIEM has to reduce all of these to a single value: event.action: "authentication_success".
• Why it matters: It lets an analyst write one query that covers Windows, Linux, the cloud and a SQL database. Without it, threat hunting becomes a fight with semantics instead of a fight with the attacker.

2. Enrichment With Business Context

Technical context (GeoIP) is the bare minimum. The real magic happens when you add business context.

• Your logs should know that the user j.kowalski isn't just a “User” but a “Domain Administrator” or the “CFO.”
• The system has to know that server SRV-005 processes payment card data (PCI DSS).
• The rule: An alert on a test server is “informational.” The same alert on a domain controller means “the house is on fire.” Your SIEM has to know the difference before it wakes an analyst up in the middle of the night.

Phase 2: Detection Strategy (Going Beyond Signatures)

This is where David Bianco's Pyramid of Pain comes in.

Most systems focus on the bottom of the pyramid: file hashes, IP addresses and domains. These are easy to detect, but just as easy for an attacker to change. They switch IPs, and your rule is useless.

A modern SIEM has to aim for the top of the pyramid: TTPs (Tactics, Techniques, and Procedures).

1. Behavioral Detection (UEBA)

Stop looking for "bad files." Start looking for "strange behavior."

• Don't ask: “Does this file contain a virus?”
• Ask: “Why is winword.exe (Word) trying to launch powershell.exe and connect to the internet?”
• This approach catches zero-day attacks that don't have signatures yet. Word launching PowerShell is always suspicious, no matter which exploit was used.

2. Hunting Hypotheses

Threat hunting isn't about staring at a screen and waiting for inspiration. It's a scientific process.

• Hypothesis: “I think attackers may be using Pass-the-Hash to hop between servers.”
• Experiment: “I'm searching the logs for Windows event 4624 with logon type 3 and NTLM authentication, where the source machine isn't an admin workstation.”
• Conclusion: Confirmed (an incident) or refuted (all clear).

Phase 3: Operationalization (Detection as Code)

If your detection rules exist only in your SIEM's point-and-click interface, you're a hostage to that vendor.

CI/CD for Security

Treat detection rules (for example, YAML files in Sigma format) the same way you treat application code:

1. Commit: An analyst writes a new rule to detect Log4Shell.
2. Test: An automated pipeline checks that the rule's syntax is valid and that it doesn't fire a billion alerts against historical data.
3. Deploy: The rule is pushed to production automatically.

This eliminates “configuration drift” and the kind of situation where someone accidentally disabled a critical alert three months ago and nobody noticed.

Phase 4: Tiered Architecture (Hot, Warm, Cold)

We have to balance greed (we want all the data) against economics (we don't have NASA's budget). Let's flesh out the storage model:

• Hot (NVMe/SSD): The last 7–14 days. This is where 90% of analyst queries happen. Responses have to be instant.
• Warm (HDD): Up to 3–6 months. The data is available, but a query may take a few minutes. Ideal for trend analysis and hunting “low and slow” threats (stealthy APT attacks).
• Cold/Frozen (S3/Tape): Years. The data is compressed and dirt cheap to store. You only pull it out when a prosecutor or auditor comes knocking. Restoring it (known as rehydration) takes hours but costs pennies.

From the Field: Detection as Code in Practice

The theory sounds good, but what does it look like at the keyboard? Instead of clicking around a GUI, we write a rule. Here's a simple Sigma rule that detects Word launching PowerShell (a classic Initial Access pattern):

title: Suspicious PowerShell Child Process
id: 1234-5678-90ab-cdef
status: experimental
description: Detects PowerShell being spawned by Microsoft Word
author: SecurHub
logsource:
    category: process_creation
    product: windows
detection:
    selection:
        ParentImage|endswith:
            - '\winword.exe'
        Image|endswith:
            - '\powershell.exe'
    condition: selection
level: high
tags:
    - attack.execution
    - attack.t1059.001

The file goes into a Git repository. A CI/CD pipeline converts it into a query your SIEM understands (for example, Elastic DSL or Splunk SPL) and deploys it. No manual clicking at all.

What Should You Build It With? (The Open Source Stack)

You don't need a bank's budget to get started. You can build a powerful SOC and pay only for electricity and engineering time. Here's the modern threat hunter's "starter pack":

1. Wazuh: The heart of the operation. It works as both an EDR (an agent on endpoints) and a SIEM. It collects logs, monitors file integrity and detects vulnerabilities.
2. TheHive: The brain of the operation. An incident response and case management platform. Alerts land here, and analysts document their investigations here.
3. MISP: The memory of the operation. A threat intelligence platform where you store information about malicious IPs, domains and hashes and share it with the community.
4. Cortex: The hands of the operation. An analysis engine that, at TheHive's request, can scan a suspicious file on VirusTotal or check an IP against a reputation database.

The Bottom Line: People > Tools

Finally, the most important element, and one you won't find in any configuration file: mindset.

The best SIEM is one that takes repetitive, tedious work off people's plates (automation, filtering out false positives) so they can do what AI is still bad at: connecting the dots, thinking creatively and understanding context that isn't in the logs.

Don't build a data museum. Build a command center.

Aleksander

Sources:

1. Elastic Common Schema (ECS) Reference
2. Open Cybersecurity Schema Framework (OCSF)
3. Sigma - Generic Signature Format for SIEM Systems
4. MITRE ATT&CK Framework
5. The Pyramid of Pain - David Bianco
6. SANS Institute - Hunting Maturity Model

FAQ

How is threat hunting different from regular SOC monitoring?

A traditional SOC is reactive: it responds to alerts generated by security systems. Threat hunting is proactive. You assume your defenses have already failed and actively search for signs of an intruder, working from hypotheses rather than ready-made signatures.

Do I need an expensive SIEM to start threat hunting?

No. A SIEM makes it easier to centralize data, but you can start threat hunting by analyzing endpoint logs (for example, Sysmon or EDR) or with simple open-source tools. What matters most is the analyst's knowledge and a clear understanding of what you're looking for (TTPs), not the tool.

What is the "Pyramid of Pain" in cybersecurity?

It's a model that shows how different types of indicators of compromise (IoCs) relate to how much pain detecting them causes the attacker. At the bottom are hashes and IP addresses, which are easy for an attacker to change. At the top are TTPs (tactics, techniques and procedures). Detecting and blocking those forces the attacker to replace their entire toolkit, which is "painful" and expensive for them.

How can I automate threat hunting?

By adopting a "Detection as Code" approach. When a hunting hypothesis pays off (you find a threat or confirm that a behavior is suspicious), turn it into an automated detection rule (for example, in Sigma format) and deploy it to your SIEM so it can catch that threat on its own in the future.

Why is data normalization so important?

Without normalization (for example, to the ECS or OCSF standard), an analyst has to write a separate query for every system: Windows, Linux, the firewall. Standardized field names let you search the entire environment with a single query, which dramatically speeds up anomaly detection.