Blog on Catscrdl

Blog on Catscrdl https://catscrdl.io/blog/ Recent content in Blog on Catscrdl Hugo en Fri, 07 Aug 2026 00:00:00 +0000 Stop Making Your Detection Engineers Triage Alerts https://catscrdl.io/blog/whotriagesthealerts/ Fri, 07 Aug 2026 00:00:00 +0000 https://catscrdl.io/blog/whotriagesthealerts/ <h1 id="intro">Intro</h1> <p>There was a question I saw the other day asking if detection engineers (DE), the folks who author alerts, should also be primarily responsible for their triage. There was large support for having the detection authors also play a large role in initial triage, often as part of the oncall. I largely disagree with this stance and am going to use some time Friday to quickly argue why you’re likely compensating for technology and process failures if you do that.</p> Container Escape Telemetry, Part 1: Isolation Primitives and the eBPF Observability Model https://catscrdl.io/blog/containerescapetelemetry/isolation/ Tue, 31 Mar 2026 00:00:00 +0000 https://catscrdl.io/blog/containerescapetelemetry/isolation/ <p>This is Part 1 of the container escape telemetry series. The <a href="../intro">series overview</a> has the project goals, key findings, and reading guide. If you’re already comfortable with namespaces, cgroups, capabilities, and how eBPF ring buffers work, skip ahead to <a href="../methodology">Part 2: Methodology and Tool Architecture</a>.</p> <p>This post is the primer. It covers what containers actually are at the kernel level, what “escape” means in concrete terms, and how eBPF-based tools get telemetry out of the kernel. That last part matters more than you might think, because the mechanism that delivers telemetry to your security tool has its own failure modes, and an attacker who understands those failure modes can exploit them.</p> Container Escape Telemetry, Part 2: Methodology and Tool Architecture https://catscrdl.io/blog/containerescapetelemetry/methodology/ Tue, 31 Mar 2026 00:00:00 +0000 https://catscrdl.io/blog/containerescapetelemetry/methodology/ <p>This is Part 2 of the container escape telemetry series (<a href="../intro">overview</a>). <a href="../isolation">Part 1</a> covered the isolation primitives and eBPF observability model. This post covers the lab architecture, the three tools under test, the 15 escape scenarios, and the detection coverage matrix. If you want to skip straight to what the telemetry actually looks like, jump to <a href="../deepdives">Part 3: Per-Scenario Deep Dives</a>.</p> <h1 id="the-lab">The Lab</h1> <p>Each tool ran on its own dedicated Hyper-V VM: 3 GB RAM, 2 vCPUs, Ubuntu 22.04 on kernel 5.15.0-91-generic. One tool per VM. This was non-negotiable. eBPF programs from different tools will contend for the same kernel hooks, and if you’re trying to measure event ordering and timing, that contention corrupts your data. You can’t draw conclusions about “tool A generated event X before tool B generated event Y” if both tools are fighting over the same kprobe attachment points on the same kernel.</p> Container Escape Telemetry, Part 3: What Each Tool Actually Captured https://catscrdl.io/blog/containerescapetelemetry/deepdives/ Tue, 31 Mar 2026 00:00:00 +0000 https://catscrdl.io/blog/containerescapetelemetry/deepdives/ <p>This is Part 3 of the container escape telemetry series (<a href="../intro">overview</a>). <a href="../isolation">Part 1</a> covered isolation primitives and the eBPF observability model. <a href="../methodology">Part 2</a> covered the lab, the tools, and the detection coverage matrix. This post is the meat: per-scenario telemetry breakdowns showing what each tool actually captured, where the qualitative differences behind the checkmarks become concrete.</p> <p>Not every scenario gets a deep dive. Some produced straightforward results that the coverage matrix already summarizes. I’m focusing on the scenarios where the cross-tool comparison reveals something interesting about how these tools work, where they disagree, and what the disagreements mean for detection engineering.</p> Container Escape Telemetry, Part 4: Volume, Signal-to-Noise, and Choosing a Tool https://catscrdl.io/blog/containerescapetelemetry/production/ Tue, 31 Mar 2026 00:00:00 +0000 https://catscrdl.io/blog/containerescapetelemetry/production/ <p>This is Part 4 of the container escape telemetry series (<a href="../intro">overview</a>). <a href="../isolation">Part 1</a> covered isolation primitives and the eBPF observability model. <a href="../methodology">Part 2</a> covered the lab and detection coverage matrix. <a href="../deepdives">Part 3</a> walked through per-scenario telemetry. This post is about the operational questions: how much data do these tools produce, what percentage of it matters, and which tool should you actually deploy?</p> <h1 id="the-volume-question">The Volume Question</h1> <p>Parts 2 and 3 focused on what each tool sees. But there’s an equally important dimension: how <em>much</em> does each tool generate, and what’s the signal-to-noise ratio of that output? If you’re going to deploy any of these tools in production, the telemetry volume has direct implications for storage, bandwidth, and whether an analyst can actually find the escape signal in the noise.</p> Container Escape Telemetry, Part 5: Tuning eBPF Tools From Defaults to Detection https://catscrdl.io/blog/containerescapetelemetry/tuning/ Tue, 31 Mar 2026 00:00:00 +0000 https://catscrdl.io/blog/containerescapetelemetry/tuning/ <p>This is Part 5 of the container escape telemetry series (<a href="../intro">overview</a>). <a href="../isolation">Part 1</a> covered isolation primitives. <a href="../methodology">Part 2</a> covered the lab and tools. <a href="../deepdives">Part 3</a> was the per-scenario data. <a href="../production">Part 4</a> covered volume, signal-to-noise, and tool selection. This post is for the practitioner who just installed one of these tools and wants to know what to do next.</p> <p>The gap between “tool installed” and “tool detecting container escapes” is larger than any of these projects’ documentation suggests. Here’s what that gap actually looks like and how to close it.</p> Container Escape Telemetry, Part 6: TeamPCP and What the Lab Predicted https://catscrdl.io/blog/containerescapetelemetry/teampcp/ Tue, 31 Mar 2026 00:00:00 +0000 https://catscrdl.io/blog/containerescapetelemetry/teampcp/ <p>This is Part 6 of the container escape telemetry series (<a href="../intro">overview</a>). Parts <a href="../isolation">1</a>-<a href="../tuning">5</a> covered isolation primitives, methodology, per-scenario telemetry, production considerations, and tuning. This post takes the lab findings and pressure-tests them against a real threat actor operating in the wild right now.</p> <p>TeamPCP compromised Trivy on March 19, 2026. Five days later they backdoored LiteLLM’s PyPI package. As of late March 2026, incident response teams across the industry are still assessing the blast radius. But the container escape techniques TeamPCP uses aren’t novel. They’re the same patterns our 15 lab scenarios were designed to detect. That’s either a validation of the research or a coincidence. Having spent weeks staring at this telemetry, I don’t think it’s a coincidence.</p> Container Escape Telemetry: Series Overview https://catscrdl.io/blog/containerescapetelemetry/intro/ Tue, 31 Mar 2026 00:00:00 +0000 https://catscrdl.io/blog/containerescapetelemetry/intro/ <p>As a detection engineer, container escapes have always fascinated me. Detections focused on control bypasses can be both extremely difficult to make and extremely valuable when they fire. The challenge is that most discussions about container runtime security focus on <em>whether</em> a tool detected an escape – a binary yes/no that doesn’t tell you much about the underlying telemetry that makes detection possible in the first place.</p> <p>There’s also a marketing problem. I’ve seen vendors demonstrate container escape detection against one or two scenarios – usually cgroup release_agent or nsenter – and present that as comprehensive coverage. It isn’t. A tool that detects cgroup escapes but misses DirtyPipe, or catches namespace transitions but is blind to capability abuse, provides a false sense of security. You can’t evaluate coverage without testing breadth.</p> The Pillars of Security Monitoring: A Comprehensive Guide https://catscrdl.io/blog/typesofsecuritymonitoring/ Sat, 03 Aug 2024 00:00:00 +0000 https://catscrdl.io/blog/typesofsecuritymonitoring/ <h1 id="the-pillars-of-security-monitoring-a-comprehensive-guide">The Pillars of Security Monitoring: A Comprehensive Guide</h1> <p>As you may know by now if you’ve read a few of my blog articles, I’m a big fan of decomposing ideas in threat detection into various frameworks. This latest blog post will touch on another area that I feel strongly about - specifically, that detection engineers need to take a step back and realize that they’re part of a larger “security monitoring” capability which consists of related but distinct pillars. We’ll explore these interrelated pillars of security monitoring: threat detection, posture management, compliance monitoring, abuse detection, insider threat, and vulnerability management. Understanding these differences isn’t just academic – it’s essential for crafting an effective, nuanced security strategy. Importantly, it will lead into an often opinionated discussion on SOC vs SOC-less models.</p> Detection as Code: A Maturity Framework https://catscrdl.io/blog/detectionascodematurity/ Thu, 25 Jul 2024 00:00:00 +0000 https://catscrdl.io/blog/detectionascodematurity/ <h1 id="detection-as-code-a-maturity-framework">Detection as Code: A Maturity Framework</h1> <h2 id="introduction">Introduction</h2> <p>In the ever-evolving landscape of cybersecurity, the methodologies and tools we rely on are continually advancing. One of the latest trends is Detection as Code (DaC), which promises to revolutionize threat detection by integrating software engineering practices into security operations. Having worked in an organization that practices DaC for several years, I’ve come to realize that it’s not simply a matter of whether or not to adopt DaC.</p> There are Only Two Rules https://catscrdl.io/blog/thereareonlytworules/ Fri, 22 Dec 2023 00:00:00 +0000 https://catscrdl.io/blog/thereareonlytworules/ <h1 id="intro">Intro</h1> <p>In the field of detection engineering, understanding and applying the right frameworks is crucial for effectively identifying and responding to threats. Mental models provide a structured way of categorizing and approaching the myriad of security alerts and anomalies we face daily. These models help us dissect and understand the behavior of potential threats, allowing for a more targeted and effective defense strategy. In this blog post, I’ll explain how I use one mental model to clearly classify every detection rule into one of two categories and how that helps shape the decision of what kind of detections I build.</p> Building Resilient Detection Suppressions https://catscrdl.io/blog/buildingresilientsuppressions/ Wed, 20 Dec 2023 00:00:00 +0000 https://catscrdl.io/blog/buildingresilientsuppressions/ <h1 id="intro">Intro</h1> <p>Raise your hand if you’ve worked on suppressing a false positive for a detector and then the very next day, it’s back in your “fix” queue with another false positive. I’d expect almost everyone who’s worked in detection engineering for any period of time has their hand raised or is nodding along. Why does this happen? Well, sometimes the detector itself isn’t very good and it’s alerting on a lot of benign behavior. But sometimes, the answer is that the suppression that was implemented was too specific and wasn’t properly generalized to capture the benign behavior.</p> How to Write an Actionable Alert https://catscrdl.io/blog/howtowriteanactionablealert/ Mon, 03 Oct 2022 00:00:00 +0000 https://catscrdl.io/blog/howtowriteanactionablealert/ <h1 id="intro">Intro</h1> <p>“If a tree falls in a forest and no one is around to hear it, does it make a sound?”</p> <p>This mantra about possibly non-existent trees very much applies to detection engineers. If you write a great rule to detect malicious behavior, but no one (or nothing) acts upon the alert, did you really catch the threat actor? Or worse, an analyst acts on the alert but responds incorrectly. In this article, I will cover the importance of not only writing solid detections, but also solid alerts. The target audience of this blog post is security professionals who write custom detections in their SIEM or other similar system.</p> Real Time vs Scheduled Query Detections - A Guide For Detection Engineers https://catscrdl.io/blog/streamingvsscheduling/ Mon, 15 Aug 2022 00:00:00 +0000 https://catscrdl.io/blog/streamingvsscheduling/ <p>Most modern SIEMs offer 2 primary methods for running their queries: real time rules and scheduled queries. Each option offers a variety of pros and cons that you should consider as you develop your detection. Before we dive into that, let’s clarify what we mean by each.</p> <h1 id="definitions">Definitions</h1> <p>A real time rule, often called a streaming rule, runs on a stream of data. Essentially, as the data enters the SIEM, each data point will get processed against the rules that correspond with that kind of data. It’s important to note that these are actually more accurately “near” real time rules. There’s delays due to log generation, ingestion, parsing, running the rule logic, and delivering the results. Let’s take a look at some examples.</p> A Shmoo's Guide to DC https://catscrdl.io/blog/ashmoosguidetodc/ Sat, 26 Mar 2022 00:00:00 +0000 https://catscrdl.io/blog/ashmoosguidetodc/ <p>Some of these recommendtions work best for Shmoocon in March vs Shmoocon in the cold of January.</p> <p>Orienting Yourself <br> Below is a map of the neighborhoods. The gold star is the Hilton hotel. When you hear someone reference a neighborhood, hopefully this helps. The most common ones you’ll probably hear are Adams Morgan (Admo), Dupont Circle (Dupont), U St, The Mall, and Capitol Hill.</p> <p><img src="https://catscrdl.io/static/img/2022/DC_neighborhoods_map.png" alt="Neighborhood Map"></p> <p><strong>Getting Around</strong> <br> Metro</p> <ul> <li>Dupont station is going to be closest and that’ll get you to the red line.</li> <li>You probably won’t need to take the Metro unless you’re heading to Amtrak or heading out to Virginia or DCA (Airport). <img src="https://catscrdl.io/static/img/2022/MetroMap.jpg" alt="Metro Map"></li> </ul> <p>Buses</p> Azure Flow Log Analysis https://catscrdl.io/blog/azureflowloganalysis/ Fri, 16 Jul 2021 00:00:00 +0000 https://catscrdl.io/blog/azureflowloganalysis/ <h1 id="intro">Intro</h1> <p><em>Disclaimer</em> I currently work at Snowflake and use the product on a daily basis for log analysis and threat detection. At the time of this writing, that probably adds bias to my article.</p> <p>At Snowflake, we’re a multi-cloud environment. As part of the threat detection team, it’s my job to ensure that we’re monitoring. One of my tasks was to monitor our Azure flow logs for unusual behavior. Information on Azure Network Security Group (NSG) flow logs can be found <a href="https://docs.microsoft.com/en-us/azure/network-watcher/network-watcher-nsg-flow-logging-overview">at this link.</a> When I started exploring this data source, I was frustrated to learn that, unlike the AWS VPC flow logs, there was no <a href="https://docs.aws.amazon.com/vpc/latest/userguide/flow-logs.html">field that ties the flow log to the instance ID</a>, or in the case of Azure, the Virtual Machine. The recommended solution from Microsoft was to use the IP address. However, the architecture I was working in had thousands of worker nodes created and destroyed frequently meaning that we would often see duplicate IP addresses within a short period of time due to NATing. If I couldn’t automatically tie the IP to a VM, the logs would be useless as our IR team needed that information in order to conduct automated response in case of a malicious event.</p> Regarding SMS 2FA https://catscrdl.io/blog/regardingsms2fa/ Wed, 29 Jul 2020 00:00:00 +0000 https://catscrdl.io/blog/regardingsms2fa/ <p>I think an important question that Tavis either explicitly or accidentally omitted is “for whom”. I am not sure why he did not include this as it’s a critical component to his argument. If Tavis is stating that “SMS 2FA is ineffective for an enterprise”, then I would agree. The threat model that he is operating from is that an organization is being explicitly targeted by a motivated (though not necessarily extremely capable) attacker, who only needs minimal access to cause great harm. However, if Tavis is instead simply stating “SMS 2FA is ineffective” with no caveats, I would disagree.</p> Adding to the Dialogue - On the Release of Offensive Security Tools (OST) https://catscrdl.io/blog/opensourceredteamtooling/ Tue, 24 Dec 2019 00:00:00 +0000 https://catscrdl.io/blog/opensourceredteamtooling/ <p>Update 1 - I’m clarifying the definition of Advanced Persistent Threats (APTs) and Financially Motivated Actors (FMAs). I combined the two groups in the previous version. The content and focus of the discussion primarily centers on FMAs. APTs and FMAs can overlap in terms of TTPs, capabilities, personnel, countries, etc. What distinguishes them is motivation. FMAs, as their name implies, are financially motivated. APTs can have a number of motivations including financial, political, etc.</p> Open Source SaaS Reconnaissance Utilizing Subdomains https://catscrdl.io/blog/opensourcesubdomainrecon/ Wed, 03 Jul 2019 00:00:00 +0000 https://catscrdl.io/blog/opensourcesubdomainrecon/ <p>On a recent <a href="https://www.praetorian.com/corporate-security/purple-team-detection-response">Purple Team</a> engagement, I was accessing the client’s Splunk cloud instance. Being my normal typo-filled self, I fat fingered the URL and went to clieent.splunkcloud.com instead of the correct client.splunkcloud.com. Instead of being redirected to the login portal, I received a DNS resolution error, fixed the typo, and moved on. A little while later, I was thinking about the typo again and wondered how this information leakage could be utilized by an attacker. To backup a little, a key part of an offensive campaign (and many say the most critical part) is recon. During this phase, the Red Team or malicious actor tries to obtain as much information about the target as possible. Key items of interest include organizational charts, employee lists, email formats, external address space, and technologies utilized by the organization. During this phase, the attacker wants to limit their interaction with the target as much as possible in order to avoid tipping off defenders. Most attackers avoid active scanning or interaction with the target during this phase.</p> Getting Started with Praetorian’s ATT&CK™ Automation https://catscrdl.io/blog/gettingstartedwithautomation/ Fri, 10 May 2019 00:00:00 +0000 https://catscrdl.io/blog/gettingstartedwithautomation/ <p>Blog link: <a href="https://www.praetorian.com/blog/getting-started-with-praetorians-attack-automation/">Getting Started with Praetorian’s ATT&CK Automation</a></p> Why Praetorian Benchmarks to MITRE ATT&CK™ and Why You Should Too https://catscrdl.io/blog/attackbenchmark/ Wed, 01 May 2019 00:00:00 +0000 https://catscrdl.io/blog/attackbenchmark/ <p>Blog link: <a href="https://www.praetorian.com/blog/why-praetorian-benchmarks-to-mitre-attack/">Why Praetorian Benchmarks to MITRE ATT&CK™ and Why You Should Too</a></p> Demonstrating the Future of Command and Control with Wikipedia https://catscrdl.io/blog/wikipediac2/ Sun, 14 Apr 2019 00:00:00 +0000 https://catscrdl.io/blog/wikipediac2/ <p>This work was inspired by my coworker’s (<a href="https://twitter.com/Jabra">Josh Abraham</a>) work to demonstrate alternative forms of Command and Control. Some of his POCs included demonstrations using Slack and ICMP for C2. While these techniques aren’t revolutionary, the simplicity of the POC and how hard it was to detect confirmed my suspicion that the future of C2 is going to be tunneling traffic via features of well-known applications.</p> <p>In addition, there was a recent Twitter post from <a href="https://twitter.com/randomuserid/status/1113088976263569408">@randomuserid</a> that got me thinking about what other sites besides Slack could be utilized. So, I browsed through the <a href="https://www.alexa.com/topsites">Alexa top sites</a> and started seeing what features could be abused to facilitate C2 traffic. The first few were all linked in @randomuserid’s Google sheet, so I moved onto <a href="wikipedia.org">wikipedia.org</a>. It didn’t take me long to stumble onto the <a href="https://en.wikipedia.org/w/api.php">Wikipedia API</a> page.</p>