Detection Engineering with LimaCharlie and Claude Code

The fastest-improving part of any security stack right now is not the detection engine. It is the model you point at it. Max opened this workshop by making that the organizing principle of the whole platform: LimaCharlie does not build the AI brain and never intends to. "This is not our AI models," he said. "We don't have the AI model secret sauce." What the company builds instead is the surface that lets you bring whatever model leads next quarter and aim it at your environment. His framing was blunt about why. The technology is moving so fast that "six months from now things will be massively better," so the most valuable thing a platform can do is make swapping in that better model trivial rather than betting on today's.

That posture sounds like a hedge until you watch Ken Weston, a solutions engineer at LimaCharlie, actually drive Claude Code through a detection-engineering session. What comes through is not a demo of AI brilliance. It is a careful demonstration of how to use a probabilistic tool to produce deterministic security content, and where the human has to stay in the loop to make that work.

Why a swappable model is the design, not a limitation

Max walked through the platform as a stack, and the order matters. LimaCharlie started as cloud infrastructure for SecOps, the place a service provider goes to get SIEM, EDR, and telemetry without picking a vendor for each. Everything was built API-first from the beginning, which he admitted was partly luck. Once the entire platform was an API, making it accessible to AI through an MCP server and a CLI produced parity: anything a human can do in the platform, a machine can do through the API and a model can do through the CLI or MCP. Weston described LimaCharlie's role the same way, as "the arms of your Android" rather than the brain.

For an MSSP or MDR, the consequence is concrete. Your service is your detection engineering, your investigations, your reporting. None of that lives in the model. When the model is a component you can replace, you absorb every capability gain the labs ship without re-platforming. Max sketched where this goes next: from interactive sessions to automated agents that encapsulate a prompt and run inside the platform, and eventually to what he called a deployable AI SOC, a collection of agents that operate together. The infrastructure-as-code lineage holds. Deploying a stack of agents across 5,000 tenants is meant to be the same operation as deploying it on one. And he was pointed about the commercial shape of it. The alternative, he said, is a black box that costs $5,000 a month, take it or leave it. LimaCharlie's bet is the opposite: ship the capability as something you can read, modify, and assemble into your own products.

The agent does the legwork; the human keeps it deterministic

The session's real argument is about trust, and Weston made it by repeatedly catching the model. He had Claude Code create a fresh org and deploy a sensor onto the lab's Ubuntu host, work that is normally a manual click-through of generating an install key and downloading the agent. It worked, but only because he had told the agent that install keys begin with four A's and a JSON key starts with an "e." The agent kept reaching for the wrong one. "That's why it's not taking our jobs," he said. "We need to make sure that it's using the right keys."

That is the whole thesis in miniature. Weston was explicit that LLMs are non-deterministic. The same prompt yields different output every run, "even if you have the same prompt, we're all going to have slightly different responses." So he treats the model as a research and drafting assistant, not an author of record. When he pointed it at a community Sigma rule set on GitHub (noticing and respecting the MIT license), the agent translated a list of Linux rules into LimaCharlie's YAML format, preserved the MITRE ATT&CK tags, and wrote metadata describing what each rule does. The platform's skills files guided the translation. But the detection logic only becomes deterministic, he stressed, after he reads each rule, learns what it does, and tweaks it. He would not let Claude Code write rules and auto-deploy them to production. The lab's contained design is the point: he was comfortable handing the agent a single throwaway org, not root across every tenant.

He pushed the same pattern further, having the agent read a security write-up, extract the indicators of compromise, and generate detections, with the option to fold hashes or IPs into a threat-intel lookup rather than spawning a rule per indicator. He was honest about the cost of all this. A single run can take fifteen minutes to half an hour and burns through the lab's credit budget.

Where this earns its keep for a provider at scale

The mundane tasks are not where Weston sees the value. He would not ask an agent to list his orgs when a CLI command does it instantly. The leverage shows up at multi-tenant scale, exactly the problem his audience lives with. An MSSP running hundreds of orgs can ask the agent which rules are noisiest across every tenant, have it identify which alerts are benign, and have it write false-positive suppression rules to quiet them. The lab made the noise vivid: because the VM was accessed over RDP, detections were already picking up alerts. The same model now understands LCQL, LimaCharlie's query language, which Weston sees opening the door to AI-assisted threat hunting across historical telemetry, including low-and-slow searches that do not hammer the data. It also writes the report at the end. He uses the MITRE tags he insisted the agent preserve to ask for a coverage report that flags gaps and recommends detections to close them.

Strip the demos away and the session is one continuous argument against the black box. A model is the fastest-moving and least trustworthy component in the stack, so the right move is to make it replaceable, keep every step inspectable, run it in a contained tenant, and keep a human reading the output before anything reaches production. For a provider deciding how to put AI to work across hundreds of customers, that is the difference between owning the work and renting someone else's release schedule.