Too philosophical
We started about two years ago with a question that's admittedly a little too philosophical to be hovering around: how do you approach knowing knowledge? We use it to describe an intuition for finding a vulnerability in software — essentially a coverage problem, or, framed another way, an exploration problem where the heuristics matter most because they set the direction and reach of the search: where it goes and how far. Taint analysis connects dangerous sinks back to inputs; fuzzing throws near-random inputs into a pipeline and waits for one to drive execution down a branch that misbehaves.
To begin with, we found that this epistemological heuristic is the biggest determinant of what and how many bugs an LLM can find — yet most LLM automation systems put almost no deliberate design into it. My guess is that we security researchers lean on it so intuitively in real-world VR that it's become invisible to us. All we really do is throw a sentence like "go and find RCEs, make no mistake" at the model and hope it surfaces zero-days that Google Project Zero missed, while handing it no clue where to start in a codebase of tens of thousands of lines. That's essentially expecting an LLM to match a Project Zero researcher with no degradation over context. P0 researchers are good because they approach where to look with far more ingenuity — mutation analysis, in-the-wild bugs, taint, fuzzing crashes — and at the end of the day it comes down to how many bugs they've already seen across complex context. Proof-of-concept development is the comparatively trivial part: once you've looked in the right place and found the right thing, the rest takes much less.
Over the past year, we've tried tying LLMs to other epistemological heuristics that we (humans) rely on, to see whether it helps them find more — and more kickass — bugs. Taint analysis, for one: see my prior work, Deductive Engine: Human Inspired Taint Reasoning (a continuation of our BH USA work, ToAST). The conclusion is that taint analysis — dataflow analysis — is a very effective intuition for LLM VR. In a sense it's almost perfect: it provides a solid epistemics toward the problem of slicing (what context we put into the LLM) via contaminations (or propagations — see Explaining taint propagation using Disney's Where's My Water?). It's as effective as how human researchers find bugs because it follows the same logical process they do — essentially an exploration problem: proving connectivity between the slice that introduces the data and the slice that weaponizes it, to establish reachability and exploitability, and thus vulnerability. For example: start with a suspicious function (say
memcpy, as a logical entry point); explore outward from it (cross-reference calls, and so on); determine whether we've reached a source; examine reachability between source and sink (source and the suspicious function — confirming no sanitizers sit in between); model the restrictions required to branch from source to the suspicious function; then craft an input that satisfies those restrictions, with payloads — all built on a dataflow ontology.
aha
We started looking at diffs while Pwno was working on an integration with GGML (llama.cpp), and we didn't approach it with the "coverage" framing at all — more as a "business model." Diffs are how we've reviewed code for the past 20 years; PRs were already the unit of attention in our development pipeline, so reviewing them for vulnerabilities just felt intuitive. It was only after doing this for a while that we noticed the first non-obvious part (#1) of Diff Rolling: diffs are a remarkably effective slicer, and crucially, they slice on a dimension no static analyzer operates on.
Traditional slicing (AST, call graph, taint) partitions code along semantic axes: what calls what, what flows into what. A diff, atomically, partitions code along intent — each commit message describes an intent, and the codebase is sliced by that shared intent. A commit pulls together lines that may be syntactically scattered, structurally unrelated, even spread across different files, and binds them under a single human-authored purpose — one stated right there in the commit message.
Take PWNO-0014, an FFmpeg EXIF heap overflow we often use as a teaching example. In commit 784aa0,
libavcodec/exif.cis effectively re-decomposed: the diff lifts a set of lines out of their original structural neighborhoods and regroups them around a new purpose — "parse additional EXIF tags" — in a way no AST walker, dataflow engine, or document ranking would. And that's the dimension LLMs natively read on. Rolling with the Diffs, in other words, slices the codebase on the one axis that aligns with how language models actually think.
Slicing was the center of Tree-of-AST and Deductive Engine's epistemics, which I'd been working on for a while, so I realized this was really the key to something bigger. But it was only later that I saw there was more here that's non-obvious.
Apart from intent alignment, another reason we found diffs work well for LLMs is that they're structured contrastively, and so natively compress context. A diff turns context into changed (new/old) lines + surrounding context + commit message. It strips away most of the semantically stable code while preserving the "boundaries" of semantic variation.
Much would have more.
Waiting on GGML's pipeline got old. We were sitting on HEAD, hoping the next push would surface a primitive worth writing up, and most days nothing did. Even when something did, it was usually a freshly landed feature (a new GGUF architecture, a half-rolled-out quant path) that hadn't shipped widely enough to matter to maintainers, or to anyone. The wait was too long and the payoff too thin.
So we got greedy: what if we stopped waiting for new commits and started scanning backwards?
The moment I tried it, the math turned absurd in our favor. There are almost too many commits to scan (#2). llama.cpp ships ~20 commits a day, and on a project actively maintained for years, that compounds into tens of thousands of diffs just sitting there. Even at a 5% rate of vulnerability introduction, the scale tilts heavily toward us: maintainers have to land every commit clean, while we only need one to slip — a tiny off-by-one in an unnoticeable corner. And crucially, all of this is embarrassingly parallel. (The part we'll mainly focus on is the application.)
Once we started scanning at a slightly larger scale, we found — to our surprise — that the distribution of findings isn't flat; they cluster toward more recent commits. Commits are naturally sorted by time, and time implicitly creates a probability gradient over the remaining vulnerabilities (#3): the later a commit is, the less likely it's been fuzzed, manually reviewed, or reported via a production crash, because it has survived for less time, so the window in which it could have been patched is smaller. Conversely, the later the commit, the closer it sits to us on the history — and the more likely we are to find something fun.
A rather ingenious corollary fell out of this: patch commits are themselves markers. (patch folding) If a later commit fixes a bug introduced by an earlier one, we mark the older commit to skip.
One subtlety worth flagging, since it looks like a contradiction at first. Earlier we said freshly landed features carry low impact because they haven't shipped; here we're saying newer commits carry higher bug density. These don't conflict — they live on different timescales (my wording might have blurred that). The not-yet-shipped window is weeks, but the gradient stretches much longer, across years.
Once we accepted that, we noticed something stranger about where the scanning was taking us: Diff Rolling's coverage is uniform at the commit level but organically weighted at the component level (#4, 5) — a duality that happens to favor security research.
Atomically, every commit is a brick in the codebase, and we don't bias any of them (we treat each slice as carrying equal weight; we call this bias in reviewing "attention"). This atomic equality matters, because vulnerability impact has no correlation with code prominence (#4). We want a slice of EXIF extra-tag processing treated the same way we treat an mp4 av_stream header-reading slice — a vulnerability in an unwatched corner carries the same latent impact as one in a popular path. This is also a latent effect of the granularity we described in #1.
Collectively, though, the distribution isn't flat. As an emergent property, actively maintained components accumulate more commits (h264 in ffmpeg, say, receives far more than some obscure 16-year-old legacy decoder nobody optimizes). So uniform sampling over commits organically induces a non-uniform, heavy-tailed coverage over components (#5), where the tail's mass tracks commit attention — which is itself a proxy for what's "valuable" from an impact standpoint. The collective-level weighting falls out of the atomic-level uniformity, for free.
The five observations above form a unified coverage space.
Slices are structured — composed by intent into a unit. (alignment) (atomic, naturally decomposed in favor of reviewing)
Effectiveness (scalability, granularity) scales with the number of commits. (abundant granular parallelism) (the better maintained a project is, the more statistically likely you are to find a bug, and the finer the granularity)
Diffs form a uniform sampling space. (flat) every individual slice is looked at (sampled) with the same prioritization.
Yield weightings are sorted temporally. (temporal weighting) (the later a commit is, the more likely the code it slices is found vulnerable)
Corollary: Patch Folding
Patch commits are themselves markers. If a later commit fixes a bug introduced by an earlier one, we mark the older commit to skip.
Attention weightings are organically accumulated. (spatial weighting) (hot, well-maintained components receive more total coverage in aggregate, but atomically they're treated with equal attention — as they should be in security research. The weighting is emergent)
We define the commit history of a codebase as a naturally enumerable vulnerability search space. Atomically, each commit is naturally structured for reviewing: constructive, intent-aligned, small enough for an LLM to review, rich enough to expose invariant change, and numerous enough to sample at scale. Yield weightings are sorted natively by time; the sample space itself is uniform at the commit level, preserving atomic fairness, while collectively, commit density forms a natural accumulation of attention.
Embarrassingly Scalable
The first idea was to roll the diffs at scale. This was still the Big Sleep era, and one of the things I wanted Pwno to do was somehow keep up with Project Zero and DeepMind's output. So I started reaching for bigger, more ambitious codebases. We began with FFmpeg — the media library billions of people rely on every day — because it ticks every box: big, well-maintained, and essential enough that people care about its security (FFmpeg has over 124,281 commits, averaging ~94 a week).
And it was at this larger scale that Diff Rolling's scalability really began to manifest. Processing commits is embarrassingly parallel — there are no dependency relationships between them, so you can technically process 20 at once — and, as we've noted, for bigger codebases there are almost too many commits to look at (#2). It's this combination of availability and scalability in the epistemic itself that makes its application insanely efficient, and thus effective: it can maximize resource utilization without sacrificing any granularity (#4)
(Similar to how the Embarrassing Parallelization of attention heads helped Attention Is All You Need*, this efficiency would be the* [have to think of best wording — the extra key factor, like that one extra rower who makes sure the boat wins?] when processing a larger, more ambitious target — the way attention won at training scale — as we scan the entire internet's infrastructure.)
Folding
While scaling Diff Rolling, we discovered more practical applications of it in real-world automated security research. Instead of scanning all commits, you can target a specific component by path (a certain codec in FFmpeg, say — and, calling back to #3, the temporal gradient can stretch much further as commits grow sparse), a specific author's commits (if a known style is more prone to particular mistakes), or a particular time of commit (Friday 5 p.m. versus Thursday 11 a.m.). This delimitation of the enumerable vulnerability search space depends only on the informative granularity of commits — and a perk of using commits as atoms is that they're information-rich yet widely arranged. Commits are designed such that even pulling out the code changes alone leaves you with something highly informative: a commit describes almost all the implicit information you'd need about a change. So we have plenty of ceiling to narrow the search space toward the kinds of commits most likely to introduce a noteworthy vulnerability.
Collectively, the way commits assemble into a codebase is itself enough to delimit the enumeration space further. Beyond Patch Folding, another interesting delimitation is what I've named the Ship of Theseus problem. Just as the classical Greek story probes the philosophical composition of objects, some commits are part of the history but no longer exist at all in the current version of the codebase. I call the process of eliminating them Theseus Folding, achievable with git blame + AST. With both folding techniques, we can actively address both the explicit decay (via Patch Fold) and the implicit decay (via Theseus Fold) of a codebase seen as a collection of commits.
Path folding itself also provides a high-quality dataset for future discovery.
With both the embarrassing scalability (available + parallelizable) and the effective enumeration-space delimitation (courtesy of how informative commits are) up our sleeves — while the sampling space arranged for that enumeration is natively advantageous for security research, and even each individual atom is shaped to favor it — the methodology ends up unintuitively effective.
Schrödinger's Box
After Rolling Diffs across Linux, FFmpeg, and V8 gave us great findings, we ran into the same problem we'd had since llama.cpp: verifying and PoC'ing them. That's even more of an engineering problem.
Since I'd been working on distributed systems already (autogdb, and RPC for llama.cpp), my first instinct was to architect isolated containers in which agents could work on PoC-as-validation for each finding — the Embarrassingly Scalable property still holds through verification. It's also intuitive to use an instrumented ground-truth validator like ASAN, UBSAN, or KASAN as proof of a vulnerability.
While we're on the definition of a vulnerability — a software bug that affects security to a degree people should worry about — our research says it comes down to two dimensions: reachability and exploitability.
Exploitability is the intuitive one: for a bug to be a vulnerability, it has to be exploitable. A mis-rendered button in the frontend isn't a vulnerability; a mis-rendering that breaks HTML escaping is. Reachability is the more essential part of triaging Diff Rolling's findings. The most common disqualifier — as with other LLM vulnerability discovery — is that a bug can be technically exploitable while its reachability keeps the component from actually being vulnerable. For instance, the diff introduces a NULL-pointer deref in a frame-processing function that's only reachable after the decoder has already allocated a frame.
To address that concern — a vulnerability's reachability in real-world scenarios — we designed the-box, a distributed system that mimics the harnessing process of CTF challenges; specifically, since we're working with memory issues, CTF pwn challenges.
The philosophy behind box is that it sets up a validation environment independent of the input. Services are built with validators (ASAN, UBSAN) integrated up front, then harnessed with a script that redirects payload I/O in a real-world manner. Unlike a CTF pwn challenge, the only way to retrieve a "flag" here is to craft a payload that triggers an ASAN-induced signal from the binary. The payload travels through the payload I/O into the harness, while the debug I/O lets agents observe the harnessed service's inner states and readjust their payload crafting accordingly.
With the separation of the input space from the validation space, box first of all stops the LLM from interfering with the integrity of the validation (cheating), and thus guarantees a valid proof of reachability and exploitability (the ASAN log), while a payload passed through the payload I/O can serve directly as a valid PoC. At the same time, the two I/O channels remove redundancy from the validation and PoC-development process. By isolating — and throwing away — the unrelated, trivial scaffolding (building, harnessing), it keeps the PoC agent focused on a single question: "Did my payload trigger the intended validation signal?" (If not: what does the debug I/O tell me about the program's internal state? Should I readjust my padding or the placement of the manipulated content — or is what I'm observing inconsistent with the finding's analysis, meaning it isn't actually exploitable?)
box, meanwhile, didn't inherit Diff Rolling's Embarrassing Scalability — but the design is scalable by nature anyway, since it's centralized in service of a distributed-systems design.
Since every box container is isolated by a kubernetes containerized design, we can only assign each container a little compute — and because of that, project construction was a problem for us at first: building a project means you have to pull it and then build it. For bigger projects like Chromium or Firefox, on a small pod the pull alone would take at least ~40 minutes, never mind the compile.
This was an even bigger challenge when I was working on upstream monitoring of larger codebases, since it means working across a different version most of the time.
Working toward a fix, it occurred to me that since we're doing diff-based analysis, all we need to maintain is a base reference of the codebase — a base build for each project. The pod pulls the built base artifacts from the server and builds from there itself; when it (the box) requests a specific or later version, we update the HEAD reference. And since we work from a centralized reference, with caching, each recompile costs far less compute and time — and the compute we save lets us centralize and maintain more projects.
