OpenHack: The Open‑Source Workflow That Turns LLMs Into Security Code Reviewers

By integrating directly with these environments, OpenHack lets developers or security teams run structured security review workflows as part of their normal coding process rather than as a separate manual audit step.

The Problem With Naive LLM Code Review

Large language models can read and reason about source code, but simple prompts like "find vulnerabilities in this repo" produce unreliable results. According to Hadrian, common issues include:

• High false‑positive rates, where models flag harmless patterns as security bugs.
• Hallucinated findings, where the model invents vulnerabilities that do not actually exist.
• Inconsistent analysis, depending on prompt phrasing or context limits.

OpenHack addresses these problems by replacing open‑ended prompts with a structured methodology that guides how the model investigates code.

Scenario‑Based Security Investigation

One of OpenHack’s key ideas is scenario‑based scoping. Instead of asking an AI to broadly scan code for problems, the workflow instructs it to analyze specific attack classes or exploitation paths.

For example, a model might be directed to:

• Trace how user input flows through an application
• Search for local file inclusion (LFI) opportunities
• Examine credential storage or cloud configuration

This targeted approach narrows the model’s attention and gives it a clear objective, improving reasoning quality and reducing generic or irrelevant findings.

Independent Triage to Reduce False Positives

Another major feature of the workflow is separating discovery from validation.

In a typical OpenHack run:

1. One AI agent identifies a potential vulnerability.
2. A separate step or agent performs triage.
3. Evidence is reviewed before the issue is accepted as a real finding.

This separation helps filter weak reports and encourages models to gather proof—such as code paths, exploit chains, or configuration evidence—before flagging a vulnerability.

Real‑World Testing in Dutch Government Code Audits

Hadrian reports that it used a similar methodology to audit open‑source applications used by Dutch government agencies. According to the company, the AI‑assisted review surfaced hundreds of security issues within hours.

One example highlighted in the research involved:

• An unauthenticated local file inclusion (LFI) vulnerability
• Exposure of Azure credentials
• A potential path toward remote code execution in an Azure environment

These results are primarily reported by the vendor itself and should be interpreted cautiously until independently verified, but they illustrate the kind of attack‑chain reasoning the workflow aims to enable.

Why Hadrian Open‑Sourced OpenHack

Hadrian released OpenHack on GitHub under the permissive MIT license, providing documentation, prompts, CLI tooling, and support for Python 3.9 and newer.

The company’s stated goal is to "level the playing field" in AI‑assisted vulnerability discovery. As AI tools become more capable at finding software flaws, defenders risk falling behind if those capabilities remain proprietary or restricted.

By open‑sourcing the workflow, Hadrian hopes security teams and developers can adopt similar methodologies to analyze their own codebases using widely available LLMs.

The Bigger Trend: AI‑Assisted Security Auditing

OpenHack reflects a broader shift in software security: using AI agents to explore codebases at scale. Modern coding assistants and AI‑native development environments can already analyze repositories, reason about architecture, and automate development tasks.

Structured workflows like OpenHack attempt to harness that capability for defensive security work—turning general‑purpose AI models into systematic vulnerability researchers rather than unpredictable reviewers.

As LLM‑driven tooling becomes more common in development environments, approaches that emphasize scoped analysis, evidence collection, and independent verification may become essential for making AI‑based security review trustworthy.