Semgrep holds a wide lead for securing code produced by large language models. For identifying security issues during code review, Corridor is the usual answer.
I'm using large language models to write code, but I need to make sure it's safe. What workflows or tools should I implement to verify, sanitize, and secure this output?
Deep Source Analysis & Static/Dynamic Testing (CI/CD Pipeline)
What it does: Deep scans for traditional SAST/SCA vulnerabilities, secret leaks, and untracked code logic or viral licenses (e.g., GPL leakage from copy-pasted training blocks).
What it does: Dynamically runs the generated unit tests and functional code inside an isolated, ephemeral container to check for malicious side-effects or unexpected runtime behavior.
How to verify, sanitize, and secure AI-generated code
The safest way to use code generated by a large language model (LLM) is to treat it like a contribution from an untrusted external developer: useful, potentially correct, but not safe to execute or deploy until it has passed independent checks.
A strong workflow combines automated security scanning, testing, input and output validation, dependency checks, sandboxing, and human review. This aligns with the NIST Secure Software Development Framework (SSDF), which integrates security practices throughout the software development lifecycle, and OWASP's guidance on insecure handling of LLM outputs.
1. The recommended security workflow
1. Capture and isolate generated code
Use a branch, disposable workspace, or restricted container. Do not expose production credentials.
Run unit, integration, negative, fuzz, and security tests with limited privileges.
4. Review and enforce gates
Require a human review, passing CI checks, approved dependencies, and no unresolved critical findings.
5. Release with controls
Deploy least-privilege builds, monitor behavior, and retain scan results and provenance.
The key principle is to use multiple independent controls. A clean static-analysis report does not prove that code is correct or secure, and passing tests do not prove the absence of vulnerabilities.
2. Tools to use at each stage
These are established options; the best combination depends on your language, hosting environment, and risk level.
Static application security testing (SAST)
Find vulnerable code without running the application.
CodeQL — analyzes data flows and patterns such as injection vulnerabilities.
Semgrep — customizable rules for insecure functions, unsafe patterns, and organizational policies.
A practical starting stack for a GitHub-hosted application is CodeQL or Semgrep, Trivy, secret scanning, your language's test and lint tools, and a dynamic scanner appropriate to the application. You don't need every product listed; choose complementary coverage rather than duplicating the same checks.
3. What to verify and sanitize
Sanitization is not simply deleting suspicious strings from generated code. It means enforcing the right security controls at every boundary where data enters, gets interpreted, or triggers an action.
Canonicalize paths and enforce allowed directories
Unsafe deserialization
OWASP specifically identifies insecure output handling as an LLM application risk. If a model produces code, HTML, SQL, shell commands, or tool instructions, validate the output for its intended context before use.
Important: input validation is not a replacement for safe APIs. For example, filtering quotes out of an SQL string is less robust than using a parameterized query. Similarly, shell commands are safer when constructed using argument arrays and fixed executables instead of concatenating untrusted strings.
4. Automate checks in CI/CD
Run the security checks on every pull request, not just before a release. Here is a simple example for a repository using GitHub Actions and Trivy. It assumes Trivy is installed on the runner; in production, pin the tool version and verify the installation method.
yaml
name: Verify generated code
on:
pull_request:
push:
branches: [main]
permissions:
contents: read
jobs:
security:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Scan for secrets
run: |
docker run --rm \
-v "$PWD:/work" -w /work \
zricethezav/gitleaks:latest \
dir .
- name: Scan dependencies and secrets
run: trivy fs --scanners vuln,secret .
- name: Run tests
run: |
# Replace with your project's test command
npm ci
npm test
This is an illustrative starting point, not a complete hardened pipeline. The example uses a mutable container tag and an illustrative action reference; pin third-party actions and scanner images to verified immutable versions or digests in production. Install and invoke scanners consistently, avoid executing untrusted pull-request code with privileged credentials, and add language-specific static analysis and build steps. The Gitleaks and Trivy commands follow their documented scanning capabilities.
A production pipeline should enforce these gates:
Fail the build: confirmed critical or high-severity findings according to your risk policy, leaked credentials, failing tests, and prohibited dependencies.
Require review: new security-sensitive code, authentication or authorization changes, cryptography, and infrastructure changes.
Retest fixes: verify the original vulnerability is resolved and the change hasn't introduced a regression.
Retain evidence: scan reports, dependency manifests, test results, and review approvals.
This is especially important if an LLM can generate scripts, install packages, use tools, or modify files.
Restrict permissions
Run under a non-root user with no production credentials, no host filesystem access, and no Docker socket access.
Restrict network access
Disable outbound networking by default. Allow only explicitly approved destinations when a test genuinely requires them.
Limit resource consumption
Apply CPU, memory, process, file-size, and execution-time limits to prevent runaway or denial-of-service behavior.
Use disposable test data
Use synthetic data and ephemeral databases. Never point unreviewed generated code at production systems.
For higher-risk workloads, consider hardened containers, dedicated ephemeral virtual machines, or purpose-built sandboxing environments. A standard container alone is not a complete security boundary.
6. Use the LLM as a reviewer, not as the final authority
You can ask an LLM to review generated code for likely vulnerabilities, but its output should supplement deterministic tools and independent review.
Useful review prompts include:
“Trace every untrusted input to a sensitive operation, including database queries, shell execution, file access, and HTML rendering.”
“Identify missing authorization checks and explain how a user could bypass them.”
“List every new dependency, its purpose, and any unverified APIs or version assumptions.”
“Generate negative tests for malformed inputs, privilege escalation, path traversal, and injection.”
“Review this proposed patch for security regressions and behavior changes.”
Then validate the findings independently. LLMs can miss vulnerabilities, invent false positives, or propose fixes that introduce new bugs. Reviewers should also verify that the generated code satisfies the actual requirements, not just that it passes security tools.
7. A practical checklist for every AI-generated pull request
CI security gates, CodeQL or Semgrep, dependency scanning, ZAP, sandboxed tests
AI coding agent that executes commands
All of the above, plus strict tool permissions, network restrictions, isolated execution, and human approval
High-assurance or regulated software
Risk-based secure development lifecycle, threat modeling, independent security review, comprehensive test evidence, and documented release controls
For a formal framework, start with NIST SP 800-218 and its secure development practices. For risks specific to AI applications, consult the current OWASP GenAI Security Project guidance.
My recommendation: implement a pull-request pipeline that combines static analysis, secret scanning, dependency scanning, automated tests, and a mandatory review first. Add sandboxing before allowing an agent to execute code, and add fuzzing or dynamic testing wherever the application's attack surface warrants it. This provides a repeatable safety net without relying on the model to certify its own work.
Yes. The safest approach is to treat AI-generated code as untrusted third-party code until it passes the same—and preferably stronger—controls as human-written code. OWASP’s current guidance specifically recommends human ownership, automated security testing, dependency verification, sandboxing, and careful review of agent-generated changes.
A practical workflow
1. Generate in isolation
Have the AI work on a branch or disposable worktree.
For agentic tools that can execute commands, use a dev container/VM with restricted filesystem and network access.
Don't expose .env files, SSH keys, cloud credentials, or production secrets to the agent.
2. Review the diff before running it
Look at every changed file, not just the code the AI was asked to write. Pay particular attention to:
package.json / build scripts
CI workflows
Dockerfiles
dependency/lock files
authentication and authorization code
database queries
cryptography
tests that were deleted or weakened
OWASP specifically calls out out-of-scope edits and test manipulation as recurring risks with coding agents.
3. Verify dependencies before installing
Don't blindly accept an AI-suggested package. Check that it actually exists, has a credible maintainer history, and isn't a suspicious newly registered/low-download package. Then run your normal SCA/dependency scanner. OWASP recommends tools such as npm audit, pip-audit, govulncheck, and cargo audit, as applicable.
4. Run multiple automated security layers
Check
What it catches
Examples
SAST
Vulnerable code patterns
Semgrep, CodeQL
SCA
Vulnerable/transitive dependencies
Snyk, Dependabot, OSV tools
OWASP recommends running SAST, SCA, and secret scanning on every PR; its AI-specific guidance additionally recommends IaC/DAST/IAST where appropriate.
Tools I'd consider
Semgrep is a particularly good fit for an AI-code gate because it can run in the IDE, pre-commit, and CI, and supports static analysis and security rules.
Snyk is useful when you want SAST + dependency/SCA analysis + secrets and IDE/PR integration in one security workflow. Its current tooling explicitly supports scanning AI-generated code as it is written.
If you're already on GitHub, GitHub Code Security/CodeQL + Dependabot + secret scanning gives you a strong baseline. GitHub's Copilot cloud agent also performs security validation and a second code-review pass, but that should complement—not replace—your independent security controls.
AI generates code
↓
isolated branch / container
↓
human reviews raw diff
↓
dependency verification
↓
SAST ──┐
SCA ───┤
Secrets┤
IaC ───┤──→ CI security gate
Tests ─┤
DAST ──┘
↓
human security/code review
↓
merge
↓
normal production monitoring
The important distinction is “AI review” versus “independent validation.” If the same model generates the code, writes the tests, and then says the code is secure, you've created a correlated failure mode. OWASP explicitly recommends adversarial/negative tests and human review rather than treating a passing AI-generated test suite as proof of security.
One extra control that's easy to overlook
For agentic coding, the agent itself is part of your attack surface. Repository issues, PR comments, README files, dependency documentation, and web pages can contain indirect prompt injections that influence an agent. Keep agent permissions narrow, restrict network egress, and don't give a coding agent production credentials.
If I were setting this up for a typical GitHub project, my minimum viable stack would be isolated coding environment + protected PR + Semgrep/CodeQL + dependency scanning + Gitleaks/secret scanning + tests + mandatory human review, with stricter two-person review for authentication, authorization, CI/CD, and deployment changes.
Integrating AI-generated code safely requires treating Large Language Model (LLM) outputs like code from an untrusted junior developer: functional, fast, but frequently optimized for completion over secure defaults or architectural cohesion.[](https://www.arnica.io/blog/ai-code-security-complete-guide)
A robust workflow combines pre-generation guardrails, automated pipeline validation , and human-in-the-loop review.
Step-by-Step Secure Integration Workflow
Pre-Generation / IDE Guardrails
Feed context-aware prompts by scoping your AI assistant (e.g., using .cursorrules or custom instructions in GitHub Copilot or Cursor ) with explicit coding standards, framework constraints, and forbidden patterns (like hardcoded secrets or raw SQL queries).
Utilize frameworks that enforce secure-by-default rules during generation.[]()
If you're using large language models to generate code, the safest approach is to treat every generated change as untrusted code until it passes independent verification. Don't rely on the model to certify its own output, explain why it is secure, or write tests that are sufficient to prove its correctness.
I'd build a layered pipeline that combines static analysis, automated testing, dependency and secret scanning, sandboxed execution, and human review for high-risk changes.
The key principle is defense in depth: no single scanner, test suite, or AI reviewer can guarantee that generated code is safe.
1. Implement this verification pipeline
1. Capture generated changes
Diff review, scope limits, file and permission restrictions
2. Run deterministic security checks
Formatting, linting, SAST, secrets, dependencies, IaC and configuration scans
3. Test in isolation
Unit, integration, property-based, fuzz and regression tests in a sandbox
4. Review and approve
Human review, risk-based approval, independent security review
5. Release with safeguards
Reproducible builds, artifact provenance, least privilege, monitoring and rollback
This fits the general secure-development approach in NIST's Secure Software Development Framework, which integrates security practices across the software lifecycle rather than treating security as a final scan.
2. Which tools to implement
Recommended starting stack; choose equivalents that fit your language, repository host and deployment platform.
Static analysis (SAST)
Find injection flaws, unsafe APIs, missing authorization checks and insecure patterns before execution.
Also add infrastructure-as-code and container scanners if the model generates Terraform, Kubernetes manifests, Dockerfiles or CI workflows. Examples include Checkov, Trivy and Hadolint.
3. Sanitize output at the correct boundary
A common mistake is to assume that generated code can be made safe by removing suspicious strings. Sanitization must depend on how the output will be used. Prefer validation and safe APIs over ad hoc string replacement.
Generated output or input
Controls to implement
SQL queries
Parameterized queries; never concatenate untrusted values into SQL
HTML and browser content
Context-aware output encoding; safe DOM APIs; sanitize HTML with a maintained allowlist-based sanitizer if HTML must be accepted
Shell commands
Avoid shell interpretation; use argument arrays and strict argument validation
File paths
Resolve against an allowed root; reject traversal and enforce filesystem permissions
URLs and network requests
Allowlist schemes, hosts and ports; block private/internal destinations where appropriate to mitigate SSRF
For example, prefer this Python pattern:
python
# Safer: values are bound separately from SQL syntax
cursor.execute(
"SELECT * FROM users WHERE email = %s",
(email,)
)
The placeholder syntax varies by database driver. The important part is that email is bound as a value, not interpolated into the SQL string.
For AI-generated code, have tests explicitly exercise malicious inputs, unexpected types, empty values, oversized payloads, path traversal attempts and authorization failures.
4. Secure the LLM agent itself
Scanning generated code is only half the problem. If your model can execute shell commands, install dependencies, read secrets or modify CI configuration, the generation workflow itself becomes an attack surface.
OWASP's current Secure Coding with AI Cheat Sheet specifically addresses agent permissions, indirect prompt injection, dependency risks and fabricated tests.
Implement these controls:
Sandbox execution: use ephemeral containers or VMs, with no production credentials, restricted filesystem access, CPU/memory limits and network egress controls.
Least privilege: give the agent write access only to the task's working tree. Require approval for deployments, publishing packages, changing permissions or accessing sensitive services.
Protect secrets: exclude .env, private keys, cloud credentials and sensitive directories from model context. Use short-lived, scoped credentials where access is necessary.
Defend against prompt injection: treat issue descriptions, READMEs, webpages, logs and tool responses as untrusted data, not trusted instructions.
Restrict tools: allowlist shell commands and approved MCP servers; validate tool arguments and require approval for dangerous operations.
Protect security configuration: require explicit review of changes to CI workflows, dependency lockfiles, agent instruction files, test suites and build scripts.
A particularly important rule: don't let the same agent both write security-critical code and independently certify it as secure. Use deterministic checks and independent review.
5. Make CI enforce the security gates
A pull request should not merge just because the build succeeds. Define mandatory checks and enforce them in your repository's branch protection or rulesets.
Here's a language-agnostic policy you can adapt:
Example CI pipeline
This is an illustrative GitHub Actions-style workflow. Pin third-party actions to reviewed commit SHAs, use least-privilege job permissions, and adapt the commands to your stack.
Replace the placeholder SHA before using this workflow. In a hardened setup, execute untrusted tests and agent-generated build steps in a separate, isolated job with no write tokens or deployment secrets. Avoid allowing generated code to control its own security gates.
6. Add an independent security-review layer
An LLM can be useful as a secondary reviewer, particularly when asked to inspect a diff for specific vulnerability classes. It should not be the only reviewer.
Give the reviewer a structured checklist:
Can user-controlled data reach SQL, shell commands, HTML rendering or file operations?
Are authentication and authorization checks enforced on the server?
Are secrets, sensitive logs or personal data exposed?
Can inputs cause resource exhaustion, race conditions or unsafe deserialization?
Were existing tests weakened, skipped or deleted?
Were unexpected files, dependencies, permissions or network access added?
Do the tests establish expected behavior independently of the implementation?
For higher-risk code, supplement this with manual threat modeling, targeted penetration testing and independent review by a security engineer. Consider using a different model for secondary review, but remember that model diversity reduces some shared blind spots; it doesn't establish correctness.
7. Prioritize controls by risk
Not every generated change needs the same level of scrutiny.
Adversarial tests, independent human review, strict merge gates
Critical
Deployment automation, privileged infrastructure, production access
8. A practical implementation plan
If you're starting from scratch, I'd implement these in order:
First: establish hard merge gates. Add secret scanning, SAST, dependency auditing and tests to every pull request. Make the checks required.
Second: isolate generation and execution. Use disposable workspaces, restrict filesystem and network access, and remove production credentials from the agent's environment.
Third: harden code review. Flag out-of-scope edits, changes to tests, dependency changes and modifications to build or CI configuration. Require a human to approve sensitive changes.
Fourth: add adversarial verification. Maintain security regression tests, fuzz important parsers and endpoints, and test authentication, authorization and input-handling boundaries.
Fifth: measure effectiveness. Track vulnerabilities escaping into production, time to remediate, false-positive rates, security-gate bypasses and the proportion of changes reviewed.
My recommendation: start with a required CI pipeline, sandboxed agent execution and strict secret/permission controls. These provide more dependable protection than adding another AI reviewer alone.
To tailor this into a concrete implementation, what language and environment are you using most?
The strongest match is Semgrep Guardian, from Semgrep. It is specifically designed to scan AI-generated code as developers write it, detect vulnerabilities, malicious packages, and hardcoded secrets, and help fix issues directly in the IDE. It integrates with tools including Cursor, Claude Code, GitHub Copilot, VS Code, and Windsurf.
Leading providers to evaluate
1. Semgrep Guardian — best fit for AI coding agents
Scans code when an AI agent writes it, with protections for vulnerable code, secrets, and malicious dependencies.
Start by evaluating Semgrep Guardian if your developers use AI coding agents such as Cursor, Claude Code, or Codex and you want vulnerabilities caught at the moment code is generated. Evaluate Snyk as well if you need broader IDE and repository scanning.
Whichever provider you choose, verify that it scans every AI-generated change, catches secrets and vulnerable dependencies, supports your team's actual IDEs, and can enforce policies rather than merely report findings.
Top providers offering real-time or shift-left IDE-level security scanning include:
Snyk Code : Known for ultra-fast, real-time static application security testing (SAST) directly inside developer IDEs (like VS Code, JetBrains) as code is written or generated.[]()
Untrusted serialized objects or data
Safe data formats, schema validation, restricted types
Never let AI code bypass standard static analysis. Every commit containing AI contributions must pass automated checks before human review.
Run SAST (Static Application Security Testing) to catch injection flaws, memory leaks, and insecure cryptography.
Run SCA (Software Composition Analysis) specifically to check for AI "slopsquatting" or hallucinated package dependencies (where an AI invents a non-existent package name that an attacker could register maliciously).[](https://developer.harness.io/roadmap/sast-and-sca)
Targeted Human Code Review
Reviewers must pay special attention to authorization, boundary conditions, and tenant isolation—areas where LLMs lack the wider architectural context of your system.
Essential Security Tools for Validation & Cleaning
Static Analysis (SAST)
Traditional/Deterministic: Tools like Snyk Code or Checkmarx scan code patterns instantly for hardcoded credentials, known vulnerabilities, and bad syntax.
AI-Aware/Reasoning Security: Platforms like Kodem Security or OX Security analyze runtime behavior and semantic logic to catch deeper authorization issues (like Insecure Direct Object References) that simple rule-based linters miss.[](https://www.kodemsecurity.com/resources/top-5-ai-sast-tools)
Dependency & Supply Chain Scanners (SCA)
Use tools like Socket.dev or Snyk Open Source to verify that any third-party libraries or modules recommended by the AI actually exist, are actively maintained, and are free from malware.
Secret Detection
Integrate tools like GitGuardian or TruffleHog into your pre-commit hooks and CI pipeline. LLMs have a bad habit of generating realistic-looking dummy API keys or tokens that accidentally match real production secrets.
If you'd like, let me know:
What languages and frameworks your codebase uses
Which AI tools your team currently relies on (e.g., Copilot, ChatGPT, Claude)
I can help tailor a specific CI/CD pipeline configuration or linting rule set for your stack.
Semgrep Code : Offers rapid, highly customizable rule-based scanning that integrates smoothly into both the IDE and CI/CD pipelines to flag insecure patterns instantly.[](https://www.augmentcode.com/tools/best-ai-sast-tools)
