BeforeShip

Your AI coding agent says the app is done. BeforeShip checks if it is safe to deploy.

BeforeShip is an open-source launch gate for solo founders and vibe coders building apps with Cursor, Claude Code, Codex, Replit, Lovable, Bolt, v0, and other AI coding tools.

It scans AI-built SaaS apps for the production basics coding agents often skip:

• auth and authorization gaps
• exposed secrets and unsafe env usage
• Supabase RLS/database isolation issues
• Stripe/Razorpay webhook safety
• rate limits, CORS, validation, and security headers
• missing smoke tests and deployment readiness
• risky AI-agent diffs like deleted tests, huge rewrites, or auth/payment changes without verification

The goal is simple:

Run one command before deploy. Know if your vibe-coded app is safe to ship.

---

Status

This repository is starting as a public product spec and research-backed roadmap for an OpenAI Hackathon project.

The first executable slice is now in progress:

npm test
npm run scan -- --no-report
node packages/cli/bin/beforeship.js scan /path/to/app
node packages/cli/bin/beforeship.js loop /path/to/app --max-iterations 3

Current checks:

• basic stack detection for Next.js, Supabase, Stripe/Razorpay, Vercel, and env files
• committed env-file warning
• secret-like token detection for OpenAI, Supabase JWT-style keys, Stripe, and Razorpay
• Next.js API route auth-guard and trusted-user-id heuristics
• Stripe/Razorpay webhook signature and idempotency heuristics
• Supabase service-role, RLS, owner-policy, and public-bucket heuristics
• API safety checks for wildcard CORS, auth-route rate limits, and input validation
• Next.js deploy-readiness checks for tests, health routes, and .env.example
• score/verdict engine
• markdown and JSON reports under .beforeship/
• bounded verification loop with named terminal states and loop memory

Demo fixture:

node packages/cli/bin/beforeship.js scan examples/flawed-next-supabase-app --no-report

The fixture should return DO NOT SHIP and show the launch blockers BeforeShip is designed to catch.

Repository-level scans respect .beforeshipignore, which excludes this intentionally flawed fixture.

The intended MVP is:

CLI first + MCP server + optional GitHub Action

Initial target stack:

Next.js + Supabase + Stripe/Razorpay + Vercel

---

Why BeforeShip?

AI coding tools make it easy to create working apps, but hard to verify production readiness.

A vibe-coded app can look 90% done while missing the dangerous invisible pieces:

• payment webhook signature verification
• database row-level security
• server-side auth checks
• safe API key handling
• rate limiting
• rollback and smoke tests
• deployment env separation

BeforeShip is the missing pre-deploy gate between:

"It works locally"

and:

"It is safe to launch"

---

Product thesis

Coding agents optimize for making code run. BeforeShip checks whether the generated app is safe to ship.

BeforeShip is not meant to replace Semgrep, Snyk, CodeQL, Gitleaks, or human review.

It is meant to orchestrate and explain the checks solo founders actually need before deploying an AI-built SaaS app.

---

Planned CLI

npx beforeship scan
npx beforeship scan --url https://myapp.com
npx beforeship report
npx beforeship fix-prompts
npx beforeship mcp

Example output:

BeforeShip score: 42/100
Verdict: DO NOT SHIP

Critical:
1. Razorpay webhook does not verify signature
2. Supabase service role key appears in client code
3. /api/admin/users has no auth guard

High:
4. No rate limiting on login
5. CORS allows all origins
6. No smoke tests found

---

Planned MCP tools

BeforeShip should expose a small number of high-leverage MCP tools, not a huge overloaded tool list.

before_ship_scan
before_ship_explain
before_ship_fix_plan
before_ship_gate_deploy

Optional:

before_ship_agent_policy

---

Planned checks

Secrets

• .env committed
• API keys in frontend bundle
• NEXT_PUBLIC_ misuse
• Supabase service role exposed
• OpenAI/Anthropic/GitHub keys exposed
• Stripe/Razorpay secrets exposed
• Cursor/agent context includes secret files

Auth

• API routes without session/auth guard
• admin pages without server-side protection
• user ID trusted from body/query
• IDOR patterns
• auth middleware exists but is unused
• role checks performed client-side only

Supabase / database

• RLS disabled
• tables with user data and no owner policy
• service role used outside server
• public anon key misuse
• migration drift
• unsafe storage buckets

Payments

• Stripe/Razorpay webhook signature missing
• missing idempotency
• subscription status trusted from frontend
• price/plan selected client-side
• billing country/currency not server-validated
• entitlement table missing
• payment success page grants access without verified webhook

API safety

• wildcard CORS
• no rate limiting
• no input validation
• no CSRF for cookie auth flows
• verbose errors
• insecure file uploads
• missing security headers
• SSRF patterns

Deploy readiness

• no health endpoint
• no smoke tests
• no rollback note
• missing env vars
• Vercel serverless timeout risks
• critical routes lack logging
• no staging/prod separation

AI-agent-specific checks

• huge AI-generated files
• repeated full-file rewrites
• deleted tests
• new dependency added without reason
• changed auth/payment/db files without tests
• destructive scripts/migrations
• no AGENTS.md / CLAUDE.md
• agent touched files outside declared scope
• diff contains temporary auth bypasses
• tests modified to fit broken code

---

Planned report files

.beforeship/
  config.yml
  launch-report.md
  findings.json
  fix-prompts.md
  evidence/

Example finding:

{
  "id": "payment.webhook.signature_missing",
  "severity": "critical",
  "file": "web/src/app/api/razorpay/webhook/route.ts",
  "title": "Razorpay webhook does not verify signature",
  "why_it_matters": "Anyone can fake a payment success event and unlock access.",
  "fix_prompt": "Add Razorpay webhook signature verification using RAZORPAY_WEBHOOK_SECRET. Reject unsigned or invalid payloads. Add a regression test."
}

---

How OpenAI fits

BeforeShip should use deterministic checks first and OpenAI reasoning second.

OpenAI can help with:

• stack-aware risk triage
• founder-friendly explanations
• fix prompt generation
• test plan generation
• false-positive reduction
• deploy-gate reasoning for AI agents

It should not rely only on LLM judgement for security.

---

Repository structure

Current repository is documentation-first.

Planned implementation shape:

beforeship/
  packages/
    cli/
    core/
    mcp/
    github-action/
  docs/
    research.md
    product-spec.md
    roadmap.md
  examples/
    flawed-next-supabase-app/

---

Contributing

BeforeShip is intended to be open source.

Good first contribution areas:

• add stack-specific checks
• add examples of common vibe-coded app failures
• improve scoring rules
• add tests for detectors
• add MCP client setup docs
• add GitHub Action workflow
• improve report UX

See CONTRIBUTING.md.

---

License

MIT — see LICENSE.