Preview Deployments: Test Before You Break Production

Production Ready — Part 14 of 30

---

The Deploy That Went Live Too Soon

A mid-size SaaS team I know was shipping fast. They had a Vercel project, a main branch, and a simple rule: merge to main, it goes to production. That's it. No staging, no preview, no gate.

One Friday afternoon, a developer merged a PR that refactored the checkout flow. The CI tests passed. The code looked clean. The PR got two thumbs-up from teammates who reviewed the diff.

What nobody caught in the diff: the refactor swapped the order of two environment variable lookups. In local dev, both vars were set, so the tests passed. In production, one of them wasn't — it was a secret only available in the production environment. The checkout form threw a 500 for six hours before anyone noticed. The on-call engineer was at a wedding.

Could this have been prevented? Absolutely. A preview deployment against a staging environment — where they could have actually clicked through the checkout before merging — would have exposed the problem in sixty seconds.

This is why preview deployments exist. Not as a bureaucratic gate, but as a last line of defense against the thing you didn't think to write a test for.

---

What Preview Deployments Actually Are

A preview deployment is an ephemeral, fully functional version of your application that's automatically spun up when you open a pull request. It lives at its own unique URL — something like my-app-git-feature-auth-fix-myteam.vercel.app — and it dies when the PR closes.

Platforms like Vercel, Netlify, Railway, Render, and Fly.io all offer some version of this. The core promise is the same everywhere:

Every PR gets its own live environment. You can test it, click through it, share it with a stakeholder, and run automated tests against it — before a single line merges to main.

For vibe coders especially, preview deployments close the gap between "it works on my machine" and "it works in production." They're the environment-parity check you forgot to write.

---

The Security Trap Nobody Warns You About

Before we get into setup, I need to talk about something that burned real teams in 2025.

In July 2025, a critical vulnerability was disclosed in Dokploy (CVE-2025-53825): preview deployments on public repositories would automatically build and deploy pull requests from any fork — including from strangers on the internet. The attacker's move was simple: fork the repo, open a PR with a malicious API route that prints process.env, wait for Dokploy to auto-deploy it, and visit the public URL. Every environment variable in the preview config — API keys, database credentials, third-party secrets — was now readable by the attacker. GitHub Security Advisory GHSA-h67g-mpq5-6ph5 documents the full details.

This wasn't a Dokploy-specific mistake. The same pattern applies to any platform that auto-builds external PRs with access to your environment variables.

Then in December 2025, when the React2Shell vulnerability (CVE-2025-66478) shook the Next.js ecosystem, Vercel's official guidance included a specific warning that production apps patched with newer Next.js versions were still at risk if their preview deployments ran older vulnerable code. Vercel explicitly told teams to "make sure that preview deployments and deployments from other environments are not used by external users without protection bypass first." Vercel's React2Shell security bulletin laid this out in detail.

The takeaway: preview deployments are only safe if you treat their security as seriously as production.

Rule 1: Never give preview deployments access to production secrets. Create a separate .env.preview (or a dedicated environment in your platform) with test credentials, sandbox API keys, and non-production database URLs.

Rule 2: Enable deployment protection. On Vercel, this means Standard Protection for all non-production deployments. On other platforms, require authentication to access preview URLs, or restrict by IP.

Rule 3: Control who can trigger previews. On public repos, require a maintainer to approve workflow runs from external contributors before the preview builds.

---

Setting Up Preview Deployments: The Practical Path

Option A: Vercel (Zero Config)

If you're on Vercel, previews are on by default. Every PR gets a deployment. The main thing you need to configure is environment separation.

In your Vercel project settings, go to Settings → Environment Variables. For each sensitive variable, assign it only to Production. Create separate preview-safe values and assign those to Preview.

# Vercel CLI — pull your current env config
vercel env pull .env.local

# Then review what's going to each environment:
vercel env ls

For protection, go to Settings → Deployment Protection and enable Standard Protection for preview deployments. This gates access behind Vercel Authentication — anyone visiting a preview URL will need to log in with a Vercel account.

For automated E2E testing against protected previews, Vercel generates a bypass secret:

// playwright.config.ts
const config: PlaywrightTestConfig = {
  use: {
    extraHTTPHeaders: {
      'x-vercel-protection-bypass': process.env.VERCEL_AUTOMATION_BYPASS_SECRET,
    },
  },
};

That VERCEL_AUTOMATION_BYPASS_SECRET is automatically injected as a system environment variable in your deployments — your tests can read it without you hardcoding anything. Vercel's protection bypass docs walk through the full setup.

Option B: GitHub Actions (Any Platform)

If you're not on Vercel, you can build the same workflow with GitHub Actions. The pattern is: build on PR open/sync, deploy to a preview environment, comment the URL back on the PR, clean up on close.

# .github/workflows/preview.yml
name: Preview Deployment

on:
  pull_request:
    types: [opened, synchronize, reopened]

jobs:
  deploy-preview:
    runs-on: ubuntu-latest
    permissions:
      pull-requests: write
      contents: read

    steps:
      - uses: actions/checkout@v4

      - name: Install dependencies
        run: npm ci

      - name: Build
        run: npm run build
        env:
          # Use preview-safe env vars, NOT production secrets
          DATABASE_URL: ${{ secrets.PREVIEW_DATABASE_URL }}
          API_KEY: ${{ secrets.PREVIEW_API_KEY }}

      - name: Deploy to preview
        id: deploy
        run: |
          # Your deploy command here (Fly.io, Railway, Render, etc.)
          PREVIEW_URL="https://pr-${{ github.event.pull_request.number }}.preview.example.com"
          echo "url=$PREVIEW_URL" >> $GITHUB_OUTPUT

      - name: Comment PR with preview URL
        uses: actions/github-script@v7
        with:
          script: |
            const url = '${{ steps.deploy.outputs.url }}';
            const comments = await github.rest.issues.listComments({
              owner: context.repo.owner,
              repo: context.repo.repo,
              issue_number: context.issue.number
            });
            const existing = comments.data.find(c =>
              c.user.type === 'Bot' && c.body.includes('Preview Deployment')
            );
            const body = `**Preview Deployment Ready**\n\n→ [Open Preview](${url})\n\nUpdated: ${new Date().toISOString()}`;
            if (existing) {
              await github.rest.issues.updateComment({
                owner: context.repo.owner,
                repo: context.repo.repo,
                comment_id: existing.id,
                body
              });
            } else {
              await github.rest.issues.createComment({
                owner: context.repo.owner,
                repo: context.repo.repo,
                issue_number: context.issue.number,
                body
              });
            }

And the cleanup job — don't skip this, orphaned preview environments accumulate costs:

  cleanup:
    if: github.event.action == 'closed'
    runs-on: ubuntu-latest
    steps:
      - name: Tear down preview environment
        run: |
          # Your teardown command here
          echo "Cleaning up PR-${{ github.event.pull_request.number }} environment"

---

What to Actually Test in a Preview

A preview deployment is only as useful as what you do with it. Here's the minimum bar I recommend:

1. Happy-path smoke test. Before marking any PR as ready for review, the author should click through the primary user flow in the preview. If you changed checkout, test checkout. If you changed login, test login. This takes two minutes and catches the class of bug that bit the SaaS team in the opening story.

2. Automated Playwright or Cypress tests against the preview URL. This is where you catch regressions. Wire your E2E suite to run on the preview deployment as part of the PR workflow.

// In your E2E test — reads the preview URL from env
const BASE_URL = process.env.PREVIEW_URL || 'http://localhost:3000';

test('checkout flow completes successfully', async ({ page }) => {
  await page.goto(`${BASE_URL}/checkout`);
  await page.fill('[name="email"]', 'test@example.com');
  await page.click('[type="submit"]');
  await expect(page).toHaveURL(/\/confirmation/);
});

3. Environment variable audit. Make it a PR checklist item: "Does this change require any new environment variables? If yes, are they added to the preview environment config?" This one question prevents an entire class of production outages.

4. Stakeholder review on visual changes. If the PR touches UI, share the preview URL with the PM or designer before merging. Preview URLs are better than screenshots — they're interactive.

---

The Secrets Sprawl Problem

In 2025, GitGuardian found 28.76 million new secrets exposed in public GitHub commits — a 34% increase over the previous year, driven largely by AI-generated code that developers commit without auditing. Security Ledger's coverage attributes much of the surge to AI tools that suggest working code with hardcoded credentials.

Preview deployments sit at the intersection of this problem. Because previews are "not real," developers treat them loosely. They hardcode a test API key in a config file, push it in a PR, and forget it's now in a public build log.

The fix is procedural and architectural:

• Never hardcode secrets in code, even for previews. Always pull from environment variables.
• Use short-lived, scoped credentials for preview environments. If your preview DB gets compromised, you want it to contain only test data.
• Rotate secrets after a PR closes if there's any chance a secret was exposed during the build.
• Scan your PRs with tools like GitGuardian or truffleHog before they deploy.

---

Preview Deployments Checklist

Before your next PR:

• ☐ Environment separation: Preview environment variables are distinct from production. No production credentials in preview config.
• ☐ Deployment protection enabled: Previews require authentication or IP restriction. They are not publicly accessible by default.
• ☐ Automated E2E tests wired to preview URL: Tests run against the actual preview deployment, not just localhost.
• ☐ Smoke test before review request: Author has manually clicked through the primary flow in the preview.
• ☐ Cleanup workflow configured: Preview environments are deleted when the PR closes.
• ☐ Fork PR protection: External contributors cannot trigger builds that access your environment variables without maintainer approval.
• ☐ Secret scanning: No hardcoded credentials in the PR diff or build output.
• ☐ New environment variables documented: Any new vars required by the PR are added to preview and production configs before merge.

---

Ask The Guild

Here's a question worth discussing with your team and the community:

What's in your preview environment that shouldn't be there?

Run vercel env ls (or the equivalent for your platform) and audit what your preview deployments have access to. Post in the Guild what you found — and what you changed. Bonus points if you catch something genuinely surprising. The most common answer I've heard is "our Stripe live-mode API key." Don't be that team.

Drop your findings and questions in the Production Ready thread.