GitHub Actions 101: Tests on Every Push

Production Ready — Part 11 of 30

---

The Incident That Rewrote How I Teach CI/CD

March 14, 2025. It's a Friday — because of course it is — and security researchers at StepSecurity notice something wrong with tj-actions/changed-files, a GitHub Action used by over 23,000 repositories. Someone had compromised the maintainer's bot account, modified the action's code, and then — here's the clever part — retroactively updated every version tag to point to the malicious commit. Every version. Going back.

The malicious code was elegant in its nastiness: it ran a Python script that dumped secrets from the GitHub Actions runner's memory directly into the workflow logs. AWS access keys. GitHub Personal Access Tokens. npm tokens. Private RSA keys. All of it — sitting in plain text in public build logs, visible to anyone who knew to look.

The vulnerability was assigned CVE-2025-30066 with a CVSS score of 8.6. Wiz Threat Research identified dozens of affected enterprise repositories with live secrets already exposed.

Here's what saved some teams: they had pinned their Actions to commit hashes, not floating version tags. No hash change, no compromise. Others had structured their pipelines so that tests had to pass before secrets were ever injected into the environment. Their blast radius was zero.

That's the lesson. GitHub Actions isn't just about automation — it's a security surface. And when you build your CI pipeline correctly, "tests on every push" is the mechanism that keeps a bad Friday from becoming your worst quarter.

Let's build it right.

---

Why This Matters More Than You Think

Before we touch any YAML, let's size the actual problem.

In 2025 alone, developers used 11.5 billion GitHub Actions minutes in public and open source projects — up 35% year over year. The platform now powers 71 million jobs per day. That's not a niche DevOps tool anymore. That's the default CI layer for most teams building today.

And yet most vibe coders I work with have the same setup: a workflow someone copy-pasted from a Stack Overflow answer that runs npm install and maybe deploys on push to main. No tests. No pinned dependencies. No gates.

The CI pipeline isn't just about automation speed (though GitHub Actions cuts deployment times by an average of 30%). It's the one place in your entire stack where you get a clean, reproducible environment to validate that your code actually works before it touches anything real. Skip that, and you're flying blind at 400 knots.

---

Anatomy of a GitHub Actions Workflow

Let's start with a real workflow file — not the toy example from the docs, but something that actually does useful work.

Create a file at .github/workflows/ci.yml in your project root:

name: CI

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    
    steps:
      - name: Checkout code
        uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683  # v4.2.2
        
      - name: Set up Node.js
        uses: actions/setup-node@39370e3970a6d050c480ffad4ff0ed4d3fdee5af  # v4.1.0
        with:
          node-version: '20'
          cache: 'npm'
          
      - name: Install dependencies
        run: npm ci
        
      - name: Run linter
        run: npm run lint
        
      - name: Run tests
        run: npm test
        
      - name: Run type check
        run: npm run typecheck

Notice the long commit hash comments after each uses: line. That's the hash-pinning that would have protected teams during the March 2025 incident. Floating tags like @v4 are convenient but they can be silently swapped to point at malicious code. Hashes can't.

---

The on Block: Trigger Strategically

The on block controls when your workflow fires. This is where most teams get sloppy:

# Too broad — runs on every branch for every event
on: push

# Better — targeted triggers
on:
  push:
    branches: [main, develop]
    paths-ignore:
      - '**.md'
      - 'docs/**'
  pull_request:
    branches: [main]
  workflow_dispatch:  # Manual trigger — always useful

The paths-ignore filter is a gift. If someone updates a README, you don't need to burn 3 minutes of compute running your full test suite. Save those minutes for code that could actually break things.

---

Matrix Testing: One Config, Every Environment

Here's where GitHub Actions earns its place in your stack. If you're shipping something that needs to work across multiple Node or Python versions, a matrix strategy runs them in parallel for free:

jobs:
  test:
    runs-on: ubuntu-latest
    
    strategy:
      matrix:
        node-version: [18, 20, 22]
        
    steps:
      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
      
      - name: Set up Node ${{ matrix.node-version }}
        uses: actions/setup-node@39370e3970a6d050c480ffad4ff0ed4d3fdee5af
        with:
          node-version: ${{ matrix.node-version }}
          
      - run: npm ci
      - run: npm test

Three jobs run in parallel. If your code breaks on Node 18 but passes on 22, you find out in the pull request — not from a user on the older runtime.

---

A Python Project Setup

For the Python side of your stack — FastAPI, Flask, scripts, whatever — the pattern is the same:

name: Python CI

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    
    steps:
      - uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
      
      - name: Set up Python
        uses: actions/setup-python@0b93645e9fea7318ecaed2b359559ac225c90a2b  # v5.3.0
        with:
          python-version: '3.12'
          
      - name: Install dependencies
        run: |
          pip install --upgrade pip
          pip install -r requirements.txt
          pip install pytest pytest-cov
          
      - name: Run tests with coverage
        run: |
          pytest --cov=. --cov-report=xml --cov-fail-under=70
          
      - name: Upload coverage report
        uses: actions/upload-artifact@65c4c4a1ddee5b72f698fdd19549f0f0fb45cf08  # v4.6.0
        with:
          name: coverage-report
          path: coverage.xml

The --cov-fail-under=70 flag is opinionated but important: if test coverage drops below 70%, the job fails. The pipeline becomes the enforcer so you don't have to be.

---

Secrets: The Right Way

Every CI pipeline eventually needs to talk to external services — a database, a staging API, a deployment target. This is where teams create disasters.

Never, ever do this:

# DO NOT DO THIS
- name: Deploy
  run: deploy.sh $MY_API_KEY  # key visible in logs forever
  env:
    MY_API_KEY: sk-abc123realkey  # hardcoded in version control

Do this instead:

- name: Deploy
  run: ./deploy.sh
  env:
    API_KEY: ${{ secrets.PRODUCTION_API_KEY }}
    DATABASE_URL: ${{ secrets.DATABASE_URL }}

Store your secrets in Settings → Secrets and variables → Actions in your GitHub repo. They're encrypted at rest and masked in logs. And critically: scope them. Use repository-level secrets for repo-specific values and environment-level secrets (Settings → Environments) to gate production secrets behind a required reviewer approval.

---

Caching Dependencies: Don't Pay Twice

The fastest way to cut your CI time in half is caching node_modules or your pip cache between runs. GitHub Actions makes this straightforward:

- name: Cache npm dependencies
  uses: actions/cache@1bd1e32a3bdc45362d1e726936510720a7c6158d  # v4.2.0
  with:
    path: ~/.npm
    key: ${{ runner.os }}-npm-${{ hashFiles('**/package-lock.json') }}
    restore-keys: |
      ${{ runner.os }}-npm-

The hashFiles function is the key insight here: the cache key includes a hash of your lockfile. If package-lock.json changes, the cache is busted and dependencies are reinstalled fresh. If it hasn't changed, you skip installation entirely. For a project with 200 dependencies, this can shave 90 seconds off every run.

---

Block Bad Merges at the Source

None of this matters if developers can push directly to main and skip the whole pipeline. Lock it down.

In GitHub: Settings → Branches → Add branch protection rule for main:

• ✅ Require a pull request before merging
• ✅ Require status checks to pass before merging → select your CI workflow
• ✅ Require branches to be up to date before merging
• ✅ Do not allow bypassing the above settings (yes, even for admins)

Now the pipeline is the gate. Nobody ships broken code to main — not by accident, and not by corner-cutting on a deadline.

---

Keeping Your Actions Secure

The March 2025 incident was a supply chain attack — your dependencies' dependencies can be compromised. Three practices that would have stopped it:

1. Pin to commit SHA, not floating tags:

# Vulnerable — tag can be silently redirected
uses: actions/checkout@v4

# Safe — this exact commit, immutable
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683

2. Use Dependabot to keep Actions updated:

# .github/dependabot.yml
version: 2
updates:
  - package-ecosystem: github-actions
    directory: /
    schedule:
      interval: weekly

Dependabot will open PRs to bump your pinned hashes when Actions release new versions. You get security updates without manual tracking.

3. Minimum permissions — always:

jobs:
  test:
    permissions:
      contents: read  # Only what this job actually needs
    runs-on: ubuntu-latest

If a job only reads code, it should only have contents: read. If a compromised Action runs in a job with write permissions to everything, the damage is much larger.

---

Your Production Checklist

Workflow Setup

• ☐ .github/workflows/ci.yml exists and triggers on push + pull_request
• ☐ Workflow runs linting, tests, and type checking (if applicable)
• ☐ paths-ignore filters out documentation-only changes
• ☐ workflow_dispatch is enabled for manual runs

Security Hygiene

• ☐ All uses: references are pinned to full commit SHAs
• ☐ Dependabot is configured to update Actions weekly
• ☐ Secrets are stored in GitHub Secrets — zero hardcoded values
• ☐ Jobs use minimum required permissions
• ☐ Environment secrets gate production deployments behind reviewers

Branch Protection

• ☐ main requires PRs before merging
• ☐ CI status checks must pass before merging
• ☐ Admin bypass is disabled

Performance

• ☐ Dependency caching is configured with hashFiles keying
• ☐ Matrix strategy used where multiple environments matter
• ☐ Coverage threshold enforced with --cov-fail-under or equivalent

---

Ask The Guild

This week's community prompt: What's the most embarrassing thing that would have been caught by a CI test running on every push? We all have one. Drop it in the thread — your pain is someone else's lesson. Bonus points if it made it to production first.

---

Sources: CVE-2025-30066 — tj-actions/changed-files supply chain attack (The Hacker News) | Wiz Threat Research analysis | GitHub Actions: 11.5 billion minutes in 2025, up 35% YoY (GitHub Blog) | GitHub Statistics 2025 — ElectroIQ