All the tests passed. What did that prove? · slopware 101

slopware 101 · step 16 of 35

All the tests passed. What did that prove?

by · 4 min read

The migration rehearsal gave us a concrete comparison: existing work should still be readable after the change. We can check that by hand, but repeating every check after every edit soon becomes difficult. A test is a small program that runs code and compares what happened with what should have happened. Its result depends on the comparison we chose.

A test that cannot fail

In a disposable folder, save the following as permissions.mjs. The .mjs extension tells Node to read an ES module; this example needs Node installed and no third-party packages. The function models one ownership rule, not a complete authentication system.

export function canEdit(user, note) {
  return note.ownerId === user.id;
}

Save this as permissions.test.mjs beside it, using the test runner built into Node. Run it with node --test permissions.test.mjs.

import { test } from 'node:test';
import assert from 'node:assert/strict';
import { canEdit } from './permissions.mjs';

test('the owner can edit', () => {
  assert.equal(canEdit({ id: 1 }, { ownerId: 1 }), true);
});

It passes. Now replace the body of canEdit with return true and run it again. It still passes. The test only checks the yes, and the dangerous direction for a permission check is the no.

The negative case

test('someone else cannot edit', () => {
  assert.equal(canEdit({ id: 2 }, { ownerId: 1 }), false);
});

Append this second test to permissions.test.mjs. Now return true fails. A negative case checks that the code refuses what it should refuse. For anything about permissions, validation or limits, the refusals are where the meaning is, and they are the cases a quick try in the browser almost never exercises.

An expected result from somewhere else

A test that computes its expected value with the code it is testing only checks that the code agrees with itself.

assert.equal(cartTotal(items), cartTotal(items));

Nobody writes it that plainly. The quieter version copies the implementation's logic into the test to work out the expected number, and if the logic is wrong, the test is wrong in the same way. The expected value should come from an independent source: a number worked out by hand, or an example from the specification. Two items at 120 and 35 with a ten percent discount should cost 139.5, and the test should say 139.5.

Mocks

A mock stands in for a real dependency, such as a database or an outside API, so a test can run fast and alone. That is useful. A test where everything around the function is mocked, though, checks the function against your assumptions about everything else. If you assumed the database returns ids as numbers and it returns them as strings, the mock and the bug agree, and the suite stays green. Keep some tests that touch real things: a real dev database, a real request to the running server.

Does the test notice?

The most useful check on a test is to break the code it covers on purpose. Flip a comparison, delete a condition, return early, and run the suite. If it stays green, that line was not protected. Do the same with a fix: take the fix out and run the test that was written for it. If it still passes, the test was never checking the fix.

Restore canEdit to the ownership comparison and run both tests again before continuing. Both should pass with the correct implementation.

CI

Continuous integration runs your tests on every push and pull request, on a machine that is not yours. It catches the things your laptop forgives: a lockfile out of date, a path whose capitalisation only works on macOS, a variable that exists only in your .env.

For the dependency-free example above, a GitHub Actions workflow can run the same command. Save this in .github/workflows/test.yml in a repository containing both .mjs files at its root.

name: test
on: [push, pull_request]
permissions:
  contents: read
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-node@v7
        with:
          node-version: 24
          package-manager-cache: false
      - run: node --test permissions.test.mjs

A project with npm dependencies should install its committed lockfile with npm ci before running its test command. The small example needs no install because it uses Node's built-in modules. A workflow reports a result; repository rules determine whether a failing result blocks merging.

If tests need secrets or a database, give CI its own, never prod's.

What green means

Green means the checks you wrote passed, on this code, in this environment. It does not mean the app is correct in the cases nobody wrote down, and it does not mean the app is safe. A suite is an argument, and it is only as strong as the cases in it.

A better question than whether everything passed is this one. For the thing you are most afraid of, a stranger editing someone else's note, a price changed in the browser, a task lost on refresh, is there a test that would turn red?

Pick the test you trust most in your project, break the code it covers on purpose, and run it. If it stays green, you have found where to write the next one.

Further reading: GitHub: checkout action, GitHub: Node setup.