Testing your program

A 6502 program can have a test suite. Not a simulation of one — the real ROM, the real BASIC, the real Kernal, booted and driven and asserted on, ten cases in about a second.

The method

Five rules. The BIOS's own suite is built on them, and so is everything below.

Boot once. Booting to the OK prompt costs about 330,000 emulated cycles. It's quick, and doing it per case is still the difference between a suite you run constantly and one you avoid.

Restore per case. Take a snapshot at the prompt and go back to it. About a millisecond, and exact — memory, registers, video, the card's changed sectors — so one case cannot leak into the next.

Wait, don't sleep. Every wait is a blocking call with a pattern and a timeout. sleep is how a suite becomes flaky: it is either too short on a busy machine or wasting your afternoon on a fast one.

Bound everything. Every send, every wait, and the machine itself get a timeout. A hung program should fail the suite, not hold it open.

Branch on exit codes. 0 fine, 2 timed out, 4 hit a breakpoint. Parsing console text for success is guesswork; the exit code is not.

A suite

Here is one, whole. Drop it in your project as test.sh:

#!/usr/bin/env bash
#
# Regression tests for a 6502 program.
#
#   ./test.sh              run every case in tests/
#   ./test.sh cases/       run every case in another directory
#
# A case is a .prg (loaded) or a .bas (typed in), with a sibling .expect file
# holding one `expect <pattern>` line per thing that must appear in the output.
#
# One machine, one boot, a snapshot restored per case. Nothing sleeps.

set -euo pipefail

cases=${1:-tests}
port=6510
state=$(mktemp)

# One machine for the whole run, on the PICOVDP (which boots BIOS 2.0), and with
# the clock pinned so a run here and a run on a build server land in the same
# place. Flow control is on unless --peer-rts ignore turns it off, which is what
# lets a long listing typed in arrive whole; don't turn it off here. It starts
# paused: the boot is over in a third of a second, and a
# machine left to run would print its prompt before anything was waiting for it.
# Its console goes to the bit bucket: the assertions below read the console
# through the debug server, so anything it echoes here is just noise.
6502 run --headless --quiet --vdp picovdp --pause --debug --debug-port "$port" \
  --rtc 2026-01-01T00:00:00 --timeout 300s >/dev/null &
emulator=$!
trap 'kill $emulator 2>/dev/null || true; rm -f "$state"' EXIT

until 6502 dbg info --port "$port" >/dev/null 2>&1; do sleep 0.1; done

# Boot to the prompt once and photograph the machine there. Restoring that is
# about a millisecond against the 330,000 cycles a boot costs — and it is
# exact, so one case cannot leak into the next.
6502 dbg wait --serial 'OK' --run turbo --timeout 30s --port "$port" >/dev/null
6502 dbg state save "$state" --port "$port" >/dev/null

failed=0

for case in "$cases"/*.prg "$cases"/*.bas; do
  [ -e "$case" ] || continue
  expect="${case%.*}.expect"
  [ -e "$expect" ] || { echo "FAIL $case — no $expect"; failed=1; continue; }

  6502 dbg state load "$state" --port "$port" >/dev/null
  6502 dbg run --port "$port" >/dev/null

  if [ "${case##*.}" = "prg" ]; then
    6502 dbg load program "$case" --port "$port" >/dev/null
  else
    # A stored program line prints nothing back, so each line waits for its own
    # echo rather than for the prompt.
    while IFS= read -r line || [ -n "$line" ]; do
      [ -n "$line" ] || continue
      6502 dbg send "$line\r" --wait "^${line%% *}" --timeout 20s --port "$port" >/dev/null
    done < "$case"
  fi

  output=$(6502 dbg send 'RUN\r' --wait 'OK' --timeout 30s --port "$port" | tr -d '\r')

  problems=()
  while read -r directive pattern; do
    [ "$directive" = "expect" ] || continue
    grep -qE "$pattern" <<<"$output" || problems+=("expected /$pattern/")
  done < "$expect"

  if [ ${#problems[@]} -eq 0 ]; then
    echo "ok   $case"
  else
    echo "FAIL $case"
    printf '       %s\n' "${problems[@]}"
    sed 's/^/     | /' <<<"$output"
    failed=1
  fi
done

exit $failed

Using it

Put your cases in tests/. A case is either a built program or a BASIC listing, with a sibling .expect saying what has to appear in its output:

tests/
  countdown.prg
  countdown.expect
  ticker.bas
  ticker.expect

expect ^10$
expect ^LIFT OFF$

$ ./test.sh
ok   tests/countdown.prg
ok   tests/ticker.bas

And when something breaks, it says so and shows you what the machine actually printed:

$ ./test.sh
FAIL tests/countdown.prg
       expected /^BLAST OFF$/
     | RUN
     | 10
     | 9
     ⋮
     | LIFT OFF
     |
     | OK
ok   tests/ticker.bas

Non-zero exit, so a build server stops on it.

Things the script is doing on purpose

It waits for its own echo when typing a program. BASIC answers OK to a statement but says nothing back to a stored program line, so waiting for the prompt after typing 10 PRINT "HI" waits until the timeout. Waiting for the line's own line number is the fix.

It pins the clock with --rtc. The clock chip is the only part of the machine that reads your computer's clock, so with it fixed the same case produces the same bytes on your laptop and on a build server.

It sends the emulator's console to /dev/null. The assertions read the console through the debug server; the copy the emulator echoes on stdout would just interleave itself with the results.

It restores before every case, including the first. Cheaper than reasoning about which cases dirty what.

Write a case that would fail

The most important case in any new suite is one that proves the suite can fail. Break an expectation on purpose, watch it go red, put it back. A suite that has never failed is not evidence of anything.

Testing things that draw

CLS, LOCATE and COLOR do nothing at all on a machine with no video card — they consume their arguments and return, which is what lets a program run happily over a serial line. It also means a test for anything visual has to boot a machine with the card fitted and read the screen instead of the console:

6502 run --headless --vdp picovdp --console video --debug --debug-port 6510 &
...
6502 dbg screen text --port 6510

Screen rows come back padded to the width of the screen — forty columns on the text screen, 32 or 40 in a layout of tiles — so anchor a pattern with \s*$ rather than $. For a picture made of tiles and sprites, where there is no text to read, compare dbg screen hash instead: the same program draws the same picture every time, and one pixel out changes the hash.

There is no console byte stream in video mode, so waiting works differently too: advance the machine by a number of emulated cycles rather than waiting for output. Emulated cycles, so the answer doesn't depend on how fast your computer is:

6502 dbg wait --cycles 2000000 --run turbo --port 6510

In continuous integration

Nothing here needs a display, an app, or a real machine, so the whole suite runs on a build server. The emulator's CLI is Node rather than Electron, which means a checkout and a build of the CLI is enough — no 100 MB browser binary needed for a job that never opens a window.

Pin your versions. A suite that goes red on a morning nobody touched it is a suite people stop believing.

Next: getting the program onto a real ACE.