Reality Check Documentation

Reality Check - framework for testing Pd realizations of electronic music

This is the documentation for Reality Check version 0.02(Thu Feb 22 04:54:11 PM CET 2024). It can be found on Miller Puckette’s website, msp.ucsd.edu, under “tools”.

Reality Check is a Pure Data (Pd) patch that can test a realization patch (that is, a Pd patch that realizes a musical piece in real time) to verify that the realiaztion patch generates the desired output given a fixed input. The input consists of a soundfile and a script. The inputs are sent through the realiation patch and whatever sounds are emitted are compared with a previously recorded output of the same patch. To test the realization, the Reality Check patch runs the realization patch, providing its inputs, and checks that its audio outpus match the pre-recorded ones. If so, the realization patch has been verified to run correctly. If not, corrective action can be taken.

The script contains time-tagged messages to send to the realization patch that simulate whatever MIDI, OSC, network, mouse, and/or keyboard events arrive during the performancem in sync with th erecorded input soundfile. The script also contains comtrol messages that instruct Reality check to open the realiszation patch, present it with the input soundfile, and compare the audio output with the reference output, as shown:

block diagram

To prepare a test, we generate a reference output soundfile by running the realization patch with a set or reference inputs. To run the test at any later time, we then present the reference inputs to the realization patch to verify that it still produces the same output, to within a pre-determined tolerance. If the test passes, this verifies not only that the realization patch is correct, but also that the real-time environment is still capable of playing the piece correctly.

This release contains the main patch, named 1.reality.pd, and two subdirectories: “doc”, containing this documentation and two related position papers, and “simple” which contains a sample “piece” and its reference inputs and outputs to demonstrate the workings of the patch.

The README.txt file contains command-line tests that can test the “simple” example: one test that should succeed and another one that should fail. Real-world tests of two pieces, en Echo by Philippe Manoury and Of Pulses and Times by Kerry Hagan, may be downloaded separately from the same website.

Running the Reality Check patch

The patch “1.reality.pd” can be run in batch mode or interactively. Normally, you would run it interactively to set up a testing regime for a particular realization, then, once it is working, arrange for it to be run automatically in batch mode to test the realization once the testing regime is in place. When run in batch mode, reality check prints out error levels (ideally zero) periodically and then exits with a UNIX-style exit code of 0 for success and 1 for failure. The run is considered successful if the maximum error is below one millionth (10^-6) in absolute value. (Ideally this error should be exactly zero but Pd has sometimes given slightly different results depending on the operating system’s default floating-point rounding mode.)

For example, to run the included test, you can type to a shell:

pd  -batch -nosound -nrt -noprefs -r 44100 \
    -send "run-batch simple simple-script.txt" 1.reality.pd 

Here, the flags to Pd indicate that it should be run in batch mode (with no GUI), with no sound input or output, no real-time enhancements, no “preferences”, and a specified sample rate. THe patch to run is “1.reality.pd” and it is to be sent a startup message,

    run-batch simple simple-script.txt

(To run interactively, suppress the “-batch” flag to Pd and send the message to “run-interactive” instead of “run-batch”). The run-batch message takes three arguments:

The test script

To test the realization, Reality Check runs a script that starts up a sub-process to run the realization patch, reads the input and control output soundfiles, and sends timed messages to the sub-process to simulate running the piece. The script is a simple “qlist” with incremental times given in seconds. For example, here is the “simple” test script included in this release:

load-patch simple/simple.pd;
running 1;
out-channels 2;
write-output 1;
open-control-readsf simple-out.wav;
open-input-readsf simple-in.wav;
writesf start;
1.5 pd~ amplitude 0.1 30;
0.5 pd~ amplitude 0 30;
0.5 writesf stop;
pd~ pd~ stop;
running 0;

Messages understood by Reality Check include the following: