Using 2000-vintage hardware, the setup of the piece is as shown:
The piano only requires MIDI output, not input. At least two companies are now selling kits to retrofit essentially any piano for MIDI output.
The six speakers are arranged as a pair close to the piano on stage (the sound of the speakers is to blend with the piano's sound) and the other four arranged in a rectangle around the audience; these are sometimes used for two-dimensional panning spatialization, and sometimes pointillistically, that is, with individual sampler voices appearing out individual speakers.
The computer processes can be divided into three parts: first, a network of effects devices (reverberation, pitch shifting, and frequency shifting); second, two sampler banks, each with a unique Markov chain device and a granular time-stretching machine; and third, an oscillator bank whose amplitudes can be controlled by the analyzed, time-varying spectrum of the piano sound, making a sort of vocoder.
These elements are arranged as shown:
The effects network consists of two reverberators, a pitch shifter (also known as a "harmonizer although that is a trademark) and a frequency shifter (which you saw in Mantra already.) Each of these can feed any of the others, and the samplers, oscillators, and piano can be fed into any of them. The gains are given names such as "ptof" for "piano to frequency shifter". The entire set of frequency shifter inputs is as follows:
ptof - piano ttof - sampler (the "t" stands for "Trevor" for historical reasons) otof - oscillator banks 1 and 2 Otof - oscillator banks 3 and 4 rtof - reverberator 1 Rtof - reverberator 2 htof - pitch shifter ("h" for "harmonizer")
These values are all on a MIDI scale, with 127 meaning "unity gain." Originally they were all restricted to the range 0-127, but this restriction might no longer be in force.
Then there are three controls to set the frequency shifter's behavior:
fsfre - frequency (actually MIDI pitch) of shifting fpos - positive frequency shift gain fneg - negative frequency shift gain
In the original patch these controls are all limited to integers; you can probably use decimal values anywhere you want now.
The frequency shifter output can go to any of the other effects, or to the speakers. The speaker sends are in the control window:
fto2 - send to stereo pair in front fto4 - send to the quad spatializer
These are not the only possible outputs for the frequency shifter; it can also be sent to any of the other effects (the two reverberators and the pitch shifter.)
The pitch shifter
The pitch shifter has the memnomic "h", and the inputs to it parallel those of the frequency shifter; so for instance "ftoh" means "frequency shifter to pitch shifter." The whole list is:
ptoh ttoh otoh Otoh rtoh htoh ftoh Rtoh
Note the "htoh" which is feedback from the pitch shifter to itself; the other modules don't have this feature. The output levels, "hto2" and "hto4," are as in the frequency shifter. The pitch shifter's intrinsic controls are:
hdel - delay, 4 milliseconds per step (so, 0-512 msec range) diramp - direct ("dry") amplitude, 127 for unit gain hamp - frequency shifter amplitude, 127 for unit gain hwind - window size, in logarythmic units, 200 msec maximum hfreq - frequency of change of delay time, one unit per 0.13 Hz., with 64 being "none". This controls pitch shift depending on window size; set this by ear.The reverberators
The two reverberators work identically; only the first "r" one is described here. The input levels are:
ptor ttor otor Otor htor ftorand the outputs are rto2 and rto4. (Use "R" to address the second one.) The intrinsic controls are:
rgate - input gate, i.e. master input send gain rout - output level gain revfb - reverb feedback, which controls reverb time. 127 gives infinite reverb.Oscillator Bank
The oscillators, found in gumbo, are only activated with an incoming signal. There are two modes, 1 and 0. Mode 1 records from the piano's signal, and mode 0 stops recording. Gumbo works like a vocoder attached to an oscillator bank, but the FFT uses a simple window that is not as effective as other windows used in typical vocoders.
There are four oscillator banks (consisting of up to twelve oscillators per bank) denoted by gumbo1, gumbo2, gumbo3, and gumbo4. A message sent to gumbo5 will employ all four oscillator banks.
Gumbo analyzes the relative strengths of frequency in the incoming signal. When a particular frequency is specified, gumbo plays the oscillator at the strength of that frequency present in the incoming signal.
To play the oscillators, send a set command with the frequencies desired gumbo, then a bang message to play them. A skew message with an argument given in milliseconds arpeggiates the pitches by causing each pitch to be initiated after the previous pitch by the duration specified.
For example, in the subpatcher "more-examples" found in the gumbo subpatch
gumbo1 set 6700 7100 7500 7800 8400 8800 9400 9700 10100 10500, bangimmediately sets the oscillators to the frequencies specified and starts them. The message:
gumbo1 set, bangturns the oscillators off.
In the following message:
gumbo1 set 6700 7100 7500 7800 8400 8800 9400 9700 10100 10500, skew 150the oscillators are started one by one separated by 150 milliseconds between each onset.
There are three types of slides, or frequency glissandi, that are differentiated by the time taken to change from the current (initial) frequencies to given target frequencies.
The first slide type is "slide 1." This particular manner starts the frequency change in all of the oscillators at the same time. However, the arrival time for the target frequencies are staggered by integer multiples of the time specified in milliseconds immediately after the slide 1 argument.
For example, in the more-examples subpatch, the message:
gumbo1 slide 1 500 6700 7100 7500 etc...causes the oscillators to start changing at the same time. However, the first oscillator will reach its target in 500 milliseconds, the second in 1000 milliseconds, the third in 1500 milliseconds, etc....
The second slide type (slide 2) keeps the rate of change the same specified by the argument immediately following "slide 2" in the message, but staggers the onset of change.
In the last slide type, the oscillators start and finish their frequency shifts at the same time.
Amplitude Whammy Bars
Although amplitude for each oscillator is determined by the FFT, there are "whammy bars" that allow the amplitudes to be adjusted. The first is a telescoping function that stretches or compresses the piano spectrum, so the ensuing amplitudes for frequencies follow the altered spectrum. A value of 64 sent to the spec message (spec1 for gumbo1, spec2 for gumbo2, etc.) is the unity value where there is no change. A value of 32 halves the spectrum; 16 halves it again. Alternately, 128 doubles the spectrum.
Another whammy acts as a moving average in which the coefficient may be
specified. The average:
Spectrum_out= lpf/127 * Spectrum_in + (1-lpf/127) * Spectrum_out_lastweights the value of the amplitude at each block based on the coefficient, lpf. For example, an lpf of 127 leaves the spectrum unchanged. An lpf at 0 freezes the spectrum. All the values in between slow the change in the spectrum. A lower value slows the spectrum changes more than a higher value. In this way, the whammy works like a lo-pass filter, hence the name.
The tab whammy takes three values: 1 for a sinusoid, 12 and 13 for a compound
waveform that is sum of cosines in harmonic series in an arbitrary relative
10 9 8 7 6 5 4 3 2 1. The tables can be
found in the fft-analysis subpatch in the DSP subpatch on the main page. They
are shown here:
And the last is an a master volume control for each bank, amp1, amp2, amp3, and amp4.
The first two oscillator banks, gumbo1 and gumbo2, are sent to the spatializing routine by a value in oto2 and oto4 from 0 (none) to 127 (full amplitude). The other two oscillator banks, gumbo3 and gumbo4, are sent by the values in Oto2 and Oto4. The 2 and 4 signify, respectively, the 2-channel static stereo pair and the 4-channel spatialized sound.
Sampler and Markov Chain Player