Multiplying audio signals

We have been routinely adding audio signals together, and multiplying them by slowly-varying signals (used, for example, as amplitude envelopes) since Chapter 1. For a full understanding of the algebra of audio signals we must also consider the situation where two audio signals, neither of which may be assumed to change slowly, are multiplied. The key to understanding what happens is the Cosine Product Formula:

$$\cos(a) \cos (b) = {1 \over 2} { \left [ \parbox[t][0.1in]{0in}{\mbox{}} \cos (a+b) + \cos(a-b) \right ] }$$

To see why this formula holds, we can use the formula for the cosine of a sum of two angles:

$$\cos(a+b) = \cos(a)\cos(b) - \sin(a) \sin(b)$$

to evaluate the right hand side of the cosine product formula; it then simplifies to the left hand side.

We can use this formula to see what happens when we multiply two sinusoids (Page ):

$${\cos(\alpha n + \phi) \cos (\beta n + \xi)} =$$

$$= {1 \over 2} { \left [ { \parbox[t][0.1in]{0in}{\mbox{}}... ...t ( (\alpha - \beta) n + (\phi - \xi) \right ) } } \right ] }$$

In words, multiply two sinusoids and you get a result with two partials, one at the sum of the two original frequencies, and one at their difference. (If the difference $\alpha-\beta$ happens to be negative, simply switch the original two sinusoids and the difference will then be positive.) These two new components are called sidebands.

This gives us a technique for shifting the component frequencies of a sound, called ring modulation, which is shown in its simplest form in Figure 5.2. An oscillator provides a carrier signal, which is simply multiplied by the input. In this context the input is called the modulating signal. The term ``ring modulation" is often used more generally to mean multiplying any two signals together, but here we'll just consider using a sinusoidal carrier signal. (The technique of ring modulation dates from the analog era [Str95]; digital multipliers now replace both the VCA (Section 1.5) and the ring modulator.)

Figure 5.2: Block diagram for ring modulating an input signal with a sinusoid.

Figure 5.3 shows a variety of results that may be obtained by multiplying a (modulating) sinusoid of angular frequency $\alpha$ and peak amplitude $2a$, by a (carrier) sinusoid of angular frequency $\beta$ and peak amplitude 1:

$$\left [ 2 a \cos (\alpha n) \right ] \cdot \left [ \cos (\beta n) \right ]$$

(For simplicity the phase terms are omitted.) Each part of the figure shows both the modulation signal and the result in the same spectrum. The modulating signal appears as a single frequency, $\alpha$, at amplitude $a$. The product in general has two component frequencies, each at an amplitude of $a/2$.

Figure 5.3: Sidebands arising from multiplying two sinusoids of frequency $\alpha$ and $\beta$: (a) with $\alpha > \beta > 0$; (b) with $\beta > \alpha$ so that the lower sideband is reflected about the $f=0$ axis; (c) with $\alpha =\beta$, for which the amplitude of the zero-frequency sideband depends on the phases of the two sinusoids; (d) with $\alpha =0$.

Parts (a) and (b) of the figure show ``general" cases where $\alpha$ and $\beta$ are nonzero and different from each other. The component frequencies of the output are $\alpha + \beta$ and $\alpha-\beta$. In part (b), since $\alpha-\beta<0$, we get a negative frequency component. Since cosine is an even function, we have

$$\cos((\alpha - \beta)n) = \cos((\beta - \alpha)n)$$

so the negative component is exactly equivalent to one at the positive frequency $\beta-\alpha$, at the same amplitude.

In the special case where $\alpha =\beta$, the second (difference) sideband has zero frequency. In this case phase will be significant so we rewrite the product with explicit phases, replacing $\beta$ by $\alpha$, to get:

$${2 a \cos(\alpha n + \phi) \cos (\alpha n + \xi)} =$$

$$= {a \cos \left ( 2 \alpha n + (\phi + \xi) \right ) } + {a \cos \left ( \phi - \xi \right ) } .$$

The second term has zero frequency; its amplitude depends on the relative phase of the two sinusoids and ranges from $+a$ to $-a$ as the phase difference $\phi - \xi$ varies from $0$ to $\pi$ radians. This situation is shown in part (c) of Figure 5.3.

Finally, part (d) shows a carrier signal whose frequency is zero. Its value is the constant $a$ (not $2a$; zero frequency is a special case). Here we get only one sideband, of amplitude $a/2$ as usual.

We can use the distributive rule for multiplication to find out what happens when we multiply signals together which consist of more than one partial each. For example, in the situation above we can replace the signal of frequency $\alpha$ with a sum of several sinusoids, such as:

$${a_1} \cos({\alpha _1} n ) + \cdots + {a_k} \cos({\alpha _k} n )$$

Multiplying by the signal of frequency $\beta$ gives partials at frequencies equal to:

$$\alpha_1 + \beta, \alpha_1 - \beta, \ldots, \alpha_k + \beta, \alpha_k - \beta$$

As before if any frequency is negative we take its absolute value.

Figure 5.4 shows the result of multiplying a complex periodic signal (with several components tuned in the ratio 0:1:2:$\cdots$) by a sinusoid. Both the spectral envelope and the component frequencies of the result are changed according to relatively simple rules.

Figure 5.4: Result of ring modulation of a complex signal by a pure sinusoid: (a) the original signal's spectrum and spectral envelope; (b) modulated by a relatively low modulating frequency (1/3 of the fundamental); (c) modulated by a higher frequency, 10/3 of the fundamental.

The resulting spectrum is essentially the original spectrum combined with its reflection about the vertical axis. This combined spectrum is then shifted to the right by the modulating frequency. Finally, if any components of the shifted spectrum are still left of the vertical axis, they are reflected about it to make positive frequencies again.

In part (b) of the figure, the modulating frequency (the frequency of the sinusoid) is below the fundamental frequency of the complex signal. In this case the shifting is by a relatively small distance, so that re-folding the spectrum at the end almost places the two halves on top of each other. The result is a spectral envelope roughly the same as the original (although half as high) and a spectrum twice as dense.

A special case, not shown, is modulation by a frequency exactly half the fundamental. In this case, pairs of partials will fall on top of each other, and will have the ratios 1/2 : 3/2 : 5/2 :$\cdots$ to give an odd-partial-only signal an octave below the original. This is a very simple and effective octave divider for a harmonic signal, assuming you know or can find its fundamental frequency. If you want even partials as well as odd ones (for the octave-down signal), simply mix the original signal with the modulated one.

Part (c) of the figure shows the effect of using a modulating frequency much higher than the fundamental frequency of the complex signal. Here the unfolding effect is much more clearly visible (only one partial, the leftmost one, had to be reflected to make its frequency positive). The spectral envelope is now widely displaced from the original; this displacement is often a more strongly audible effect than the relocation of partials.

As a special case, the modulating frequency may be a multiple of the fundamental of the complex periodic signal; then the partials all land back on other partials of the same fundamental, and the only effect is the shift in spectral envelope.