Dependence on Intensity

In general, increasing the intensity of pure tones:
   a) decreases the pitch of low frequencies
        (approx. <300Hz),
   b) increases the pitch of high frequencies
        (approx. >3000Hz), and
   c) has no noticeable effect at middle frequencies.

In the figure to the left:
   (a) A pure tone of frequency 98Hz has
         a pitch of G₂ when quiet (ppp), and
         a pitch lower than E₂ when loud (fff).
   (b), (c), and (d) show the influence of intensity
         on the pitch for pure tones with frequencies
         392Hz, 784Hz, and 3136Hz respectively.
         (in Campbell & Greated, 1987; derived
          from Stevens & Davies, 1939).

In addition, the introduction of a high-intensity "interference" tone will change the perceived pitch of existing low-intensity tones, even if the frequency of the low-intensity tones remains unchanged.
For frequencies well above the high-intensity tone, the perceived pitch will rise.
For frequencies well below the high-intensity tone, the perceived pitch will drop.

In other words, the high intensity tone pushes the pitch of low intensity tones away from it, assuming frequency separations beyond one critical bandwidth.
(What will happen if the intense and weak tones both fall within the same critical band?)

The figure to the left  offers another, simplified illustration of the average dependence of the pitch of pure tones on intensity.
In summary:
    Increasing the intensity will
    _ lower the pitch of a low frequency tone
    _ raise the pitch of a high frequency tone
    _ have no noticeable impact on the pitch of a
       middle frequency tone

Dependence on Duration

Pitch also depends on duration. A tone must last more than a minimum amount of time (~10-60ms, depending on frequency and intensity) in order to sound more than a 'click' and convey a clear sense of pitch (see below). We will return to this during the module on Timbre.

Dependence on Spectrum

Reminder: Periodic/harmonic complex tones are usually perceived as a single unit, with their multiple spectral components merging into a single tone perception rather than being perceived as a set of individual pure tones.

The pitch of  periodic complex tones with harmonic spectra (i.e. spectral components with frequencies that are integer multiples of the frequency of the lowest component, called 'fundamental') matches (in general) the frequency of the fundamental component.
This is the case regardless of whether or not the fundamental component is perceivable (i.e. even when it is too low in level and/or masked) and even if it is not physically present in the tone's spectrum at all.

Phenomenon of the missing fundamental or "virtual pitch": the pitch of a complex harmonic tone matches the frequency of its fundamental spectral component, even if this and several additional low-frequency components are missing from the tone's spectrum. 
In general, the phenomenon of the pitch of the missing fundamental or "virtual pitch" cannot be explained by place (tonotopic) theories of pitch perception (see below) and provides evidence that pitch depends not only on frequency, intensity, and duration but also on spectral distribution.

Listen to a pair of complex tones illustrating the phenomenon of the missing fundamental. Both have fundamental of 300Hz and up to 15 components (ramp spectrum: A_n = A₁/n for all components). The first tone has all 15 components while the second is missing the first 5. As you can see, removing these first 5 components does not change the tone's pitch, which continues to match that of 300Hz.

Frequency region most significant to pitch
      Note: Spectral and temporal context and fundamental frequency determine which of the criteria, below, will be more significant.

• For complex tones with a fundamental of 100Hz, the pitch sensation is altered after removing the first ~15-20 components and starts deteriorating after removing the first ~25-30 components. 
    
• For fundamentals of 500Hz and 800Hz, the same observations occur after removing the first ~4-7 and ~10-12  components respectively.
   
        Listen to this example.  It includes 13 harmonic complex tones, with fundamental of 600Hz, played in succession. 
        The first tone has all harmonic components from the 1st to the 16th and each successive tone drops one component,
        starting from the fundamental, until only the highest 3 components remain. What happens to the pitch?
    
• Experiments examining the effect of mistuning some of a harmonic tone's components on the resulting pitch have determined that the frequency region most important to pitch is between ~400 and ~1500Hz. That is, mistuning or removing components laying within this region has the most effect on pitch, while presence of frequency components within this region results in most clear pitch sensations. 

Perceiving a complex signal's individual components

As noted previously, harmonic complex signals are perceived as a unit rather than a set of multiple pure tones. However, the individual harmonics can be heard if we draw attention to them by removing them and re-introducing them (Houtsma et al.,1987).

Tuva throat singers (peoples of Tibet and the Siberian grasslands) exploit this phenomenon to create musical passages where one singer appears to produce two pitches simultaneously: one acting as a fixed drone and one performing a sort of melody. These passages are created:
a) from the single fundamental frequency produced, acting as a background drone, and
b) from harmonic components associated with this fundamental, selectively accentuated by the performer, acting as the melody line. [Optional: For more information and video/audio examples see here and here.]

Analytic / Synthetic Listening

In this example, (adapted from Smoorenburg, 1970), two complex tones with 2 components each are presented in succession.
(a) 800Hz + 1000Hz (b) 750Hz + 1000Hz.
When moving from (a) to (b):

• Some listeners hear the pitch going down by following the motion of the first component in each tone
  (800Hz 750Hz ~1 semitone drop).
  Explicit rules are employed to track the physical attributes of the two tones and determine the pitch motion.
  This is considered an example of analytic listening.
   

• Other listeners hear the pitch going up, by reconstructing the motion of the (missing) fundamental implied by the two complex tones
  (200Hz 250Hz ~4 semitone or a major third rise).
  Implicit rules are employed to synthesize a physical attribute that is implied by the rest of each tone's attributes, helping determine pitch motion. This is considered an example of synthetic listening.
  

Let's repeat the experiment, but now with two 5-component complex tones, presented in succession:
(a) 800Hz+1000+1200+1400+1600 (b) 750Hz+1000+1250+1500+1750.    In this case, most listeners only hear the pitch going up by a major 3rd, perceiving the motion of the (missing) fundamental (200Hz 250Hz). (Why?)

In Dannenbring's (1974) demonstration (masking noise bursts filling tone gaps in a steady or frequency modulated tone - from our Hearing Module):

• Listeners synthesize the sensation in a form of listening often referred to as synthetic or holistic, based largely on implicit rules (rules we are not explicitly aware of).
   

• If listeners are alerted to the fact that the presented tones actually have gaps, they may be able to perceive them by directing their attention to separate portions of the total stimulus. This 'directed' form of listening is often referred to as analytic, based largely on explicit rules (rules we are explicitly aware of).

The McGurk effect is an auditory illusion caused by ambiguous audio/visual cues. It offers another example of synthetic listening, illustrating that perceptions are the result of an experience-guided synthesis of information from our environment.

The McGurk effect indicates that our responses to stimuli do not necessarily match the stimuli. Rather, they reflect the best way such stimuli fit to previous experience, concurrent context, and the associated expectations. When staring at the speaker's lips, the effect is so robust that, even when alerted to the a/v stimuli conflict, listeners/viewers are unable to listen to the a/v composite analytically. Listening with eyes closed presents no ambiguity, resulting in a different sonic perception. Watch this BBC2 clip.

Pitch of Complex Tones with Inharmonic Spectra

Inharmonic/non-periodic complex tones, whose frequency components are not integer multiples of a 'fundamental' component, do not elicit a clear, unique pitch sensation. They may elicit more than one competing pitch sensations, may resemble chords, or may sound as noise, depending on their spectral distribution.

As illustrated below, however, changing the spectral peaks of inharmonic complex tones without changing the frequencies of their sinusoidal components can result in a distinguishable change in pitch, matching the frequency change of the spectral peaks. This highlights the perceptual salience of changing vs. static stimuli and provides additional evidence of the dependence of pitch on spectral distribution. 

Listen to three individual major chords (three simultaneous notes: C, E, and G), performed by combinations of flute, clarinet, and oboe:                     Clarinet-Flute-Oboe             Flute-Clarinet-Oboe            Oboe-Clarinet-Flute

Now, listen to this "chord melody" example, consisting of a five-chord sequence, each of which is one of the three chords, above. All 3 chords are major, include exactly the same notes (C5, E5, and G5), and have inharmonic spectra (i.e. their spectral components are not integer multiples of a single fundamental, even though the spectra of the individual notes in the chords are themselves harmonic).

The frequencies of the components are identical among chords but the spectral envelopes (i.e. relative intensities of the components) are different and depend on which instrument plays what note (a flute, a clarinet, or an oboe). 
The melody you hear (C-E-G-E-C) tracks the position of the flute in each successive chord, because the flute's fundamental frequency provides the spectral peak for each chord's spectrum.  In other words, the melody you hear matches the pitch changes corresponding to the changes in the flute's fundamental frequency. 

This effect can also be produced through spectral shaping of noise bands. Pitch perception will track changes in the spectral peak of the noise.

NOTE: Minor deviations from harmonic spectra (up to ~1-2% of frequency) and the way these interact when, for example, several instruments perform together in unison change the timbre of the combined sound rather than its pitch. They produce what is referred to as 'chorus effect': richness of ensemble sound due to slow and varying beating rates among the slightly detuned components of the complex tones involved.

Approximate value of the virtual pitch of slightly inharmonic spectra, whose fundamental or additional low-frequency components are missing:

Assume the 3-component harmonic spectrum: 800, 1000, 1200Hz (components 4, 5, and 6 of a 200Hz missing fundamental and a slightly inharmonic version of the same spectrum: 792, 1020, 1170Hz. Assume equal amplitudes for all components.
 
The virtual pitch of the harmonic spectrum will correspond to 200Hz.
 
The virtual pitch of the inharmonic spectrum will correspond, approximately, to the average of the three implied fundamentals: (792/4 + 1010/5 + 1170/6)/3 = (198+202+195)/3 = 595/3 = 198.3Hz.

Pitch Theories (for more details, see the Optional section at the end of this and the Hearing modules)

Place (Tonotopic) Theory

Pitch relates directly to the point of stimulation on the Basilar Membrane (BM).
Different frequencies resonate at different locations on the BM.
Frequency is encoded as pitch by the inner hair cells corresponding to the BM portion resonating for that frequency.

Watch these (simplified / exaggerated) animations of the basilar membrane motion in response to:
1000Hz,    8000Hz,    1000Hz+8000Hz,  &  a range of frequencies (.mov files).  
Read this brief outline of the place theory.

Pros

Applies to all frequencies.

Explains pathological conditions:
diplacusis: different pitch sensations per ear for the same frequency, due to BM structural differences btw ears;
presbycusis: pitch shift with age for the same frequency, due to BM hardening with age.

Cons

Cannot explain the coarseness of JND or the observed relationship of perceived pitch to intensity.

Unless it is modified, it cannot reliably explain two phenomena associated with the pitch of complex tones:

1. The pitch of the missing fundamental.
    
   The place theory claims that, even when missing, the fundamental frequency and other low frequency components are re-introduced as the "difference" frequencies, distortion products arising from the interaction among the existing components in the tone's harmonic spectrum (the difference frequencies between upper harmonic equals the frequencies of lower harmonics; why?).
   The difference frequencies then excite the BM locations that correspond to the fundamental and other low-frequency components.
    
   This claim is challenged by several observations, including the pitch-shift effects. These effects refer to instances where spectral modifications result in the pitch changing towards a direction that appears to be opposite to that predicted by the place theory of pitch and the "difference frequency" hypothesis.
     

2. The persistence of the missing fundamental phenomenon even when the first 8 or more components are removed.
    
   For most tones, only the first 5-8 components excite separate critical bands and can therefore elicit a unique pitch sensation. Beyond that, 2 or more components lay in the same critical band and cannot be differentiated based on a place theory of pitch. Therefore, the place theory of pitch requires energy at the BM location corresponding to the lower components for pitch sensation to register.

Temporal (Periodicity/Frequency) Theory

Periodicity/Frequency theories of pitch rely on time information, as represented on the signal itself or through its conversion to neural electrical spikes, following inner hair cell activity.

The volley theory of pitch, introduced in the 1940s by Wever, explains pitch in terms of the combination of
a) phase locking (neurons firing at a single point in the vibration cycle) and
b) neural firing synchrony (multiple neurons firing in sync with each other)
information from a large number of inner hair cells associated with a specific place on the basilar membrane.
Select the "Illustration" tab on this page for an interactive animation that describes the temporal theory.

Whether based on the periodicity of the sound signal itself or the periodicity of the sound signal's representation in the auditory nerve (e.g. periodicity of neural spike activity), temporal models of pitch assume that the hearing mechanism extracts a signal's periodicity by performing an autocorrelation function on the signal.  Qualitatively, autocorrelation provides a measure of how well a signal matches a time-shifted version of itself, as a function of the amount of time shift.  

Pros

May explain why pitch perception becomes increasingly coarse past 5kHz.

May explain the perception of the missing fundamental.

Efficiently captures pitch and loudness information based on neural firing patterns and densities.

Cons

Cannot readily explain pathological conditions (diplacusis / presbycusis), which can be traced to the BM and have physiologically-based pitch perception implications.

Cannot readily explain the pitch shift effects.

Appears to break down far above 5kHz.

Hybrid Theories

Contemporary, hybrid theories of pitch incorporate aspects of both, Place and Periodicity theories (Plomp, 1964; Terhardt, 1974; Pierce 1990). Terhardt's theory of 'virtual pitch', introduces 'previous learning' as an additional novel component that can be developed based on either temporal or place pitch cues and which provides us with pitch templates of disturbance patterns on the BM. These are used to support pitch decisions in ambiguous contexts (spectra with missing components, mistuned spectra, etc.). 

In general, it appears that the most salient/prominent components of a signal's spectrum (in terms of intensity level and frequency separation) are the most important carriers of pitch information.
In the presence of multiple complex tones, the components of each complex tone are perceptually linked together into a single percept, separate from the other complex tones. This appears to be due to each particular complex tone's spectral jitter (i.e. fast amplitude and frequency micro-variations that are almost synchronized among the components of each complex tone).

The Octave (use this virtual piano to help you with the concepts in this section)

Multidimensionality of Pitch (pitch spiral)

Pitch is multidimensional with at least three dimensions, involving a) pitch height (one dimension: frequency) and
b) pitch chroma (two dimensions, separating pitches within an octave and linking pitches an octave apart).

Note: A variable is uni-dimensional if all its values can fit on a single straight line. If for example A, B, & C represent 3 values of a variable as points in space then:
a) if |AB| + |BC| = |AC| then the variable is uni-dimensional
b) if |AB| + |BC| ≠ |AC| then the variable is multidimensional
[Does this rule remind you of another distinction?]

The western chromatic scale breaks the octave down into 12 different pitch chromas. The pitch chroma dimensions represent a circularity in pitch perception.
Due to this circularity, within a single octave, the octave interval represents the largest physical distance in terms of pitch height but smallest perceptual distance in terms of pitch chroma.
 
Pitch perception therefore wraps on the octave, with scales defining sets of different pitch chromas that repeat at different pitch heights for each new octave. This is best represented by a pitch spiral (versus a pitch scale).

The perceptual circularity of pitch is explored in Shepard-tone scales/slides (after Roger Shepard) that present the paradox of a continuously ascending (or descending) pitch.

Listen to two pitch spiral examples (Houtsma et al., 1987).

Shepard scales/slides are the auditory analog of the continuously ascending/descending staircases, explored conceptually by Penrose and artistically by Escher (see the images, below)

                          Relativity (Escher, 1953)                                                                        Ascending-Decending (Escher, 1960) 

[OPTIONAL SECTIONS]

This results in the neural signals of sinusoidal inputs
i) having a pulse-like shape with repetition rate equal to the input's period (i.e. they represent frequency in the time domain) and
ii) being rectified (i.e. they are constricted to the positive portion of the two-dimensional signal graph).

The process of phase locking is closely related to hearing's "temporal coding theory" of encoding frequency information:

The inner hair cells release neurotransmitters only when the basilar membrane moves in one direction, towards the scala media. They therefore only respond to the positive portions of incoming signals, with their stereocilia opening their ion channels when bending against the TM mostly at a signal’s positive peak.
 
Temporal coding assumes that, thanks to phase locking, the auditory system encodes periodic information through the firing rate of neurons. This rate corresponds to the incoming signal’s period (or some multiple of it) and, therefore frequency.

Since neurons are not fast enough to encode high frequencies, more than one neuron must be involved in the process. Each neuron fires at some of the peak portions of an incoming signal and, after adding the outputs of all neurons, the signal is represented to the brain in a manner similar to that shown at the bottom graph (left).