Speech and audio from the ground up
The Fourier view
Every signal is a sum of sinusoids, and the Fourier transform says how much of each. The catch is that a finite recording can only answer the question approximately.
Any shape as a sum
Fourier’s claim is that any reasonable signal can be written as a sum of sine and cosine waves at different frequencies. Each wave has its own amplitude (how loud it is) and phase (where it starts in its cycle). A violin note is a fundamental (the lowest tone) plus a stack of harmonics (multiples of that tone). A vowel is a buzz shaped by the resonances of the throat. The spectrum is the list of ingredients, and it is often a better description than the waveform. A spectrum describes one moment; it is not a recipe for the sound.
The ear does something similar. The cochlea (the hearing organ) is a bank of filters arranged by frequency, so a pressure wave is broken into parts before the brain ever sees it. So a spectrum is not an arbitrary mathematical convenience: it matches the organ that will interpret the sound. When a feature is hard to see in the time domain, the Fourier view is usually where it becomes clear. The transform is a change of basis (a different way to write the same data), and no information is created or destroyed.
The discrete transform
For N samples, the discrete Fourier transform produces N complex coefficients. A complex number here is a pair: an amplitude and a phase. Each coefficient measures the correlation (how closely the signal matches a sinusoid at one frequency), and its magnitude says how much of that frequency is present. Because the input is real, the second half mirrors the first, so only N divided by two plus one values carry information. A coefficient carries both because a sinusoid has both a size and a starting point.
The frequency of bin k is k times the sample rate divided by N. A bin is one slot on the frequency grid. With 16000 samples per second and a 400-sample frame, each bin is 40 Hz wide. The transform does not choose the frequencies; the frame length and the sample rate do. Change either and the grid moves, so features computed at different rates cannot be compared. A feature computed from a 25 ms frame is therefore tied to that grid forever.
A real spectrum in NumPy
import numpy as np
frame = np.hanning(400) * signal[:400]
spectrum = np.fft.rfft(frame)
freqs = np.fft.rfftfreq(400, d=1 / 16000)
peak = int(np.argmax(np.abs(spectrum)))
print(round(freqs[peak], 1), round(abs(spectrum[peak]), 1))The transform by hand
Run the direct transform and read the two peaks. Then change the second tone to 1500 Hz and explain why its energy no longer sits in one bin.
Resolution is not accuracy
The spacing between bins is the sample rate divided by the frame length, and that spacing is the best resolution the transform can offer. A 400-sample frame at 16 kHz gives 40 Hz bins, so two tones 20 Hz apart cannot be separated: they are closer than one bin. A longer frame narrows the bins but assumes the signal stayed steady for longer. The resolution limit belongs to the observation window, not to the algorithm.
A tone that falls between bins does not disappear. Its energy spreads into neighbouring bins, an effect called leakage, and the shape of the spread depends on the window. The rectangular window (no taper at all) leaks slowly and far; a Hann window leaks much less far but widens the main peak. No window removes the trade-off; each puts the error where you can tolerate it. The choice is a statement about which error matters for the task.
Zero-padding a frame before the FFT does not add resolution. It interpolates the same spectrum onto a finer grid, which makes peaks easier to locate but cannot separate frequencies the original frame could not resolve.
The fast transform
The fast Fourier transform computes the same coefficients as the direct sum in N log N operations instead of N squared. The result is identical up to rounding. The difference is that a 400-point transform becomes cheap and a million-point transform becomes possible. Every library you will use calls the FFT, and almost none of them calls the direct sum. The direct sum is still useful for understanding and for checking a fast implementation.
What the FFT does not do is tell you which frequencies matter. It returns a complex number per bin, and the magnitude throws away the phase. For many speech features that is exactly what you want, because the ear is insensitive to phase in short frames. For other tasks, such as rebuilding the waveform, the phase is essential and discarding it is the bug. A magnitude spectrogram is therefore not invertible (you cannot rebuild the sound from it alone).
Leakage and the window
Run both windows on a tone between bins and compare the energy that escapes the main peak. Then move the tone onto an exact bin and watch the difference shrink.
Check what you learned
One question at a time. A wrong answer explains itself — take another go.
Part 1 of 5
What does the Fourier transform give you?