Nyquist–Shannon sampling theorem

(Redirected from Nyquist-Shannon sampling theorem)

Categories: Digital signal processing | Information theory | Theorems

The Nyquist–Shannon sampling theorem is the fundamental theorem in the field of information theory, in particular telecommunications. It is also known as the Whittaker–Nyquist–Kotelnikov–Shannon sampling theorem or just simply the sampling theorem.

The theorem states that:

When sampling a band-limited signal (e.g., converting from an analog signal to digital), the sampling frequency must be greater than twice the input signal bandwidth in order to be able to reconstruct the original perfectly from the sampled version.

If <math>\Omega</math> is the signal bandwidth and <math>f_\mathrm{s}</math> is the sampling rate, then the theorem can be stated mathematically (called the "sampling condition" from here on)

<math>2 \Omega < f_\mathrm{s}</math>

Note: The above formulation implicitly assumes that the signal is a real-valued function with a fourier transform that is zero outside the intervall <math>\left[-2\pi\Omega,2\pi\Omega\right]</math>. In cases with a lowest positive frequency <math>\Alpha>0</math>, it is possible to have smaller sampling frequencies close to the double of the difference <math>(\Omega-\Alpha)</math> of highest and lowest frequency, see the section about undersampling below.

Contents

[edit]

Aliasing

If the sampling condition is not satisfied, then frequencies will overlap (see the proof below). This overlap is called aliasing.

To prevent aliasing, two things can readily be done

  1. Increase the sampling rate
  2. Introduce an anti-aliasing filter or make anti-aliasing filter more stringent

The anti-aliasing filter is to restrict the bandwidth of the signal to satisfy the sampling condition. This holds in theory, but is not satisfiable in reality. It is not satisfiable in reality because a signal will have some energy outside of the bandwidth. However, the energy can be small enough that the aliasing effects are negligible.

[edit]

Spatial domain

Image:Moire pattern of bricks.jpg
See for full size image

Although this article discusses sampling in the time domain, the sampling theorem holds for the spatial domain as well.

For example, a digital photograph of a striped shirt with high frequencies (in other words, the distance between the stripes is small), can cause aliasing between the shirt and the camera's sensor array. The aliasing appears as a Moiré pattern. The "solution" to higher sampling in the spatial domain for this case would be to move closer to the shirt or use a higher resolution sensor (for example, a CCD).

Another example is shown to the right in the brick patterns. The top image shows the effects when the sampling theorem is not followed. When software rescales an image (the same process that creates the thumbnail shown in the bottom image) it, in effect, runs the image through a low-pass filter first and then downsamples the image to result in a smaller image that does not exhibit the Moiré pattern. The top image is what happens when the image is downsampled without low-pass filtering and aliasing results.

The top image was created by zooming out in GIMP and then taking a screenshot of it. Likely reason that this works is that the zooming feature simply downsamples without low-pass filtering (probably for performance reasons) since the zoomed image is for on-screen display instead of printing or saving.

[edit]

Downsampling

When a signal is downsampled, the theorem still must be satisfied. The theorem is satisfied when downsampling by filtering the signal appropriately with an anti-aliasing filter.

[edit]

Critical frequency

The critical frequency is defined as twice the bandwidth (if the sampling condition was an equality instead of an inequality).

If the sampling frequency is exactly twice the highest frequency of the input signal, then phase mismatches between the sampler and the signal will distort the signal. For example, sampling <math>\cos(\pi t)</math> at <math>t = 0, 1, 2, \dots</math> will give you the discrete signal <math>\cos(\pi n)</math>, as desired. However, sampling the same signal at <math>t = 0.5, 1.5, 2.5, \dots</math> will give you a constant zero signal. These two sets of samples, which differ only in phase and not frequency, give dramatically different results because they sample at exactly the critical frequency.

[edit]

Historical background

The theorem was first formulated by Harry Nyquist in 1928 ("Certain topics in telegraph transmission theory"), but was only formally proven by Claude E. Shannon in 1949 ("Communication in the presence of noise"). Kotelnikov published in 1933, Whittaker in 1915 (E.T.) and 1935 (J.M.), and Gabor in 1946.

Mathematically, the theorem is formulated as a statement about the Fourier transform.

If a function <math>s(t) \ </math> has a Fourier transform with <math>S(f) := \mathcal{F} \{s\}(f) = 0 \ </math> for <math>|f| \ge 2\pi \Omega \ </math>, then it is completely determined by giving the value of the function at a series of points spaced <math>\frac{1}{2 \Omega}</math> apart. The values <math>s_n := s \left( \frac{n}{2 \Omega} \right)</math> are called the samples of s(t).

The minimum sample frequency that allows reconstruction of the original signal, that is <math>2 \Omega \ </math> samples per unit distance, is known as the Nyquist frequency, (or Nyquist rate). The time in-between samples is called the Nyquist interval.

If <math>S(f) = 0 \ </math> for <math>|f| \ge 2\pi \Omega \ </math>, then <math>s(t) \ </math> can be recovered from its samples by the Nyquist–Shannon interpolation formula.

A well-known consequence of the sampling theorem is that a signal cannot be both bandlimited and time-limited. To see why, assume that such a signal exists, and sample it faster than the Nyquist frequency. This finite number of time-domain coefficients should define the entire signal. Equivalently, the entire spectrum of the bandlimited signal should be expressible in terms of the finite number of time-domain coefficients obtained from sampling the signal. Mathematically this is equivalent to requiring that a (trigonometric) polynomial can have infinitely many zeros in bounded intervalls since the bandlimited signal must be zero on an interval beyond a critical frequency which has infinitely many points. However, it is well-known that polynomials do not have more zeros than their orders due to the fundamental theorem of algebra. This contradiction shows that our original assumption that a time-limited and bandlimited signal exists is incorrect.

[edit]

Undersampling

When sampling a non-baseband signal, the theorem must be restated as follows. Let <math>0<f_L<f_H</math> be the lower and higher boundarys of a frequency band and <math>W = f_\mathrm{H} - f_\mathrm{L}</math> be the bandwidth. Let s(t) be a real valued function with a spectrum limitied to this frequency band, that is with <math>S(f)=\mathcal F\{s\}(f)=0</math> for <math>|f|</math> outside the intervall <math>[2\pi f_L,2\pi f_H]</math>. To avoid overlapping of the sampled spectrum the sampling rate <math>f_\mathrm{s}</math> has to satisfy the following inequalities <math>\frac{2(f_\mathrm{L} + W)}{N + 1} < f_\mathrm{s} < \frac{2f_\mathrm{L}}{N}</math>, with the integer <math>N<\frac{f_\mathrm{L}}{W}</math>. Every integer value of <math>N</math>, down to zero, gives a permissible interval of sampling frequencies. Doing this results, for some values of <math>N</math>, in a sampling rate less than the carrier frequency <math>(f_\mathrm{L}+W/2)</math> of the signal and even of the minimum. In other cases of non-baseband signals the intuitive sampling-by-taking-values has to be replaced by sampling-by-taking-scalar-products, as is (implicitly) the case in Frequency-division multiplexing.

Consider FM radio to illustrate the idea of undersampling. In the US, FM radio operates on the frequency band from 88 MHz to 108 MHz. The sampling conditions are satisfied for <math>N<4.4</math>, i.e. <math>N = 4, 3, 2, 1, 0</math>. The value <math>N=4</math> gives the lowest sampling frequencies interval <math>43.2\ \mathrm{MHz}<f_\mathrm{s}<44\ \mathrm{MHz}</math> and this is a scenario of undersampling.

Note that when undersampling a real-world signal, the sampling circuit must be fast enough to capture the highest signal frequency of interest. Theoretically, each sample should be taken during an infinitesimally short interval, but this is not practically feasible. Instead, the sampling of the signal should be made in a short enough interval that it can represent the instantaneous value of the signal with the highest frequency. This means that in the FM radio example above, the sampling circuit must be able to capture a signal with a frequency of 110 MHz, not 43.2 MHz. Thus, the sampling frequency may be only a little bit greater than 43.2 MHz, but the input bandwidth of the system must be at least 110 MHz.

If the theorem is misunderstood to mean twice the highest frequency, then the sampling rate would assumed to need to be greater than 216 MHz. While this does satisfy the correctly-applied sampling condition (<math>N=0</math> gives <math>216\ \mathrm{MHz} < f_\mathrm{s}</math>) it is grossly over sampled.

Note that if the FM radio band is sampled, e.g., at <math>43.2\ \mathrm{MHz}<f_\mathrm{s}<44\ \mathrm{MHz}</math>, then a band-pass filter is required for the anti-aliasing filter.

In certain problems, the frequencies of interest are not an interval of frequencies, but perhaps some more interesting set F of frequencies. Again, the sampling frequency must be proportional to the size of F. For instance, certain domain decomposition methods fail to converge for the 0th frequency (the constant mode) and some medium frequencies. Then the set of interesting frequencies would be something like 10 Hz to 100 Hz, and 110 Hz to 200 Hz. In this case, one would need to sample at a data rate of 360 Hz — i.e. at a sampling rate of 20 Hz with 18 real values in each sample — not 400 Hz, to fully capture these signals.

[edit]

Proof

To prove the theorem, consider two continuously defined signals: any continuous signal <math>f(t)</math> with fast decay at infinity and a Dirac comb <math>\delta_T(t)</math>.

Let the result of the multiplication be

<math>

f^{*}(t) = f(t) \delta_T(t) = f(t) \sum_{n=-\infty}^{\infty} \delta(t - n T) =\sum_{n=-\infty}^{\infty} f(nT)\delta(t - n T)</math>

and taking the Fourier transform and applying the multiplication/convolution property

<math>F^{*}(\omega) = \mathcal{F} \{f^{*}(t)\}</math>
<math>= \frac{1}{\sqrt{2 \pi}} F(\omega) * \mathcal{F}\{\delta_T(t)\}</math>
<math>= \frac{1}{\sqrt{2 \pi}} F(\omega) * \left\{ \frac{\sqrt{2 \pi}}{T} \sum_{n = -\infty}^{\infty} \delta (\omega - n \omega_\mathrm{s}) \right\}</math>,

and by the shifting property of the Dirac delta distribution under convolution,

<math>F^{*}(\omega) = \frac{1}{T} \sum_{n = -\infty}^{\infty} F(\omega - n \omega_\mathrm{s})</math>

where <math>\omega_\mathrm{s} = \frac{2 \pi}{T}</math> and is the sampling rate.

Looking at the combed function in a second way, we see that

<math>F^{*}(\omega)

= \frac{1}{\sqrt{2 \pi}}\int_{-\infty}^\infty \sum_{n=-\infty}^{\infty} f(nT) \delta(t - n T) e^{-i\omega t}\,dt = \frac{1}{\sqrt{2 \pi}}\sum_{n=-\infty}^{\infty} f(nT) e^{-i\omega n T} </math>. The integral is to be interpreted as application of the compact distribution <math>\delta</math> to the bounded, continuous function <math>e^{-i\omega t}</math>.

The end result is a summation of shifted <math>F(\omega)</math>, which is equal to some Fourier series with values of f at equally spaced points as coefficients. This is just the Poisson summation formula.

Let <math>\omega_{\max}</math> be the maximum frequency of <math>F(\omega)</math>, then the support of <math>F(\omega)</math> is bounded to <math>\left[ -\omega_{\max}, \omega_{\max} \right]</math>. The bandwidth of <math>F(\omega)</math> is then <math>2 \omega_{\max}</math>. In order for a replicated <math>F(\omega)</math> shifted by <math>\omega_\mathrm{s}</math> to not overlap then the condition <math>2 \omega_{\max} < \omega_\mathrm{s}</math> must hold true. In this case <math>F^*(\omega)=F(\omega)</math> on <math>\omega\in[-\omega_\mathrm{s}/2,\omega_\mathrm{s}/2]</math> and

<math>

f(t) = \frac1{\sqrt{2\pi}} \int_{-\omega_\mathrm{s} / 2}^{\omega_\mathrm{s} / 2} F(\omega) e^{i\omega t} \, d\omega =\frac{T}{2\pi}\sum_{n=-\infty}^{\infty} f(nT) \int_{-\omega_\mathrm{s}/2}^{\omega_\mathrm{s}/2} e^{i\omega (t-nT)}\,d\omega </math>

<math> = \sum_{n=-\infty}^{\infty} f(nT) \, \frac{\sin(\pi(t/T-n))}{\pi(t/T-n)}
[edit]

\sum_{n

-\infty}^{\infty} f(nT) \, \mbox{sinc}_N(t/T-n)

</math>, the last expression being a cardinal series with the cardinal sine function as interpolation kernel.

Otherwise, if <math>\omega_\mathrm{s}</math> is not sufficiently large then the terms of the summation will overlap and aliasing will be introduced.

Although the theorem states that the sampling rate must be twice the bandwidth, it can readily be seen that this proof still holds. One has just to make sure that <math>F^*(\omega) = F(\omega)</math> on the frequency support, i.e., of all the <math>F(\omega + n \omega_\mathrm{s})</math>, <math>n\in\mathbb Z</math>, only one should be nonzero for all <math>\omega</math>. For real bandlimited signals one must assume that the frequency support is centered about zero.

[edit]

See also

[edit]

References

  • E. T. Whittaker, "On the Functions Which are Represented by the Expansions of the Interpolation Theory," Proc. Royal Soc. Edinburgh, Sec. A, vol.35, pp.181-194, 1915
  • H. Nyquist, "Certain topics in telegraph transmission theory," Trans. AIEE, vol. 47, pp. 617-644, Apr. 1928.
  • V. A. Kotelnikov, "On the carrying capacity of the ether and wire in telecommunications," Material for the First All-Union Conference on Questions of Communication, Izd. Red. Upr. Svyazi RKKA, Moscow, 1933 (Russian).
  • C. E. Shannon, "Communication in the presence of noise," Proc. Institute of Radio Engineers, vol. 37, no.1, pp. 10-21, Jan. 1949.
[edit]

External links

es:Teorema de muestreo de Nyquist-Shannon fr:Théorème d'échantillonnage de Nyquist-Shannon it:Teorema del campionamento di Nyquist-Shannon he:תורת הדגימה nl:Bemonsteringstheorema van Nyquist-Shannon ja:標本化定理 pl:Twierdzenie Kotielnikowa-Shannona ru:Теорема отсчётов Уиттакера—Найквиста—Котельникова—Шеннона fi:Nyquistin teoreema