This is a term frequently used in audio measurements but it has vibration applications as well. If you pass a sine wave excitation through a linear time-invariant system, you get out a sine wave at that same frequency. But if you pass a sine wave through a non-linear system, you also get energy at different frequencies. In many cases, this energy will be at integer multiples of the original frequency (that is, at the harmonics of the frequency). Total harmonic distortion measures the amount of power at the higher harmonics versus the fundamental sine wave. Typical definitions may be (V2^2+V3^2+V4^2+...)/V1^2 or (Vrms^2 - V1^2) / (V1^2) where Vn is the voltage at the nth harmonic and Vrms is the root-mean-square voltage. See wikipedia for other definitions and more discussion.
