Although undergraduate physics majors are introduced to concepts such as the fluctuation-dissipation theorem in statistical mechanics courses, they are almost never presented with a situation (with the possible exception of Johnson noise) in which useful information is extracted from a noise measurement. The experiment presented here demonstrates how detailed spectroscopic information about an atomic system can be extracted from a noise spectrum. The experiment is essentially a noise-based analog to the popular optical pumping experiment already present in many undergraduate laboratories. In this case, Faraday rotation is used to probe spin fluctuations of a gas of alkali atoms in thermal equilibrium in a small magnetic field. The demonstration will focus on Rb-85 and Rb-87, for which the low-field spectrum in a magnetic field is described by the Breit-Rabi formalism. Transitions between Zeeman-split hyperfine levels can be observed as peaks in the noise spectrum of the Faraday rotation signal, which is read out by a fast digitizer card and then Fourier-transformed. The experiment provides an excellent introduction to atomic physics, noise measurements, and signal processing.