Algorithms

Samples that are measured using diffuse reflectance often exhibit significant differences in the spectra due to the non-homogeneous distribution of the particles. In fact, multiple spectral measurements of different aliquots of the same sample can look completely different. In many cases, the scattering can be an overpowering contributor to the spectrum, sometimes accounting for most of the variance in the data.

The degree of scattering is dependent on the wavelength of the light that is used and the particle size and refractive index of the sample. Therefore, the scattering is not uniform throughout the spectrum. Typically, this appears as a baseline shift, tilt and sometimes curvature, where the degree of influence is more pronounced at the longer-wavelength end of the spectrum.
 

Figure1. A set of 50 Log(1/R) NIR spectra of ground wheat samples measured using diffuse reflectance. The concentrations of the constituents of interest fall in a relatively narrow concentration range. However, note that the light scattering causes the spectra to appear quite different.

Standard Normal Variate Transformation (SNV) and Detrending
Standard Normal Variate correction and Detrending are other methods that attempt to remove the major effects of light scattering from the spectra. However, the calculational methods used are quite different.

This correction is actually two separate algorithms that are usually applied together. SNV is applied first to correct for the effects of the multiplicative interferences of scatter and particle size, similar to Multiplicative Scatter Correction. Detrending usually follows to attempt to remove the additional variations in baseline shift and curve linearity typically present in diffuse reflectance spectra. Although, in some cases, applying SNV alone can be more useful for interpretation of the final model vectors. Detrending is not usually applied alone; it is either used after the SNV correction, or not used at all.

SNV uses a different approach than MSC to correct for the scattering effects. Here, no external "ideal spectrum" is required. Instead, the scattering is removed by normalizing each spectrum by the standard deviation of the responses across the entire spectral range:

Mean Response:

SNV Correction:

where A is the n by p matrix of training set spectral responses for all the wavelengths, Ai is a 1 by p vector of the responses for a single spectrum in the training set, is the average of all the spectral responses in the vector, n is the number of training spectra, and p is the number of wavelengths in the spectra.
 

Figure 3. The SNV corrected spectra of the same data in Figure 1. Note that this is very similar to the results of MSC; however, no external reference ("ideal" spectrum) is required in order to calculate the correction.

The next step is to apply Detrending. As with SNV, each spectrum is treated independently of the others in the training set so that there is no external reference. It is a relatively simple calculation; a linear least squares regression is used to fit a quadratic polynomial to the responses in the spectrum. This curve is then subtracted from the spectrum to give the result. As mentioned earlier, the quadratic curvature component attempts to correct for the effects of particle size and sample packing.
 

Figure 4. Spectra of the data from Figure 1 after SNV correction and Detrending.

As with MSC, this method is only applicable to spectra that have responses that are fairly linear in concentration. Any spectra collected in Reflectance units should first be converted to Log(1/R). However, since each spectrum is corrected independently, this method may allow for greater variability in the training spectra composition than MSC.