Dimensionality reduction through random projections for application to the retrieval of atmospheric parameters from hyperspectral satellite sensors

Physical inverse problems found on appropriate forward models, which can have highly systematic errors. As an example, in remote sensing from satellite observations, the forward model depends on spectroscopy of atmospheric gas molecules and radiative transfer modelling, whose accuracy is not perfect. The problem of correctly addressing both error components (instrument and forward model) is one of major concern in retrieval methodology. Until now, the treatment has relied on ad-hoc strategies, which makes the retrieval algorithms sub-optimal or nonoptimal at all. Optimal estimation is based on the Gaussian assumption for noise, which is normally not satisfied in presence of forward model error. In this paper, we will show that a proper Random Projections approach can provide a) an unified and coherent treatment of systematic and random errors; b) a compression tool, which can reduce the dimensionality of the data space; c) a noise model which is truly Gaussian therefore, making it possible to apply rigorously Optimal Estimation and derive the correct retrieval error; d) a simplified treatment of the inverse algebra to get the final solution. The present paper addresses the specific point of how to fully exploit the compression capability of random projections to develop an inverse algorithm able to deal with big data, and minimal loss of information content. The approach will be exemplified for IASI (Infrared Atmospheric Sounder Interfermoter) and we will show the very first physical retrieval scheme, which exploits the full IASI spectral coverage for the simultaneous retrieval of surface and atmospheric parameters. The methodology can be applied to any inverse physical problem dealing with high-dimensionality data space, how normally arises in astrophysical and Earth remote sensing science. The performance of the methodology for the retrieval of temperature and water profiles has been assessed through comparison with radiosonde observations. The retrieval accuracy, for a tropical atmosphere, is better than ± 1.25 K and ± 1.5 g/kg for temperature and water vapour, respectively. We have also performed a retrieval exercise for the Eastern China and we have shown that air quality gases, such as CO, SO2 and NH3 can be simultaneously and confidently retrieved, meaning that Random Projections preserve information content of data.