Difference between revisions of "Filling in gaps in the observations"

This page is part of the Climate Change AI Wiki, which aims provide resources at the intersection of climate change and machine learning.

Historical record provides valuable information for evaluating the performance of climate models with respect to the observed changes. However, especially early historical observations are available only for sparse regions. ML can help with filling in the gaps in observations to provide a complete record for different climate variables, such as ocean carbon uptake[1][2] or surface air temperature using neural networks[3], Kriging[4][5], or Empirical Orthogonal Functions[6].

Kriging

Kriging is a statistical method of spatial prediction that (usually) relies on the second-order properties of the process analyzed. The aim of this method is to predict the value of our process at any location of interest ${\displaystyle s_0}$ within a domain ${\displaystyle D}$, using the available information of our sample. Under specific assumptions, it will be the Best Linear Unbiased Predictor (BLUP), e.g. the estimator with the minimal error variance. The main idea of Kriging is that nearby locations will tend to be similar to those more distant, which is a prediction framework it translates into a higher relevance to those nearby observations. One of the most common formulations of the Kriging estimator is:

${\displaystyle \hat{Z}(s_0) = \sum_{j=1}^{n} \lambda_j Z(s_j)}$

This means that the value at the prediction location ${\displaystyle s_0}$ will be a linear combination of our sample. The parameters ${\displaystyle \lambda_j}$ aim to encode this 'relevance' for the observation ${\displaystyle Z(s_j)}$ over the location prediction ${\displaystyle \hat{Z}(s_0)}$. Clearly, this encoding should (and can) consider the spatial dependence structure of our process, which makes this method stand out from others e.g. an inverse distance weighting approach. The assumptions and restrictions we made over our process and over the parameters ${\displaystyle \bold{\lambda}}$ respectively, will define different variations of Kriging. The most commons are Simple Kriging, Ordinary Kriging, and Universal Kriging.

A bit of history

This method was first employed empirically by Danie G. Krige, a South African mining engineer, on the task of determining the ore grades of panels, highly relevant since the cost of its extraction should be considerably lower than the value of the metals on the panels extracted. The work done by Krige wrapped up with his Master’ Thesis at the University of the Witwatersrand called: 'A statistical approach to some mine valuation and allied problems on the Witwatersrand' in 1951. The generalization of this method was done by the French Engineer Georges Matheron nine years later, assigning proper weights to each sample, where these weights are determined to minimize the estimation variance under a set of constraints.

References

1. Landschützer, P.; Gruber, N.; Bakker, D. C. E.; Schuster, U.; Nakaoka, S.; Payne, M. R.; Sasse, T. P.; Zeng, J. (2013-11-29). "A neural network-based estimate of the seasonal to inter-annual variability of the Atlantic Ocean carbon sink". Biogeosciences. 10 (11): 7793–7815. doi:10.5194/bg-10-7793-2013. ISSN 1726-4170.
2. Gregor, Luke; Lebehot, Alice D.; Kok, Schalk; Scheel Monteiro, Pedro M. (2019-12-10). "A comparative assessment of the uncertainties of global surface ocean CO2 estimates using a machine-learning ensemble (CSIR-ML6 version 2019a) – have we hit the wall?". Geoscientific Model Development. 12 (12): 5113–5136. doi:10.5194/gmd-12-5113-2019. ISSN 1991-959X.
3. Kadow, Christopher; Hall, David Matthew; Ulbrich, Uwe (2020-06). "Artificial intelligence reconstructs missing climate information". Nature Geoscience. 13 (6): 408–413. doi:10.1038/s41561-020-0582-5. ISSN 1752-0908. Check date values in: |date= (help)
4. Cowtan, Kevin; Way, Robert G. (2014). "Coverage bias in the HadCRUT4 temperature series and its impact on recent temperature trends". Quarterly Journal of the Royal Meteorological Society. 140 (683): 1935–1944. doi:10.1002/qj.2297. ISSN 1477-870X.
5. Morice, C. P.; Kennedy, J. J.; Rayner, N. A.; Winn, J. P.; Hogan, E.; Killick, R. E.; Dunn, R. J. H.; Osborn, T. J.; Jones, P. D.; Simpson, I. R. "An updated assessment of near-surface temperature change from 1850: the HadCRUT5 dataset". Journal of Geophysical Research: Atmospheres. n/a (n/a): e2019JD032361. doi:10.1029/2019JD032361. ISSN 2169-8996.
6. Benestad, R. E.; Erlandsen, H. B.; Mezghani, A.; Parding, K. M. (2019). "Geographical Distribution of Thermometers Gives the Appearance of Lower Historical Global Warming". Geophysical Research Letters. 46 (13): 7654–7662. doi:10.1029/2019GL083474. ISSN 1944-8007.