Filling in gaps in the observations: Difference between revisions

From Climate Change AI Wiki
(Very first commit... still to do. Feel free to change /rewrite as much as you want.)
 
Line 4: Line 4:
   
 
Historical record provides valuable information for evaluating the performance of climate models with respect to the observed changes. However, especially early historical observations are available only for sparse regions. ML can help with filling in the gaps in observations to provide a complete record for different climate variables, such as ocean carbon uptake<ref>{{Cite journal|last=Landschützer|first=P.|last2=Gruber|first2=N.|last3=Bakker|first3=D. C. E.|last4=Schuster|first4=U.|last5=Nakaoka|first5=S.|last6=Payne|first6=M. R.|last7=Sasse|first7=T. P.|last8=Zeng|first8=J.|date=2013-11-29|title=A neural network-based estimate of the seasonal to inter-annual variability of the Atlantic Ocean carbon sink|url=https://bg.copernicus.org/articles/10/7793/2013/|journal=Biogeosciences|language=English|volume=10|issue=11|pages=7793–7815|doi=10.5194/bg-10-7793-2013|issn=1726-4170}}</ref><ref>{{Cite journal|last=Gregor|first=Luke|last2=Lebehot|first2=Alice D.|last3=Kok|first3=Schalk|last4=Scheel Monteiro|first4=Pedro M.|date=2019-12-10|title=A comparative assessment of the uncertainties of global surface ocean CO2 estimates using a machine-learning ensemble (CSIR-ML6 version 2019a) – have we hit the wall?|url=https://gmd.copernicus.org/articles/12/5113/2019/|journal=Geoscientific Model Development|language=English|volume=12|issue=12|pages=5113–5136|doi=10.5194/gmd-12-5113-2019|issn=1991-959X}}</ref> or surface air temperature using neural networks<ref>{{Cite journal|last=Kadow|first=Christopher|last2=Hall|first2=David Matthew|last3=Ulbrich|first3=Uwe|date=2020-06|title=Artificial intelligence reconstructs missing climate information|url=https://www.nature.com/articles/s41561-020-0582-5|journal=Nature Geoscience|language=en|volume=13|issue=6|pages=408–413|doi=10.1038/s41561-020-0582-5|issn=1752-0908}}</ref>, Kriging<ref>{{Cite journal|last=Cowtan|first=Kevin|last2=Way|first2=Robert G.|date=2014|title=Coverage bias in the HadCRUT4 temperature series and its impact on recent temperature trends|url=https://rmets.onlinelibrary.wiley.com/doi/abs/10.1002/qj.2297|journal=Quarterly Journal of the Royal Meteorological Society|language=en|volume=140|issue=683|pages=1935–1944|doi=10.1002/qj.2297|issn=1477-870X}}</ref><ref>{{Cite journal|last=Morice|first=C. P.|last2=Kennedy|first2=J. J.|last3=Rayner|first3=N. A.|last4=Winn|first4=J. P.|last5=Hogan|first5=E.|last6=Killick|first6=R. E.|last7=Dunn|first7=R. J. H.|last8=Osborn|first8=T. J.|last9=Jones|first9=P. D.|last10=Simpson|first10=I. R.|title=An updated assessment of near-surface temperature change from 1850: the HadCRUT5 dataset|url=https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/2019JD032361|journal=Journal of Geophysical Research: Atmospheres|language=en|volume=n/a|issue=n/a|pages=e2019JD032361|doi=10.1029/2019JD032361|issn=2169-8996}}</ref>, or Empirical Orthogonal Functions<ref>{{Cite journal|last=Benestad|first=R. E.|last2=Erlandsen|first2=H. B.|last3=Mezghani|first3=A.|last4=Parding|first4=K. M.|date=2019|title=Geographical Distribution of Thermometers Gives the Appearance of Lower Historical Global Warming|url=https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/2019GL083474|journal=Geophysical Research Letters|language=en|volume=46|issue=13|pages=7654–7662|doi=10.1029/2019GL083474|issn=1944-8007}}</ref>.
 
Historical record provides valuable information for evaluating the performance of climate models with respect to the observed changes. However, especially early historical observations are available only for sparse regions. ML can help with filling in the gaps in observations to provide a complete record for different climate variables, such as ocean carbon uptake<ref>{{Cite journal|last=Landschützer|first=P.|last2=Gruber|first2=N.|last3=Bakker|first3=D. C. E.|last4=Schuster|first4=U.|last5=Nakaoka|first5=S.|last6=Payne|first6=M. R.|last7=Sasse|first7=T. P.|last8=Zeng|first8=J.|date=2013-11-29|title=A neural network-based estimate of the seasonal to inter-annual variability of the Atlantic Ocean carbon sink|url=https://bg.copernicus.org/articles/10/7793/2013/|journal=Biogeosciences|language=English|volume=10|issue=11|pages=7793–7815|doi=10.5194/bg-10-7793-2013|issn=1726-4170}}</ref><ref>{{Cite journal|last=Gregor|first=Luke|last2=Lebehot|first2=Alice D.|last3=Kok|first3=Schalk|last4=Scheel Monteiro|first4=Pedro M.|date=2019-12-10|title=A comparative assessment of the uncertainties of global surface ocean CO2 estimates using a machine-learning ensemble (CSIR-ML6 version 2019a) – have we hit the wall?|url=https://gmd.copernicus.org/articles/12/5113/2019/|journal=Geoscientific Model Development|language=English|volume=12|issue=12|pages=5113–5136|doi=10.5194/gmd-12-5113-2019|issn=1991-959X}}</ref> or surface air temperature using neural networks<ref>{{Cite journal|last=Kadow|first=Christopher|last2=Hall|first2=David Matthew|last3=Ulbrich|first3=Uwe|date=2020-06|title=Artificial intelligence reconstructs missing climate information|url=https://www.nature.com/articles/s41561-020-0582-5|journal=Nature Geoscience|language=en|volume=13|issue=6|pages=408–413|doi=10.1038/s41561-020-0582-5|issn=1752-0908}}</ref>, Kriging<ref>{{Cite journal|last=Cowtan|first=Kevin|last2=Way|first2=Robert G.|date=2014|title=Coverage bias in the HadCRUT4 temperature series and its impact on recent temperature trends|url=https://rmets.onlinelibrary.wiley.com/doi/abs/10.1002/qj.2297|journal=Quarterly Journal of the Royal Meteorological Society|language=en|volume=140|issue=683|pages=1935–1944|doi=10.1002/qj.2297|issn=1477-870X}}</ref><ref>{{Cite journal|last=Morice|first=C. P.|last2=Kennedy|first2=J. J.|last3=Rayner|first3=N. A.|last4=Winn|first4=J. P.|last5=Hogan|first5=E.|last6=Killick|first6=R. E.|last7=Dunn|first7=R. J. H.|last8=Osborn|first8=T. J.|last9=Jones|first9=P. D.|last10=Simpson|first10=I. R.|title=An updated assessment of near-surface temperature change from 1850: the HadCRUT5 dataset|url=https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/2019JD032361|journal=Journal of Geophysical Research: Atmospheres|language=en|volume=n/a|issue=n/a|pages=e2019JD032361|doi=10.1029/2019JD032361|issn=2169-8996}}</ref>, or Empirical Orthogonal Functions<ref>{{Cite journal|last=Benestad|first=R. E.|last2=Erlandsen|first2=H. B.|last3=Mezghani|first3=A.|last4=Parding|first4=K. M.|date=2019|title=Geographical Distribution of Thermometers Gives the Appearance of Lower Historical Global Warming|url=https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/2019GL083474|journal=Geophysical Research Letters|language=en|volume=46|issue=13|pages=7654–7662|doi=10.1029/2019GL083474|issn=1944-8007}}</ref>.
  +
  +
=== Kriging ===
  +
Kriging is a statistical method of spatial prediction that (usually) relies on the second-order properties of the process analyzed. The aim of this method is to predict the value of our process at any location of interest <math>s_0</math> within a domain <math>D</math>, using the available information of our sample. Under specific assumptions, it will be the Best Linear Unbiased Predictor (BLUP), e.g. the estimator with the minimal error variance. The main idea of Kriging is that nearby locations will tend to be similar to those more distant, which is a prediction framework it translates into a higher ''relevance'' to those nearby observations. One of the most common formulations of the Kriging estimator is:
  +
  +
<math>\hat{Z}(s_0) = \sum_{j=1}^{n} \lambda_j Z(s_j)</math>
  +
  +
This means that the value at the prediction location <math>s_0</math> will be a linear combination of our sample. The parameters <math>\lambda_j</math> aim to encode this '''relevance''<nowiki/>' for the observation <math>Z(s_j)</math> over the location prediction <math>\hat{Z}(s_0)</math>. Clearly, this encoding should (and can) consider the spatial dependence structure of our process, which makes this method stand out from others e.g. an inverse distance weighting approach. The assumptions and restrictions we made over our process and over the parameters <math>\bold{\lambda}</math> respectively, will define different variations of Kriging. The most commons are Simple Kriging, Ordinary Kriging, and Universal Kriging.
  +
  +
==== A bit of history ====
  +
This method was first employed empirically by Danie G. Krige, a South African mining engineer, on the task of determining the ore grades of panels, highly relevant since the cost of its extraction should be considerably lower than the value of the metals on the panels extracted. The work done by Krige wrapped up with his Master’ Thesis at the University of the Witwatersrand called: '''A statistical approach to some mine valuation and allied problems on the Witwatersrand''<nowiki/>' in 1951. The generalization of this method was done by the French Engineer Georges Matheron nine years later, assigning proper weights to each sample, where these weights are determined to minimize the estimation variance under a set of constraints.
  +
 
==Background Readings==
 
==Background Readings==
   

Latest revision as of 16:35, 9 April 2021

🌎 This article is a stub, and is currently under construction. You can help by adding to it!

This page is part of the Climate Change AI Wiki, which aims provide resources at the intersection of climate change and machine learning.

Historical record provides valuable information for evaluating the performance of climate models with respect to the observed changes. However, especially early historical observations are available only for sparse regions. ML can help with filling in the gaps in observations to provide a complete record for different climate variables, such as ocean carbon uptake[1][2] or surface air temperature using neural networks[3], Kriging[4][5], or Empirical Orthogonal Functions[6].

Kriging[edit | edit source]

Kriging is a statistical method of spatial prediction that (usually) relies on the second-order properties of the process analyzed. The aim of this method is to predict the value of our process at any location of interest within a domain , using the available information of our sample. Under specific assumptions, it will be the Best Linear Unbiased Predictor (BLUP), e.g. the estimator with the minimal error variance. The main idea of Kriging is that nearby locations will tend to be similar to those more distant, which is a prediction framework it translates into a higher relevance to those nearby observations. One of the most common formulations of the Kriging estimator is:

This means that the value at the prediction location will be a linear combination of our sample. The parameters aim to encode this 'relevance' for the observation over the location prediction . Clearly, this encoding should (and can) consider the spatial dependence structure of our process, which makes this method stand out from others e.g. an inverse distance weighting approach. The assumptions and restrictions we made over our process and over the parameters respectively, will define different variations of Kriging. The most commons are Simple Kriging, Ordinary Kriging, and Universal Kriging.

A bit of history[edit | edit source]

This method was first employed empirically by Danie G. Krige, a South African mining engineer, on the task of determining the ore grades of panels, highly relevant since the cost of its extraction should be considerably lower than the value of the metals on the panels extracted. The work done by Krige wrapped up with his Master’ Thesis at the University of the Witwatersrand called: 'A statistical approach to some mine valuation and allied problems on the Witwatersrand' in 1951. The generalization of this method was done by the French Engineer Georges Matheron nine years later, assigning proper weights to each sample, where these weights are determined to minimize the estimation variance under a set of constraints.

Background Readings[edit | edit source]

Conferences, Journals, and Professional Organizations[edit | edit source]

Libraries and Tools[edit | edit source]

Data[edit | edit source]

Future Directions[edit | edit source]

Relevant Groups and Organizations[edit | edit source]

References[edit | edit source]

  1. Landschützer, P.; Gruber, N.; Bakker, D. C. E.; Schuster, U.; Nakaoka, S.; Payne, M. R.; Sasse, T. P.; Zeng, J. (2013-11-29). "A neural network-based estimate of the seasonal to inter-annual variability of the Atlantic Ocean carbon sink". Biogeosciences. 10 (11): 7793–7815. doi:10.5194/bg-10-7793-2013. ISSN 1726-4170.
  2. Gregor, Luke; Lebehot, Alice D.; Kok, Schalk; Scheel Monteiro, Pedro M. (2019-12-10). "A comparative assessment of the uncertainties of global surface ocean CO2 estimates using a machine-learning ensemble (CSIR-ML6 version 2019a) – have we hit the wall?". Geoscientific Model Development. 12 (12): 5113–5136. doi:10.5194/gmd-12-5113-2019. ISSN 1991-959X.
  3. Kadow, Christopher; Hall, David Matthew; Ulbrich, Uwe (2020-06). "Artificial intelligence reconstructs missing climate information". Nature Geoscience. 13 (6): 408–413. doi:10.1038/s41561-020-0582-5. ISSN 1752-0908. Check date values in: |date= (help)
  4. Cowtan, Kevin; Way, Robert G. (2014). "Coverage bias in the HadCRUT4 temperature series and its impact on recent temperature trends". Quarterly Journal of the Royal Meteorological Society. 140 (683): 1935–1944. doi:10.1002/qj.2297. ISSN 1477-870X.
  5. Morice, C. P.; Kennedy, J. J.; Rayner, N. A.; Winn, J. P.; Hogan, E.; Killick, R. E.; Dunn, R. J. H.; Osborn, T. J.; Jones, P. D.; Simpson, I. R. "An updated assessment of near-surface temperature change from 1850: the HadCRUT5 dataset". Journal of Geophysical Research: Atmospheres. n/a (n/a): e2019JD032361. doi:10.1029/2019JD032361. ISSN 2169-8996.
  6. Benestad, R. E.; Erlandsen, H. B.; Mezghani, A.; Parding, K. M. (2019). "Geographical Distribution of Thermometers Gives the Appearance of Lower Historical Global Warming". Geophysical Research Letters. 46 (13): 7654–7662. doi:10.1029/2019GL083474. ISSN 1944-8007.