Building ChatGPT-style tools with Earth observation

[Pelican Press 0]

Building ChatGPT-style tools with Earth observation

A giant, geological wonder in the Sahara Desert of Mauritania is pictured in this satellite image. The 40 km-diameter circular Richat structure is one of the geological features that is easier to observe from space than from down on the ground, and has been a familiar landmark to astronauts since the earliest missions. Once thought to be the result of a meteor impact, researchers now believe it was caused by a large dome of molten rock uplifting and, once at the surface, being shaped by wind and water into what we see today. Concentric bands of resistant quartzite rocks form ridges, with valleys of less-resistant rock between them. The dark area on the left is part of the Adrar plateau of sedimentary rock standing some 200 m above the surrounding desert sands. A large area covered by sand dunes—called an erg—can be seen in the lower-right part of the image, and sand is encroaching into the structure’s southern side. Zooming in on the southern side of the bullseye, we can see individual trees and bushes as tiny dots. These follow a river-like structure that appears to have been dry when this image was acquired, a few weeks after the rainy season. Some areas to the south and east of the Richat appear to be covered with temporary lakes, which are dry for most of the year. Credit: JAXA/ESA

Imagine being able to ask a chatbot, “Can you make me an extremely accurate classification map of crop cultivation in Kenya?” or “Are buildings subsiding in my street?” And imagine that the information that comes back is scientifically sound and based on verified Earth observation data.

ESA, in conjunction with technology partners, is working to make such a tool a reality by developing AI applications that will revolutionize information retrieval in Earth observation.

A digital helping hand for data

Earth observation generates vast volumes of vital data every day, but it is difficult for humans alone to ensure that we obtain the best value from that data. Fortunately, AI helps in interacting with such large and complex datasets, identifying key features and presenting the information in a user-friendly format.

I*STAR for example, an activity co-funded by the ESA InCubed program, developed a platform that uses AI to monitor current events like earthquakes or volcano eruptions so that satellite operators can automatically plan the next data acquisitions for customers.

The SaferPlaces AI tool, again supported by InCubed, creates flood maps for disaster response teams by merging in situ measurements with satellite data. SaferPlaces was crucial to damage assessment efforts during last year’s floods in Emilia-Romagna in Italy.

In the last few years, the progress of AI has accelerated tremendously, with the advance of tools such as ChatGPT and Gemini even surprising experts in the field. To take advantage of this transformative innovation and capture the opportunities enabled by this technology, a natural next step is to build a ChatGPT-style text-based enquiry with Earth observation data.

Along with various partners from the fields of space, computing and meteorology, ESA is currently developing an Earth observation digital assistant that will understand human queries and respond with human-like answers—known as natural language capabilities.

Not surprisingly though, there are a number of pieces of the jigsaw puzzle to complete to create such a digital assistant, starting with the powerhouse that underpins it, the foundation model.

The motor roaring under the bonnet

AI models work by training and improving over time, but in more traditional machine learning, the machine has to be fed with large sets of data that have been labeled, often by a human.

Enter foundation models, which take a very different approach. A foundation model is a machine learning model that trains, largely without human supervision, on sizeable and varied sources of unlabelled data. Foundation models are quite general, but can be tailored to specific applications.

The result is a flexible, powerful AI engine, and since their inception in 2018 foundation models have contributed to a huge transformation in machine learning, impacting many industries and society as a whole.

ESA Φ-lab has several ongoing initiatives for creating foundation models dedicated to Earth observation-related tasks. These models use data to provide information on environmentally critical topics such as methane leaks and extreme-weather-event mitigation.

One foundation model project, PhilEO, started at the beginning of 2023 and is now reaching maturity. An evaluation framework based on global Copernicus Sentinel-2 data, and soon the PhilEO model itself, are being released to the Earth observation community in order to stimulate a collaborative approach, advance development in the field and ensure the derived foundation model is extensively validated.

The image above shows the Richat Structure, the type of feature that the PhilEO model has learnt to recognize without human supervision.

The human interface

Separate ESA initiatives are looking into the human end of the jigsaw puzzle—creating the digital assistant that will take a natural language question from a user, process the right data through Earth observation foundation models and produce the answer in text and/or images.

A

This is the hidden content, please

• Sign In
• or
• Sign Up

has recently demonstrated that its digital assistant prototype can carry out multimodal tasks, searching among multiple data archives such as Sentinel-1 and 2 to compare information.

An ESA Φ-lab activity due to start in April will explore natural language processing for extracting and analyzing information from verified Earth observation text sources, together with interpreting queries from both experts and general users. This activity will ultimately lead to the creation of a fully functioning digital assistant.

“The concept of an Earth observation digital assistant that can provide a broad range of insight from varied sources is a tantalizing prospect, and as these initiatives show, there are a number of fundamental building blocks to put in place to achieve that aim,” comments Head of ESA Φ-lab Giuseppe Borghi.

“Given the extremely encouraging progress already achieved with PhilEO and the digital assistant precursor, I fully expect the new projects to yield game-changing results in the near future.”

Provided by
********* Space Agency

Citation:
Building ChatGPT-style tools with Earth observation (2024, March 25)
retrieved 25 March 2024
from

This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no
part may be reproduced without the written permission. The content is provided for information purposes only.

This is the hidden content, please

• Sign In
• or
• Sign Up

Science, Physics News, Science news, Technology News, Physics, Materials, Nanotech, Technology, Science
#Building #ChatGPTstyle #tools #Earth #observation

This is the hidden content, please

• Sign In
• or
• Sign Up