How NASA and IBM Are Using Geospatial Data and AI to Analyze Climate Risks

Kevin Murphy:

If I had a crystal ball and I could look forward five years or six years and to see how these types of models might really fundamentally change how we do science, it would be that we could have a natural language interaction with the petabytes of spatial data or geospatial data that we collect each year.

Michael Torrance:

Welcome to sustainability leaders. I'm Michael Torrance, Chief Sustainability Officer with BMO Financial Group. On the show, we will talk with leading sustainability practitioners from the corporate investor, academic, and NGO communities to explore how this rapidly evolving field of sustainability is impacting global investment business practices and our world.

Speaker 3:

The views expressed here are those of the participants and not those of Bank of Montreal, its affiliates or subsidiaries.

Michael Torrance:

So, welcome, Kevin and Raghu, to the podcast, and I'd love to learn more about the both of you and the work that you're doing with NASA. Maybe Kevin, why don't we start with you? If you could just tell us about your background and what you do at NASA.

Kevin Murphy:

Michael, thanks a bunch. My name's Kevin Murphy. I'm the Chief Science Data Officer at NASA. I'm within the science mission directorate. My job is really to make all of the amazing data collected by NASA available to as many people as possible. One of the things that people really don't know that NASA does in terms of science is provide critical global observations of the earth including information on fires and sea level change along with the atmospheric conditions. And we try to make that available to as many people as possible.

Michael Torrance:

Thanks, Kevin. And Raghu, what about you? What does your scope of work tell?

Raghu Ganti:

Thanks, Michael. My name is Raghu Ganti. I'm a Principal Research Scientist at IBM Research, and I co-lead the foundation model platform for our Watson X mission. I think I'm sure by now everybody has heard about Watson X from IBM. And how you do training and validation and tuning of these models is where my scope is. And I work very closely with the PyTorch community as well. It's all about how to train models faster, how do we train larger models, how do we make these happen for not just languages, but also other domains, including geospatial and so many other various kinds of data that we are looking at?

Michael Torrance:

Kevin, starting with you, in terms of data collection, is data being collected from satellites, and can you give us a sense of much data collection is happening, what are the areas of focus, how long has it been happening for? Just to give us a sense of the depth and scope of the information that you're collecting.

Kevin Murphy:

Sure. NASA collects a view of the earth from the seas to the tops of the atmosphere every day. When we do that in a variety of different ways, with over 20 satellites and instruments that constantly orbit the earth, we generate about 25 petabytes of new data each year that we use to understand how the various environmental systems of the earth interact and work together and are influenced by one another.

Michael Torrance:

And so, when you're collecting data about the earth, is it through satellites or do you have any other types of sensors that can pick up and collect data about the earth?

Kevin Murphy:

We collect data not only from satellites, but from airplanes as well as on the ground. And we do this collection in that multi-scaler approach so that we can make sure that what we do detect from satellites is actually what we see on the ground. We do that pretty often. We calibrate our measurements so that they're very precise and really represent what the environment is telling us.

Michael Torrance:

And then, what do you do with this data that you're collecting?

Kevin Murphy:

All the data that we collect is available publicly to anyone for any purpose. We take the raw data from the satellites and we turn it into higher level products, and eventually, maps, which people can download or visualize.

Michael Torrance:

In terms of your audience, who do you find actually uses your data?

Kevin Murphy:

We have users from scientists all the way to decision makers or people that are influencing policy. We work with different organizations within the US government and other governments to provide this critical information.

Michael Torrance:

So, Kevin, then the data that you're collecting could, I imagine, be used for a wide range of purposes. One of the things that we are using geospatial data for is understanding physical climate hazards, so things like flood risks, potential wildfires, extreme weather events. Is NASA data being used for those purposes, and are there any projects or initiatives that you can speak to?

Kevin Murphy:

NASA collects this data, which really tells us about the environmental conditions as they existed when the satellites did their overpass. But we also invest in developing models that help predict or evaluate the impacts of exactly the things that you were just talking about. We both collect the observations, and then, develop the models, and then, run the models and evaluate those models against the observations which we collect. We support research into everything from sea level change, to the changing fire dynamics, to water availability. And all of those things, I think, are really critical to assess geospatial environmental risks.

Michael Torrance:

What types of teams do you have at NASA for understanding and building out those models?

Kevin Murphy:

So, this is a necessarily interdisciplinary across agency and partnerships with academic institutions and international bodies types of work. We have grantees, we have science teams with both NASA, and other agencies, and academic participants who really focus down and provide their various types of expertise to understanding the data we collect and the development of those models.

Michael Torrance:

Is there a role for the private sector, Kevin? Do you work with any private companies on how to utilize this data for planning or other purposes?

Kevin Murphy:

We provide a user support to anyone who comes to one of our archives to use the products. We have a variety of different resources on our websites which teach people how to use the products and how to use them correctly for different types of activities. One of the things that we have available that a lot of people have liked lately is our data pathfinder series. And those data pathfinders really walk people through the correct products for things like wildfires or sea level change and how to use those.

Michael Torrance:

Oh, that's fantastic. Raghu, for your work with IBM, do you work directly with these NASA data sets?

Raghu Ganti:

In the context of geospatial model? I think this is a first of a kind because foundation models, I'm sure everybody has heard, in the context of language, you've heard of ChatGPT, obviously. People are familiar with language, and we ask the question, "Hey, the same kind of innovations that are happening in the language space, how do we bring them to the world of geospatial?" So, that's where we started. People have been applying deep learning models on these kinds of data, but never a foundation model where you're doing the self supervised training. So, that's basically what defines a foundation model. And the popular thing that people do today is using transformer architectures. Can we pick data from geospatial world? And of course, naturally, NASA was the best partner because they know that data the best. So, we ask the question jointly, what would be a good starting point to build that model and demonstrate what it is capable of? Can it do something better?

Michael Torrance:

And so, can you tell a little bit more about that? What is that project? What does that involve, and what are your goals?

Raghu Ganti:

Absolutely. Can we create a model? That's a basic question that we are asking. Initially, working with NASA, we identified one specific data set that has a broad user base, which is the Harmonized Landsat and Sentinel-2 data. And it was one of those products where the earth science team said, "Let's start with this data set, and can we apply the transformer technology and foundation models to build a model?" So, that's where we started. And the question was as simple as does this model learn something?

Michael Torrance:

And just to be clear, this is using artificial intelligence, that's what the foundation model is for?

Raghu Ganti:

Absolutely. It is using generative AI principles in the context of geospatial data.

Michael Torrance:

That's an innovation because, Kevin, I think when you're talking about the models that NASA has built, those are scientific models that are able to interpret data. Are the types of models you were referring to, are they utilizing AI or is this an innovation that would take it a step further and use this new technology to try to develop deeper insights into the data that you've acquired?

Kevin Murphy:

I think partnerships with companies like IBM who really embody the same values as NASA, in terms of openness and sharing of information, are key to exploiting all of this information we have. So, you're right, historically, we've used physics-based models to do a lot of the type of modeling that we were talking about earlier. But the utilization of foundation models for understanding environments is a new area that shows a lot of promise. And we're really happy to be partnering with IBM to really investigate how well that will work for a lot of these types of issues we have.

Michael Torrance:

What would you say the end state of this work could be? Would it allow for predictive capability that would be an advancement on the current state? Are there other objectives that you have for this?

Kevin Murphy:

If I had a crystal ball and I could look forward five years or six years and to see how these types of models might really fundamentally change how we do science, it would be that we could have a natural language interaction with the petabytes of spatial data or geospatial data that we collect each year. It's very difficult today for more novice users to interact with that information. And if you were able to have a natural language interaction saying, "What does fire in California look like over the past five years? Is that increasing or decreasing, and what's causing it?" Would be an amazing tool that will really broaden the ability for people to understand how the climate is changing and how they might be able to interact with it.

Michael Torrance:

Raghu, what are your thoughts?

Raghu Ganti:

The other side of the coin is really understanding how the technology in AI evolves with addressing a problem of this scale. And there is accessibility from a language domain, which is being well understood, but I think language AI is giving us the direction of how to do the same, but with many differences. And the difference is that you're looking at physics-based data. And the question is, does the model actually learn the physics aspects of it? Is it learning something which is just memorizing the whole thing, and then, predicting some things very well? Or is it actually learning the aspects of weather changes that are resulting in a hurricane or resulting in a wildfire?

So, we are not yet there, but as we stand today, we know that foundation models can apply to geospatial data. They have shown promise. They have delivered in the very first time that we have attempted to build a model of this size and quality. Now, this opens up the door to a whole new dimension. So, can we start getting to this holy grail that Kevin is talking about? Accessibility to the large data sets that NASA has and having a natural language interaction using a multimodal model will be the Holy Grail. And we are at a starting point of, yes, foundation models have delivered on their first iteration.

Kevin Murphy:

As you said, Raghu, we are in the very early stages of this, and as we move forward to understand how these foundation models and large language models can be utilized by science, we still want to apply the same scientific rigor to the results as we would with any type of research activity. So, we're really invested in making sure that not only are these capabilities accessible to people, but they're also accurate.

Michael Torrance:

And Raghu, you've mentioned wildfires as one of the use cases. Are there other topics, flooding or other kinds of physical climate hazards, for example, that you're focused on?

Raghu Ganti:

One of the things that we have done at IBM and NASA is to release this foundation model in open source through Hugging Face. And when we worked together, we developed two specific use cases. One was flood mapping, which was led more from the IBM side, and the other was burn scar mapping, which was more led from the NASA side. And in collaboration with Clark University, a third use case that a variant of that model has been put again back into Hugging Face is this multi-crop classification. So, you can take different categories of crops, and then, classify, and tag, saying that this region is this particular crop, and so on and so forth. So, it's a wide variety of use cases, and I think multi-crop classification is addressing more in the agricultural domain. Burn scar mapping is more in the wildfire and disaster kind of domain. Whereas flood mapping is of course in the disaster domain, but very different kinds of disasters.

Michael Torrance:

For these projects, do you have any other partners than NASA that are working with you?

Raghu Ganti:

We are looking at two ways of expanding. One is through our EIS product, which is part of our sustainability software portfolio, looking at others being able to leverage these models, consume it in their enterprise use cases. And the second is through the open science community being able to put that model in the open source. The goal is for us to attract all the climate researchers being able to come use this model. We are hoping that releasing this model will cause a snowball effect, that more and more people come up with more and more use cases and start demonstrating what the model is capable of and what are the gaps in the model so that we can think about what will the future look like?

Michael Torrance:

And in terms of bringing this forward, when do you think this would be past the experimental phase?

Raghu Ganti:

I would say in the next three to six months. Right now, it is already out in the open, and people can go access it from Hugging Face, which is by far the most popular community for these kinds of AI models. We are hoping that in the future, we will have other kinds of partnership based activities, which can boost the leveraging of these models.

Michael Torrance:

Kevin, what do you know about the way that your work generally is being used or influencing the policy space? We're kind of coming at this from different perspectives. I'm working for a financial institution. You're working for one of the world's premier organizations focused on science. There's organizations like the IPCC that are setting out models and analysis that are being used by policy. That policy is driving the engagement of regulators, and that's driving engagement of companies like BMO in the financial sector. But you're also on the science side of it, and I would imagine that the models and the data that you're collecting could be informative of those policy processes. Are you aware of IPCC, or the UN, or other organizations that are leveraging your data and your models to do their own work?

Kevin Murphy:

We certainly do support all of the efforts to really model the environment within NASA and with other government and international organizations. So, yeah, our information is incorporated into those reports. I think that if you look through the IPCC or any other ones, you'll see a lot of references to the critical observations and model output NASA makes.

Michael Torrance:

That's fantastic. Raghu, for the work that you're doing, is it only AI that you're focused on or are you using any other techniques to draw insights from all of this data?

Raghu Ganti:

I think the primary focus is AI, but AI, in general, will require a lot of pieces to come together. The model itself, you can think of it as a heart, but surrounding it, making sure that the model is capable of doing something meaningful is not AI, and that's also where a platform comes into play. I think that is going to be a critical aspect. So, AI on its own is worth only so much. It is about how do we build the scaffolding around it, how do we make it operational? Those are becoming very, very important.

Michael Torrance:

Kevin, what other things are you working on these days in your role?

Kevin Murphy:

Oh, wow. We have so many different scientists in so many different communities that we're working on all sorts of different things. Everything from the next discoveries for the James Webb Space Telescope, to looking at the sun and understanding its dynamics, to the future explorations of the moon and Mars. So, we have a lot of things going on. Certainly, foundation models in our collaboration with IBM in this regard is a really, really fruitful area, and we hope that we can apply these same types of techniques to some of the other data sets that we have.

Michael Torrance:

Great. Well, thank you both very much. I think that was a really interesting discussion about kind of laid the foundation for NASA's data collection work, the fact that it's publicly available, and then, Raghu's project with AI. I thought that was super interesting.

Kevin Murphy:

Thank you for having me.

Raghu Ganti:

Yeah. Thank you, Michael.

Michael Torrance:

Thanks for listening to Sustainability Leaders. This podcast is presented by BMO Financial Group. To access all the resources we discussed in today's episode and to see our other podcasts, visit us at bmo.com/sustainabilityleaders. You can listen and subscribe free to our show on Apple Podcasts or your favorite podcast provider, and we'll greatly appreciate a rating, and review, and any feedback that you might have. Our show and resources are produced with support from BMO's marketing team and Puddle Creative. Until next time, I'm Michael Torrance. Have a great week.

Raghu Ganti:

For BMO disclosures, please visit bmocm.com/podcast/disclaimer.