Machine Learning Applications for Remote Sensing

8 min readMar 4, 2021

As part of my Environmental Science program I took during college, I took a few courses on remote sensing. For those who haven’t heard this term, remote sensing refers to the use of sensors to detect information about the earth’s surface or atmosphere from a distance. The classic example is the use of satellites, but there are also sensors used on aircraft, and there’s an emerging application of drones and other small vehicles for remote sensing.

Introduction

I’ll go over a bit of the history of the field, and then get into what kinds of data you can get with remote sensing imagery. Then, I’ll cover some really interesting things we can do with these data, and some case studies of how researchers and scientists have used these data to do some amazing things that you might not intuitively expect could be done using these kinds of images.

Early History

The first thing that we could really call remote sensing was the use of balloons to take aerial photos. This was in the late 1800s. There wasn’t really any way to scientifically analyze these photos, but it’s still pretty cool that we can look back on images from so long ago.

“Boston, as the Eagle and the Wild Goose See It” taken by James Black in 1860.

Military History

The military and intelligence agencies poured a lot of resources into this field. This was all done by plane at first. While these planes took photos using film, the resolution is still amazing. Analysts used microscopes to examine the pictures, and could achieve resolution letting them see things like the serial numbers on top of military vehicles or parked airplanes, all from an altitude of 70,000 feet, around twice the altitude of a passenger plane.

A photo of Aleppo, Syria, taken by a U2 spy plane in 1959.

The First Satellites

Then came Sputnik and the era of satellites. This provided a more systematic way to photograph the Earth’s surface. If you can believe it, satellites were loaded with reels of film and would take photos ordered by radio commands. When rolls of film were full, the satellite would literally drop the film downward with a parachute. Then, a plane would fly by and snag the bucket of film for processing. These were still used for military purposes, and focused on getting extremely high resolutions over a very small area. It wouldn’t be until later that the other lane of satellite image technology emerged, focusing on increasing the area of the image rather than the resolution.

The LANDSAT program

Of course we’ve made a bit of progress from the 50s. Now, the big player in the game is NASA’s LANDSAT program. That’s LANDSAT 9 on the left there, scheduled to be launched next September. Since the 1970s, the LANDSAT satellites have created a complete archive of the entire surface of the Earth, creating digital images that we can look back on. The satellite follows an orbit around the Earth that’s carefully calculated so that the Earth turns under the satellite and the satellite can capture images that slightly overlap with one another. These swaths that the satellite can capture are quite large in a single go. It still takes LANDSAT 16 days to capture the entire Earth’s surface. This means that we can work with time series data for a specific location.

Resolution

The resolution of these satellites is much more coarse than the spy satellites that I’ve described, due to the fact that there’s so much ground to cover. The current generation of LANDSAT, LANDSAT 8 has a pixel size of 15 square meters for visible light. That means that these images are best for studying very large-scale phenomena, such as deforestation or the growth of deserts or urban areas. You can also get these for free from the US Geological Survey which is nice. You can see that these images are huge, and it’s impressive that a satellite can get this much coverage in a single snapshot. The little white box there is Seattle, and the resolution really isn’t so bad considering the area of the photo. You can even see large ships in the bay if you look closely. The image on the left is tilted because the course that the satellite takes isn’t perfectly north to south. Also, if you know anything about Washington state, it’s rarely cloud-free like this, but the USGS provides ways to filter images and find ones without cloud cover over your target area.

LANDSAT 8 image taken over Washington State.

How to Use the Data

When it comes to machine learning and data science applications, these images are pretty easy to work into the methods that we’ve learned so far. The LANDSAT platform has a variety of cameras that all capture a specific range of energy. The visible light band is just one of many that the cameras capture. Each range can be used as a feature for analysis, and there’s some interesting feature engineering that can be done here. There are vegetation health indices that use a linear combination of multiple ranges to determine the water content of vegetation from the imagery. This image is a vegetation health index that I created from photos taken from a drone over farmland in Washington State. The areas of red correspond with areas where the vegetation has a higher water content. This can be done because every material or combination of materials absorbs a different proportion of light in each specific energy range. By combining all the ranges together, we can get a good idea of what we’re looking at.

True-color photo of Washington State farmland.

Land Cover

This image is a land use classification map that I created as part of my course, covering the city of Bellingham in Washington State. Just from the way the material absorbs light on the ground, we can correctly determine the amount of farmland in yellow, urban land in grey, and even the difference between old trees and those that have been recently harvested, the different shades of green here. This was done using unsupervised classification, meaning that I had to examine a true-color image of the area, and manually assign all the spectral classes to land cover categories. If you want to do something like this, you could just use a single image. You could expand on that by using images across a timespan for a region in order to see the changes in land cover or land usage. But there are other kinds of remote sensing data that can get much more detailed, at the cost of a smaller survey area.

LIDAR

If you want higher resolutions without joining the CIA and getting the really good stuff, you have to get closer to the surface. This is where LiDAR enters the picture. Instead of using the light that bounces off the Earth, LiDAR uses lasers to get a picture of the surface. This is often done from planes that fly at low altitudes over the target area. Unfortunately, this makes it a lot more expensive than the LANDSAT stuff, which is free. But with the extra price tag, you get much much better resolutions. LiDAR can be accurate down to the centimeter, and can actually create a 3-d model of the terrain that the plane flies over. You could fly over Manhattan and create a 3D model that would be able to detect a loose quarter on the sidewalk.

Applications: Archaeology

That leads me to the first really cool application of this technology. LiDAR is very powerful because we can do more than just see what the surface looks like. The sensors can detect a so-called “first return” — the first laser to return to the sensor after bouncing off something. But the sensor can also distinguish a “last return” — the last returning laser. This can be used to see through dense forest canopies, since a few beams of light make it through the canopy to the forest floor. This allowed researchers in Mexico to fly over the jungle, where they were able to detect 20,000 previously unknown Mayan archeological sites, doing in 45 minutes what would take a decade of backbreaking work on foot.

Forest canopy and underlying Mayan sites detected using LIDAR.

Applications: Famine Detection

I wanted to give another example of how one could use the low-resolution, large-scope images to do something really useful. Using satellite data from central Africa, a team at USAID was able to create a famine early warning system that could correctly predict the effect of drought on crop production without the need for on-the-scene information gathering. This kind of tool allows the international community to send aid before the famine even begins, without the need for expensive monitoring teams in regions that are often very hard to get to.

USAID near-term food insecurity monitoring system.

Conclusion

The reason I wanted to share some of the history of the field and some of the really interesting things that have been done with the technology is to maybe spark some of your interests in looking into this as a possible project idea, either for the passion project or for a future project. I know we haven’t gone super in depth on working with images, but I just wanted to make sure that people knew these sorts of images were out there for free. I haven’t even gotten into the sorts of preprocessing that we have to do with these images, like removing cloud cover and performing color correction when combining images, or the many many platforms that exist besides LANDSAT, but this is just a huge field and there’s no way I could get to everything. I’m certainly excited to learn more about what kinds of problems we can solve using these resources and this technology. If anyone is interested in the nitty gritty of getting this stuff to work in python, there are some free tutorials offered by Earth Lab that I’ve been dipping into. Earthpy is an extensive library for this sort of thing that’s worth checking out if you’re interested.