A little over five years ago, when Navy SEALS entered Osama bin Laden’s compound in Abbottabad, Pakistan, they knew exactly where to go, how many people were in the compound, what weapons were there, and whether the place had been booby trapped. They didn’t have a mole inside or a secret agent feeding them information, and they didn’t have cameras or other hidden equipment inside the compound.
How did they know where to go to find Bin Laden? They had pictures — in real time — from a special camera that can see things that the human eye can never see. Put simply, where people can only see color through visible light in three bands – red, green and blue, hyperspectral imaging (HSI) divides the spectrum into dozens or hundreds of bands, covering a wide range of wavelengths of the electromagnetic field. The result: The possible combinations of wavelengths are infinite and discoverable in ways that can detect and identify everything from a blade of grass to a toxic gas leak.
Related: Meet the 'Seal Team 6' Who Killed Bin Laden
The digital photos taken before and during the operation from MH-60 Black Hawks by classified HSI experts gave the SEALS a complete blueprint of what to expect in advance. Backed by this technology, the SEALS completed their mission.
This valuable technology is expensive and out of reach for most businesses and consumers. That’s a problem a young scientist on the research staff of PARC, a Xerox company in Palo Alto, decided to solve.
I met Alex Hegyi, one of MIT Technology Review's 2016 “Innovators Under 35,” at EmTech, a conference held at MIT’s Media Lab. He said his goal was to make hyperspectral technology “cheaper than anything that's out there and also small enough to fit inside a cellphone. That could dramatically change the way just about anyone looks at the world.” Here are a few of those changes, according to Hegyi:
- In agriculture, you could look at your crops and see if they need more or less water, more or less fertilizer, and predict crop yields.
- In medicine, you could look at your skin and actually see tissue oxygenation, or look at a mole and see if it’s cancerous.
- In defense, you could easily spot camouflaged targets.
- In planetary exploration, you could spot signs of life.
- In the fine arts, you could look at a painting and see the different layers of composition.
Related: A Robot Is My Co-Pilot: The Pentagon Tests a New Way to Fly
The list of potential applications is endless. One of my favorites is identifying counterfeit currency. Hegyi explains, “Someone hands you a hundred-dollar bill. You don't know if it's real or not. If you had a general purpose hyperspectral camera on your cel lphone you could download the app, take a picture of the bill and know whether or not it's counterfeit, based on the spectral signatures you get back.”
He continued, “When people are printing counterfeit money they're just matching the colors. They're not matching the spectral signature. It turns out there are anti-counterfeit measures in the bills that we didn't know about until we started taking hyperspectral images.”
I suppose Hegyi could find a way to install his technology in everything from ATMs to cash registers, saving businesses and the government a bundle.
The Early Days of Photonics
Hyperspectral imaging isn’t new. It’s been used by the oil and gas industry and geologists to pinpoint exactly where the motherlode is located.
NASA - In this wavelength region in California, clay, carbonate, and sulfate minerals have unique absorption features, resulting in distinct colors in the image. For example, limestones are yellow-green, and purple areas are kaolinite-rich (kaolinite is a clay mineral).
One of the more recent uses for the technology is to detect leaks in gas pipelines. The Canadian Energy Pipeline Association says hyperspectral cameras can detect evaporated hydrocarbons that can’t be seen by the human eye.
Although Hegyi’s camera has a somewhat limited wavelength range, Marian Dudek, a senior manager at Accenture, explained, “Hyperspectral images can show extremely low volumes and concentrations of evaporated hydrocarbons within a one-mile radius of the camera.”
Bending the Cost Curve in Health Care
As the U.S. and other developed countries struggle to keep health care costs in check, a technology that identifies disease symptoms before they’re visible to the naked eye and before they blossom into full-blown (read: expensive) diseases could be invaluable.
Related: Obamacare Rate Hikes for 2017 Even Bigger Than Expected
Hegyi says, “Hyperspectral imaging can be a tool like a stethoscope.” For example, “Blood changes color depending on if it's oxygenated or not, so scanning diabetics to find ulcers that are forming on their bodies can prevent one of the biggest co-morbidities of diabetes — amputation. That has almost an incalculable cost associated with it. So if you could find those early and well before they become clinically significant, let's say, that would be significant.
“Hospital readmissions after tissue grafts is another big one. Taking pictures of tissue after you do a skin graft to see that the blood is perfusing through it and making sure it's viable tissue immediately allows you to know if that surgery was unsuccessful and that they have to be admitted for a second surgery. So that changes that indicator of success vs. non-success from something that's 12 to 24 hours out to something which is immediate.”
HSI helped save a patient’s injured toe, which he initially thought would require amputation. HSI was used to monitor oxygen levels in the tissues, which provided instant analysis of this patient’s foot.
For consumers, Hegyi offers lots of workable “healthy” ideas: “You could have different medical apps like the one that allows you to take a picture of a mole and upload it to a doctor to figure out if it's cancerous.” He said if you put a drop of blood or saliva on a colorimetric test strip with hundreds of colorimetric indicators and take a picture of the test strip, the doctor could perform multiple analyses simultaneously.
How can color answer these hidden secrets of the human body and beyond?
Engineers have built “hyperspectral sensors and processing systems for applications in astronomy, agriculture, biomedical imaging, geosciences, physics and surveillance.” In other words, everything has a unique signature or fingerprint that can be identified.
Hyperspectral images are sometimes referred to as “image cubes because of the large number of measured wavelengths. Each pixel has a known code or identity of what is in the cube, be it animal, vegetable or mineral.
Source: Dr. Nicholas M. Short, Sr. - NASA
Consumers Reap Benefits, Too
For consumers, the technology could be incredibly practical. Someday, if Hegyi has his way, you’ll know immediately if that cantaloupe is ripe enough to eat before you buy it by taking an HSI of the melon with your smartphone while shopping in the supermarket.
Related: The Surprising Food Trend That’s Driving the Restaurant Recession
Want to know which tomato is ripe or sweet? Are those flowers fresh enough to last a few days? The hyperspectral camera on your smartphone could have the answers. There’s a downside, too. You’re having dinner at a four-star restaurant with your new friend and the gray sole tastes, well, fishy. If you photograph it and learn it’s been frozen and for two weeks, what would you do?
If we use this technology wisely, the U.S. could shed one its dubious distinctions — the country that leads the world in food waste. From farm to table, knowing when to water and fertilize, and when to buy and eat could avoid tossing roughly 50 percent of all the produce we grow and buy into a landfill.
Moreover, much of the food that must be tossed because it’s passed its expiration date could be sold at a discount or given to food pantries with a new label: Certified Edible Until Date: XX by Hyperspectral Imaging.
