The next game-changer is likely lurking in the data we already have—but it will take scientists years to uncover it
Earlier this year, astronomers stumbled upon a fascinating finding: Thousands of black holes likely exist near the center of our galaxy.
The X-ray images that enabled this discovery weren’t from some state-of-the-art new telescope. Nor were they even recently taken—some of the data was collected nearly 20 years ago.
No, the researchers discovered the black holes by digging through old, long-archived data.
Discoveries like this will only become more common, as the era of “big data” changes how science is done. Astronomers are gathering an exponentially greater amount of data every day – so much that it will take years to uncover all the hidden signals buried in the archives.
Sixty years ago, the typical astronomer worked largely alone or in a small team. They likely had access to a respectably large ground-based optical telescope at their home institution.
Their observations were largely confined to optical wavelengths—more or less what the eye can see. That meant they missed signals from a host of astrophysical sources, which can emit non-visible radiation from very low-frequency radio all the way up to high-energy gamma rays. For the most part, if you wanted to do astronomy, you had to be an academic or eccentric rich person with access to a good telescope.
Old data was stored in the form of photographic plates or published catalogs. But accessing archives from other observatories could be difficult—and it was virtually impossible for amateur astronomers.
Today, there are observatories that cover the entire electromagnetic spectrum. No longer operated by single institutions, these state-of-the-art observatories are usually launched by space agencies and are often joint efforts involving many countries.
With the coming of the digital age, almost all data are publicly available shortly after they are obtained. This makes astronomy very democratic—anyone who wants to can reanalyze almost any data set that makes the news. (You too can look at the Chandra data that led to the discovery of thousands of black holes!)