Ever since I can remember, I've been interested in thinking about the stars and thinking about the universe and how it's put together As a blind person, my in for knowing about that was to read science fiction or to watch space documentaries on TV.

You are about to take a journey out of this world.

When a sighted person looks up at the night sky, there's a visceral connection I think, that people feel to the universe around us.

For me, sonifications are the way that I get to have that visceral sensory experience of the universe.

Sonifications are life changing for me I get to have that experience of the universe.

Listen to the universe Data sonification in its simplest form is just the translation of information that we can see into something that we can hear.

It's this idea of extrapolating out that information into sound My name is Kimberly Arcand, and I'm a visualization scientist and emerging tech lead for NASA's Chandra X-ray Observatory.

All that means is that I get to take data from our universe and tell a scientific story with it.

In this case, I'm sticking with my good friend Cassiopeia, a this beautiful exploded star.

And what we've done is we've used the slice data with the silicon sulfur, the calcium, iron in the high energy, and mapped each different layer to a different sound, a different tone, We worked with people who are blind or low vision to make sure that they could extrapolate their own meaning.

If you are just listening to this object that the scientific story would still be communicated in a way that made sense and that was also pleasing.

And then we test, test and test again because we don't want anything that's going to hurt people's eardrums, not make sense if it's too fast, for example, if it's too slow.

But it is like when you're making images.

There's, you know, choices to be made.

Sonification is important because it gives us a different way to experience and perceive information.

And in particular, it's very important for people who are blind or visually impaired because it gives them access to information that they wouldn't otherwise have.

So the idea is to convert relations or features in data, into something we can hear, like the changing pitch of a note or the changing volume It turns out that it's not just people who are blind or visually impaired that find value and meaning in the sonification.

They're popular almost universally with the general public.

And perhaps that's because they help people connect something that's far away and unfamiliar in astronomy with something that's deeply personal and emotional, such as music.

So all of this type of data that we're getting from these various NASA telescopes, we have to translate it from something humans can't easily naturally detect into something we can either see or hear or even perhaps touch.

So it's this process of translating information from one form to another.

And when we're doing that, we can create a visualization or we can create a sound, or we can create something that could be 3D printed for example, so that we could touch it.

But it's kind of the idea of, you know, if you're speaking French and you need to translate into Mandarin, you're just taking that information and you're just moving it into another form.

So just another way of understanding.

For me, the process of getting to know a sonification is...

It's demanding and rewarding and it takes time because instead of just looking at an image and maybe reading a caption and then listening, I want to read through what each sound means.

The way the sound is panned gives a lovely sweeping sense that the image is big, or at least what the image is referencing is big.

And so this one is very rich the listening experience, even if you just listen to it, not knowing what it was, it's a lovely, lingering, meandering sort of melodic listening experience and when you add in the meanings, it becomes so much richer and more meaningful, more important.

Most of what we're looking at in the universe is in a light or a type of light that humans can't naturally detect.

I like to think of these telescopes that are up in space as just being kind of super friends, right, with their various superpowers.

And Chandra's superpower is to look at really energetic regions, really hot regions of space.

But what's really helpful is be able to look at the entire universe with a different set of eyes, looking at many different kinds of light.

So, for example, though, Chandra looks in X-ray light.

There are other telescopes that look in infrared light, like the James Webb Space Telescope or the recently retired Spitzer Space Telescope.

Hubble looks at optical light and also little bits of ultraviolet and infrared light as well.

There are radio telescopes.

Gamma ray telescopes.

And the whole point is you need all of these different kinds of telescopes to be able to fit in the various pieces of the puzzle.

The Perseus cluster of galaxies is this beautiful system, hundreds of millions of light years away from us, where it's a cluster of galaxies that has a very high powered supermassive black hole at its core.

And what Chandra found back in 2003 was that that black hole is essentially burping out into the hot gas around it.

And what that burping into the hot gas is doing is causing pressure waves, which are sound waves.

The image itself shows hot gas glowing around this galaxy cluster.

It's not true that there is no sound in space.

In some cases, like in this galaxy cluster, there is enough ambient gas surrounding the galaxies that sound can travel through it.

And because these ripples are visible in the image, we were able to extract their shape and re-synthesize a sound.

The black hole is essentially creating a sound.

It is a B-flat about 57 octaves below Middle C. So I love to sort of think that there are black holes out there like these divas in the universe, you know, singing this really, really deep, deep song because 57 octaves below middle is hundreds of keyboards too low for humans to hear.

So I dug back into the research and I found images showing these ripples much more clearly.

And then I spent a long time learning how to cleanly extract those waves.

I knew it was something special.

In a way, it's a very accurate sonification because the sound waves we're recreating came from the black hole.

I did not expect it to go viral at all.

We had released it for NASA's Black Hole week and it was on the news and that was lovely.

And then on a slow news Sunday in August, a colleague at NASA Exoplanets just tweeted about it.

And for some reason it just caught on fire.

And by the end of it, it was like almost 2 billion people had seen or, you know, come across the story And that just blew me away.

That was very windy and gusty.

And so there was less of a sense of discrete phenomena like stars or galaxies and more of a sense of movement.

because the sound and this is so uncommon that it gave me a little bit of the creepiness where you start to get the sense of your own minuscule, tinyness when trying to comprehend big, huge forces like that.

It was so exciting that a project that we had started during the pandemic as a way to connect with a community resonated.

Pun intended, I guess, with a much wider part of our planet.

Nowadays, nature feels.

It feels as if all nature serenades with a mystic melody, and then it gets better.

The spectacle gets better as I hear how galaxies arch across the celestial dome, What happens is that the information that the satellites gather is light.

And usually people like me who dedicate their lives to physics and astronomy, etc., convert that light, that information gathered by the satellites into numbers, and those numbers can be converted into a sound.

So I have a very dear friend, Dr. Wanda Diaz.

She is an astronomer and a computer scientist, and she's been blind since she was a teenager.

She was really my inspiration.

That's how I first learned of data sonification.

She has told me these stories of how she would be in math class at University, and her professor would just be writing equations on a board and she could not access that information.

So for her to study the stars, she relies on data sonification so that she can translate that information to something she can hear and therefore process and therefore study.

So for her, data sonification is an actual step of scientific research.

Sound effectively increases the sensitivity of people to signal detection, and I think it's crucial that the training begins in school.

Let's not leave out people with disabilities.

Even if we're not scientists, I think that that curiosity or that awe or that attempt to understand the cosmos and infinite space, I think that's something that all humans have in common.

It's kind of like listening to a poem in a language you don't know because if you're not a scientist those concepts are, won't say foreign, but they're just not in your everyday parlance, they're not things you deal with everyday.

So it's dense, there's so much to take in.

Yeah.

Worth the effort though, right?

Yes.

Yeah.

Yeah.

It's so rewarding.

I first met Matt and Andrew from System Sounds at a visualization conference that we had all attended a couple of years before the pandemic, and I really admired their work.

System Sounds is a science art outreach project.

So the basic idea is that we take data from space and convert it into music and sound for outreach, to inspire, to educate and to make space more accessible.

We just decided to to work together to try a few of these data sonifications based on Chandra data and to see what happens, because I really suspected that, you know, with over two decades of operation, we have this treasure trove of a data archive with Chandra and with other telescopes from NASA as well.

and we just started playing.

We started experimenting with translating this information into new sounds, new ways of knowing.

So, the first time that I heard a completed piece all the way through it was of the Galactic Center, this sort of inner 400 light year region around Sagittarius A star, our very own supermassive black hole.

And it's this wonderful, dense, busy, active downtown region of the Milky Way.

Right, there's exploding stars.

These X-ray binaries, there are these beautiful loops of material, all these massive stars.

There's so much going on.

It's kind of like Times Square, right?

And I'm so used to that data and understanding it visually that the first time I heard the sounds, there were things in the data set that I had never really noticed before.

For the Galactic Center.

We, we started by looking at it, looking at what the data looks like, what are the patterns.

And the important thing is to find out what's the story we're going to tell.

It was a beautiful multi wavelength image.

So it had layers of recorded light of optical, infrared and X-ray light all combined into one.

In most cases, the sonification results in music that is not humanly playable by anyone.

And so we have to use computers and synthetic instruments to bring this music to life.

But there are some examples where we actually do some real recording.

We assigned each a telescopic instrument to a musical instrument.

So they had different qualities of sound.

One very common question I get when talking about the sonification project is from musicians who ask, “Can we play this data ourselves?” So that was the next goal to get sheet music into the hands of musicians.

I grew up in Toronto, and I've been interested in music for my whole life.

I started playing piano when I was, like, five years old.

Eventually, I went to an arts high school, and that's where I met Matt, who is the other half of System Sounds.

Andrew is by far my longest friend.

We met in grade nine when I was 13.

We just happened to sit next to each other in chemistry class.

Then we found out we were also in the same music program in our high school, and years later, after high school, we joined the same band with a few other people from that high school.

And so we've been creating music together in some form for a very long time.

In 2017, Matt and I converted a musical solar system they had found called TRAPPIST-1.

TRAPPIST-1 is a very exciting system of exoplanets.

There are seven earth sized planets, all orbiting a nearby Red Dwarf star, and this system happens to be the most musical solar system ever found because these seven planets are locked in an orbital resonance.

So it's like they're playing the same song.

They're all locked into a rhythm and playing a certain harmony.

A lot of it's trial and error.

You look at what's important to the image or what's important to the data.

So what's the image about?

Is it about how many of something it is, what had happened?

We just look at what the story of the image is Find out what sounds can bring that out.

And then we just try version after version after version until we get something that represents the data.

It sounds like the image looks, but also it stands out on its own as a piece of music.

I like that one.

Yeah.

I know.

So the first step is I sit down with Andrew and we look at the data or the image and we think about all the possible different sonification mappings.

So we do a lot of sketching and diagrams and really try to do this design stage very slowly and methodically.

And then we get into the computer programing.

And that can sometimes take a while.

From there we do a bit of music production to put the elements together and then we send what we have to Kim and also to Christine for their consultation and their feedback.

Christine Malec is amazing.

She gives the best feedback we could ask out of anyone.

Her insight and her thought process and just how much she loves space and how much she loves the science behind it.

Her feedback, really does shape a lot of what we do, and it gives us a whole new perspective on anything we make.

South by Southwest was a context I never imagined myself participating in Kim asked if I would attend also It was great to be in the same room as Kim and Christine because as you know, this this project started during COVID.

And so the three of us were working mainly remotely, and so we all got to be in the same room.

And Andrew was one of the four of us.

You always listen to the composite first.

Or do you like to hear the layers first and then put it together?

I would listen to the composite first and then definitely the layers.

I always listen to each of the layers So here we go.

The image associated with the Perseus Galaxy Black Hole resembles the view down in the center of a purple cotton candy funnel cloud.

Being there as a panel participant was extremely exciting.

People in the blind community have a different, slightly different experience of these sonifications in that for us it's more directly educational.

So I was super honored, I was very grateful for the opportunity.

It was extraordinary.

Thank you again so much for coming.

We really appreciate your time.

I came across the opportunity to experience the work of Matt Russo many years ago.

We both live in Toronto and a local blindness organization was promoting a presentation he was giving on the sonification he had done of the TRAPPIST-1 Solar System, and I had never heard of sonifications, so I was eager to go.

And he gave a sort of multisensory presentation at a local little planetarium, and I was completely blown away.

I had goose bumps on my goose bumps.

It was I'll never forget the experience because it was my first direct sensory experience of cosmic phenomenon and that really got me hooked into how else can we or other members of the blind community experience these phenomena and learn about them more deeply?

In this very modern digital era of astronomy and computer science, binary code is that language that we use to talk to each other.

Once we get that binary code, that digital suitcase, it's then unpacked into, say, a table of information.

The table of information catalogs the energy, the intensity, all of the location information, the time of every photon, every packet of energy that was captured during the observation.

And then from there, it goes down the pipeline to be created into an image, into a sound, into whatnot.

Data sonification itself is also not, you know, holding up a big space microphone.

It really is just like the image translation part.

It is a translation into sound.

One of the most fun steps is that whenever we start a new sonification, I get to kind of put my researcher hat back on.

I get to dig into the actual scientific papers and try to understand the scientific story behind the data that I'm looking at or the image that I'm looking at.

And so I still get to kind of flex that muscle a little, and that informs our sonification choices.

My feedback is often about, can we make this pitch a bit higher or can we choose, can you choose a different instrument, or could it be oriented this way or that way?

So for me, it's informational.

It's about how can the sonification be more of an educational tool for someone whose only sensory experience of this phenomenon is going to be the sonification.

And you've also got a few of our mistakes.

Stereo panning.

-You say this, but I think is he telling me the truth?

No, we've had some stereo panning issues.

And you're listening so carefully that you can tell this has to be on the left or the right, and that's happened a few times.

Okay.

When I look at images that we've created, descriptions of or sonifications of, I have a completely different appreciation for them.

And I think any time any researcher has a new avenue to go down, there's a reward at the end of that street because it pushes how you process things.

It pushes how you understand things.

So I'm kind of addicted to it at this point.

Like, what else can we do?

I think now that blind or low vision kids who see things like image descriptions and sonifications, feel more included and feel that they're part of the learning world and that maybe a career path that seems plausible to them in a way that it didn't seem plausible to me.

I think that as always happens, when you make something more inclusive and accessible for one particular group, you positively affect a lot more people than you think you have.

When you take the time to create something of value to one community, oftentimes you know that “rising tide lifting all boats.” Well, that's a phrase for a reason, because a lot of other communities have expressed that these types of outputs matter for them as well.

That was something that we particularly learned from users who are blind or low vision that they wanted to make sure it wasn't just “this is a duck in a pond,” because that can be done automatically.

But being able to capture some of the visual poetry of your dataset was really critical I'm very happy that we're at a place now where a blind astronomy enthusiast or even just a blind kid checkin' stuff out could go, “oh astronomy.

yeah, okay, so I can read this image description, I can listen to this sonification, I can maybe touch a 3D print of something.” So having all of these access points is a dream come true for a lifelong astronomy lover like me.

Chandra is the result of this international collaboration of scientists.

We are managed by the Marshall Space Flight Center, situated at the Center for Astrophysics.

Our colleagues here at the Smithsonian Astrophysical Observatory and all of our colleagues at the Marshall Space Flight Center and NASA's Universe of Learning.

So many people, so many experts that really give their heart and soul into making this data possible and to helping this data get out into the world.