The Limitations of Simulation in Space

In the 1980s, entrepreneur John P. Allen conceived of Biosphere 2 as a way to simulate the realities of space settlement. The steel and glass dome structure was built in Arizona in the late 1980s, and eight people lived inside from 1991 to 1993, though the seal was effectively broken at sixteen months, when oxygen fell to 14.2% and outside air had to be injected to keep the crew safe.

Today, Biosphere 2 would most likely be run as a series of tests by an AI model. Could it account for the way Biosphere 2’s second mission ended, in a 1994 management dispute? Probably. But you wouldn't want to risk your life on it.

We believe the entire history of space exploration rests on the ability to test something you aren’t going to touch again. As humanity’s aspirations grow, the demands on simulation become higher and higher. This is not only an engineering problem, but an existential one. In an era when everything from weather to war can be simulated, the ambiguity of physical matter persists. However many simulations you run—whether it’s a relationship, a hurricane, or a voyage to space, you can’t be sure how it ends.

Into this uncertainty steps Bryce Strauss, cofounder of Nominal, a data platform for hardware teams. One might say Strauss sells software to people that don’t trust software. “You can go read blog posts about how ‘the world is in silica’ or ‘we’re going to be able to simulate the whole world’ and that is a really stupid take,” he says. “You won't really know what's going to work and not work until the metal is bent, until you've done something in the God's-honest physical world.”

Strauss worked for Lockheed Martin and supported the NASA team immediately out of college. He was 22 years old and working on a mission called Juno, a spacecraft in polar orbit around Jupiter. He was only in fourth grade when most of the technical decisions about Juno had been made—when they decided what battery it was to have; what rocket propulsion system was going to help it get into orbit around Jupiter, known as Jovian orbit. 

He was only in fourth grade when most of the technical decisions about Juno had been made

Juno approached the planet at 36 miles per second. It only needed to shed about a third of a mile per second to be caught by Jupiter's gravity. If it didn't, Juno would fly right by the planet and the whole mission would be a failure. Slowing down the spacecraft was achieved by firing a rocket in the opposite direction from which the spacecraft was traveling and then being captured by Jovian orbit. Strauss’s team spent a full year doing nothing but preparing themselves for those 35 minutes.

“Doing a thing with any complex piece of hardware should practically never be the first time you've done that thing,” Strauss says. “It should look and rhyme like something that's happened in the past, and hopefully you were able to simulate that environment or situation, and you did it, and you called it testing.”

One of the limitations of simulation is that it treats testing and operating as separate functions. Strauss disagrees with this dichotomy. “The operation is just a really high-stakes test,” he says.

According to Strauss, NASA agrees. They say the words, “Test like you fly.” This philosophy is reminiscent of aeronautical engineer Kelly Johnson, who founded and led the Skunk Works inside Lockheed during WWII.

Johnson made his team test equipment like their lives depended on it by literally making their lives depend on it. Strauss paraphrases Kelly: “Hey, if you're going to build an airplane, you're going to ride co-pilot. We're going to put two seats in it, and the engineers are going to have to ride along with a pilot.” 

Strauss applied this to InSight, the mission that put a robotic lander on Mars in November 2018. NASA was willing to spend a lot of money to test InSight because the team only had one shot for a lot of the tasks the robot would complete on the surface of Mars. They built one satellite that would go to Mars and a duplicate that stayed on Earth, just for testing.

“That means you have to buy two flight-grade batteries and two computers and two things that cost many millions of dollars,” Strauss says. 

The team that developed the lander’s robotic arm practiced the first five days on Mars by sitting in mission operations and operating the Earth lander remotely. “This is just like it's on Mars. You can’t just manually flip a switch. You only have software updates to fix problems,” says Strauss. 

How do you test things you don't know are going to break? You break them.

How do you test things you don't know are going to break? You break them. You take them to the brink of failure. If it's something really expensive, you probably can't break it. But you can put it in the closest possible operating environment. There's no such thing as complete confidence, but you can have confidence in change as long as you're changing the right-sized thing, explains the founder.

Strauss also worked on a NASA mission called OSIRIS-REx. OSIRIS-REx retrieved America's first sample from an asteroid. NASA simulated the mission by building huge walls of different types of substances on Earth for the robot arm to practice on. They had a lot of hypotheses about what would happen if the surface was a little softer, or brittler, or something else. It was Biosphere 2 if Allen had thought to test whether the concrete in the structure was absorbing CO2.

Strauss personally thinks the world has dramatically over-rotated on what it thinks is possible with just software. “I need a better word than simulation, because everyone, when they hear simulation, thinks of The Matrix,” he jokes. 

“This is the whole point of why Nominal exists, but a broader theme here is that an organization's willingness to invest in constant, consistent evaluation and operation of the system is the key indicator of whether their products are going to work for real,” Strauss continues. 

What actually makes Strauss worried is this: there are a lot of organizations in the world—more big institutions than small startups, but some small startups—that have put all of their dollars, tokens, and chips into, “We're just going to simulate only on computers.” If you were asking Strauss, “Would I put my kids in a self-driving car that only went through ten thousand only-computer simulations? No. Add twenty well designed IRL scenarios? Yes.”