Why Serial Is Like Science (Part 1)

serial-social-logoLike everyone else in the known universe, I recently finished listening to season 1 of the Serial podcast. Yup, for the first time ever I’ve actually been on the forefront of a viral phenomenon, thanks to the fortuitous intersection of my NPR addiction with a set of long international flights.

For those of you who’ve been living under a rock or something (e.g. me for any cultural trend not involving NPR), Serial is a podcast whose first season was devoted to investigating the 1999 murder of a teenage girl, and trying to determine whether the man currently serving a life sentence for this crime is guilty or innocent. Honestly, I’m not exactly sure what it is that’s gotten everyone hooked on this podcast. I guess it’s because it’s a true crime drama, a real-life episode of CSI (or one of its 37 spinoffs that are never more than a channel click away, no matter how hard I try to lose them).

As you may have deduced, I’m not a fan of crime dramas. They’re just too gruesome and creepy for my taste. So all along I wasn’t really sure why I got hooked on Serial. I appreciate their unbiased reporting, their investigation of the question from every possible angle, and of course the whole unsolved mystery thing gets addictive. But then around episode four I realized it: the Serial story is a lot like the story of being a scientist.

By the way, if you’re planning on listening to Serial in the future, I don’t think reading this post will ruin it for you or anything. But I will be mentioning things that could be considered spoilers, so if you don’t want to hear them then stop reading now and come back once you’re caught up!

In the shadow of doubt

Serial is framed around a single, basic question: is Adnan Syed really guilty of murdering Hae Min Lee? Serial’s host, Sarah Koenig, begins the series thinking that this question should be pretty straightforward to answer. As she describes in the final episode, “Certainty, one way or the other, seemed so attainable. We just needed to get the right documents, spend enough time, talk to the right people, find his alibi.”

Scientists begin in the same way; you start with a conceptually simple but important question that you’re convinced you can answer based on a clear set of experiments. If you’re a young grad student, it’s usually your advisor who sweet-talks you into a project by explaining how it’s going to be a piece of cake to figure out the answer; how you’ll publish a Nature paper and graduate in three years and become the star of your program.

But in the case of both Serial and science, things rapidly change once you actually start investigating your question. First of all, you realize how hard it is to even do the experiment in the first place, let alone to obtain relevant data. In science, we spend months or even years trying to lay the groundwork to do the “real” experiment: building equipment, writing code, obtaining reagents, rebuilding stuff when the first version turns out to be crap, rewriting the code, and so on. This is the phase of grad school when you drink a lot of beer.

experiment flowchart

Flowchart of the steps needed before you can even start doing experiments.

I’m not a reporter, but it seems like investigating a story is similar; you first have to lay the groundwork by tracking down the people you want to talk to, the documents you want to read, etc. This doesn’t sound easy, especially if you’re investigating an event that took place 15 years ago.

Then if you’re lucky, you finally get some data. Yay! You do a happy dance, probably drink some more beer (celebratory beer always tastes better than consolatory beer).

But then you start actually analyzing your results, and find that they rarely represent the clear-cut evidence that you expected. This happens a lot in Serial. Like when they dug up Adnan’s cell phone records, which partially support the prosecution’s story, but also show that the prosecution’s key witness is lying. And once you start questioning whether Adnan even had possession of his phone in the first place, you start wondering whether this evidence is even relevant, let alone which side it supports.

Science is the same way. Many experiments that you began with high hopes, full of excitement and anticipation, don’t turn out to yield any meaningful result. Kind of like every date you’ve ever gotten from OkCupid.

Even after nearly a decade in science, this always surprises me. I always begin an experiment taking for granted that it’ll give me some kind of answer, one way or the other. It’ll either support my hypothesis or refute it. That’s what we were all taught in science class, right? But honestly, the most common result is that you don’t get any real answer at all. Maybe the technique you’re using doesn’t work, or the data are too variable to make a real conclusion.

For example, my experiments frequently involve observing the behavior of mutant flies, i.e. flies that lack the function of a specific gene that I’m interested in. I always figure that I’ll either see an effect or I won’t; either the gene is involved in the behavior or it’s not. Instead, I’ll get weird non-answers, like the mutants show a difference but so do the controls (flies without any mutation), which are supposed to be normal! Or the mutants show a difference but they also act weird in general, so how do I know that the gene is specifically involved in MY behavior as opposed to ALL behaviors?

Assembling the puzzle

Things get even more hairy when you start collecting data from different experiments. In Serial, this means putting together all the cell phone records and witness statements and evidence about whether there was or was not a freaking pay phone at the Best Buy. You’d think that because an objective truth does exist—either Adnan is innocent or he’s guilty—the evidence should generally support either one side or the other. But as many of you know (and here comes the main “spoiler”), all the mountains of data that Serial collected ultimately failed to conclusively support a single answer. The evidence went back and forth, at times supporting Adnan’s story or casting doubt upon it, and Serial was never able to come to a definite conclusion.

At the beginning of the podcast, didn’t we all think it was obvious that Adnan was innocent? The early data seemed to support his story. But the more evidence that Serial gathered, the more it made us question what was really true.

This is a lot like science. You think that the more experiments you do, the more you’ll amass support for one answer or the other. But often the opposite occurs: the more experiments you do, the more conflicting information you get, and the more you don’t really believe any of the possible answers. I know, this doesn’t sound like the scientific method that you were taught in school.

For example, in grad school I studied drunk fruit flies. Yes, really. Yes, flies do get drunk. Yes, it’s entertaining to watch them get drunk, until you’ve been doing it for 6 years straight and just want to get your freaking PhD already. Can I go back to my story now?

Ok, so at one point I performed an experiment where I artificially activated a specific group of a neurons in the fly brain. I found that this caused alcohol-exposed flies to get drunk faster than usual, suggesting that the firing of these neurons promotes intoxicated behavior. I then wanted to confirm this conclusion by artificially silencing those neurons while flies were getting drunk, which should have the opposite result: flies should get drunk more slowly than normal. Instead, flies with the silenced neurons also got wasted super-fast!

So do these neurons actually promote drunknenness or soberness? The short answer is, I still have no idea. Paradoxically, if I had performed fewer experiments then I would have been more convinced of a single answer—but it might not have been the right one. That’s the thing that frustrates me (and probably Sarah Koenig, too): sometimes the more work you do, the farther you get from the answer. Or at least from believing you know the answer.

Of course, some people do manage to get clear results that all fit together nicely. Every new experiment adds another piece to their puzzle, which locks in perfectly with the existing pieces to slowly reveal the overall picture. I don’t know how they do this. My puzzle is always full of holes and the pieces I have don’t fit at all, and I can’t tell whether the picture is supposed to be the Grand Canyon or SpongeBob SquarePants.

Some will assume that those folks with the pretty results must be better scientists than the rest of us; others will say they must be hiding something. While there might be some truth to both explanations, I think it’s probably just that they’re studying problems that have relatively simple answers. Like if Serial had decided to investigate the O.J. case instead.

But most of biology is messy, and so are human interactions and behaviors and crimes. After all, biology is what we’re made of. I guess we just have to be ok with that, and realize that answers are hard to come by.

Stay tuned for Part 2

Wait, I’m not done! Somehow this post got crazy long, so I’ve decided to save the rest of my thoughts for a separate post next week. In the meantime, go finish (or start) listening to season 1 of Serial!


Comments

Why Serial Is Like Science (Part 1) — 4 Comments

  1. oh my god so true!! Stupid infuriating show just like my stupid infuriating experiments.
    Now Sarah Koenig knows what it’s like to be a scientist! But she can get the most-listened-to podcast in history without finding the answer while we can’t get a nature paper with conflicting results.

    • Haha yup. The part I left for next week actually talks about the screwed-up publication process… but it needs some work so it doesn’t sound too depressing :)

  2. Great post! Like the podcast, a scientific story can be pretty interesting if it is well researched and presented in a entertaining and accessible way, even if the ultimate finding is kinda..meh.

    Greetings from Belgium. Really enjoying the blog.
    – V

    • Thanks! Hope you’re enjoying your postdoc and life in Belgium!
      And good point- I always like hearing talks where the speaker discusses all the twists and turns that their project took, and doesn’t pretend like everything just worked out perfectly. But I feel like those kinds of stories don’t usually make it into published papers.

Leave a Reply to Vinoy Cancel reply

Your email address will not be published. Required fields are marked *