Once upon a time, there was a magic pizza shop. The pizza shop was magic because it was so efficient — their oven could instantly cook an unlimited number pizzas at almost zero cost. The kitchen staff never ran out of ingredients, nobody ever had to wait long for their order, and every single pizza came out of the oven just as hot and delicious as the one before it.

It truly was a magical pizza shop.

The owners of the shop loved pizza. They were passionate about making excellent pizza. Their dream was to share pizza with as many people as possible.

So the owners decided to give their pizzas away for free to anyone who asked. No charge. No strings attached.

The neon sign outside the shop glowed 24/7. “Free Pizza,” it said.

🍕✨

The owners of the magic pizza shop knew they were providing a widely-used, valuable public service. But they didn’t know how many pizzas they were giving away.

So the owners hired somebody to sit outside their magic pizza shop and count the number of pizza requests that came in. Jimmy McMetrics was his name.

In theory, Jimmy’s job was very simple: count the number of daily requests for a free pizza.

But in practice, it proved to be much more difficult.

OnJimmy’s first day, it didn’t take long before people started to arrive to ask for free pizzas. As they entered the magical pizza shop, Jimmy began to count their requests. 1, 2, 3, 4, 5… and so on.

As the day went on, Jimmy noticed something unusual: he began to recognize repeat customers. Some people were coming back more than once to ask for a free pizza. One especially hungry person visited the pizza shop seven times that afternoon!

“Hmmm,” Jimmy thought to himself. “The owners of the pizza shop want to understand how many people they’re serving. So if the same person comes in more than once a day to ask for a pizza, I’ll only count them once.”

But Jimmy quickly realized that this was easier said than done. The magic pizza shop was very busy, and he couldn’t always remember which customers he’d seen before.

So he made a decision.

“For every person who visits the magic pizza shop, I’ll write down their shoe size, and the color of their shirt,” he thought to himself. “That way I can uniquely identify every person who comes in. And if the same person asks for a pizza more than once per day, I’ll only count their requests once.”

(You might think the combination of shoe size and shirt color isn’t a particularly accurate way to identify unique customers, given that many different people have the same shoe size, and many people don’t wear the same color shirt every day. You’d be right.)

At the end of Jimmy’s first day on the job, he told the pizza shop owners how many pizza requests he’d counted. The owners loved knowing this number, but they didn’t ask many questions about how Jimmy had arrived at it.

Jimmy’s second day at the magic pizza shop started much like his first. Customers arrived, asking for free pizzas, and Jimmy counted each pizza request, making sure to take note of their shoe size and shirt color so he wouldn’t double-count requests from customers who visited multiple times a day.

Then Jimmy noticed something he hadn’t seen before. Some customers were leaving the shop with a single slice of pizza instead of an entire pie.

Jimmy stopped one of the customers. “Why do you only have a slice of pizza?” he asked. “This shop gives away entire pizzas!”

“I didn’t want a whole pie,” the customer replied. “I only wanted a slice. Why do you care how much pizza I ask for?”

Before Jimmy could answer, he noticed another customer leaving the shop. But instead of carrying an entire pizza, or even a slice, she was carrying a tiny crumb of pizza crust.

“Well, I’m not even going to count her pizza request,” Jimmy said to himself. “Any reasonable person knows that a tiny crumb of crust isn’t the same as a whole pizza.”

So right then and there, Jimmy McMetrics decided he would only count customers who left the magic pizza shop holding at least one slice of pizza. Anything less simply wouldn’t count.

At the end of Jimmy’s second day, he told the owners how many pizza requests he’d tallied up. Once gain, the owners were delighted to have a number to attach to their shop’s success.

Every day for the next five years, Jimmy showed up to work and dutifully tallied the number of pizza requests that came in.

And at the end of each day, he reported back to the owners, who came to rely on the pizza request numbers.

Some days, when the kitchen staff were cooking especially creative and delicious pies, traffic to the store doubled or tripled. Other times, when the kitchen hadn’t come up with a new recipe in a while, the number of pizza requests went way down.

Jimmy liked those days — it was easier to count.

Asthe years went on, Jimmy McMetrics saw his fair share of unusual activity at the magic pizza shop. For example, one time somebody with a brown shirt and size 11 shoes came to the shop five thousand times in a single hour.

“That’s unusual,” thought Jimmy. “But, if they have a brown shirt and size 11 shoes, they must be the same person, so I’ll just count them once.”

Another time, Jimmy saw the same customer multiple times over the course of a month. That customer never came in more than once per day, but each time, they left carrying a single slice of pizza. “I guess I’ll count them once per day, every time they come in,” he said to himself.

For the first several years on the job, Jimmy McMetrics believed he worked at the only magical free pizza shop in the world.

But over time, he started to learn about other magical pizza shops in other towns. These other shops also gave away free pizzas to anyone who asked. And these shops also employed people just like Jimmy to count pizza requests.

Jimmy learned that all of his counterparts had similar problems when it came to counting. But each shop addressed those challenges in slightly different ways.

Jimmy kept track of customers’ shoe size and shirt color. But other shops used customers’ phone numbers and and body weight. Other shops only counted customers who left the shop with two slices of pizza, instead of one. And other shops would only count a customer once per week, instead of once per day!

Jimmy didn’t understand why each magical pizza shop counted their pizza requests differently.

To be fair, all of the pizza shops had the same basic idea: a pizza request would only count if it came from a unique customer within a certain period of time, and it would only count if the pizza request resulted in a meaningful amount of pizza being taken away from the shop.

But each magical pizza shop meant slightly different things when they talked about “unique customers” and “a meaningful amount of pizza” and how long between multiple requests before you count a customer twice.

So Jimmy McMetrics and his counterparts at the other pizza shops got together, and they agreed on a common set of definitions for what a “pizza request” really meant. They agreed on:

• a standard definition of “unique customer”
• a minimum amount of pizza the customer would need to get in order to count
• a standard window of time, during which multiple requests from the same customer would only count once

As with most standards-setting groups, the process was fraught, and in the end, nobody was 100% happy with the final definition. Everybody preferred their own original definition of a “pizza request” and wished they could stick with that. But as the free pizza industry grew, everybody recognized the benefit of a shared way to measure their work.

“Sheesh,” Jimmy muttered to himself as he implemented the new standard. “The owners of the pizza shop rely on these numbers, and they will be mad when I tell them we need to start measuring in a new way. What if, under the new measurement system, our numbers go down? What does that mean for the success of the magical pizza shop?”

But when Jimmy sat down with the owners, their reaction surprised him.

“Remember Jimmy,” they said as they all sat down in front of a piping hot margherita pizza, “We opened this pizza shop because we love pizza. We’re really good at making pizza. And we want as many people as possible to share in our love of pizza.”

The owners were right.

The pizza was still hot and delicious.

The pizza was still free.

Customers loved the pizza. They raved about it to their friends, and they kept coming back, time after time.

Yes, the number of pizza requests matters. But it’s not the only thing that matters.

What does this have to do with podcasting?

OK, it’s not a perfect analogy. Podcasts aren’t pizzas. Shoes sizes aren’t user-agents, shirt colors aren’t IP address, and slices of pizza aren’t incomplete downloads.

But…

The entire podcast industry is moving towards a new measurement standard: the IAB’s Podcast Measurement Technical Guidelines 2.0 for measuring downloads. Some podcast hosting companies claim to be “IAB 2.0 compliant.” Others have been certified by the IAB. And several others are currently going through the certification process.

Adopting the IAB 2.0 guidelines means growing pains.

Nobody likes when their numbers go down. And retroactive changes can be a tough pill to swallow.

If your shows are facing this type of change, think about Jimmy McMetrics and remember:

1. The entire industry is moving to the new standard. When everyone uses the same yardstick, we get better apples-to-apples comparisons across episodes, shows, and networks.
2. If your download numbers change as a result of IAB 2.0, your audience hasn’t changed. Nobody went away. The yardstick changed.
3. Every other non-download metric of success is still relevant. Your Spotify and Apple Podcasts average completion rates don’t change. Your ratings and reviews don’t change. Positive emails and social media mentions don’t change. If you’ve done a brand lift study, the results don’t change. The only thing that’s changed is the download yardstick.
4. For brands especially, downloads can’t be the only metric of podcast success. Downloads should be part of a holistic view. If you’re only focused on downloads, you may have lost sight of the real reason podcasting is such a powerful medium for brands to tell stories and connect with audiences.

🍕✨

Sign up for the Pacific Content Newsletter: audio strategy, analysis, and insight in your inbox. Once a week.