Photo by Mika Baumeister on Unsplash
If you’ve read the other posts in this series, you may have asked, “Why go through all the trouble of collecting this data? Why not just ask what potential backers want?”
Well, there are a couple of reasons for preferring hard data over surveys. Let’s go over a few of them.
The Case for Hard Data
Response Rates
Getting people to respond to survey’s can be difficult. Personally, my usual channel is Twitter and/or Reddit. So far, the highest number of responses I’ve gotten to any question I’ve asked has been about 10 people maybe, and that’s just not a lot of info to work with. To be honest, even if I got a phenomenal number of responses, it would still pale in comparison to the literal thousands of projects that I can analyze on Kickstarter.
Surveys Can’t Answer Everything
If I surveyed 100 people, and asked them “What’s the best month to launch a Kickstarter campaign?”, most answers would probably congregate around 6 different months or so. Sure, most would say December isn’t a good time to launch, but what about August? Or March? That’s a bit harder to know, and can really only be answered by analyzing the hard data.
Stated vs Revealed Preference
There’s often a real and measurable difference between what people care about and what they say they care about. For example, people may say they like blue boxes better than red, but if you measure their actions, you may find that red boxes are actually bought more often. Or, to give a real life example, someone may say that data privacy matters… until they’re offered some pizza.
This disconnect could easily affect Kickstarter as well. For instance, when you ask backers about shipping, usually the majority say they prefer to have shipping “baked in” to the overall price, instead of added separately. But what if doing that takes the price of your game from $30 to $42? That might start triggering the “sticker shock” reaction and actually cause people to not pledge. Even though $30 + $12 shipping = $42, seeing the $42 all at once can definitely have an effect.
That might seem odd, but the human mind is sometimes an odd thing. This, of course, was just a made up example. Usually, the disconnect is a little more subtle than that, but it is definitely there. Because of that disconnect, measuring actions is often better than measuring opinions.
The Case for Surveys
Don’t get me wrong, surveys are still incredibly useful and are sometimes vastly superior to hard data. One of their biggest advantages is their ease of access. What might take months or even years of effort to gather through other approaches could be a single question on a survey.
For instance, if I wanted to know the most popular color in America, there are a number of hard data points I could use to guess it. In theory, I could broker a deal with credit card companies to get everyone’s purchase history and scan it for the colors of the clothes and bath towels they buy. I could use google maps to collect the color of everyone’s house. I might even be able to access a vehicle purchasing database to see what color car everyone owns.
Well, not only would that be super creepy, it would also be super difficult. Why not just ask you what your favorite color is?
When playing board games, what’s your preferred player color?
— David (@ddaviddx2) May 4, 2020
(More colors below)