Many people, especially those handicapped with a graduate level education, like myself, think that data is only interesting and can only be acted upon if it is “statistically significant”. In the context of early stage Customer Development, I believe this is well-intentioned, but ultimately, misguided.
Evidence comes in a diversity of forms. It can be anecdotal, it can be in aggregate or it can be a trend line. If you take an open-minded approach to the types of evidence you’ll accept, and adjust for their biases/problems/problems accordingly, you’ll likely fare better in the chaos of startup-land than just simply jettisoning what you feel is low-quality data.
In my opinion, trying to make CustDev formulaic, well–that’s a road to perdition. Some of the techniques that may work now, might not work as well in two, five, ten years, so there is no point writing a hyper-specific rule book because no book can know exactly the context you are operating in and when you are operating there.
There are a lot of other good points made, generally pointing to the fact that not all business decisions can be reduced to a statistical A/B test, and how techniques and methods for customer development need to remain contextually sensitive to be useful. It's worth reading the full article.
If you read this far, you should follow my RSS feed here.