Data is a dangerous tool in making decisions. You need to understand the different ways in which it can fail you before you can make good use of it. Otherwise, you might make completely wrong data-driven decisions.
And then, of course, there's also some decisions (for example, high-level design decisions) which must be made based on something other than data.
This article outlines three possible failure modes of data-driven decisions, with Digg's catastrophic failure earlier this year as the backdrop for the points.
- Ask boring or loaded questions which can lead to self-fulfilling results;
- Get lazy and sample something too small or too favorable to the answers you hope to find;
- See patterns and correlations where there are none.
