Matt Mazur, of jMockups and Preceden, ran something that he calls a test, comparing freemium to paid-only plans, and draws the following conclusion:
Users who were able to test Preceden before paying were 347% more likely to upgrade to a paid account than those who weren't.
Unfortunately, as you will see if you read the full article, the test is completely and thoroughly invalid, for the following (and more) reasons:
- The two options were not run alongside each other, so external trends could have influenced the results
- The 'paid only' option was run during december, i.e. during the thanksgiving to new year period, which could explain the conversion rate changes all by itself
- Multiple things were changed
- The selling experience was still tuned to the freemium sales pitch; there was no demo, the sales copy didn't clearly state that the user was being sold something as opposed to being asked to sign up for free, and in general there was no tuning of the site to this very different selling experience
Accordingly, I'm switching back to the freemium version starting today.
The decision to switch back to freemium is as irrational as the way this "test" was conducted.
Calculating conversion rates does not magically lend validity to your test. If it is fatally flawed in so many ways, the fact that you present a table of results which looks like data doesn't make the data meaningful in any way.
Next: I put a heavy block of metal on my horse-drawn carriage to simulate a car and it did not perform any better. Here are some percentages.
If you read this far, you should follow me on twitter here.
