Facebook Cookbook - Jay Goldman [19]
Measuring About Page success
The concept of A-B testing will be familiar to anyone who has tested speakers in a stereo store or built a sophisticated online marketing or e-commerce site. The name actually comes from the stereo test, in which two sets of speakers are connected to the same amplifier and are tested by flipping the A-B switch back and forth while the same music plays. The online version takes a little more work but a lot less crawling around in the dark trying to remember which side of the speaker wire is the positive connector. The same general principle applies: try two versions of something and see which gives the better result. This concept will come up again in Chapter 10 as a more general tool to stick in your tool belt, since it works equally well for things like invitations.
Think of this as an experiment in which you’re going to gradually refine your About Page content until you’ve tweaked your visit-to-install ratio into install-base nirvana. We’ll keep this example short by focusing on the Application Description. This will give you the general idea, which you can then apply throughout your app design. You’ll need to track your success, so create a spreadsheet with different Descriptions in each row of column A, and then columns for each day you’ll be running the experiment in columns B and up. I’d suggest keeping each iteration short but long enough to take into account different Facebook usage patterns on different days of the week. If you can spare a full seven-day period for each iteration, you’ll have all your bases covered; if not, you can probably assume that most weekdays are the same and leave weekends out. Running a round is simple: at the start of the test period, swap the next version of the Description into your application, and then diligently record the number of installs over the interval (don’t fret if you miss a day, since you can always look up your numbers through Facebook’s Insights tool or on a third-party tool such as SocialMedia).
NOTE
You may be wondering why you need to go to the trouble of tracking this yourself, since there are other options out there that will track it for you. Good question! If you’re content with the analysis they provide, save yourself the extra work and go with their numbers. If you’d like to be able to easily extract your stats into other reports or if you will be generating your own complex analysis or graphs, you’ll want your own data set in an easy, spreadsheet-like format.
As with experiments in the real world, you have to follow scientific principles here and make sure you change only one variable at a time. If you change your app’s Description and Photo at the same time, you’ll have no way of measuring which one was responsible for an increase in uptake. Since we’re focusing on the Description, try to keep everything else static while you refine just that piece. If you see a precipitous drop in installs right away, feel free to abandon all scientific reason and logic (no doubt setting countless scientific forebears spinning in their graves) and skip to the next variant. This kind of data unfortunately tends to be a closely guarded trade secret, so there’s no established metric I can tell you to aim for. This makes it especially important to define your success criteria before you start or you’ll be doing this forevermore. Establish a ratio that will make you happy, and stop when you hit it (but keep an eye on the ratio over time, since it will likely start to drop off and you’ll need to begin again).
Navigating the Applications Menu
Problem
The Applications menu seems really sophisticated and slick. How does it