ANOTHER UPDATE: I just published a new posting based on the recent BBC and TechCrunch articles that includes a more recent communication from Facebook customer support and a method for avoiding booklicants.

NOTE: I hope this post is helpful. However, we are in the app business, not the advertising business, so if you have a chance please help us out by signing up for our beta. Thanks.

UPDATE ADDED 24 MAY: I finally heard back from Facebook. It is an onerous but friendly response and better than radio silence. I posted their reply and my revised request for refund here: http://wahanegi.com/10-percent-of-fb-revenue/#comment-186.

UPDATE ADDED 23 MAY: Still no word from Facebook. However, after doing more research I think I understand the problem better now. This post is still largely accurate, and you may find this useful as well: Is 10 Percent of Facebook Revenue Generated by Booklicants?

This is an entirely true warning and partly complete solution for entrepreneurs and business people seeking to use Facebook advertising to do market testing and build audiences for new businesses.

It is partly complete because I have not yet received an answer from Facebook to my extensive inquiries into what seems to be a high percentage of invalid clicks that cost money, wasted time and provided bad data. Perhaps this posting will help get a response from them, though I am sure they are busy with their IPO looming and their business booming. I wish them well, their success will be good for all of us.

Nonetheless I wanted to publish this info since people keep reading my Twelve-Steps to Lean Startup Success blog posting and I now know that Step Three does not work very well and will give you bad information.

The Problem

There is a tribe of Facebook profiles out there that click on ads obsessively and without discrimination. And I mean obsessively – the most obsessive profile I found in a sample of 10 percent of our Facebook fans also ‘liked’ 346 other things the same day they ‘liked’ Wahanegi. On average it seems that the 500+ profiles that liked our page as a result of our test campaigns liked an average of 42 and a median of 25 other random pages the same day they liked ours. They may be real people and thus legally considered ‘valid’ clicks, but they are worse than useless for an entrepreneur because they will suck up your time and confuse you.

It is easy to replicate the problem, so I suspect that it affects other “non-lean” advertisers to some extent. The problem hit me just right so that we spent hundreds of dollars more than originally planned based on what seemed to be surprisingly promising early results – how many times must one learn that if it seems too good to be true, it probably is? It appears that tens of thousands of other advertisers were affected by the same problem, since we can see the detailed list of likes in many of these profiles, including time and date stamps. The list of likes does not say whether they came from paid ad campaigns, but given the pattern of randomness and speed in many of the likes, that seems likely.

Some of the profiles must be bots of some sort, but others may represent real people who really enjoy clicking the word ‘like,’ or maybe they are doing it just because they are bored. In the end it doesn’t matter, because the impact they have on campaigns and campaigners, especially in the early days, is significant and not good. I honestly can’t tell if these profiles are real people or not at first glance, so I am calling them ‘booklicants‘ in honor of the movie Blade Runner.

For those who don’t understand the ‘booklicant’ Blade Runner reference, the movie is about a character played by Harrison Ford who is tasked with hunting down and ‘retiring’ human-like escaped ‘replicants’ and ends up facing a moral dilemma when he falls in love with one of the targets, an attractive replicant who fully believes she is human. The association came to me when I was showing my 8-year-old daughter the Facebook data and we were coming up with conditional rules to find invalid profiles and she said “Why don’t you just ask them a Wahanegi question? If they answer too fast or their answer doesn’t make sense, then they aren’t real.” In her lexicon, a Wahanegi question is a question about something you want, have, need or give. Unfortunately Facebook makes it hard to get in touch with your fans individually so I didn’t bother to try, but my daughter’s idea* is eerily similar to the fictional “Voight-Kampff empathy test” that Harrison Ford used to ferret out replicants. When I remembered that the original novel by Phillip K Dick is titled “Do Androids Dream of Electric Sheep?” and set in the general vicinity of Facebook’s headquarters, the connection was made and booklicants were named.

For the purposes of a lean startup entrepreneur trying to get useful market validation and segmentation from Facebook ads, it doesn’t matter if booklicants dream of electric sheep, it only matters that their clicks do not give you useful validation and segmentation data. So here is what you can do to still leverage the simplicity of Facebook ads without getting as much crap data. Unfortunately it will cost more and take longer than my original method but there is no such thing as a free lunch and it is still cheaper than building a widget that no one wants.

The Solution

Here is a method to reduce the confounding effects of booklicants on your Facebook marketing campaigns. I did one small test of this method and it seemed to work; however, I am waiting to hear back from Facebook before I spend any more money with them, and I have plenty of other stuff to work on for now. Here you go:

1. Target your campaigns by demographic and interest as specifically as you can, even if you don’t really know which demographics will be most interested. This is an obvious advertising recommendation, but not obvious if you are doing market testing. For instance, to save time and money I wanted to run generic campaigns for several different product concepts across very broad demographics for 3-4 days, and then use Facebook’s demographics report to see which specific segments were most interested in which concepts, after which we would have built very specific campaigns for each concept x demographic and run a final bake-off to see which was the most successful. Unfortunately, that doesn’t work. Besides Facebook’s slow demographics reporting which takes 2-3 days to update, this doesn’t work because when you go broad for a short time with somewhat bland ads on Facebook, you get a lot of clicks from booklicants all across the spectrum, since they seem to be just waiting for your ad to show up so they can click on it no matter what it says. That means your initial click rates and CPCs look better than you expect, you pat yourself on the back and feel excited about spending more money to get more clicks, but your data is garbage.
2. If you want to get the most data from your tests then advertise your Facebook page. This is also exactly what Facebook wants you to do, which is nice, since that means your CPC rates will be lower. But more importantly, your reporting will be much richer, since Facebook provides you with some nice reports on the people who liked your page. The main two downsides are that you won’t be able to do Unbounce-style landing page tests, and you won’t be able contact your new fans 1-on-1 as easily as you can by collecting email addresses.
3. If you want the cleanest list of customers, then advertise an external landing page and collect email addresses there. From what I can tell from six thousand or so clicks across fifteen campaigns, booklicants don’t leave email addresses. I can see clicks that bounce off our landing pages with somewhat suspicious speed…why bother even clicking…but the double-opt-in email addresses all seem legit. HOWEVER, your conversion rates from click to email address will be _very_ low, so low that it will be difficult to use the rates to compare different concepts or positioning statements with any statistical confidence. The rates are depressed by both the behavior of the booklicants and the natural problems involved with getting people to enter email addresses into semi-professional landing pages. Said another way, those email addresses will be very expensive, probably not worth it in my opinion. If all you want is feedback from potential customers you are probably better off just accosting your friends and acquaintances like I do rather than trying to attract potential customers on Facebook so you can talk to them. Or maybe do the old stand-by and advertise in Craig’s List, “We’re building an X product and need to talk to Y-type potential customers for $Z and a great cup of coffee for N hour(s) of your time.” I can talk to just about anyone about Wahanegi and learn something useful, so I haven’t had to do that, but we did it in the past at Adobe and Macromedia and it worked fine.
4. Ignore the first 3-4 days of your campaign results. This is particularly frustrating if you are trying to iterate/fail/learn quickly as part of a lean startup mentality, especially since the data is going to look surprisingly good and make you feel smart, but unfortunately it seems to be necessary. From what I can tell, the booklicants click as soon as they see your ad, after which Facebook will no longer show them the ads from that campaign. Booklicants also seem to surface at least once every four days (though that is just based on my limited tests…perhaps some hibernate longer than that…e.g. if they are real people, maybe they do it on the weekends or something, or only while sitting behind the cash register at a boring once-a-week convenience store job). So if you have a very targeted demographic and you bleed out the booklicants at first, then afterwards you will get data that is more representative of your intended target segment…at least until Facebook starts putting your ad into a broader rotation. Regardless, the last thing you want to do is make important business decisions based on those first few days, including the decision to spend more money on Facebook ads. This will be hard to do, because as you create your clever ads you will expect that a lot of people will like your stuff and the initial data will support your hypothesis. Unfortunately, the real number will be lower than that. Get used to it, and move on.
5. OPTIONAL: When you are done, remove the first 3-4 days worth of fans from your Facebook Page. This is optional since maybe you want to appear popular even if these aren’t real people, and maybe some of them are real people who just have unusual Facebook habits and were always waiting for a widget just like yours in addition to liking 42 other things each day. Plus, as you start to get more people liking your page, and your ads start getting the “your friend so-and-so also likes this” blurbs next to them, your click-thrus will go up and your costs will go down and who really cares if the booklicants are dreaming of sheep or not if they help you get customers. If you are a Blade Runner on the hunt for booklicants and want to be more careful, then any Admins of your Facebook Page will be able to review the public profiles of the people who like you to double-check them individually to see how prolific and random they are in their liking behavior. This is time-consuming using the current interface, feel free to suggest to Facebook that they make the process easier. You can ’remove’ a user which means they can come back (not good if you care about them polluting future tests) or ‘ban’ them which means they have a black mark on their record and can’t come back (not good if she was a nice cleaning lady from Torrance who just clicked ‘like’ because you have a pretty logo). I haven’t decided what to do. I’d like to hear more from Facebook about the issue before I do anything. For now, Wahanegi looks popular because of them, and we paid money for them, so we’ll leave them alone.

These are significant constraints, the steps will cost you money for clicks that you will know are meaningless if your goal is to understand market interest in your value proposition, and the process will take a bit more time and be a bit more loosey-goosey than it seems it should. However, all learning is good, the Facebook ad platform is incredibly easy to use, and no one else will be able to tell that your Facebook fan base is comprised of an army of booklicants.

Now you know, and you can decide. And now I have discharged my moral obligation, and I can get back to building Wahanegi.

For Anyone Wondering If I Am Blithely Ignoring the Obvious

No, I am not. Obviously Facebook needs to fix the problem and/or be more up-front about it, and as they do so I assume they will continue to refund money from any invalid clicks out there. From googling around a bit and searching their help documentation, it is apparent that they are working on the problem and have been for some time. They must benefit a lot from having extremely good information, since a profile has to log in before clicking on any ads, and as my tests show it is very easy to replicate and find the problem so it would also be very easy for Facebook to create algorithms to stop it. Here is my suggested booklicant algorithm based on a quick analysis of our fans:

Mark as booklicants any profiles that have liked more than 499 pages in the past 12 months, OR that have liked more than 9 pages on the same day they liked our page, OR that have liked more than 149 pages in the past 12 months AND more than 5 pages on the same day they liked our page.

As my proposed solution indicates, the problem is probably pretty small as a percentage of clicks, especially for longer and bigger campaigns…it’s too bad that so many booklicant clicks come at the beginning of a campaign, nicely timed to raise your entrepreneurial expectations and slow down your lean startup data collection.

From what I can tell compared to more mature Google AdWords, the Facebook Ad Manager provides relatively little information to advertisers about their attempts to reconcile invalid clicks, and there is also relatively little information in their documentation. Also unlike the SEO problems Google faces, it is hard to see how/why anyone would engage in this activity. There is some speculation that this sort of click fraud is sometimes executed by criminal business people trying to deplete competitors’ ad budgets; apparently people have even gone to jail as a result of doing that on other advertising networks. However, that seems unlikely in this case since the relative variation in CPC and the effectiveness of ads on Facebook are so low…I also don’t see why fraudsters would be so interested in the ad budgets of Filipino advertisers.

Finally, unlike Google searches where it is hard to imagine real people obsessively searching for ‘CRM’ or ‘web conferencing’ or ‘marketing analytics’ and obsessively clicking on the AdWords results Google serves up, it is not completely impossible to imagine that some otherwise normal human beings find themselves obsessively clicking on ‘likes’ in the ads that are shown to them by Facebook each day just because it makes them happy…I don’t know.

However, I do know one thing for sure - If you are trying to get market insight, don’t believe the like.

If any booklicants read this, please ping me at twitter.com/erikdlarson or drop us a note on our Facebook page to help us understand why you do what you do. It is very interesting behavior, and I imagine many people would like to know more about it.

*If Facebook wants to license a series of Wahanegi questions to use as an empathy CAPTCHA per my daughter’s idea, feel free to contact me at my first name at wahanegi dot com. Otherwise, please just keep working on the problem and refunding our money as you do so. And reply to my emails, please.