Stats Referer Spam Moderation, Like Blogger Comment Moderation, Is Collaborative

Some blog owners just do not understand the realities of referer spam moderation.

How can I set my dashboard, to not track pageviews from referring sites like "vampirestat . com"?

Not everybody understands that Stats was not designed for catering to individual preferences, in deciding what statistics should be displayed.

Stats spam moderation was not designed to allow for individuals to decide on what figures to display - or ignore. Like Blogger Comment moderation, Stats is collaborative, fuzzy, and heuristic.

Collaborative systems work best when everybody is subject to the same restrictions. Blogger / Google can best fight referer spam by monitoring all activity, against all blogs, uniformly.

Stats displays show the top 10 entries, in every list.

Some referer spam, like some genuine traffic, is not displayed, simply because at least 10 other traffic sources (bogus or genuine) are displayed.

Since newer blogs, with less established real traffic, are more vulnerable to referer spam, owners of newer blogs are more likely to demand individual filtering options. Owners of more mature blogs, who have devoted more time to publishing their blogs, don't see the spam, and don't worry about filtering it.

If individual blog owners were given the choice of what "traffic sources" should be filtered, for each different blog, they would waste more time examining their Stats displays, recording what sources appear spammy, and updating their filters. This would leave them less time for maintaining and publishing their blogs.

Given this possibility, referer spammers would be encouraged to tune their attacks, and to focus their attentions upon newer blogs, to encourage the newer owners to waste more time maintaining filters. In some cases, this could even enable referer spammers to conduct DOS attacks against blog owners who are naive enough to believe that they can effectively identify and filter referer spam.

With individual blog owners given the ability to individually filter Stats displays, referer spammers would be better able to conduct DOS attacks against innocent, third party blogs and websites.

Like every other righteously needed Blogger enhancement, adding individually maintained Stats filters would make Stats more complex. This would require Blogger Engineering to spend more time developing a feature demanded by a minority of blog owners - and spend less time developing and maintaining other, more essential features.

It does not require too much imagination to suspect the legitimacy of some of the unhappy blog owners.

Why does Blogger not give me the ability to decide, for myself, what traffic sources that I want to monitor?

Some of these might even be the referer spammers, forcing Blogger Engineering to develop features that, in the long run, would benefit the spammers as much as the blog owners.

>> Top

Comment Spam And Referer Spam Has Various Purposes

One of the most intriguing subjects discussed in Blogger Help Forum: Something Is Broken involves the seemingly purposeless spam which torments us in our comments and our Stats logs.

I see this random garbage on my blog - and the sites advertised are complete junk. Is there any actual reason for this?

Interestingly, both nice blog spam (in our comments), and referer spam (in our Stats logs), have purposes - though with the continually varying content used in both, it all looks very much like random garbage.

If we analyse the content, structure, and volume of the various spam attacks, we can see patterns - and an understanding of the overall purpose of the spam.

Both our comments - and our Stats logs - are continually assaulted by seemingly random and useless noise, in large volume. This is similar to the random spam which attacks our email.

If you look at enough comment and Stats log spam, you will start to see patterns - and reasons for the spam. The spam, though apparently random in content, is not purposeless.

First, some warning may be appropriate.

1. If you wish to examine the sites advertised in comment or referer spam, learn to use a proxy server.
2. Unless you are a computer security expert, with a properly protected (or intentionally sacrificial) computer, do not surf these sites advertised, using an unprotected computer. Avoidance is the best way to protect yourself.
3. If you do, even inadvertently, surf a comment or referer spam site, quarantine and scan your computer as soon as convenient. If you must surf the sites advertised, while unprotected, diagnosis is the next best way to protect yourself (and others around you).

Examining the comment and Stats based spam diagnostically, you'll see various purposes behind the spam.

• Spam delivery. It is called "comment spam" / "referer spam", after all.
• Malware delivery. This is the secondary purpose, known by many people - and the reason why I provide the above advice.
• Attacks against third party, non spammers. Various third party blogs and websites, who do not subscribe to the service, may be maliciously targeted, by some spam.
• Strategic malware delivery. Some malware may be packaged in portions, delivered through multiple attacks.
• Filter poisoning. Some spam may be simply intended to overwhelm the malware / spam detection systems.
• Email address mining. Some very special comment spam, which I call "Nice Blog" spam, is a very imaginative email address mining technique.

The best way to identify comment and referer spam may be to simply follow the various forum discussions - and observe which spam techniques are reported by multiple blog owners, being assaulted in identical style. As the old saying goes,

If it quacks, it's probably a duck.

>> Top

Confusion About Referer Spam Can Have Unexpected Consequences

As referer spam continues to be reported in Blogger Help Forum: Something Is Broken, we see occasional signs of confusion.

Some blog owners, knowing how to use WhoIs lookups and similar online utilities, to their advantage, are looking up the referer spam targets - and are reporting the targets to their registrars and hosts. What these blog owners may not realise is that not all apparent "customers" of referer spam "services" may have actually contracted to be featured in the services.

In October - November 2011, this blog was featured in one wave of referer spam.

Somewhere, buried deep in my comment moderation queue for this blog, may be some odd comments

Chuck, I clicked on a link in my Stats log, and got this blog. Your blog does not link to my blog - so why do my Stats displays link here?

When I saw one such comment, I thought that it was just some spammer, messing with me - as many do (which is why I moderate comments, aggressively). As more such remarks appeared, both here and in Blogger Help Forum, I realised that what I had idly expected, long ago, had actually come to pass.

I've been using this blog, since I started it, as a weapon against spamming activity in general. In February 2011, I started using it as a weapon against referer spam. I did both knowing well that if I was ever seriously effective in my fight, the guys who I became effective against would eventually attack me.

In October 2011, I realised that my dream had become true. I was honoured by the providers of referer spam, as someone worthy of their attention. In spam fighter parlance, I was the victim of a "Joe Job".

Just as surely as I, personally, was a Joe Job victim, I know that other honest and righteous blog and website owners are similarly being attacked by referer spam, in other Joe Jobs. I know that I am not the only fighter of spam, worthy of the attention of the spammers.

Once we understand that not all blogs or websites featured in referer spam activity may be actual intentional customers of referer spammers, we have to realise that unfocused reporting of blogs and websites, mentioned in referer spam, will actually play right into the hands of the referer spammers.

What the referer spammers are doing, in their Joe Jobs, is also called a "Smurf Attack". The spammers are hoping that thousands of angry Blogger blog owners will report the blogs and websites, featured in the spam, as spam customers. This will cause the immediate recipients of the Smurfing, the registrars and Internet Service Providers, to discontinue service to the reported blogs and websites. In 2011, with such reporting taken seriously, this blog might have been damaged.

Hopefully, most ISPs and registrars know of Joe Jobs and Smurf Attacks - and know how to verify the nature of any individual service being reported for spamming. However, that's simply a complication which can have other effects.

The bottom line is that anybody seeing the referer spam, in their Stats logs, has to understand that not all blogs and websites should be reported for spamming. When you report all referer spam, indiscriminately, you are simply working for the spammers.

Please, don't work for the spammers. The existence of this blog (and other, much more significant blogs and websites) may depend upon your discretion.

>> Top

Why Does Referer Spam Get Reported, Primarily, By New Blog Owners?

One of the details of referer spam, that intrigued me for a while, involved who was reporting the spam.

Most - but not all - referer spam is reported by newbie owners. People who just setup a blog, and are anxiously waiting for their first readers outside their own country, are suddenly astounded

Wow, thousands of people are reading my blog - from China and Russia! I am an international celebrity, in my first week!!

The euphoria quickly vanishes, when told in the forums

Nobody is actually viewing your blog from these sites - and nobody knows about these links but you, since only you can see your Stats logs.

The general response to the cold water, dashed in their faces, is generally acceptance

Thank you for the prompt and informative reply - I'm very relieved to learn that the site is not actually linking to my blog.

The latter is the general response - but not the only response. Nobody should be surprised at that, as not everybody is the same. What is surprising, though, is who is not so accepting of my advice.

Of the people who do not seem to appreciate my advice - that the owner of the new blog is not an immediate international celebrity - not all of them are actual first time blog owners.

Many blog owners, who scorn my advice, turn out to be long time blog owners.

You are an idiot! I have other blogs, which I've been publishing for years - and none of them get this crap! How dare you tell me that this traffic is not targeting my new blog, intentionally?

Why would people who have published other blogs not see the spam traffic, in their other blogs? Particularly in other blogs that are well known, by the world, in general? And why would experienced blog owners not know of - or even believe - my advice?

The principle of Occam's Razor tells us to look for a simple explanation, to the questions. Instead of concluding that so many experienced blog owners are morons - or do not check their Stats logs like everybody else - or wondering if referer spammers carefully tune their spammy activities to only attack new blogs - the simple explanation is that every blog is subject to referer spam, in the same volumes.

Not every blog owner sees referer spam, in their Stats logs, simply because the volume of referer spam is not so heavy, compared to genuine traffic. If I check the Stats logs for this blog, repeatedly, throughout the day, I can generally find the currently mentioned spam domains, reflected in my pageview counts.

To reliably find the referer spam attacking my blog, I have to check my Stats logs frequently. In my case, when I note the domains in question, they will be listed in the "Now" time range - and occasionally in "Today" - when "Now" reflects early morning UTC (just after the daily counts reset).

If the referer spammers attack this blog during any other time of the day, the volume of legitimate traffic generally exceeds the referer spam traffic, any time outside "Now". Thanks to the limited function of the Stats logs, referer spam is simply not always visible, because legitimate traffic is more visible.

Thanks to regular changes by the spammers, and to ongoing spam blocking by Google, the same domains are not active for more than a week or so. The "Week" and "Month" normal volumes, for this blog, simply exceed the referer spam volumes.

And that's the case for everybody's blog. Check the logs for your blog, if you don't believe me. Just check frequently, and regularly, during a week, for the most accurate conclusion.

And that brings me to my last bit of advice, about the spam.

Concentrate on publishing your blog, and getting legitimate traffic - and the referer spam will not be an important issue.

That's not condescension, or snark - it's simple truth.
>> Top

Blog Owners, Marking Stats Log Entries As Referer Spam, Won't Accomplish Much

We see the occasional optimistic suggestion in Blogger Help Forum: Something Is Broken, from blog owners tired of the referer spam, in their Stats logs.

Why can't we just tell them which Stats entries are spam?

This is, admittedly, such a simple suggestion - it surely must present part of the final solution.

If one considers the purposes behind referer spam, one can see a problem with having individual blog owners designate specific pageview entries, in their Stats logs, as referer spam.

There are actually multiple purposes for referer spam.

Look carefully at the entries, in your Stats logs. Are all of the websites, which are being advertised by non existent links to your blog, actually spammy? Chances are, if you look closely enough, you'll find one or two which just don't seem to fit the pattern.

• The volume of hits makes the pageview counts look suspicious.
• There's no link to your blog, from the websites.
• Looking closely at the content of the blog or website, it just does not look like the typical referer spam target.

This blog was honoured, as a false target, in October 2011. I have no doubt that other blogs and websites are also the targets of similar, deliberate attacks.

With referer spam used strategically, the spammers could simply adjust the volume of their attacks, to make innocent victims more visible, and more vulnerable to false accusation by the less observant blog owners, who simply want all referer spam stopped.

To guard against referer spam being used strategically, to attack innocent blogs and websites, any designations "This is referer spam!" would have to be complemented by "This is not referer spam!". Ultimately, all Stats pageview entries would have to be verified as "Spam!" or "Not spam!", to prevent referer spam from being successfully used as an attack technique. How many blog owners would want to scan their entire Stats log, daily, and mark all pageview entries as "Spam!" or "Not Spam!"?

Understanding the nature of referer spam, one sees that the only way to effectively combat it is to have Google globally examine all pageview activity for all blogs, over long periods of time, to identify actual referer spam - and to avoid falsely designating innocent third party blogs and websites as being intentional referer spam customers.

>> Top

Stats Time Ranges, And Pageview Count Recalculations, Cause Confusion

Periodically, we see evidence of confusion about Stats and the pageview counts, as provided in the various displays and time ranges, expressed in Blogger Help Forum: Something Is Broken.

My pageview counts keep going down!

or

My counts go up during the day - but in the afternoon, they drop to zero!!

or

My counts keep going down! There's no end to the drop!!

All of these complaints - and more - involve blog owners, who don't understand the relationships between the displays and time ranges.

We've explored the results of the ongoing referer spam war, of normal viewer activity, and of the daily pageview count reset which is applied, simultaneously, to all blogs. Besides those 3 issues, there is some confusion about the relationships between the various time ranges.

There are 6 different displays - and opportunities for confusion.

1. The Dashboard "Overview" page.
2. The Posts "All time" per post pageviews count.
3. The Stats "Overview" page.
4. The Stats "Posts" page.
5. The Stats "Traffic sources" page.
6. The Stats "Audience" page.

The 4 Stats display pages each offer 5 different "Time range" displays.

• Now (the last 2 hours).
• Day (the last 24 hours).
• Week (the last 7 days).
• Month (the last 30 days).
• All time (since May 2008, when Stats pageview counts were first extracted) - though possibly missing one year or another.

Some blog owners try to compare the Stats "Overview" histogram (in its 5 time ranges), with the "Overview" counts ("today", "yesterday", "last month", and "all time history"). Unfortunately, the graphs and numbers are recalculated (by Blogger) on various schedules.

• Some are recalculated on a minute by minute basis.
• Some are recalculated hourly.
• Some are recalculated daily (but not necessarily at midnight, for everybody).

Confusion arises, when a blog owner refreshes a display, without understanding when the numbers behind the display may have been recalculated, relevant to reader activity.

• When reader activity occurs, the graphs / numbers (when refreshed, following a recalculation) will indicate an increase.
• When reader activity expires from relevance, the graphs / numbers (when refreshed, following a recalculation) will indicate a decrease.
• When reader activity is removed because it was bogus, the graphs / numbers (when refreshed, following a recalculation) will indicate a decrease.

Like the inability to balance the various per post pageview counts, the task of balancing the various graphs and numbers may be beyond the ability of some blog owners. Each count or graph has to be considered on its own.

Understanding any two displays, in context with each other, simply may not be possible.

>> Top

Fluctuations In Stats Pageview Counts Are More Obvious, To Owners Of Newer Blogs

Some owners of newer Blogger blogs spend time reading their Stats logs, and worry that their "All time" pageview counts don't always go up - they go down, too.

Why do the counts go down? Is there one number, that I can believe?

They don't understand that with Stats, you need to look for trends - not absolute readings. In the beginning, fluctuations are more obvious, and trends are less obvious.

Thanks to the referer spam war, and to normal visitor activity, even the "All time" numbers will go up and down - for newer blogs, which have less genuine and constant visitor traffic. And the constant rise and fall is scary - until you get used to it.

Newer blogs have less established readers, which makes both the periodic referer spam and normal visitor activity generated spikes more visible. With the 4 limited time ranges (Now, Day, Week, and Month), the spikes will "move" in and out of context - and make pageview counts in these time ranges fluctuate. When a spike in numbers "moves in" to a time range, the pageview counts will rise, suddenly. When a spike "moves out" of a time range, the counts will fall, just as suddenly.

With newer blogs, the fluctuations exert a "double whammy".

1. The fluctuations are more visible, because overall genuine visitor activity is lower. Less readers = less visitor activity.
2. Worrying over the fluctuations takes time away from working on the blog. Less time working on the blog leads to less blog content, attracts less new readers, and leads to less visitor activity.

More experienced blog owners know to use Stats to watch for trends - not immediate numbers.

The answer is simple, really. Monitor Stats periodically - not constantly. Watch for trends, and know how to interpret the trends. Spend more time working on your blog, because that's where the value of your blog lies.

>> Top

Fluctuations In Reader Activity Level Are Normal

One sign of panic, seen regularly in Blogger Help Forum: Something Is Broken, comes from people who spend a lot of time viewing their Stats logs.

My pageview counts are always going up and down. Why is Stats so flaky?

It's good to monitor your reader activity - sometimes this will warn you of a major problem in the Internet, or with Blogger. But keep it in perspective.

No matter what time range you view, in Stats - "Now", "Day", "Week", "Month", or "All time" - you can expect to see fluctuations - peaks and valleys - in pageview counts.

• If you spend time working on your blog this week, and your work is successful, your readers may enjoy the blog more next week. Conversely, if your work is unsuccessful, they may enjoy it less.
• As Google deals with the constant waves of referer spam, you'll see a drop in counts across all time ranges. And as new referer spam campaigns come online, you'll see the counts rise.
• Unless your blog is completely homogenous across all world wide cultures, you'll have one or two predominant audiences. Your audiences will eat, sleep, and go to work - and all real life activity will affect their viewing of your blog.

It's good to watch the counts, understand your audience, and be prepared to take action if you see a significant rise or fall - but don't obsess over the details.

Remember that your readers will appreciate content, so keep publishing to the blog. And, learn to understand the trends.

And, to best avoid confusion, use more than one visitor log / meter.

>> Top

Your Blogger Blog, And The Wave Of Spam Behind You

The sport of ocean surfing has been described as something like "constantly falling down an ever advancing wall of water", while trying to stay upright and ahead of the wall of water (aka the "wave"). Publishing a Blogger blog, and dealing with the spam, is similar to surfing.

I am frequently advising people, who complain about the constant waves of spurious spam classifications, blog clones (aka "content thieves"), and referer spam, to keep looking ahead - and keep publishing their blogs. Like ocean surfing, you want to keep looking ahead and remain standing - don't fall down, and let the wave behind wash over you. The latter leads to drowning.

Blogger blogging provides a great environment for spammers to operate.

• The Blogger address space provides great cover for the spam.
• We publish our blogs, with constantly updated content.
• We advertise our blogs, and provide our updated content in our blog feeds.
• We read our Stats logs, see the referer spam, and give the spammers free inclicks.

What more could any spammer ask for?

• Free address space.
• Content provided by other people.
• Advertising provided by other people.
• Activity provided by other people.

And that is the wall of spam, that we, as publishers of Blogger blogs, have to stay ahead of.

How do we stay ahead of the wall, constantly approaching? Keep publishing your blog, keep watching your reader audience, and ignore the spam. Look ahead, stay upright, and don't fall down.

And that's surfing, in the ocean of Blogger blogs.

>> Top

Concentrate On Your Readers, Not On The Spammers

We've been exploring the nature of referer spam for over a year now. I've provided various suggestions about the problem - what you should not do, the observation that it does not actually show traffic to your blog, and even why Stats is vulnerable to this abuse.

To the most frequently asked question

What do I do about it?

the answer is generally

Don't click on the links.

And that's just half of the answer, if we think about it.

We've slowly realised that referer spam is probably here to stay - so "Don't click on the links" is similar to "Close your eyes, and you won't see it". Maybe, we should try a better answer.

How about we tell everybody to just get back to work on their blogs?
This will have 2 benefits.

• You won't have time to click on the links.
• Your blog will get some real traffic.

In time, with actual traffic from actual readers, you won't see the referer spam. You'll have actual reader activity, in your Stats log. Isn't that really what you want?

Win - Win - Lose (You - Your readers - The spammers).

>> Top