The Daily Whim

The Daily Whim

Sun. Dec 12, 2004

MT Plus Comment Spam Equals Dead Site

Let’s start by explaining our terms (and thus likely narrowing our audience). If you don’t know that the “MT” in the title stands for “Movable Type” (a very popular program used in creating weblogs), this article likely won’t interest you. As for “comment spam,” Adam Kalsey explains: “Usenet news succumbed to spam long ago. Email was next. Now spammers have turned their attention to weblogs and comment forms. In order to increase search engine rankings you are posting advertisements to our Web pages.” Note, his “manifesto” was written over a year ago, and the problem has only gotten worse since then. Much, much worse.

In fact, it’s now stressing web servers so greatly that a number of hosts are shutting down comments in Movable Type, or shutting down Movable Type itself. So, if you run a weblog using Movable Type, and have comments enabled (even with MT Blacklist, as you’ll see below), you’ve got a problem. Or rather, you may be causing one at your web host, and you may get shut down with no notice.

I’ve had to restore permissions on MT for a friend who got shut down at Pair due to comment spam, and the server my site is on at TextDrive has been taken down (briefly) twice in the past week or so by thousands of MT processes run amuck. And we’re talking industrial strength web servers, like the one discussed below:

You mean the persistent disruptions of my weekend luncheons? Yes.

All MT. All mt-comments.cgi.

We typically run at a [server] load less than 1 just about everywhere. Despite having millions of emails and millions of web hits every day now. is a relatively expensive server with dual xeons, 6GB hard wired RAM, 6×73GB RAID-arrayed SCSI 10K Seagate Cheetahs and a FreeBSD kernel optimized for running web and database servers. It’s also on a 100Mbps switch with two Gigabit ethernet cards. I have a Linux-threaded static build of MySQL and I load that build into 1GB of RAM when the database server loads up. I have a fast static build of Apache in front of that.

The server is designed to be the kind of thing that could do 20,000,000 requests a day without a problem. In many ways no one uses servers like these for shared hosting.

If you were to lease’s hardware and it’s network connection, if you would cost you about $12,000+ a year. If you were to contract someone like myself to build it and get it up and running, it would cost you about $10,000. If you were to hire someone like me to watch over it, it would cost you well into the six figures a year.

So that said…

MT-comment.cgi, it’s inherent nature and the fact that it’s targeted so can push this server to loads of nearly 300…

Jason at Textdrive Forum: MT comments on one are off

Jason also says “MT-Blacklist sucks,” and has the screenshots and figures to illustrate (a site that gets 300-400 legitimate comments per month is getting over 46,000 hits on mt-comments.cgi, while MT-Blacklist has less than 200). And it’s not just at TextDrive, as Joe Katzman reports:

...We have taken other measures to combat the 20,000 comment spam attempts we’ve seen in the last 2 weeks. For reasons we’re still trying to figure out, the spams are causing problems for our hosts at Total Choice Hosting due to server load.

Over the last 2 weeks, Blacklist may have blocked 18,000 spams, but it also forced moderation of another 2,000 or so, and in many cases they were already-blacklisted items that got through due to a flaw in the system.

Blacklist also changes in one more important way. Instead of comparing new comments against a text file blacklist, it stores the blacklist items (in our case about 2,750 items and 60 programmed “catch alls” for various things, and we aren’t unusual) in MySQL. This forces MySQL database calls whenever a comment is submitted. 1500 database hits a day may not mean much, but if you get 100 from various IPs in about 10 seconds, is that a problem? I don’t know what’s happening elsewhere, but it has been a problem for us at Total Choice Hosting.

Meanwhile, if you’re considering following in our recent technical footsteps [an “upgrade” to MT 3.x and Blacklist 2.0], a word of friendly advice: DON’T.

And elsewhere, from Eric Rice: “I woke up this morning to pissed-off people. My ISP was faced with Movable Type (MT) scripts bringing down the server, as the newest wave of comment spam seemed to make anything near this server not respond — DNS, Email, WWW, SSH — you name it [...] The culprit? MT-comments.cgi was running tons of processes, taking half the memory for a light-trafficked community blog site.

And from Chris Lehmann:

I came home and found us in the middle of a blog spam attack. The load on our machine was up around 200 by the time I was able to get a root prompt and shut down httpd. I had to let the machine calm down, restart httpd and then log into MT to be able to add the offending spammer to our blacklist. Each time I restarted httpd, the attack started again, and I had to shut it down after getting one step closer.

The whole process took about an hour of my life that I can’t have back.

Now, right now, Beacon can’t afford to upgrade to MT 3.1. So either we have to turn off comments, stop blogging or deal with the fact that we all will have really disgusting email messages in our inboxes every day and have periodic shutdowns of our system.


Pathetic, indeed.

Elise Bauer has become well know for her helpful tutorials on using MT, and she saysSpammers are getting more aggressive with Movable Type blogs everyday. I have found the only really effective measure to completely block spam is the combination of using MT3 and TypeKey to require approval of comments before they are posted and the MT-Blacklist to keep your inbox from being swamped by hundreds of spam comments waiting for approval.

I don’t mean to criticize Elise, as I’m sure what she says is true. But from what I’ve seen, Typekey is a real barrier to a lot of would-be commenters, who will simply go away rather than sign up (plus those who do create an account, and still have trouble leaving a comment). And then if the visitor successfully jumps through that hoop, the only way left for the site owner to block spam is to approve each and every comment manually, even though you’re running server hogging “automated” apps? That’s not a commenting system, that’s a commenting bureaucracy. On the front end, you’ve got a hurdle for the visitor who just wants to bang out their thought, and on the back end you’ve got a hurdle for the site owner, who must manually approve each and every comment. Like it’s a job, or something.

If it has truly come to that, why not just put an e-mail link at the bottom of each article that says “if you have a comment, please e-mail it to me,” and then paste them into a passworded form yourself. Because that is in essence what you’re doing with the above “system.”

In the saddest irony, posts from the developers themselves are afflicted with trackback spam (scroll to the bottom). Including the one announcing that Jay Allen is joining them to combat comment spam. I’ve watched spam appear on those threads, be removed, and then one week later, it’s back. On the developer’s site.

Not very encouraging, since Six Apart is the obvious direction MT users turn when they face this issue, and many of them have paid money for a license this year. To Six Apart’s credit, they did hire Jay Allen, and one would assume they’re hard at work on a solution. I don’t know, I’m not programmer, but it seems to me they are so deep in the Perl Soup that the whole MT Community may take some heavy blows before any viable solution is publicly available. It’s going to take more than a band aid. Again, I’m no programmer, but it seems as long at the rendering of comments involves static files rather than dynamic display (as well as some throttling of MySQL requests under the stress of a spam attack), there will continue to be big issues.

But I don’t think you can lay all of this on Six Apart. First of all, how can you damn them for the fact their software has become so popular? It is, in effect, the “Outlook” of blogging tools, and it is therefore targeted by Black Hats, just as Outlook is. For the same reason. Simple predominance. It’s hard to blame Six Apart for that.

Then, of course, you have the cockroaches of the Internet, the spammers themselves, who deserve nothing less than to be cast into a swirling sucking pit of despair, where they will spend eternity taking 80% of the overdose level of Phentermine, Viagra, and Rogaine, while being forced to play a version of Texas No Hold ‘Em Poker where the losers will get forcible breast implants … if they’re male … and penis enlargements for the rare spamming woman.

Finally, we have Google. In fact, they are the Patient Zero of this plague. These spammers leave comments with links in weblogs because many weblogs have a relatively high Page Rank (the way Google sorts returns for any search), and by creating a link within that highly rated site, they “steal” some of that Page Rank, in hopes of increasing their own search returns for their various nefarious schemes.

I hear those guys at Google are pretty smart, and gots lots of computers. I’m betting they could figure out a way to filter these spam comments from their index, based on keyword or URL, or even establish a common protocol where anything wrapped in a certain tag/id/class (like the whole list of comments) would have no URL’s indexed by the Googlebot. Like I said, they’re pretty smart, and I feel certain they could provide a solution to this.

So … why haven’t they? Well, my guess is because Google owns Blogger. And Blogger competes with MT. So why would Google go out of its way, or be in any hurry at all to help a competitor with this problem? Perhaps especially now that it’s causing MT to be shut down in some places.

Meanwhile, in just my limited personal experience or reading over the past two weeks, five hosts have in some way disabled MT or MT comments because of the server load they were creating. Not five little Mom & Pop hosts, at least three of them I’d consider serious to top-notch hosts. One of them, Pair, has been around forever, has a serious rep, and as late at this May, claimed they were very “MT friendly.” Yet last week I had to log in to a disabled MT install at Pair, and get the permissions back up again so the author could switch over to Typekey. Which has generated problems and complaints from visitors.

If you’re an MT user, I’m not sure what advice to give you. I still use MT, but for two “miniblogs” in the sidebar with no comments, and one that does have comments enabled, but only gets a legitimate one maybe once per month (it’s never gotten spammed either). Still, my primary weblog, what you’re reading right now, is powered by Textpattern. In addition to its various built-in spam countermeasures, it’s a dynamic system rather than static. No rebuilds. Wordpress is similar, in that it is dynamic and has spam countermeasures.

Of course, neither app’s user base is as large as MT’s, and therefore they are less targeted. That could well change. But if it does, I think both have some advantages that MT lacks. Both are open source, and both have devoted communities. The collective response to a serious problem would be organic, broad based, and swift. With Six Apart, it’s not open source, it’s a corporation’s property. Though they’ve been hiring coders at an increasing clip, Six Apart still has a limited number of man hours to throw at any problem, given the profit demands of product growth.

While I very much hope Six Apart can pull a mean fanged rabbit out of their hat, I wonder if they can do it in time. The problem is rapidly escalating, and users are having their sites shut down. Afterwards, they face the same problem with a choice of MT solutions that is very limited, and overly complex for many users.

Frankly, I’m out of the MT support business, simply because I’m now dangerous. I don’t know MT 3.x (just up to v2.6, for me), don’t know Typekey, and don’t know MT-Blacklist. Don’t wanna know. Don’t need to know. I’d rather spend the time learning about things with which I can earn money, and I think it’s pretty conclusive that there never be a Blogger Caste living in mansions in Beverly Hills.

You, however, may be an MT user who hasn’t had a problem. Or, have only had a few comment spams. With the number of blogs out there (especially with less than attractive Page Ranks), the odds are in your favor. You just have to wait and see. And the same with a new solution from Six Apart. I have no doubt they will do something. You just have to wait and see what it is, and how long it takes.

Just know that the e-mail could come with no warning: “Due to problems it was causing on the server, we were forced to disable the following script located in your account…

So you can wait. Or you can move forward. You’re going to invest time in it either way, so pick yer poison.

And be aware that MT is becoming less than popular on many web hosts out there on the InterWeb. It’s taken down this site twice in a couple of weeks, and this site wasn’t under attack. Its MT-using server-mates were.

Later: An example of how Typekey and MT-Blacklist can frustrate the hell out of a user.

Even Later: From Anil on the Six Apart site: “There are a variety of ways to deal with spam, ranging from technical to legal to social methods, and we’ll discuss them all [...] We’ll have more details today, and a full overview within 48 hours.

Important Update: The solution is coming.

Peanut Gallery

1  Greg Greene wrote:

I run my own site on MT, but I spend more and more of my time in Textpattern because of work, and ... well, I have to say the switch is starting to look tempting. When you consider that I get about 40 comment spams a day — not to mention trackback spam attacks — for a site that I’ve taken a site that I’ve taken a month-long holiday from … well, I shiver at the thought that it could get worse.

A friend just asked me to build a political site for him, and my first thought was to run it in WordPress. I have a hunch that a lot of MT loyalists like me are starting to have the same thought.

2  Paul wrote:

Typekey is a real barrier to a lot of would-be commenters, who will simply go away rather than sign up (plus those who do create an account, and still have trouble leaving a comment). And then if the visitor successfully jumps through that hoop, the only way left for the site owner block spam is to approve each and every comment manually, even though you’re running server hogging “automated” apps? That’s not a commenting system, that’s a commenting bureaucracy.

The great advantage to that set-up, in addition to reducing spam, is that it makes a great idiot filter. If you provide only one obstacle to filing a comment (beyond the so-easy-it’s-foolproof-way), then you’ve just reduced the idiot content of your comments section by at least 75%.

I would go futher and develop or find a script that automatically closes the comments section when it reaches 20-25 comments, because experience shows that the only fruitful discussion happens within the first 20 comments, everything after that is redundant filler or flamewars. The advantage to this system is that if your comments are closed, spammers can’t attack them.

Lastly, I’d rename mt-comments.cgi to something else and disable pop-up comments. IIRC, most spammers to my old MT site keyed-in on that specific file more than anything else.

3  Reid wrote:

Paul, I understand your intent, but rather than place a mechanical obstacle in place, I find it’s better to just use big words and write about obscure topics idiots don’t care about. However, your “Comment Quantity Filter” has some real merit to it.

As for renaming mt-comments, I advised that, too, but I’m afraid it has escalated beyond that. If you ping, as everyobdy does, these spambots will follow that to find the new individual entry and get its ID number, find the form and POST method, and then start a chain submit incrementing the ID number. They can leave 200 in no time.

There are people using all existing methods to block MT comment spam, and still being shut down by their host. For some MT users (those with a Page Rank of 6/10 or higher, we’re finding), there is simply no available solution that even slows the onslaught.

4  Reid wrote:

I would also note there has not been an electron emitted on the Movable Type home page or the Six Apart homepage for two months. No hint if there’s even an awareness of this issue, nevermind work on a solution.

5  Melanie wrote:


The attack I had earlier this week got through closed comments, closed from the main menu. It was awful. Blacklist has been working well since, however.

But even that’s a problem, as I’m using more cpu cycles than ever in the past and may get shut down for that.

6  Adam Kalsey wrote:

The spam problem has gotten worse in the last year mainly because so many people do nothing about spam. Those who ignore comment spam and allow it to remain on their sites are as much to blame for the increase in spam as Google is. If spam were not tolerated on blogs, then the spam links wouldn’t last long enough to be particularly effective.

With email spam the 0.01% of people who actually respond to spam messages make spam profitable and cause the rest of us to get thousands of junk messages daily. Likewise, the small percentage of people who allow spam comments to site on their sites ensure that comment spam is a viable promotion technique.

The simgle most effective thing a person can do to combat comment spam on their MT blog is to rename their comment script. By far, the majority of spammy comments come from bots that exist only to send POSTs to mt-comments.cgi at random hosts using random entry ids.

Despite having a relatively popular blog that enjoys fantastic Google rankings for common search terms, I receive comparitively little comment spam. Maybe 20 spam messages slip through to where I see them each month and of those, only one or two a year actually hit the pages of my blog.

This is due to several things. First, I renamed the comment script. I get thousands of 404 hits a day to mt-comments.cgi, so I know this measure works. Second, I have a check that ensures commenters are actually submitting my comment form, preventing the random entry ID POST attacks. Third, I reject comments from known open proxies and email-spam friendly hosts thanks to a combination of Brad Choate’s DNS Blackist plugin and some of my own custom tricks. Fourth, if a comment makes it through those lines of defense, MT Blacklist does a fantastic job of preventing it from appearing on the site.

7  Reid wrote:

Thanks for your comment, Adam, and I agree with the “broken window” theory. In fact, I’d bet there’s some script that checks to see if a spam comment is removed before Google indexes it, and if not, they send it back for more. Lots more. Bust out a few more windows in that old building, since the first ones broken never got fixed.

And your list of suggestions is great … but … how many MT users are willing or able to deal with it at that level of complexity? Like I said, I got brought in on one site because CHMOD was a challenge, so how many would be able to hack in a check that commenters are actually submitting a comment form from their site (a great idea that ought to be default)?

I’d like to see a lot more things like what you list available by default in MT. But, apparently, it’s near impossible to get any response from them on this issue, from what I’ve heard. If I was still a heavy MT user, I’d be somewhere between concerned and pissed.

8  Rusty wrote:

When I finally moved away from Blogger to “real” software about a year ago, I tested out both MT and Wordpress on my local machine and found MT to be painfully slow by comparison because of all the static rendering.

9  Richard wrote:

As of 3.1, Movable Type has dynamic rendering using PHP.

Last year, Mark Pilgrim wrote an article on comment spam arguing that comment spam is a losing battle.

10  Anil wrote:

First, thanks Reid for taking the time to write this up. For an immediate fix, I’d recommend that people do enable TypeKey and/or MT3.x with comment moderation. Comment spam attacks seem to come in waves, and we’re definitely seeing an increased volume recently.

You can let TypeKeyed comments go through without having to moderate them, so the site admin isn’t burdened with responding to each comment individually. I do understand that’s a barrier to some commenters, but so is moderation of any sort (“my comment didn’t appear right away!”).

Jay Allen has just about finished his latest update to MT-Blacklist (the beta is released to our Professional Network right now) and he’s very aware of the massive amounts of spam attacks going on right now. I believe he was waiting to post about this once we have an improved solution, but I’ll make sure he knows it’s reached a critical level again right now.

Regarding spam on our own site, we’re actually running the site without MT-Blacklist, and I’ve personally been watching the spam attacks on our site. We’ve always been very up-front about being transparent with the feedback on our site, and we’re not going to start hiding criticisms or weaknesses now. It’s not that we’re not embarassed or disappointed, but we are working on it.

There’s a couple of other points of failure here. First, spammers are clearly scraping the update monitor sites. (,, and most of the others have been known sources of spambot traffic.) A simple authentication being required to view the update lists on those sites might go a long way towards limiting which sites get hit. Second, Google’s PageRank algorithm seems to have almost completely stopped updating. Let’s not forget, these spammers are trying to increase their PageRank. Given the semantic similarity of the markup used by most comment pages, it wouldn’t seem to be incredibly difficult to have PageRank evolve to stop rewarding comment spammers.

Also, I talk a lot to the creators of almost all the popular blogging tools, and it seems like spam is increasing at the same rate on every platform. The only solution that’s consistently been successful is comment authentication, and despite the pushback we got when we pre-announced and then launched TypeKey, the adoption of sign-in by Blogger and MSN Spaces seems to bear out this assessment.

Finally, in regards to the dynamic/static debate which never seems to end, MT does let you choose whichever you prefer. To be clear, you’re either building once, at publish time, or building many times, with each visit to your site. However, if your site is getting automated hits from any kind of malicious script, the system used to generate your pages is probably not going to be the deciding factor as to how much server load you’re generating.

There’s a lot of FUD and misunderstanding on this point, so let me reiterate: If you’re building a page on every view, that’s usually going to be the same amount of load as rebuilding the page dynamically.

11  Anil wrote:

Sigh, sorry ‘bout that confusing last sentence. That should be:

There’s a lot of FUD and misunderstanding on this point, so let me reiterate: If you’re building a static page on every comment, that’s usually going to be the same amount of load as rebuilding the page dynamically.

12  Johan Svensson wrote:

I googled a phrase I saw in comment spam, with an obvious typo that spammers most likely use to track it.

205,000 hits. That’s just depressing.

Anil wrote: If you’re building a page on every view, that’s usually going to be the same amount of load as rebuilding the page dynamically.

There’s a major difference between MTs rebuilding and dynamic generation: MT forces a rebuild for each and every comment that gets added.

Dynamic generation just throws the comment into the database; there’s no page generation until someone actually watches the page in question. And the spam scripts just do a driveby; I doubt they actually fetch the page after the deed is done.

13  Michael Pate wrote:

I recently moved some sites I had on a virtual server I have been managing over to DreamHost. MT installed correctly but I could not get MT-Blacklist to function correctly. Yesterday, I spent an hour deleting trackbacks manually every few minutes (much easier than under 2.x but still annoying).
After reading this thread, I followed Adam’s advice and renamed mt-comments and mt-tb to semi-logical alternatives. I found Jay’s announcement and upgraded to his beta of MT-Blacklist (I know he said it was non-production ready, but I decided to risk it).
I still look forward to the day when either MT better deals with this stuff or the alternative platforms are better alternatives for the sites in question, but until then, it seems it will be an ongoing battle.

14  Greg Greene wrote:

I edited the name of mt-comments.cgi quite a while ago. I did the same for Mel, who Reid helped with the site shutdown that he mentioned in this post. That solution, we can safely say, is no longer operative—at least not by itself.

I hear you on TypeKey, too—but it can also serve as an inadvertent perfectly-normal-commenter filter. I can attest to that, thanks to my experience with Mel’s site after Reid resurrected it from the spam attack. No matter how often I log into TypeKey, the site never manages to recognize me. «sigh»

[In fairness to Mel and to Six Apart—hi, Anil—that problem may be a template issue. Mel, I’ll work on a solution to that as soon as possible.]

15  Ryan wrote:

Anil wrote: If you’re building a page on every view, that’s usually going to be the same amount of load as rebuilding the page dynamically.

Agreed, but the manner of building the page is what’s causing the problem. PHP handles it fairly well, but it’s the old non-threaded perl cgi script that’s causing our server loads to top 200

16  Kevin wrote:

The fix is to remove the benefit for the spammer. Put your comments in a format that Google (or any other spider) will ignore. Use javascript to open your comments page. List your comments using a javascript include and tell users that’s what you’re doing. I’ve had zero problems with comment spam in my over two years of using Movable Type.

The Moderate plugin for MT is also a good option. It turns on moderation for all posts older than X days, which should keep the number of places spammers can hit to a minimum.

And some of the blame DOES fall on Six Apart for, even after knowing they had a problem, keeping the benefit in the default template (which most people probably never change, or change enough to remove the benefit to the spammers).

17  Richard wrote:

I can confirm what Anil is saying about other publishing tools receiving a lot of spam, that is, an increase from zero. Drupal-powered sites (to take an example out of thin air) have been targetted as well.

The solution may not be authentication, as Anil suggests, but distributed authentication. Drupal has this “out of the box”, and other systems will have this when Identity Commons and Sxip become more popular, giving administrators more options to allow comments from trusted users (or, rather, trusted “types” of users). It’s early days for identity-based solutions to comment spam, but like Anil suggests, they hold a lot of promise.

18  Scott Chaffin wrote:

The joy of the internets just never stops, does it?

This, by the way, is a canned response from The Fat Guy:
Until we users get serious about doing something about it, it’s going to continue, and for all platforms, Textpattern included. If you want to stop spam of any sort—comments, email, pop-ups, or usenet—we’re going to have to fund and support the ‘net equivalent of the Texas Rangers. Not-so-white hats out there on the frontiers doing justice without waiting for the circuit judge to get back to town and bless the hanging. There’s no question that’s harsh, and innocents will catch a bullet every now and then, but what do you want? I doubt that we’d even need funding…I’ve got a bare-bones Linux system sitting here connected at very high speed 24/7 ready to do some damage for free. I ain’t the only one.

I mainly say this because I’ve been in the internet security wars for about 15 years now. Absolutely nothing has changed in those 15 years except the rapidity with which the black hats can operate and adapt to the humane measures we are taking so far. We’re all still living inside the stockades and building more walls that have to be climbed, but nobody is out killing Comanches.

19  Reid wrote:

Damn, Scott. I’m nominating you for Director of Homeland Security. Just one question.

Do you have a maid or nanny?

20  Lisa Williams wrote:

I use WordPress but I get plenty of comment spam, about 40-50 a day.

I agree that this should be laid at Google’s doorstep. Because they haven’t fixed the hole that lets spammers increase their page rank by spamming my comments, I’ve spent hours deleting spam comments and installing, configuring, and reconfiguring plugins to combat comment spam. Google is, in effect, outsourcing the work of making sure their search results reflect real links to me and thousands of other bloggers, and I haven’t gotten so much as a mousepad ;).

Plugins are good, but they’re a bandaid solution. I’ve moved from using Colin Devroe’s Optional Comment Moderation, to Dr. Dave’s Spam Karma Plugin.

Spam Karma and OCM work. That’s great. Spam Karma, in particular, has stopped spam comments from appearing on my site, but the spambots still hit it all the time.

And I’m still spending a lot of time messing with spam comments that I’d really rather spend doing anything else; and because I’m not really certain what all of the options do, I misconfigured one of the plugins, which meant a lot of people who were not spammers got their comments deleted. I’m still not quite sure I fixed it properly, and I figure it will take me a few hours to wade through all the documentation and puzzle it out.

Also, last month, my husband and I shut down WP on the colo server we share with 3 other people, because fetches from search engines that were ignoring robots.txt and hits from comment spammers were making everything so slow. So we installed prerendering, which, if I’m reading this correctly, takes away the advantage of WP being dynamic(?).

21  Richard wrote:

they [Google] haven’t fixed the hole that lets spammers increase their page rank by spamming my comments, I’ve spent hours deleting spam comments and installing, configuring, and reconfiguring plugins to combat comment spam. Google is, in effect, outsourcing the work of making sure their search results reflect real links to me and thousands of other bloggers

I’ve read this argument before, and I’m still not convinced. The spammers are not exploiting a hole in Google, but rather are exploiting how PageRank is designed. In other words, this is a feature, and not a bug.

The only 100% effective solution to eliminating comment spam is to stop allowing comments and encourage others to write their own weblog post in response.

22  Lisa Williams wrote:

I think it’s a bug because it’s not good for Google, either. How does Google and its search users gain from having online casinos and offshore pharmacies rise in page rank to unrelated search terms? It’s not good for the search end of the business either.

PageRank wasn’t passed down the mountain on stone tablets. If it produces spam on hundreds of thousands of websites and produces bad search results, why can’t it be changed?

23  Al wrote:

How does Google and its search users gain…

Your thinking these two things have anything to do with one another?

Comment by Al · 12/13/2004 09:33 AM
24  Richard wrote:

Which unrelated search terms did the spammers rise in the rankings for?

25  Scott Chaffin wrote:

No maids or nannies, but I do have one guy in indentured servitude down at the ranch. He’s from San Diego, though.

Maybe people should turn off the Google crawler and ignore PageRank? I don’t have a clue what my PageRank is, and I don’t have a clue what it could mean to my ‘umble efforts. Now I’ve only had one attack, and I renamed the comment/trackback module in response, and the problem went away. Between Blacklist and CloseComments, I get about one spam a week, and I’m usually on top of it right away. Yeah, Blacklists slow things down, but um, it’s a comment on a blog, not a heart/lung machine remote monitor.

I also wonder how much of this is a result of concentrated blog servers, like HostingMatters or something like that. I’m on an eensy, resold-six-times shared server, and again, only one concentrated attack. Now,what that may say about my popularity is not something I want to explore. heh heh

26  Adrian wrote:

Since last week, I’ve been averaging close to 400 spam comments a day, and this is on a site that’s not very well known.

This is up from around 2 or 3 comment spams a day in June.

I run WordPress, and out of need, I’ve had to go back and turn comments off on all posts older than 1 month simply because the spambots were slamming the site from the shear amount of traffic between crawling the comments rss feed and posting comments.

A few characteristics I have noticed… The pool of IP addresses between all the spam comments on my sites is less than 400 unique IPs, and many of the comments have at least one two word phrase (or one long string) that is common among the bulk of the posts from the same source that positively identifies the comment as coming from the same source. Heuristically speaking, that is very significant. I certainly hope that many of the blogging packages out there take note of this and allow some heuristics analyzing into the spam that gets deleted to better catch future common spam. It’s amazing how simple many of the comment spammers are being, and tracking them should be fairly easy to build into the blogging packages. The side effect is this will force them to get more sophisticated, thus causing more comment spam to get through, but I’d rather them have to work harder for it than to simply let them spambot us to death with little to no effort.

One last thing I’ve noticed, at least here on the WP powered side of things… They tend to trawl the backwaters where you’re less likely to notice (if you’re not paying attention), and they do spam the same post multiple times if one does get through.

27  Usman wrote:

Why not use the “enter the code you see in this image” trick that I see all over the web? That’s easy enough for users and won’t generate complaints. Spammers still haven’t figured how to OCR images to extract those codes.

A slightly more difficult method is used by blogs on – they require your e-mail address, then you have to click on a link they send you via e-mail to confirm your comments. That’s pretty easy too.

I thought MT was the king of blogging software, aren’t there plugins available to do all this stuff?

Comment by Usman · 12/14/2004 02:09 AM
28  DanteDante wrote:

I agree with Usman; but there are some problems with that approach:

1. Users won’t like extra typing (especially if it’s a mix of letters and numbers)

2. Generating the images dynamically with PHP is impossible without the GD library. I don’t know wether most servers have this or not, I think mine does. AFAIK GD comes with PHP 4.3+, and my server just upgraded to PHP v4.3.8, so I guess I have it. But I don’t how popular the GD Library is.

What I am going to do is just look for repetive instanses of popular comment spam phrases (ie Texas Poker Holdem, Viagra, Loan payment). Not sure if it’ll work, as they may just end up pulling the crap they used to get past filters (T3xa5 P0k3r, v1agra)

29  Justin French wrote:

MT is the king of nothing. MT may be the most popular software out there, but it can’t last forever. I had a much longer comment typed up, but I think this short one will do just fine.

30  Bernie Goldbach wrote:

I block 460 IPs from commenting and have found that Six Apart’s industrial strength spam controls have sliced my comment spam burden from 20 a day to no more than 6 a week.

But I rarely get more than 2000 unique views a day, so I imagine I’d be facing more of a hassle with higher traffic flows.

I think there’s no need to show “recent comments” on all blog pages and no need to keep comments enabled on topics more than two weeks old.

31  Yams wrote:

On my MT blog I had to resort to going to the individual entry template and removed the code that allowed the comment form to be allowed at all and instead just replaced it with text saying the comments were closed, and to email me if someone wanted to add something. I have comments open on my main page, however, as I have never received any spams to new entries (I’ve received about 10,000 in the last month or two to old entries).

Comment by Yams · 12/14/2004 04:01 AM
32  Tom wrote:

I’m in the middle of a flat out battle with my webhost on this right now. They’ve dropped my site twice in the last couple weeks – without ANY notice – and barely answer me when I ask them what happened. That’s a whole different discussion, though. It seems that my problem isn’t even the comment spam – it’s blacklist. I’m getting huge slams of spam attacks, left and right. I’ve changed MT-Comments.cgi’s name twice in the last couple weeks. I swear that someone comes to my site, gets the new name, and posts it on a message board somewhere that “Search Engine Optimizers” – not the good kind, either – get their ideas for the day.

What they don’t realize, or at least their “customers” don’t, is that most of this garbage is deleted within minutes when it does land on a site. When it sticks around, it is because the site’s owner hasn’t been going back and modding the comments or trackbacks – while it’s not optimal to do so, it comes with the territory. Six Apart’s own site having this problem is kind of sad, don’t you think?

So anyway, part of this is a hosting issue, part is not. I’m in the middle of moving hosts, upgrading to MT 3.x, and going from there. I don’t like WordPress or Expression Engine, really, and may end up using a different kind of CMS if it comes down to it.

It’s just so unfortunate that there are people – who are getting ‘paid’- who are literally ruining the efforts of writers and publishers with their junk. I have declared war on them so many times it’s not funny. Unfortunately, just as fast as I’m good with technology, so are they. Guess that’s what happens when the smarts are on both sides of the game.

Comment by Tom · 12/14/2004 04:08 AM
33  dan wrote:

Just a note that the link to Joe Katzman is blank. It has the ‘a’ tag, but no href.

Comment by dan · 12/14/2004 04:51 AM
34  Reid wrote:

It’s been interesting to read the comments as this article has gotten linked elsewhere, so let me try and “gang address” them.

Anil” “Jay Allen has just about finished his latest update to MT-Blacklist … and he’s very aware of the massive amounts of spam attacks going on right now. I believe he was waiting to post about this once we have an improved solution, but I’ll make sure he knows it’s reached a critical level again right now.

Thanks for the comment and info, Anil. I feel a bit out of place on this one, since I’m no longer a “primary” MT user, but I think any communication is good on a critical topic like this, even if it’s just “we know, it’s bad, we see the evidence too, and we’re hard at work on it. More soon.” Until your comment, I’d seen some doubt about that from others. So, for whatever it’s worth, transparency is your friend…

Anil’s suggestion to “enable TypeKey and/or MT3.x with comment moderation” may be fine for many users, but it’s already clear that it’s equally problematic for many users (visitors and site owners), and not a long term solution. However, when I read Richard’s philosophy thatThe only 100% effective solution to eliminating comment spam is to stop allowing comments and encourage others to write their own weblog post in response”, I was reminded of my 11th grade biology teacher, who said the only 100% effective form of birth control was the word “No.” A perfectly factual statement, but not realistic advice to a roomful of raging teen hormones. They’re gonna do it, so you’d best give ‘em some protection.

Even the suggestion of changing the name of mt-comments.cgi seems obsolete, as these spambots are now finding the new name via the form POST.

But Bernie and others have mentioned, one of the best things you can do is close comments on entries (X) days old, no matter what program you’re using (I use 6 weeks). I started doing that a long time ago, and not just because of comment spam. I got tired of late night Google searchers trolling a 1.5 year old topic they’d stumbled upon. Reducing the target size for spammers was a side benefit. After a relatively short amount of time, most discussions have covered the topic.

Kevin suggestedPut your comments in a format that Google (or any other spider) will ignore” which will indeed keep spam comments from being indexed … but the spam bot trying to deliver its load won’t know that. It will drop its merry load anyway. I thought about adding a meta robts tag with the instructions “index, no follow,” which would cause the Googlebot to index the page, but not follow any links on it. But again, the spambot won’t know that, or care. They aren’t that sophisticated. Yet.

Scott saysMaybe people should turn off the Google crawler and ignore PageRank”. I suppose that’s an option, just ban the Googlebot in robots.txt, and wait for their next index shuffling to remove you. Since there’s some evidence that sites with a Page Rank of 5/10 or lower don’t seem to be targeted by most of these spambots, one would think not being in the index at all would protect you. But it will “protect” you from a lot of traffic, too. I get about 30% of mine through Google returns.

Usman asksWhy not use the ‘enter the code you see in this image’ trick that I see all over the web”. In addition to the PHP/GD issue, there’s an accessibility issue. Screen readers will not be able to “pass the test,” nor will users with image loading turned off.

Yes, 98% of the time, it wouldn’t be a problem, but it’s just another speed bump, another escalation that will be matched, and then we’ll be looking for another speed bump. We can address this by continuing to invent hurdles to place in front of the user/bot trying to leave a comment, or in front of the site owner on the back end (manually moderating each comment), until we create a system so complex no one wants to use it anymore.

Or we can see if there’s another way. One not dependent on any individual blog program’s efforts, or those of Google. Because this has become an assault on the resources of web hosts. They can shuffle it off for now by shutting down the problem, site by site, day by day. But like the poor site owner clicking frantically to delete the damn spam, they, too, will reach a point where that “solution” just isn’t viable any longer. It consumes to many man hours, too many resources. It reactive, not proactive.

There’s an effort underway to address this at the server level. To reject these attempts at comment spam before they even hit MT/WP/TxP, just as mail servers drop clearcut e-mail spam on the floor without it ever hitting your inbox. It will likely require Apache 2.0, mod_security, and people a lot smarter than me. Because that’s about all I can tell you about it. But I know who these smart people are, and I’m going to do whatever I can to help them. Because it seems to me the only way. We should continue to hope that Google/Six Apart/whoever will offer individual solutions … but we shouldn’t wait for them. We need a collective server level solution.

35  Elliott Back wrote:

I make clients compute a hash in javascript when they submit a comment. This gets checked with the server, and if it doesn’t match, the comments is thrown out. This cuts down on the serverside problems, as spam is rejected before it gets to major processing.

36  Scott Chaffin wrote:

I have to seriously question how valuable those random Google searches are to you, Reid. I seriously, after seriously looking at them for my site, could care less about them. Hits are nice for chest-puffing, but at the end of the week, they ain’t adding anything to anyone.

Now, I can say this because for the time being, I’ve cured my spam problems. I’m active about it, I’ve taken all the normal documented steps to kill it, and it’s worked. I’m only dealing with two blogs, though, not 20. And I’m still letting Google crawl me, but I don’t see any huge benefit from that, outside of whatever ad revenue I pull in from the schmoes.

I’m telling you, though—if it gets crazy, I’m not sitting on my heels. Their attacks can be turned right back around on them. I’ll order 5 billions tabs of C1@1i5 for the cows one ranch over, and someone, like the Home Depot, will have the biggest boobs in the county. Perhaps I’m just uniquely lucky in knowing my neighbors will shoot first and ask questions later…well, cest la guerre. Cest la guerre.

37  Adam Rice wrote:

I’m an MT user; I long ago adopted the “rename the comments script” approach, and have since layered on Jacques Distler’s “forced preview” approach, with throttling. Not for the faint of heart. But it works—the only comment spam I get now is actually pasted in by inconvenienced humans, about once/month.

Based on this, I can imagine a few remedies for MT:
1. Forced preview and throttling.
2. Randomize the name of the comment and trackback scripts for every install.
3. Design the comment form so that there are 100 buttons that apparently will post the comment (but are hidden through CSS and could never be seen by a human). If anything clicks those buttons, it is blacklisted.
4. Batch-mode rebuilds: if a site does get hammered by spammers somehow, don’t rebuild with every posting, rebuild every 10 seconds at most.

38  Reid wrote:

While I haven’t tried everything you mention, Adam, there is clearly some variance in what works for some, and not for others (and I won’t even address the variance in capability users have in deploying these tactics).

At one MT site that had been afflicted with spam, I changed the name of mt-comments.cgi, closed comments on posts more than 30 days old, and forced previews by removing the post button. That worked for a month or so … and then a spammer dropped over 50 comment spams, despite all those steps.

Oh, and as to Scott’s point about Google, this particular site has a Page Rank of *0*/10. As good as “not listed,” in Google terms. Yet spam is an ongoing problem there.

39  Jacques Distler wrote:

I can reiterate Adam’s experience. Some pretty low-tech steps (I , II , III ) are devastatingly effective against the current generation of comment spambots.

I’ve written up some even more sophisticated techniques , which Adam has adopted, but—even today—they provide very little incremental benefit.

Still, it’s good to stay two or three steps ahead of the spammers. I have some even more challenging anti-spam techniques deployed on my weblog.

I’ll get around to writing them up when I see the slightest hint that the spambots have gotten sophisticated enough to breach my previous lines of defence.

40  Jacques Distler wrote:

“At one MT site that had been afflicted with spam, I changed the name of mt-comments.cgi, closed comments on posts more than 30 days old, and forced previews by removing the post button.”

Removing the POST button does nothing to force previews. It is moderately effective against unsophisticated humans. It is useless against spambots.

41  Vincent wrote:

The way I’ve managed to defeat comment spam for the moment is to use session data when the page loads setting a flag allowing that user to comment so if the spammers bots just keep hitting the post comment URL and there is no session data flag allowing them to comment the script just exits with a notice, this won’t stop spammers totaly but seems to work for the moment.

42  Duncan wrote:

At the end of the day the only way to solve the problem is to ditch MT. After ditching Im getting 50,000 hits to mt-comments.cgi not found which shows how big the problem is. Try WordPress and install Spamkarma, it gives you the power of a blacklist combined with a number of other tools which allow legitimate comments and blocks the spam. You’ll also find that with WordPress you’ll be targeted less and the server load is lighter as well.

43  Mark J wrote:

Vincent, this is another thing like JavaScript or CAPTCHA that is going to be a hurdle for legitimate users. Sessions require cookies, and not everyone has cookies turned on.

Right now, WordPress plus Spam Karma is as good as it gets.

44  James wrote:

Getting to the heart of the Google issue, one solution which really intrigued me when I saw it was Lachlan Hunt’s proposal of a metadata profile which defined a link relationship of ‘unendorsed’ to indicate that the linked page should be given no additional search-engine weight as a result of the link. It’d be ludicrously easy to implement for commenting in blog software, and Google and others probably whouldn’t have a hard time with it either.

45  Kevin wrote:

1. Install Movable Type outside of your cgi-bin folder if your host allows it (Dreamhost does).
2. Rename mt-comments.cgi (make sure to change mt.cfg to point to the right one).
3. Use javascript to create the link to your comments page, and keep the comments separate from the post itself. Spiders won’t follow the links, and your post won’t be linked to the comments, even if the comments do get indexed.

I’ve done two out of three of those (and will do the other today), and get next to no comment spam, and my blog’s had the same url for four years.

If the spambots aren’t sophisticated enough yet, let’s take advantage of that. Move stuff around, keep it out of sight, and they’ll never find it.

46  Nick wrote:

I’m using a similar technique involving sessions and JavaScript. I’ve made one modification though in that users with JS disabled can still submit comments—they just go to my moderation cue instead of hitting the front page. I totally agree with Mark J’s assessment that we are harming legitimate users’ interactions with our sites when we require cookies and JS, but I’m also willing to argue the harm done filling out extra form fields, clicking “preview” before “post,” identifying a CAPCHA, or even wading through thousands of comment spams is worse than a process that should be transparent.

47  Kathy K wrote:

Note to WordPress users:
Try Kitten’s Spaminator and Spamwords. Works wonders.
Here is her dev blog

48  orangeguru wrote:

Thanks for the great article! I have several clients using different blog tools – but mostly MT and WP. MT has the most problems with spam & speed – apart from the usual silence from SixApart regarding addressing the problem in public.

I just wanted to applaud here all those developers who write great tools to tackle the problem.


49  rturner wrote:

I recently gave up on Wordpress (and switched to Textpattern) because it was taking too long to delete 100 comment spams per day. Yeah, I could’ve upgraded and gone for a plugin, but I opted to switch anyway.

Just a comment on how bad the comment spam has gotten….hold on a minute, I’m adjusting my tinfoil cap….okay, reception’s coming in fine…Anyway, I found another reason why Google might not want to get rid of the comment spam.

AdWords. My 3 business sites that have been on the web about a million years used to come up routinely in the top 10. Not any more. Depending on the keyword there are many sites ahead of me now, uh, except it would appear that they’re variations on 2 or 3 sites. I haven’t had the time to track them down and investigate, but suffice it to say, if I was a spammer, the type of stuff they sell would appeal to me.

At any rate, just to keep my head above water in web sales, I had to institute an adword program. I’m just a little guy, and my Adwords bill last month was over $600.00. Imagine what a real business would pay?

50  John C. Aldrich wrote:

It looks like simply adding the offending host to MT-Blacklist is not going to stop the problem. While it may provide a line of defense against malicious comments actually making it to your front page, either thru gaps in the Blacklist plugin or the abscence of filters in the master blacklist, the only truely viable option would be to block the traffic directly at .htaccess.

Drastic solution, but would you rather have your site summarily taken down, or would you rather risk alienating a couple of legitimate users? I think that call’s a no brainer…

51  Kehaar wrote:

My newspaper uses MT for blogs and we had a problem with comment spam like everyone else. We implemented MT-scode to prevent people from commenting unless they typed in a graphical code. It’s totally eliminated spam. It’s available here:

52  John C. Aldrich wrote:

The only real system I can think of beyond using MT-scode and blocking at .htaccess is setting up some kind of mod re-write rule to scan for incoming domains that match the list and then just make the incoming think that the server went away. that could atleast get it down to one hit per day…or hour.

53  Reid wrote:

The best .htaccess option is unfortunately not available at all web hosts. But you can very effectively use mod_security and POST_PAYLOAD if your host is running Apache 2.0 with that module. This is the direction TextDrive is heading (along with a few other server level directives), and it should be effective for a range of blog apps, not just MT.

54  John C. Aldrich wrote:

Adam Kalsey and I were discussing mod_security earlier today, and your right. That would seem like the most viable option. Second only to packet filtering at the networks inbound access point, though even that would be a hit on network resources. I’d imagine that if you could reliably determine that a given address is a known source of spam, or look for certain markers in the transmission itself, you could dynamicly block/allow hosts/ips based on that information so that they can’t launch spam attacks on specific blogs on the network. Just a theory.

55  Reid wrote:

As the links to this article have accumulated, I’ve seen more and more evidence how widespread this problem is, and the impact its having. Here’s just a few I’ve noticed:

Reid News “Due to a huge influx of comment spam, I have disabled comments on tnir. This affects all blogs hosted on tnir…”

Ned Edwards (permalinks aren’t working right now, but it’s the Dec. 15 entry) “I had some trouble today – seems there was a Comment Spam attack, which caused my installation of MT and MTBLacklist to crash the hosting server. So, my domains were suspended.”

Jason Toney “The load was such a burden on my site host that they shut down my site”

And this “Several MovableType installations on Employees.Org were attacked with Content Spam, which brought the machine to its knees. If you install MovableType on your user account, please delete the mt-comments.cgi script until further notice”

56  Reid wrote:

And, hot off the press, a post from Anil on the Six Apart site, with some developments worth reading.

57  Barry wrote:

Great post. I finally decided to eliminate all commenting today after experiencing my biggest spam attack to date. It’s really getting bad. For a long time MT blacklist combined with vigilance was adequate, but the problem is growing exponentially. I tried addressing the problem by turning my comment submission form into a service agreement wherein each spam comment was an ad, with an advertising fee of $10,000, but I didn’t have the time to pursue the matter when the spammers accepted my agreement. It’s an interesting idea, however. If you make the submission of each comment a legally binding agreement, how difficult would it be to pursue the spammers for payment?

58  elise wrote:

Thank you Reid for this thoughtful post.

Regarding renaming the scripts…

Renaming the trackback and comment cgi scripts are no longer effective because the spammers have found a way to scrape off the new names of the scripts from the entry page.

I changed my trackback cgi name and also changed the words leading up to the trackback URL on my entry page and that seemed to stop the flood of trackback spam pings.

Don’t know how to do the same with the comment cgi as the name of that script is in the comment form that appears on the entry page. I’m somewhat clueless when it comes to most HTML. Can I change the order of the form elements and have the form still work? If so, doing that would help keep the bots from finding the new name of the comment script.

Regarding moderation and TypeKey authentication…

I’m getting about the same amount of comments on all of my blogs as I did before I required approval on every non-TypeKey authenticated comment. Many (most) of my visitors on the blogs other than Learning Movable Type are not bloggers. It is perfectly reasonable to the average person to expect that the website owner would want to approve a comment before it posted. This is the way it is on commercial sites (e.g. Epicurious) that accept comments. In fact, you usually have to register.

The problem is that those of us who are used to using comments as conversation get aggravated by having to wait to see our comment display. It is as if a freedom is being taken away from us. I for one am more than willing to trade real-time comment posting for keeping comment and trackback spam completely off my site.

To all…

I’m trying to keep track of some methods MT users can employ to fight blog spam here:

If there are other ideas you might have that you think would be useful to MT users, please let me know at elise at elise dot com. Thanks!

59  John C. Aldrich wrote:

Quoting from Elise Bauer…

The problem is that those of us who are used to using comments as conversation get aggravated by having to wait to see our comment display.

Exactly. It really burns me that we even have to moderate comments. Granted Blacklist does an adequate job of automaticly screening out most of the cruft, but ideally there shouldn’t even be a need for Blacklist or Typepad. I know full well that we don’t live in an ideal world, so it’s a pipe dream at best.

With these bugs in Movable Type it only escalates the problem. MT appears to be generating twice the server load as one would normally get in a spam attack. I’m not placing blame by any means on SixApart, as it’s an honest oversight. I’ll be waiting for the patch to fix this bug. For the meantime, I’ll more than likely continue to tightly moderate comments on my own site until the spam issue settles down a little bit and the bugs are corrected.

60  Camilo wrote:

I agree with you 100%, and the subtle kick about a Blogger Caste in extremes of richness is quite apposite.
Perhaps an Open Source approach to blogging could actually solve the spam problem, perhaps not. I remember the answer from Mark Pilgrim to Jay Allen, when Mark grimly foresaw a scenario very similar to the one we have now.

Anyway, just to tell you, photodude, that you gave me a hell of an idea for a story. Details to follow.

61  John C. Aldrich wrote:


The problem is that MT has an open codebase but is not open source. The distinction being is that we as users can not go in and modify the source ourselves (well, if you want to get technical we can, but it violates the software agreement if I remember correctly, and needless to say that the source is modified to some extent by plugins. We just can’t modify the underlying API). I’d love nothing more than to go in and fix the bugs myself, but for the sake of maintaining a functional copy of MT and upholding the terms of the software agreement, I haven’t.

62  Brad Choate wrote:


I don’t know how you got that impression. MT has always been open to make custom modifications, as long as they are not redistributed. The license states:

“Although you may modify or alter the Software for your own use (including copies that extend, or enhance the Software), you may not distribute, transfer, or resell the modified or derivative copies of the Software”

So, making your own changes will not violate your agreement, since the agreement grants you the right to do just that.

And you’re also welcome to submit any bug fixes you make back to us! I promise you that avenue of distribution won’t violate your license agreement either.

63  dbs wrote:

I have to take exception to a lot of the comments in your posting. You’ve miscategorized a lot of information, and I also think you’re running on a poor set of assumptions.

First of all, you’re running on MT 2.6. That version is over a year old, and has many inherent problems. Complaining about issues with the system when you’re not running the most recent version, and show no interest in doing so, really underlines that you’re less interested in finding a solution, and more interested in just complaining.

Any good sysadmin knows there are ways to fix and manage any application. In your case, the application is MovableType. 30 seconds of googling will reveal fixes you as a simple administrator can do to make the software more stable.

On the issue of comment-spam, the number one thing to do is to rename the mt-comments.cgi script to something slightly obfuscated (mt-script-that-does-comments.cgi or whatever), edit mt.cfg to point to the new script, and rebuild. That’s it. When we were being hammered on our host (we host 4-5 blogs), doing this ONE change cut us from 2000 spams a day to 1-2.

This will necessitate uprading to MT3, which you should have done long ago.

Comment by dbs · 12/17/2004 05:23 AM
64  Reid wrote:

Complaining about issues with the system when you’re not running the most recent version, and show no interest in doing so, really underlines that you’re less interested in finding a solution, and more interested in just complaining.

I was not complaining that my personal install was being spammed. In fact, the only thing I said about my install on this site is “it’s never gotten spammed.”

My complaint was that the server my site is on has been taken down by assaults on other MT installs. I also point to a half dozen web hosts who were shutting down users. Users who were running MT 3.x and MT-Blacklist.

So you must not have read very closely if you think the source of this article is my personal problems with spam on an old install of MT. I haven’t had any.

Follow a few of the links in the blockquoted text in the article, or in this comment and go tell those people their problem is the version of MT they’re running. Go tell the web hosts who’ve shut them down.

Any good sysadmin knows there are ways to fix and manage any application. In your case, the application is MovableType. 30 seconds of googling will reveal fixes you as a simple administrator can do to make the software more stable.

Funny, Six Apart today reported to bugs in the latest versions of their apps that they are working hard to fix/test right now. Bugs that are beyond the control of any sysadmin, and which they admit have caused extreme server loads during spam attacks. It’s all linked here

Maybe you can go tell them they’re running the wrong version, too.

On the issue of comment-spam, the number one thing to do is to rename the mt-comments.cgi script to something slightly obfuscated (mt-script-that-does-comments.cgi or whatever), edit mt.cfg to point to the new script, and rebuild. That’s it.

As has been pointed out repeatedly, even by Six Apart, that is only a partial solution, as these spambots are following pings from, grabbing the POST method (with the new name of mt-comments) and article ID number, and then dropping up to a hundred spams in no time. Yes, there are still a few dumb bots who look for the default name in the default location. But there are much smarter ones, too.

The site I helped resurrect at Pair (whose sysadmins certainly know what they are doing) had mt-comments renamed. It got hit so hard Pair shut it down.

This will necessitate uprading to MT3, which you should have done long ago.

MT 2.6 still serves my needs perfectly, and as I’ve said, spam is not an issue for me. The comment thread you are posting in right now is generated by Textpattern.

I’m not running the latest version of it, either.

My concern is with the larger community, and the impact this problem is having on it. You may not see it, but I can point you to dozens who are. And now Six Apart is working hard on what they acknowledge is a big problem for their users.

So I’m having a hard time seeing what you’re bothered by.

65  Isaac wrote:

Here’s my two cents. Switch to WordPress. Use Spam Karma AND TrencaSpammers plugins

TrencaSpammers is a
“Security Graphic Generator for Comments

* Name: TrencaSpammers
* Version: 1.0
* Description: This plugin ads a graphic generated number which you need to read and then write on a new field of the post to be able to post. This way only humans will be able to comment a post.

# Author: Gabriel Ortega
# Plugin Site:
# Author Site:
# Compatibility: WP 1.2 & WP 1.3
# License: n/a
# Download URL: ”

Comment by Isaac · 12/17/2004 06:40 PM
66  Jay Allen wrote:

“Here’s my two cents. Switch to WordPress.”

Yes, because, as we all know, WordPress with those plugins is invulnerable to spam… :-)

67  Reid wrote:

Now, boys.

One thing I hope this thread has made clear is that comment spam isn’t an app-specific problem. If it was, why would WP have all those plug-ins? Obviously, the target is not the app, the target is an open comment form that might bring a smidgen of Page Rank. Other than that goal, spammers don’t care about your app wars. You might as well respond “get a Mac.”

We’re not going to make progress against these bastards by arguing among each other about our particular blogware. There’s got to be app level solutions, server level solutions, and user level solutions.

Because we’re all on the same servers. Your app can take down mine, so fixing mine doesn’t solve the problem. I don’t get comment spam, but I’m not alone on an island, I’m on a shared server like you.

68  adrienne travis wrote:

Not wanting to contribute to the blogware argument; however, ExpressionEngine has some nice features when it comes to catching comment spam.

This page
has all the spam-blocking methods that can be used in EE, by default and without any hassle on the site creator’s part. Below are the highlights:

There’s a built-in way to verify that the comment comes from an actual form submission, which was discussed above as a homegrown solution some people have created for MT.

You can also set an interval for allowing comments from the same IP, and/or filter out comments with duplicate data.

There’s also a blacklist/whitelist feature that can be updated from a central list kept by pMachine/ExpressionEngine, which makes it a great help at hitting the moving target of comment spam. Any registered EE user can submit IPs or domain names to the blacklist, and those can then be distributed to ANY EE user who wants them.

Comment by adrienne travis · 12/17/2004 10:42 PM
69  John C. Aldrich wrote:

_“On the issue of comment-spam, the number one thing to do is to rename the mt-comments.cgi script to something slightly obfuscated (mt-script-that-does-comments.cgi or whatever), edit mt.cfg to point to the new script, and rebuild. That’s it. When we were being hammered on our host (we host 4-5 blogs), doing this ONE change cut us from 2000 spams a day to 1-2.”_

Expound upon that and set it up as a plugin that kicks in on page rebuild ( or per each comment ) and also edits the mt.cfg file to reflect the change to the comment script name. that should eliminate it, while allowing Blacklist to catch anything that this doesn’t. Couple lines of defense. :-)

70  Ariel wrote:

Renaming mt-comments.cgi is a short-term fix. Bots will find the renamed file and start hammering it. For me, it took only 2 days for spam bots to start hammering the renamed file.

71  Collin wrote:

I’ve completely eliminated my spam problem by using this trick from a friend’s blog.

While I’m sure it won’t protect against all spambots, it certainly helps without causing much server processing at all.

72  John C. Aldrich wrote:

“Renaming mt-comments.cgi is a short-term fix. Bots will find the renamed file and start hammering it. For me, it took only 2 days for spam bots to start hammering the renamed file.”

Then my only other suggestion would be to call it thru script in a method similar to what Collin is showing, OR, citing from a conversation I had with Brad Choate, define a mechanism where communication is TWO-WAY. That way, a response is required instead of the fire and forget system we have in place now.

73  Alex wrote:

I’ve posted some analysis of long-term spam prevention methods here.

My argument is that authentication is doomed to fail as a comment spam prevention method, regardless of whether it is centralized or distributed.

74  Reid wrote:

And for anyone arriving late to this hub-bub … Six Apart has released an important update, MT 3.14 and they recommend all MT users upgrade as soon as possible.

Comments are closed for this article
Contact me to find out more