UPDATE BELOW: How Google Is Killing An Arts Site (A Twist On "Too Big To Fail")
On the web, Google gives the words "too big to fail" new meaning. The company's dominance in directing the flow of information and people around the web is so big that if Google decides to block information, then there's not much you can do about it. And once you get caught up in its gears, it's difficult to extricate yourself without getting ground up in the process.
I run a small news site called ArtsJournal. Every day we look at a thousand or so arts stories from all over the world and aggregate the best of them into a daily digest of arts and culture news. I started the site in 1999, and was one of the first wave of aggregator sites of this sort. In those days I had to hand-code all the pages and paste each story into the html. No content management systems then.
Over the past 10 years we've become the leading digest of arts news, and cultural leaders and arts journalists all over the world use us to keep up with cultural news. We're also home to almost 60 arts bloggers, including some of the leading arts journalists on the web. And from time to time we host conversations about issues of the day. We don't get Andrew Sullivan numbers or Boing Boing numbers or Gizmodo or Mashable or Gawker numbers, but we reach a very specific niche. What Romenesko does for journalism news, we do for arts news.
Monday night about 10:30 I had just sent out our daily newsletters when I got an email from a professor at the University of Oregon that he was getting blocked trying to get to ArtsJournal and that a notice had come up saying ArtsJournal was a "Reported Attack Page." The notice is red and scary-looking. We had been hacked.Now, I'm a journalist, not a techie, but I have picked up enough over the years doing ArtsJournal that I can usually figure out the technical side. I followed the directions Google pointed to for how to scrub a site of malware. I made sure the software powering the site was latest-issue, and went through all the pages Google had flagged as being infected. When I couldn't see any of the code they referenced, I went back to the Google Webmaster page and submitted the site for review. A few hours later Google reported that we were still infected.
I redoubled my efforts and discovered that
I removed all the malware scripts, then disabled the ad server, then shut down access to the ad server altogether. I went to every blog and took the ad call scripts out. Okay, way too much detail here, I realize, but I wanted to give at least a bit of a sense of the methodical work I went through to get rid of the malware.
After I was sure everything was fixed about 3 PM on Tuesday, I tried to resubmit ArtsJournal for Google review, but was told on the site that I had to wait "several hours" before resubmitting. Okay -- I get that there has to be a limit, otherwise people would be resubmitting over and over again. So I kept checking back every hour, and it wasn't until 3 AM Wednesday that Google would allow resubmission. Great, I thought. I resubmitted and thought we'd be back up in a few hours.All day the Google page said the review was "being processed" or "pending." The Google report page on AJ said that malware hadn't been detected since Tuesday morning. Review still "pending." By mid-afternoon, the site still wasn't unblocked and I went to the review page and asked what was up. By late evening, still "pending." This morning 3:30 AM still pending. All morning Thursday, still "pending" or "being processed."
But A Larger Issue?
Maybe this is a good place to say how amazing I think Google is (no, I'm not trying to suck up, as you'll see in a moment). The company has transformed my life. I use Gmail to manage my email. Google Calendar to manage my appointments. Google Reader has been a godsend in helping to collect stories for ArtsJournal. Google Analytics helps me keep track of how ArtsJournal works. Google Voice helps manage my calls. And of course the various Google Search services feed my seemingly endless need for information. And did I mention YouTube? Or the indispensible Google Toolbar? Or Google Sync? In short, Google has become nearly impossible to do my work without. I spend more time using Google on the web every day than anything else.
And therein may lie the problem. "Too big to fail" when you're talking about banks means a bank so big that if it fails it might bring down the whole financial system. In the information business, "too big to fail" means that our free access to information is too dependent on the actions (benign or not) of a single company. You want to talk media consolidation? Google has become all media consolidated. I don't in any way think Google is singling me out. I don't think Google is being malicious. Indeed, I'm glad Google blocks malware (bastards!) to help contain it and protect others.
And why shouldn't Google try to be as big and powerful as it can be?
But should any one company have this much ultimate control over what we have access to? In my case (and probably for many others), the Google system for getting unblocked from the rest of the web sucks. Google offers low information about your status once you've been blocked and no information after you've done what they've suggested. For a news site that depends on constant updating, three days being blocked from most of the web is devastating.
That most of us are now tied into Google's ability to sort and deliver information is testament to the amazing things Google has been able to create. That most of us are now dependent on Google for our access to information is maybe not such a good thing. Has any one company ever had such control over the world's information? (And I haven't even mentioned Google's book project.)
Yes, you can see ArtsJournal on Internet Explorer. And a third of our visitors use IE, so they can still see the site as we continue to update it. But do a search for ArtsJournal and you get a nasty warning. And many of our regular visitors have been scared off by the big ol' nasty red "Attack Site" warning.
It's 10:47 AM PDT (Day 3) and I continue to wait.
UPDATE: As of 6:56 AM Friday morning, Google seems to have lifted the block. (the Google Review page still says review of the site is "pending"). So no official word, but we appear to be back.