WordPress surfing led me to another interesting sob story from a penalized webmaster and my reply got so long it deserved to become a post:

Marshall Sponder wrote:

Take Know More Media’s case – you have 100+ blogs and 2+ years of content – that’s easy, 30,000 to 50,000 blog posts and Google, with just one or two paid links that pass PageRank, is going to throw the entire blog network out of it’s index over that?

Yep, it appears that’s it – that’s the reason.  But is it fair?  No.
Strictly from a users point of view I think it is very hard to justify technical penalties on good content.    Few users know or care what “hidden text” is, so if a mom and pop webmaster uses this tactic and Google deletes the otherwise informative, relevant website it is hard to argue that users are served well.    Even if a black hat SEO created a site filled with illegal tricks but also full of highly relevant quality content I think Google’s case against including that site is weak.  As a user I want  *quality content* and I don’t care about the site’s technical construction.    Where Google is simply banning sites for using spammy tactics I’d agree with Marshall that to be faithful to user centricism they really have to take it a step further and look at the content they are excluding.   Even if the content contains paid linking and other violations if it unique, quality content Google cannot exclude it without violating their stated “prime directive” of providing the best for the users.

However, Google has to manage about one trillion URLs, so obviously they need shortcuts in ranking and one of them is a page from AZ Senator Barry Goldwater’s playbook when – many years ago – he tried to justify an escalation of the Vietnam war, perhaps to nuclear level.   Google’s coin of the famous Goldwater phrase would be: “Extremism in the defense of the algorithm is no vice”.

I don’t think penalties are generally *fair* or *user friendly*, but I’m willing to concede they may be necessary for Google to function as profitably as they do since it would take a lot of human intervention to help every mom and pop determine what’s wrong with their sites.

However, I feel Google continues to fail in their obligation to communicate more effectively with penalized sites although I think they are s-l-o-w-l-y  catching on to the fact that most webmasters of penalized sites remain unclear as to why the site has been penalized or downranked.    Removal offers you a shot at “reinclusion” and (very rarely) possible webmaster tools staff feedback.   Downranking is algorithmic and Google will not generally offer any advice to help downranked sites.     In this case you generally want to re-read the webmaster guidelines and experiment with different approaches in an effort to boost rankings.

My view is that as many thin content database sites have flowed online Google is holding online material to a higher standard of quality, especially if it’s at a new website.    This helps explain why you can find well ranked pages that are inferior to pages at a new website.

There is a solution to all of this in my opinion, which is for Google to include a lot more community input and feedback into the process than they currently appear to do.    I’d guess the recent discussions to aquire DIGG may have been in part to gain more community feedback tools and data.     Historically Google has been brilliant at using algorithms to determine ranking and advertising, but has fallen short of brilliance in their ruthlessness in dealing with website practices they don’t like, leaving a lot of collateral damage – especially related to sites involved in “paid linking” and variations on that complex theme.

At SES San Jose 2009 I’ll hope to get to ask Matt Cutts more about this in person.   Matt is Google’s top spam cop and always very open to conversations about ranking and search.    In fact the best event of the conference is the Google Party where engineers are on hand to discuss search related issues – including complex ranking technicalities that are sometimes brought to Google’s attention as part of the search conference circuit.