Google economist on Google’s success: Huh?


Hal Varian is an economist at Google, and I’m sure he’s a good one.   However his Freakonomics and Google blog analysis of why Google has done so well in search leaves a lot to be desired.    After knocking down a few straw man items that obviously have nothing to do with Google’s search   monopoly   dominance, he goes on to conclude that Google is just better than the competition because they have been doing search for so long.

Hal – Excuse me but you call that economics?    I doubt this would be your internal Google explanation (assuming you want to keep your economics job, let alone your degree).  In fact it was so thin and almost bogusly “cheerleading” that it raises for me the ongoing questions about Google’s questionable mantras about doing no evil and transparency:   Transparency in all things except those that might affect our bottom line!

As I’ve noted ad nauseum I do NOT think Google has more than a modest obligation to be more transparent, but I’m tired of how often Google *witholds information* to protect Google and then pretends this is in the interest of users.  Google screws users and webmasters regularly – this is common knowledge in the search community.   The most glaring challenge is with ranking errors, mistakes, penalties, and rules.   In this area literally tens of thousands of mom and pop websites, and sometimes larger enterprises, are indexed in questionable ways by Google leading to serious economic challenges.   Unlike almost any other business however Google has only a tiny team of specialists who generally can only offer vague and often useless canned information, even when the problems are fairly obvious to an experienced search person.   

But I digress into ranting….!  

My working hypothesis about Google’s success is simple and I think would hold up far better than Hal’s silliness:  Humans are creatures of habit, and Google was the best search at the time when most formed their internet search habits.   Yahoo, LIVE, and even Ask are only marginally inferior to Google search now, but there were dramatically inferior a few years ago when the online ranks swelled with people looking for information.   Google provided (and still provides) high quality, fast, simple results. 

This hypothesis helps explain the following facts:
Google is not the search of China where Google.cn traffic is dwarfed by Baidu.com
Even as Yahoo improved search quality they did not improve their search market share. 
Quality differences are slight, yet Google search share in USA is very large.
 

Another indirect factor in the Google success equation is that Google’s monetization remains superior to the competition by a factor of more than 2  (per Mike Arrington .09 vs .04 per search at Yahoo).   In this monetizing sense Hal’s “we are better from experience” would ring very true, and if he had written about *economics* he would have noted that Google’s brilliancies in monetization are a lot more notable than in other areas, and are more of a key focus area at Google than is generally talked about.    In fact such a focus area that they are downright opportunisic in the effort to monetize the heck out of the searches.  My favorite examples are when Google violates their own guidelines to bring users …. non-information from advertisers.   I ran into this last week with the following search for airline tickets.   

Google Query: “Xiamen to Beijing”

The top result on the left side, which is supposed to be reserved for non-commercial results, at first seems helpful, giving you the ability to order tickets from several places:

Flights from Xiamen, China to Beijing, China

Departing:   Returning: 

CheapTicketsExpediaHotwireOrbitzPricelineTravelocity

Unfortunately though, you can’t order the tickets because at least some of those clicks lead to commercial websites that do not offer that route.  

No big deal?  I guess not, but this is a clear violation of the Google Guidelines which call for clicks to a page where you can really get the thing advertised.  Also it would be refreshing for me if Google stepped down at least half way from the high horse of claiming they never put money ahead of users, and more importantly used some of the enormous profits to bring more transparency and helpful information into the mix.

In summary I want to be clear:  Google has the right to make big money online.   They also have the right to be very aggressive in making money.   However with their success goes an obligation for quality communication and transparency.   They are failing in that obligation and perhaps as importantly are not even recognizing that they are failing.   Google is a great company.  But they can do much better by users whose habits have made Google the most successful company of this generation.

Yahoo Tech Ticker – Arrington on Yahoo


Yahoo’s got an interview with Mike Arrington who provides some excellent and concise commentary on Yahoo’s demise, including at the end of the interview his prediction that the interviewer will be …. out of a job soon.

I think the *key* point Mike makes is that where Google gets about .09 per search on average, Yahoo gets about .04.   He notes this is partly demographic and partly due to Yahoo’s search monetizing deficiencies that were supposed to be rescued by project “Panama” but were not.  I think this is per search query rather than per click on a search ad, but the point is the same – Google makes more than twice as much per search action, and this is a crushing advantage to have over Yahoo.

So, what is the endgame for Yahoo in the Microsoft deal?   We should know soon if the rumors of a News Corp deal are well founded or hyped up.   Some are suggesting that it’ll be very hard to fend off Microsoft in any case as they are likely to bid $35 per share soon which will about equal the rumored News Corp deal of about a 50 billion valuation for Yahoo.  

Disclosureizing:  Long on Yahoo

Death by Google


My Airport Codes Website, AirportCityCodes.com , was completely removed from the Google index last month.   Not at all clear why and I’m hoping it’s just a a fluke.    The site was very stable and although it was somewhat uninspired it offered airport code and other information on about 9000 airports.     Google traffic has become so critical to a website’s success that without Google a site is generally almost “dead” in terms of traffic and revenues.

The site had enough sloppy construction and odd duplication across directories – problems that I had simply left intact after taking it over several years ago – that there could be hundreds of reasons the index didn’t like the site, but usually Google reserves a complete deletion like this for a major transgression against Google guidelines.    

I’ve posted questions over at the Google forum and the answers should be interesting.  

Your kids are not so smart after all


A prevailing assumption of the past several years holds that young internet users are very computer and search savvy, but  recent study of children and internet in the UK suggests otherwise.    ARS Technica Reports

The study of young searchers found they preferred visual over text information, liked to cut and paste, and tended to do simple searches.

My own observations of how kids use computers lead me to think this study is correctly characterizing use and also offers some profound insights into the future of computing – a future that is *less*, not more intellectually sophisticated in terms of how people interact with the internet and with each other.

Several forces are conspiring to make the average internet user “dumber” than in the past:

* Entertainment usually trumps education, and as entertainment value of online environments becomes increasingly compelling kids (and adults) will increasingly spend time “playing” rather than “learning”.   

* Short attention spans now rule everywhere, and this trend is unlikely to see any reversal.    We are replacing contemplation and reflection about experiences with … more experiences.

* Human nature.   We are not designed for personal enlightenment and long term planning – rather for short term gain and satisfaction which until very recently was a better survival strategy.

So, strap in and watch out, because things are going to get a lot …. dumber!

Search Ranking Factors


Rand Fishkin’s SEOMOZ has been doing some of the best work collecting data from prominent SEO folks and groups of experts and then analyzing that data.     Back in April I missed this report about SEO ranking factors but it’s a great read, especially for those who have little idea about how to optimize a website and web pages for better placement in search engines.    Note that experts do not agree.    Also, my fairly extensive experiences have convinced me that Google changes the ranking rules regularly simply to make it impossible to reverse engineer them.   But it’s still important to follow these basic recommendations which include what I’d argue are now the “prime directives” for optimizing websites:

Create pages that are of high and unique content quality.

Use URLs and Titles that are highly relevant to the queries you wish to rank for.

In bound links are still very important – seek external links and create internal incoming links using your desired keywords as anchor text.

Tend to exaggerate the keywords you are targeting.   ie the best writing will NOT result in the best optimization due to defects in the way machines process word information.     

Why Blogs are better than Google.


Today, as I searched for some breaking news and technology insights, I was struck by how much better informed you tend to be after reading a few blogs targeted to a topic (and following related links and sites and ask questions) than when you simply search Google (or Yahoo or MSN or, if you enjoyed the silly and short lived TV campaign, ASK).    

Don’t get me wrong – I like the search engines and I love the way you can quickly winnow through billions of pages down to the handful that are relevant and good for your topic.     But I’m noticing how increasingly I wind up turning to blogs *first* for the best news, links, and insight.    I’m beginning to understand why I’m doing that, and why it’s a big deal.

There are the obvious advantages to blogs over websites.   They are fresher (ie recent and new content) – especially compared to Google searches that often yield so much old content.   They usually offer some community components so you feel like you are “where the action is” on topics.   This is usually true for major blogs.  TechCrunch is a key watering hole for startups, HuffingtonPost.com for liberal political folks, etc.

However these advantages are secondary to the fact that as blogs mature they offer an excellent “human powered search engine” for your niche of interest, and as we all know humans still beat out computers in terms of understanding what information is most relevant to our inquiry when it is a broad field of interest.

Again, the TechCrunch Technology blog is a great example of this.  A search in Google for “startups” or “technology news” or “venture capital” will give some good results, but even a careful study of those results won’t give you nearly the insight you’ll get from a one hour session at TechCrunch.     Even a Silicon Valley startup new arrival – or distant silicon startup wannabe, could sound like a veteran if they simply kept up with the parade of posts from Mike Arrington and his clever crowd at TechCrunch.

I think this blog advantage breaks down as you move into very specific topics, but it’s going way up as an advantage in the study of general topics as blogs explode and gather traction and community.     Of course there are caveats to this.   Learning in any form takes time, and you would never simply stuble into a blog about a topic without checking other blogs and sites related to that.   But my point is that once you find “the key blogs” about a topic, even if it is a contentious one, you’ll find through those blogs links, references, breaking news, and a community of other interested parties.   This complex, interactive, cross referenced community experience is how humans learn best, and the internet is making that type of learning exponentially easier to obtain.     

TechMeme Secrets


TechMeme has rapidly become one of the key techno watering holes in the blogosphere thanks to how it helps sift through tech blogs and posts to winnow out those getting maximum buzz.     Todd recently suggested he thinks a lot of SEO firms  are gaming techmeme, especially by post plants from A list bloggers – the implication is that they are paid for this.

He’s largely wrong about this and has given *way* too much credence to the always inflated claims of SEO companies (his inspiration for the post was a small SEO firm in Honolulu).

Of course Tech blogging, and most of the web for that matter, and much of the offline world, have been “damaged” with respect to objective quality content by various tactics that come about as the inevitable result of content monetizing.

But take a look at the prominent TechMeme posts tonight – it’s clear that these are generally spawned from sincere interests and not “planted” as part of advanced SEO tactics. Do any plants happen? A few, but in SEO you have to balance the chance you’ll “sneak in” a good plant against the greater chance that you’ll permanently tarnish the blogger’s reputation cause a scandal (Wal Mart’s Edelman fiasco), or simply spend a lot of time and money for a marginal result. The best SEO strategies rely more than ever on getting legitimate content and placements.

Now, Robert Scoble has a great video post today that is a lot more interesting because he’s trying to reverse engineer TechMeme, something a lot of people in tech are interested in for several reasons.   Robert also manages to feed the new little Scoble during his impromptu 2am advanced tech blogging lecture, which is really a fun statement about how far social networking and life/work integration has come in the past few years.

I hope Gabe responds to Robert to clear up some confusion though he may want to keep the TechMeme algorithm top secret, following in Google’s footsteps.

Some key points by Robert as he speculated about the TechMeme algorithm:
Tech blog database of perhaps 10,000 blogs.
Blog rankings (see TechMeme leaderboard) used to reflect their authority and thus “weight” the power of outbound links from  those blogs.
Reciprocal linking is not as heavily weighted as one directional outbound linking.

Robert suggests an experiment to test some of his ideas and I hope he does it, though Gabe may simply shut down that post or (if he wants to mess with his TechMeme folks) manually override the algorithm so it does funny things that lead to wrong conclusions.  Scoble’s Breeeport experiment was fun a few years ago, and this stuff can be a great way to bring more transparency to the mysteries of content ranking.

China redirects searches to Baidu? OR NOT!


TechCrunch is reporting today that China is redirecting internet searches from Google, Yahoo, and MSN and I assume all other engines – to Chinese search engine Baidu.   However I can’t find anything but little anecdotal posts to support this.   Looks to me like some videos and blogs have been affected, but that the big search engine issues may have related to a temporarily problem or testing of DNS stuff.

They suggest this may relate to the recent award given to the Dalai Lama I’d guess China is spending a lot of time thinking and experimenting with ways to maximize their search revenues, and this redirection, if it really did happen as dramatically as some suggest, would probably be testing ways to gather data on how well Baidu monetizes search compared to the agreements they have with other players.

Wait – here’s a blogger in Beijing, China saying that he’s getting to places TechCrunch says have been sent to Baidu, like Google.   Not sure what’s up …

Is this a false alarm?   I think so, though it might be another example of how China’s centralized socialist economy can create power and monopoly conditions the most ruthless old style US capitalists could only dream about.    Increasingly control of the online landscape is control of the business landscape, and as China’s massive economic expansion continues it will be very interesting to see how the China wields her power.

Note – I just edited this post quite a bit thanks to the new info.  Still dunno what’s going on.