SES San Jose “Meet the Crawlers” Session



SES San Jose “Meet the Crawlers” Session

Originally uploaded by JoeDuck.
Though poorly attended the SES San Jose “Meet the Crawlers” section offered some give and take with key search engineers from MSN, Google, Yahoo, and ASK. All engines seemed to “agree” on these points:

Avoid paid linking
Use 301 redirection for site moves.  Generally this will pass authority (though IMHO you should avoid 301 moves if possible due to losing rank).

Make sure robots.txt file is in order and is allowing all bots  you want “in”.
In another session it was said that ASK downranks sites without a robots.txt.

L to R in picture: MSN’s Eytan Seidman, Yahoo’s Sean Sucher, Google’s Evan —, and Ask’s Peter? Lindsey (I’ll correct these names later with my notes)

Mary is Meeker than yesterday on revenue estimates?


I’m still digesting this amazing story by Henry Blodgett  about how Morgan Stanley analyst Mary Meeker’s seems to have 1) inadvertently miscalculated YouTube revenue potentials by a factor of *1000* and then,  adding insult to injured fuzzy math, seems to have reworked the calculation to “back into” a new number that is closer to the original than the one you’d get from the original assumptions.

I need to read her side of this but Blodget is no stranger to the perils of fuzzy math and I remain amazed at how few in the media question how the big players estimate this stuff.  This certainly indicates that for many years big players have used bogus valuations, fueled by the casino-like buying behavior of clients.     Without more critical review this will keep on trucking for some time.

SES San Jose – links built the web – are SE’s saying links are now ruining the web?


Search Engine Strategies 2007 has wrapped up and I think the hottest topic this year was linking and how links are treated by search engines.

The irony of the link debate is that regardless of your view about *paid linking*, we now see that the paid linking abuses combined with aggressive anti-paid linking policies of the search engines have distorted how *unpaid linking* works, and this is nothing short of tragic because links are the key to the web.

I’m very concerned about how Google’s obsession with paid linking and other linking schemes has dramatically changed – and often poisoned the waters – around legitimate unpaid linking of the type done in the early web. I don’t have a solution other than much greater guidance from Google about what they see as legitimate linking patterns. NOFOLLOW rings hollow to me as anything approaching a solution here.

Hyperlinking was the brilliancy that launched the web. Tim Berners-Lee is sometimes credited with this concept of the hyperlink as the foundation of internet relevance.

Enter Google’s Larry Page who developed pagerank as a measure of a sites general level of community interest. This was a spectacular insight, based on the notion of academic citations. Page realized that ranking sites partly with a measure of the web’s own linking patterns was a great way to enhance the relevancy for a query. In a sense this was a global Web 1.0 social network where user interactions were measured and factored into the ranking mix. In this early web world links flowed fairly freely and without monetary considerations. Links were a vote for other sites or a favor to friends with new websites or just a way to play around with HTML.

Enter the power of the internet as a medium for commerce, leading to loads of cash, from many players who wanted to *rank high at Google* so they could sell more stuff or simply set up sites that would rank high so that highly ranked site could sell ads or use affiliations with others who were selling stuff. This led to an explosion in paid linking, off topic linking, massive reciprocal linking, and other link schemes and scams designed to raise ranking for sites using non-natural linking that would trick the Google algorithm into thinking the site was really more popular (ie more linked to), than otherwise.

Enter Google’s “NO PAID LINKS!” policies and aggressive crackdown on the practice of buying and selling links. This takes many forms including site penalties, “no pass pagerank” penalties, and Google’s recommendation that the “nofollow” tag be applied to any link that is paid as well as many others such as ‘self referencing’ links from comments at blogs.  Blog comment NOFOLLOW is a good example of how Google policies may be distorting the logical growth of the web.  If somebody leaves an intelligent and extended comment at a blog with a link back to their site they have created legitimate web content and the linking structure of that content should be incorporated into the web linking patterns.  As a user *I want* people who actively engage in blog comments to rise to the top and I want them to reference their own blogs!  Also, by simply making many forms of blog commenting irrelevant to ranking I think we’ve seen  *diminished* tendency for people to comment because as a blogger they want to rank and they know this won’t help them.  NOFOLLOW at Wikipedia is another great example of a problem, since in many cases a WIKIPEDIA link is an *excellent* quality signal that is destroyed with NOFOLLOW.

Most significant is the fact that most onliners now understand how valuable links are in a commercial sense and therefore are resistant to linking for this reason. This is the real tragedy. There are exceptions like Robert Scoble or Jeremy Zawodny who go to some length at their blogs to link extensively to new and interesting content. However on balance many bloggers – especially those on the infamous “A list” – now reserve links for their friends or for indirect commercial uses such as helping other sites get a rank boost.

This last point seems lost on Google as well as many A list bloggers when they discuss the implications of paid linking schemes and pay to post blogging. Indirect monetization is still monetization and changing the links game seems to be leading people to change – quite dramatically – the way they publish and link. Or perhaps more importantly changing people so they do not link like they used to do in the good old days. This is my main beef – people don’t link like they would have in the old days because they think they are “giving away big value for nothing”.

Linking, once the very heart of the web, are now the wampum of the web, and this is leading to a lot of undesirable consequences.

Wrapping up day 3 of SES San Jose Search Strategies Conference


For great session coverage of SES see the following sites:

SEO Roundtable

Search Engine Land

SEOmoz

Bruce Clay

Technorati “SES” tag

TopRank Blog

Yesterday’s “Is Buying Links Evil” was by far the most interesting and heated of the sessions. Google’s Matt Cutts was under heavy fire from Todd Malicoat and Michael Gray regarding Google’s aggressive policies on paid linking and the application of the NOFOLLOW tag. The best question came from Rand Fishkin who asked Matt Cutts if it would be preferable to do without NOFOLLOW and have better, scalable, algorithmic ways to determine link relationships. Matt indicated it would and this gets to the huge middle ground in the paid linking debate. I think SEO folks, especially those who worked back in the gravy days of massive paid linking, should have expected Google to crack down on the practice but I would *strongly* criticize Google for not bringing more transparency to this issue by clarification of their paid link penalty structure and what appears to be a lot of leniency for paid linking in many situations. Many links, such as those a brief aquaintance might give to another person who opens a new website, are probably in line with guidelines but are essentially identical in structure to a paid link. In this case adding nofollow is totally inappropriate since the goal is to indicate a mild endorsement of the new site. I suspect this type of link is treated favorably by Google and I’m wildly guessing that they err on the side of not penalizing this type of link, but that is not clearly indicated in the policy statements or in the talks I’ve had with Google search folks. This failure to clarify, combined with Google asking for “help” in finding paid links, has led to more frustration in the Webmaster community than Google thinks it has caused. One indication of this was the huge applause given to parts of the “anti Google” presentations yesterday. As always Matt Cutts handles this with great composure and I think a very sincere desire to make things work well for all players, but I’d recommend that Google really examine the linking policies carefully and issue a detailed and full clarification of “legitimate linking practices” with, literally, thousands of examples. Will this be reverse engineered for SEO benefit? Yes, but if it’s written correctly it can improve the web rather than leading to confusion about linking and the rampant continued use of paid linking schemes.

Links are a big theme here and I’m now off to Danny Sullivan’s session on “Search Engine Q&A On Links”

Marissa Mayer on intersection of strong AI and search


Marissa Mayer of Google gave today’s Keynote conversation. It’s no wonder Google does such wonders when people like this are in charge. I did get a chance to ask about the intersection of search and AI and got a fantastic answer – she thinks they will intersect, and this could happen within about ten years. Also interesting was that she said they are now seeing things that “look like intelligence” emerging from the search algorithms. This is not thought, but she indictated that it’s possible to have thought like processes emerge in this fashion rather than with the massive computational approaches that were popular several years ago. This is consistent with Kurzweil’s notion that it’ll be massive parallel processing and not massive supercomputing that will probably bring the mechanical mind “to life” within the next decade or so. I’m glad Marissa Mayer seems to agree and I hope this is a focus for Google in the future (I got the idea it’s not a focus now).

I had a chance to ask Matt Cutts of Google engineering fame the same question yesterday and he was not as optimistic, thinking that it could take another 50 years to get conscious computing. But Matt correctly noted that Marissa would be more optimistic than he was because his Master’s program at University of North Carolina was lacking in much AI content due to the AI skepticism of the architect of that program.

Larry Page’s recent remarks suggesting that a viable human thinking algorithm may appear fairly soon are more in line with Marissa’s optimistic view that within a decade we could see mature, conscious, artificial intellects. The staggering implications of conscious computing are lost on many people in computing for reasons I simply don’t understand, but I think are related to the current focus on computing science as an engineering and calculation paradigm rather than a biological one. As the brain is reverse engineered and we begin to enhance neurons with forms of programming it seems reasonable to assume things are going to get … very interesting very fast.

Google Party 2007 and SES


The link buying session was extremely intense and interesting.   In short, Matt continued to suggest that link buying was distorting the natural patterns of the web and is a bad SEO practice while the SEOs on the panel argued that links do work and Google has no right to police them so severely. 

Unfortunately and as I’ve always seen, the debate tends to dwell on extremes on both sides rather than the important middle ground.  I have a lot more to say about this but it’s time for the Google Party!

SES San Jose “Mini-Interviews”


I’ve had a chance to talk in depth to several folks and will post that later, but wanted to check in before the session on “Is link buying evil” which will feature Matt Cutts from Google and some notable advocates for strategic link buying.    I’ve been surprised to hear from some really good SEO folks here that link buying still works well as part of their strategy, though I think they’d agree it’s very difficult to find the types of links that “work”, and from my perspective you always have a potential gun to your head from SE’s which do not like this practice.   So perhaps the best advice for most is to avoid link buying unless you want to live dangerously.

I had a nice talk at lunch today with Matt Cutts about his view on AI and  severak search themes but no time now to spell out the details.  

A key theme here is the number of SEM firms – many that seem fairly inexperienced.  Lanzone mentioned that it’s  increasingly common for large clients to buy out their SEM firm to bring it all in house and I think that may be a new strategy for the players in SEM.

SES San Jose – Jim Lanzone on Ask’s upcoming billion dollar search deal


ASK CEO Jim Lanzone was the first keynoter here at Search Engine Strategies San Jose, and Lanzone gave a lot of insightful answers to Chris Sherman’s excellent series of questions about ASK’s future in search and advertising.   A few highlights:

“It’s not a zero sum game” said Lanzone, noting their cooperation with Google in a 100,000,000 ad sponsorship deal and saying the next deal will be in the billions and could be with other players as well as Google.  

ASK 3d is leading to some interesting findings, esp. that 50% of the ASK 3d activity is not in the search listings portion.  Lanzone feels the sweet spot is in the “Collective Context” that billions of searches are bringing to the table now.    ASK’s new “Edison Algorithm” will seek to make sense of the maelstrom of data ASK has from their search property as well as the dozens of separate IAC online businesses.

“Search is now your co-pilot”, said Lanzone, and suggested that the value of search based ads is still very high compared to traditional media.  

Sherman noted that Lanzone’s “Etour” was similar to StumbleUpon.   Lanzone said it was before it’s time and was “Darwined out”.   No plans to revive it are pending. 

Search Engine Strategies – Google Party


Day one of the four day SES conference is wrapping up although a lot of the conference action takes place at restaurants and bars after hours.   I think for most people the highlight of SES is the huge Google Party which will be held tomorrow night at the Googleplex in Mountain View.   “Meet the Engineers” is one of only a handful of times each year when you can talk directly to a large number of people on the Google search team – the other is WebmasterWorld’s “PubCon”  in Las Vegas.

One thing I learned today is the Google’s Marissa Mayer is an expert in Artificial Intelligence (yikes – ValleyWag says Marissa IS an Artificial Intelligence!), and I’m hoping I’ll get a chance to ask a few questions tomorrow after her keynote about where she sees Google’s AI efforts heading over the next 5-10 years.  Larry Page was recently quoted as suggesting that a human-like thinking “algorithm” could well be cracked fairly soon, and Google is one of the places where this type of innovation might actually take place.  That said, based on my talk with Matt Cutts a few years back I don’t think AI as a search driver is a Google priority.  I was surprised then to hear that Matt felt quality AI driven search was still many years away.    Google has to maintain a practical edge to things so they probably can’t put a huge effort behind a “conscious computing” effort, though I get the idea from Kurzweil’s book that a Googley “massively parallel” info architecture may be more likely to bring consicousness to a machine than, for example, the IBM Blue Gene style supercomputer.

Blogs covering or writing about the SES Search Conference

Session coverage roundup from Barry at Search Engine Land

Search Engine Strategies

SES San Jose


I’m excited to be at the Search Engine Strategies Conference here in sunny San Jose.  This is the *big event* of Search Marketing and it’ll be fun to cover it as a press person.   Rather than intense live blogging I’m going to try to summarize the day each evening with a post and extensive links out to others who will be doing more authoritative coverage, like Barry and the Search Engine Roundtable gang.    Barry has been doing simply outstanding play by play blogging of search conferences for several years and you can get a great handle on things following conferences from his website.    Even when I’m *attending*, as here in San Jose, I like to read the Search Engine Rountable session summaries.