Google Says ...

Switching to new SEO Theory blog

2006-12-09T11:16:00.000-08:00

I've long been wondering if I should not stop focusing on Google (with this blog) and expand the scope of my SEO blogging to cover all the major search engines. In fact, at least one Googler has suggested in my comments here that would be preferable.

So, while I'll still have many things to say about Google in the future (including, I hope, many positive things), I'll be saying them as part of a much broader commentary on search engines. Accordingly, I have set up a new blog on SEO Theory and will subsequently share my search engine-related thoughts there.

And I thought I should explain my situation a little better. In September I was offered and accepted a position as Director of Search Strategies with an Internet Marketing firm here in Seattle. We discussed my online visibility and how that might impact relationships with clients, or just my job performance in general. At the time it was agreed that I should become less active in the online SEO community.

Since then, I've been told it's okay to discuss SEO principles in general, but obviously with the expectation that I stay focused on my job. So I still won't be nearly as active and visible in the forums as I once was. But I plan to be more active on SEO Theory than I have been on Google Says ... over the past few months. I'll leave this blog in place for as long as the service allows it, since there are inbound links to the articles.

Thanks to all of you who have stopped by to share your thoughts with me. I hope you'll find the new SEO Theory blog to be at least as valuable and interesting.

SEO Discussion Search Tool and Google Woes

2006-12-08T02:04:00.000-08:00

I have been creating custom search engines with Google's little Custom Search Engine tool. All that prevents me from turning them out en masse is the amount of time I have to spend evaluating Web sites. I look at hundreds of sites whenever I create one of these search engines.

The latest one is mostly for my own personal benefit because I'm sick and tired of sifting through listings I cannot see the real content for without having to register for memberships. That bugs me. I don't have time to buy my way into every crowded venue that may occasionally generate some useful information.

In fact, I treasure SE Roundtable because they often save me the trouble of having to find interesting forum discussions. And then encapsulate the important points pretty well, too.

In that vein, it would be nice if Threadwatch were more like the original site Nick Wilsdon created. Nick didn't always appreciate having me around (or maybe he did in a link baity way) because I tend to disagree with most SEOs' conventional wisdom (usually for good reason), but he had a great resource. Nowadays, Threadwatch is mostly a rant-center, albeit an important one as long as its signal-to-noise ratio remains relatively high.

So, I set up an SEO Discussion Search engine at my SEO Web site (which, for all intents and purposes, is in an imposed stage of dormancy since I am now the Director of Search Strategies for an Internet Marketing firm -- those pesky non-compete contracts keep me a little idle in the evenings).

Anyway, I scoured the Web for interesting SEO blogs and search engines. I wanted SEO communities that were open to public scrutiny, relatively active, and/or extremely useful. By that I mean I deliberately included a few sites that don't get much traffic (or at least not much comment) but which still produce a lot of worthwhile information. Bill Slawski's SEO By the Sea blog is a must-read for anyone who likes to prognosticate about where search technology may take us -- and that is my full-time job, now.

I left out some of the more popular SEO blogs because I know someone out there has already indexed them in an SEO blog CSE and because I didn't feel those blogs really contribute much useful information for search engine optimization. For example, Danny Sullivan is extremely popular, but Daggle.com is not loaded with deep insights in search. I did include Searchengineland because it will soon be loaded with search news.

But the purose of this CSE is not really to help people find the latest search news. Rather, I am constantly searching forums and blogs for specific things I know I have read somewhere at some time. I just cannot remember where or when. By narrowing the index to a handful of sites, I reasoned, I may have a pretty good chance of finding what I am looking for.

And since I am sure other people share that occasional frustration, I decided to expand the list of included blogs and forums to make sure the search engine has a pretty solid coverage. I tried not to exclude anyone's site on the basis of personal bias, but frankly some sites are so in-your-face with ads or nonsense or vitrolic ramblings I just don't see any value in them. So a few very well-known, very popular sites joined the other excluded very well-known, very popular sites solely because I just don't see any value in them.

I don't ever search those sites for anything useful. Take that however you want to.

Nonetheless, in setting up this new search engine, I have noticed some more issues I'd like to see resolved or at least clarified. For example, if I specify a sub-directory on a large content domain, how much of that domain will actually be included? Google made it easier to include sub-domains in an index but I'm not sure of what is actually being indexed if I just specify part of a domain.

And while Google also made it possible for people to include subscribed links to their custom search engines, what I would like is the ability to override Google's filtering for the CSE.

You see, one of the very best SEO forums for years has been Spider-Food, launched by J.K. Bowman about six years ago. J.K. is no longer as active in SEO as he once was, but he stays in touch with a core group of old buds (including me). A few years ago, Spider-Food was penalized by Google for using hidden divs -- J.K. was pretty good at both the White Hat and Black Hat stuff before anyone used such nicely worded cliches to distinguish between people who followed the search engine guidelines and people who exploited the algorithmic holes.

So because Spider-food had thousands of inbound links, J.K. pretty much ignored the Google ban. But when he stopped actively participating in the forum himself, people began dropping off. J.K. always said I had a lot to do with keeping the forums active. I usually publish my most original research there, for example. But in my opinion, J.K. is and will always be the life and heart of Spider-Food. He had a very no-nonsense approach to SEO and he was especially good at the under-the-hood SEO that few people today really appreciate.

Most of J.K.'s advice was sound and ethical, and much of it would still apply today. And for over a year J.K. has been promising to remodel Spider-Food, clean it up, and ask for reinclusion. He's just a bit too much of a perfectionist, I think, as well as dragged down by other demands on his time.

Well, Google has for years indexed the Spider-food forums and I've been able to find the threads I needed to get to through Google. Not any more. I don't know if the delisting is permanent or temporary. I'm not sure of why it happened, although I know that spammers were hitting the forums pretty hard the past few weeks. J.K. finally took steps to prevent the robots from dropping any more links. But maybe he acted too late.

I don't know.

All I know is that a great resource has been delisted. Since Google just rolled out some sort of update a couple of weeks ago, and since they usually recrawl the Web after they finish an update, I'm hoping to see Spider-food come back into the index. I know it's penalized, but as long as I can do site searches, that's fine by me.

But let people sit up and take notice, because Matt Cutts did warn the SEO community a few months ago that some very serious changes were on the way. This past week at SES Chicago Rand Fishkin of SEOmoz was quietly told clean up the outbound links in your profiles or suffer some consequences. I have no doubt the Googler who conveyed that warning was doing so out of a legitimate concern to help a widely valued resource from losing search engine value.

Nonetheless, it comes across as an act of bullying. While I have always maintained that Google has every right to do whatever it pleases with its search engine, Webmasters have a right to do what they please with their Web sites. Before there were search engines there were Web sites and even without search engines there are still Web sites. But as the lines of communication open up between Webmasters and search engines, pre-emptive warnings about impending algorithmic doom only confirm what conspiracy theorists have said for years: Google acts like it owns the Web.

In fact, without the Web there is no Google, and Google cannot honestly reach into every Web property and contact the right person. For example, if they were to try to send me a warning email as they send such emails to some Webmasters, it would never reach me. I've disabled all the traditioncal admin accounts because of email spam. Many other Webmasters have, too.

I'll know if Xenite.Org hits the skids only after the fact. Shame on me for linking to sites I think are valuable but which Google may not.

Now, in Rand's case, he was apparently told that some really undesirable sites were being linked to through SEOmoz. I don't link to sites like that. I suppose I'm not in any danger from that kind of algorithmic assessment. And I've been gradually closing off directories where I don't feel Google needs to be going over the past year anyway. Old URls that no longer exist except to redirect people following old links don't really need to be indexed by the search engines. Nonetheless, my redirects have always led me to wonder when or if the axe will fall.

Frankly, if a Googler were to say to me, "Dude, fix your site or get axed," I'd say, "It's your search engine but my site."

Google only sends me a fraction of the traffic I receive. I, on the other hand, provide Google with a lot of great content. It's more their loss than mine. Besides, I could easily enough build up content on other domains that would tell people where to find my orphaned domain. I've been promoting Web sites far longer than Google has been around.

But what is the happy medium? After all, I see the Google warning to SEOmoz as a friendly interventive action intended to benefit the entire Web community. The problem is that not everyone will get such friendly warnings, and the ominous clanging of shuttered windows and doors in various blogs and forums reflects the essentially suspicious attitude of many people toward Google.

Which leads me back to Spider-food.

I want to include their threads in my custom search engine. I'm not asking Googe to lift the ban completely. I just want access to good content -- content that I believe does not violate Google's guidelines. The forums are hosted on a sub-domain.

After years of my openly complaining about how Google has treated sub-domains as if they are independent domains, has Google changed the status quo? Or did the spam robots that hammered an innocent forum get it delisted?

Or am I simply jumping the gun and the forums will be recrawled and reindexed soon?

Well, I have to get some sleep. Thanks for listening to my rant.

And I hope that the CSE I set up does, actually, help other people in the SEO community. Even the folks who didn't make the cut. I was just trying to create a tool with a different value from the one I had already read about.

Google's Supplemental Pages and other neat things

2006-11-24T20:06:00.000-08:00

When I signed off a couple of months ago, I did say I would probably occasionally post something new. I've been working as the Director of Search Strategies for a Seattle-based Internet Marketing firm, and one of my responsibilities is to keep an eye on the SEO community.

While my company is not particularly interested in what most SEO theorists have to say, I do need to stay aware of the community's concerns and current theories.

I have to admit that I was surprised to see hardly anyone react to the apparent update Google rolled out this week. It was breifly mentioned on SERoundtable and maybe a couple of other places, and that was it. Well, I suppose most people were either pre-occupied with the holidays or else they have simply gotten used to see the results change.

But I've noticed an increasing amount of discussion about Google's Supplemental Pages, and to be honest I find the answers being given to the usual questions somewhat inadequate. I feel the inadequacy is due more to boredom than anything else. Forum regulars are simply tired of speculating about the Supplemental Index, so they are now handing out brief, trite responses.

With one exception, who shall go unnamed, although I pretty much consider the guy to be an idiot, that is. The exception suggested one of the most outlandish ideas I've seen from him in quite a while. Well, after rolling my eyes I thought, "I could write a better answer than that in my sleep".

So I challenged myself to do it (while awake, though). And all that is to say that I have now posted a more in-depth discussion of Google's Supplemental Pages on my SEO site. And while I am at it, I might as well point out that I have slightly reorganized the site, moving this new page and a couple others into a new SEO Information section.

The bottom line: if you want to move a page from Supplemental Results to the Main Index, point a few good links at it. The links can come from anywhere, including your own pages, as long as the linking pages are not in the Supplemental Index. So far, that has worked for me, even if I'm wrong to suggest that the links must come from pages in the Main Index.

I met some Googlers at the Seattle MindCamp a couple of weeks ago. I wasn't sure what to say to them. When they asked me what I do, all I could think of was, "I manipulate your search results", which is entirely true. But since they manipulate their own search results (I mean that in a good way), I figured it was a safe response.

So, I hope everyone in the U.S. (or from the U.S.) who celebrates our Thanksgiving holiday has had a good holiday weekend so far, and I wish you all the best.

I may post something here again before the end of the year, but I cannot be sure of that.

Farewell for now...

2006-09-26T18:35:00.000-07:00

Things have moved so quickly for me since the end of August. I have more projects to complete than I have time to work on them.

This week I accepted a position with an SEO-related firm that is doing the kind of work I enjoy and have been seeking out. They specifically like my theoretical approach, although they keep their clients' best interests at heart (no so-called "black hat" SEO).

Unfortunately, I won't be able to maintain this blog. I'll leave it up for posterity's sake, and perhaps to make an occasional (very occasional) comment in the future.

I have been gratified by the interest in my work and my ideas expressed by so many of you.

Thnak you all.

Interviewed by Sootle...

2006-09-20T11:00:00.000-07:00

The Sootle Web Directory has been publishing a series of interviews with SEOs. They asked me for an interview and I am officially SEO Interview Part Five.

Thanks to Darren at Sootle Web for the interest. Other interviewees so far have beedn Bill Slawski (number four), Michael Grey (number three), Dave Davies (Beanstalk - number two), and Joe Balestrino (number one).

Each interviewee is asked six SEO questions.

Great series, Darren! (And not just because I'm included -- you have some top-notch names in the list.)

The uneveness of Google blogging

2006-09-19T17:10:00.000-07:00

It's tough when you put all your eggs into one basket. I've occasionally wondered if I should not have called this blog "The Search Engines Say..." because Google doesn't always say anything noteworthy from my point of view. I don't want to do a roundup of every announcement that Google makes, either officially or unofficially. But Google being Google, it seems worthwhile to me to say something about their ponderances and pronouncements on occasion.

I have, however, noticed a pattern that demonstrates just how large and cumbersome Google is becoming. Some of their blogs are updated frequently and some of them are updated about every time a remake of "King Kong" is produced.

The main Google blog sometimes carries posts that don't seem very relevant to me. Such as today's Congratulations, Luis von Ahn post. Alan Estace, Google's VP of Engineering, claims that Google has built upon von Ahn's work, and they extend worthwhile recognition to him. But one does not get a full sense of how much people at Google feel they owe to von Ahn.

Maybe I should say I don't get that sense because I'm so used to reading gushing fan site copy that when someone sings a simple verse of praise, I react with a jaded, "That's it?" It's a worthy note, but doesn't shed much light on Google's mysterious ways to an algorithm chaser like me.

Nonetheless, at least the official blog gets updated on a frequent basis. That's important. It keeps many people coming back. There are other Google blogs that don't update very frequently. Surprisingly, one of those infrequently updated blogs is Blogger Buzz. Their last post as of this writing is September 11's Flickr Support for Blogger in Beta. Seems to me they could find something more to say for Blogger users. I'm not very good about digging into manuals and FAQs any more. I'd rather read about all the cool stuff I should be doing with my blog in contemporary posts.

The Google Analytics blog posts an update about once a week. That's almost enough to keep me coming back, but I have to admit that there are weeks where I forget to do that. So, I did see last week's Spotlight on: How to read the ROI column but I didn't really get excited. I actually liked the Tip: Tracking 404 Pages post more because I do have a lot of 404 issues and I should be paying closer attention to them. Analytics might be a better tool for that than sifting through my error log.

Google Base is not as consistent as Google Analytics in updating their blog. The Open House tip they shared yesterday is actually very intriguing. Can the highly competitive real estate industry benefit that much from Google Base? I may have to go out and recruit a few real estate sites just to test it. contact me if you want help promoting a real estate site. I'll give it some thought.

Normally, I shy away from real estate, as do so many other people. It's a tough, tough field. I do better with promoting insurance Web sites. But I digress. Google Base's blog offers helpful tips on how business site operaors can maximize their visibility in Google, and I wish they offered just a little more frequent information. 2 posts a week would be good for me, even if some of them repeat previous topics or just recap information from the past 3 months.

In fact, I think a quarterly recap of topics would be a good idea for all the Google blogs.

Pacing is a real issue for these blogs. For example, Google Checkout had a flurry of posts last week but has been dry ever since. That's not a good thing.

Google Code updates its blog quite often, but it took me a while to realize they did because their home page looks more like an information portal. Did you know that they have posted a Summer of Code update? BTW guys -- your permalink page layout doesn't look so good in my browser. The solution is not to tell me to switch browsers. Still, one of the great things about this post is that they point you to Drupal's project list, where you can download and test new software.

Speaking of coding, there is now a Google Data APIs blog. Leslie, I hope you and your team see what I am getting at here. Consistency is more important for building a readership than waiting for that whambang product release or update that absolutely has to be mentioned. Pace yourselves with the new blog. I would aim for 1-2 posts per week.

Google Enterprise does a good job of posting to their blog frequently. New version of Google Search Appliance! is one of those whambang product release announcements I referred to in the previous paragraph. But they posted about the Xythos OneBox and Webinar on Sunday and did a post on Friday, and so on. They keep the information flowing and I look forward to seeing what comes next from that blog.

The Google Maps blog is another of their 1-2 times a month updaters. How many people noticed that they posted an announcement about New Satellite Imagery and Performance on Friday, September 15? This post will give that post its first Permalink. Their last post occurred on September 6. Rip Van Winkle would not be likely to check in on a blog so seldom updated.

I have pretty much given up on Google Reader's blog. It hasn't had a post since August 3. Is that all there is to say? No tips on great feeds to find, or ways to find feeds, or how to manage your feeds? Come on, guys, surely you use your own tool, right? Share something.

Same for you, Google Scholar Blog. I'm tired of seeing the N-Gram headline. The joke was funny the first time I read it on August 3 and cute the next time I read it. Now I want more.

I don't understand the Google Video Blog, so I don't even read it -- oh, wait. You can't read it. That's the issue.

Google Web Toolkit blog almost had a September update. Only 11 more blogging days left in the months, folks.

I'm not sure I want to link to Google AdSense blog's September 19 post because I think I'm going to do this. It just sounds too cool to pass up. But it will depend on how complicated it really becomes in the implementation.

Alas! My most favorite of Google Blogs, Inside Google Book Search, has not updated since September 12. What happened, guys? You had a great thing going! I hope you didn't get banned from Blogger or something. That would just be so unfair. Inside Google Book Search is the most innovative of the Google blogs. They actually show people how to use their product 9 different ways from Molly. It's a great propaganda tool for a controversial service because it really does show the value the service provides.

The Google Desktop Blog is another disappointment. No posts since September 5.

And just as I was about to complain that Webmaster Central's blog isn't updating enough, Amanda posted their very cool and informative Debugging Blocked URLs tips. Because of Webmaster Central's name, this blog has the potential to become the most closely read blog of the group, as far as search engine optimizers and Web site promoters are concerned. That's the power of branding. But the power of expectation will impel the Webmaster Central staff to be more active on the blog if they want to keep their audience happy.

The real reason for this roundup is that Google hasn't given me much to write about lately. Sure, Matt Cutts wrote about Changes in URL queries yesterday, but frankly I found that topic to be a little boring. It's not really controversial. We see Google make changes to its interface every day. Changing a query syntax once in a while is no big thing, and I can easily accept the reasoning Matt offers.

In fact, this recent change will probably make my URL reference research a little easier. Unlike some people in the SEO industry, I have sense enough not to expect Yahoo! to tell me which links Google has indexed.

That whole concept still scares me, because those people get paid to do research. That's like looking at the Democratic Party's mailing ilst to see who contributes to the Republican Party.

So, Google, here are your orders: Post more stuff on your blogs. Say something. Give me a purpose, because I really don't know how to change the name of this blog to "The Search Engines Say...".

Using Google To Help Others

2006-09-17T20:59:00.000-07:00

Google announced a United Negro College Fund scholarship program last week. I think our higher education system is way too expensive and am always happy to see more scholarships appear.

But in thinking about what other types of scholarship programs I could imagine Google funding (say, for Native Americans, Hispanic Americans, unwed mothers, orphans, and victims of drunk drivers -- not necessarily in that order), it occurred to me that there is one important segment of American society that never gets any attention: experienced workers whose skills are no longer required. I'm not just talking about unskiled laborers. I'm talking about college graduates.

Last year, Bill Gates whined about a shortage of skilled IT workers in the United States. Don't be fooled by the low percentages for unemployed IT workers mentioned in the article. In 2001, 500,000 American IT workers permanently lost their jobs. Estimates published since then indicate that as many as 400,000 of those workers have been unable to return to the IT field.

Many of those jobs went to India because the cost of hiring Indian programmers is considerably less than the cost of hiring American programmers. Now, I've been to India in a professional capacity and the programmers I met there struck me as being competent, capable programmers. Nonetheless, there is a growing backlash among Americans against the outsourcing of jobs to India. To his credit, Mr. Gates doesn't want to outsource jobs to India -- he wants to bring Indians to America on H1-B work visas and pay them American salaries.

In the meantime, approximately 400,000 former American tech workers find it very difficult to get jobs because they are "overqualified and will leave at the first good opportunity". Try submitting a resume that shows 25+ years' experience (including management positions) in the IT field to an unrelated industry and you'll see what the problem entails. There is a great deal of technical experience and skill languishing in this country because CEOs like Bill Gates are too cheap to invest in teaching skilled programmers how to work in a new language.

Google and other large corporations could give thought to tapping an immense resource of experienced talent that is currently not being exploited.

But enough about scholarships. Google AdWords' Blog has launched the first in a series of blog tips about how to create effective video ads. I don't think many companies have thought about what it would take to sell video ad space on their Web sites, but I've often wished I had the resources to put together some randomly available videos fo Xenite.Org. In about 3 years, these tips may look naive and misguided to experienced hands, but for now they are all that business advertisers have to work with in this emerging field.

If you operate a large multimedia site, think about how embedding cross-promotiona video ads in your own content can help you improve your on-site conversion ratios.

In August, Inside Booksearch published Finding Gems In Your Library. I have now seen this feature in action. I mainain a discussion group about Middle-earth and occasionally post news stories to the list. Tonight I shared a story about Amanda McKittrick Ros, an author whom J.R.R. Tolkien and his friends (the Inklings) read for amusement. I wanted to see if Google Books had any of her works (they don't, although they do have my first book Visualizing Middle-earth).

When I scroll down the first results page for Amanda McKittrick Ros, however, I see three searches of library catalogues. I shared that search with people on the Endor list and I think this marks the first time I have recommended Google Books to people. Don't assume I'm now wholly in favor of the service, but it would be crazy to pretend it is not there.

Google Enterprise says Xythos has developed a Google OneBox module without having to use a Google Search Appliance. If they share enough of their technology in their October 18 Webinar, other developers may be able to follow their lead.

And, finally, Google Webmaster Central reports that the login function for Public Search Service has been temporarily disabled. It's interesting that Google says they are not aware of any malicious exploits of the search function. But I have to wonder if this update was inspired by SEOMoz's 20 government links in 20 minutes post. There are Googlers who read SEOMoz. If they noticed that Public Search was being exploited, SEOMoz may have just helped Google close what could have become a serious loophole.

If you've never looked into what Google Webmaster Central offers, check out their resources. They make the SiteMaps Site Status tool available for free without requiring a login. Many people who don't trust the service may want to reconsider using this tool.

While it's true that Google most likely saves the queries, I doubt they use them to tag sites that are not compliant with Webmaster guidelines. They aready have filters to do that.

Inside Google Magazine...

2006-09-14T20:45:00.000-07:00

Google needs a monthly magazine. Google needs a consumer-oriented voice that can do two things: tout the strengths of Google and Google products and services; and present a more balanced assessment of the various controversies that surround Google.

This year, Google has found itself buried in legal actions and news stories over several topics. While the corporate attorneys would prevent Google from speculating madly in print (and I'll adress what I mean by "magazine" in a moment) and thus poisoning their legal options, a Google Magazine or Inside Google would give alternative voices a fair platform in which to present their concerns. Not to argue their legal cases in front of the public, but to help explain what's stake if Google prevails in some way.

This is a bold step for a corporate entity, but it's a step that would help Google build trust around the world. And they badly need trust for their long-term financial success. Google is now large and ponderous in some ways. If they roll over they are almost certain to crush some little person who cannot run out of the way fast enough.

But while a monthly print magazine would be interesting, I think it would be better if Google distributed a .PDF or Flash magazine from Google.com. The front page. And they should set a standard for the publishing world by publishing the November issue in November (I'm sick of buying next month's magazine this month -- that makes absolutely no sense and no one is fooled by it).

The reason why Google needs to set up an autonomous media production group is that their "official" Web sites are so loaded with propaganda that no one trusts them. In fact, even their employees' private Web sites are viewed with considerable distrust. I have no idea of why people say Matt Cutts cannot be trusted. He has never been caught in a lie but a lot of people in the SEO world say, "You cannot trust Matt Cutts". I mean, literally, they say that in the open on forums.

Who has Matt betrayed, and why doesn't that person come forward to warn the rest of us?

If you look at Google Book Search's Legal section, Google makes an amateurish attempt to bamboozle visitors into believing that everything is okay and authors and publishers are really not going to get hurt by the Library Project, etc.

Oh, and just to be fair, after you read all the testimonials about how great Google Library Project is, there are two bare links leading to specific statements from opposing groups. It's like doing a search for some product you think you have heard about, finding a page listed in the search engines that says it sells the product, and then you click on the link only to find an endlessly scrolling page loaded with fake testimonials from anonymous people. You have to scroll, scroll, scroll, scroll to get down to the, "And it just costs $49.99 to buy this cheap crap!" button.

Google, you can do better than that. You owe it to us, your using public, to do better than that.

The closest thing to a magazine right now is the collection of Google blogs, but they are not updated on a consistent basis. Google Book Search Blog is updated frequently, with loads of great content. I like this blog because it takes marketing seriously. Okay, I just criticized them for playing fast and loose with the facts on their legal page. But overall, the Book Search people seem to understand marketing better than all the other sections at Google.

Look at the "Useful links" they post in the margin. They index the official Google Book Search site for you, point right to interesting and useful content. How much better it would be if they pointed to this month's latest Inside Google Magazine articles.

I would look for this magazine every month. What's more, Google could sell advertising in it. What would you pay for each download, Mr. Business Operator, Ms. Business Founder? Would you want to put your smiling face in front of 10 million people a month. Of course you would. There's a medium for Google advertising waiting to be invented.

There are a lot of powerful things Google could do with a monthly multimedia magazine. I think the Internet is ready for it. In fact, if they don't do it, someone soon will. PodCasts and Google Video becoming passe. A downloadable magazine with embedded video content would be the utmost cool thing.

Until next year, when we would have to have Google television for cell phones.

Think about it Eric. I'm available if you want to talk....

Google expands its crusade to legitimize Google Book Search

2006-09-13T09:11:00.000-07:00

Note: I have toned down some of the rhetoric. I hope it's clear I am expressing opinions and writing for effect.

I think Google is cranking up the volume on Google Book Search. In doing so, they are relying on the age-old principle of, "The more you say something, the more acceptable that saying becomes". Another variant is, "The more often you repeat a lie, the more believable it becomes". I'm not suggesting that Google is lying about anything -- far from it. But propagandists have long understood that repetition is critical to establishing believability and credulity.

Advertisers and marketers study how effective repetition is. We have to constantly find new ways to repeat a message because people ultimately tune out the message, although some experts suggest that even when repetitive messages become white noise they still have a subliminal impact.

Google is not being subliminal or subtle at all in its campaign to establish the legitimacy of Google Book Search. This week they are playing the liberal card with a Google Blog post titled Celebrate Your Freedom To Read. The post, written by Leslie Burger, President of the American Library Association, is a clever use of a conservative voice to make a liberal point: let's all oppose banned books.

I wonder if the ALA would support adding The Satanic Bible or Al-Qaeda's training manual to the works of literature that high school students should have access to.

It's not a question of whose morals are right, but of whose books are more popular with decision-makers. American schools have been banning books for as long as we've had American schools. The ALA advocates the popularization of reading banned books outside of school during Banned Books Week. But they try to direct your attention toward banned books that are at least acceptable to a sizable minority, if not a majority of Americans.

All this liberal-versus-conservative controversy fits nicely into Google's campaign to persuade everyone that Google Books really is a great thing for us all. As I have said previously, I am ambivalent about the service. Google really doesn't seem to appreciate just how important intellectual property rights are to other people. On the one hand, they strive to hide the secret sauce that controls their search results. On the other hand, they have an active policy of intimidating authors by reporting all DMCA requests to the Chilling Effects Web site. Chilling Effects is a very deceptive site. It presents itself as a moral watchdog. But the site's primary function appears to be to bully people into not protecting their intellectual property rights, and Google seems to support that bullying wholeheartedly without reservation.

So the message Google sends to everyone is, "We get to protect our intellectual property rights, but we're going to trample yours and make you look like petty little self-important twits for wanting to protect your own."

Is that harsh? Well, let me praise Google for how they are conducting their propaganda campaign. It's an advertising campaign. It's a branding campaign. It's an educational campaign.

Google Book Search is a powerful resource and I hope that ultimately a lot of people benefit from it. I'm not about to boycott the service just because it may be violating intellectual property rights. I'll leave it to the courts to figure out what the boundaries of my rights are. I hope and trust the various parties suing Google really do have my best interests at heart -- or at least share interests with my own.

But the Google Book Search campaign demonstrates in a positive way (in my opinion) just how effective repetition can be. They are using both the Google Book Search blog and the official Google Blog to tell people they can Explore Banned Books Through Google Book Search. They are showing Web marketers how you can assemble multiple voices of influence to hammer a message home repeatedly.

In time, as Google brings more voices of moral acceptance to its side, the message will become acceptable to many people -- eventually it will become persuasive. People will become so used to hearing and seeing the message that they'll accept it as part of their normal world experience. And when a message becomes normal and comfortable, it feels safe. And when a message feels safe, it becomes believable.

Google doesn't have to be right about anything. All it has to do is keep pressing home the message that Google Book Search is useful and valuable and eventually everyone will be saying it's useful and valuable.

How useful and valuable won't really matter. I say it's useful and valuable, but I also say it's a bit snarky. After all, Google is encouraging millions of students across the United States of America to read books in violation of their school districts' policies (these books can be read online, and therefore from school computers).

That is a little short-sighted on Google's part. But then, bullies and visionaries alike seldom think about whether what they are doing is wrong.

The lesson I want Web marketers to take away from this post is that you can employ powerful propaganda and marketing techniques to brand your service or products. Google is providing a textbook example of how to do this. But I hope you choose not to pick a morally divisive service to promote. Google is large enough that it can afford to tick off a few people.

Most small business operators cannot really afford to take that risk.

Google refines the art of content spamming...

2006-09-11T15:03:00.000-07:00

Not to be mean, as Google's tools are well-intentioned and actually neat. But I find that more and more Google stuff is geared toward helping people create mashups (Web pages consisting of gizmos supplied by other Web sites). The Web went through a mashup movement in the late 1990s, and it was not a very pretty thing to see.

In fact, you could say that in some respects Xenite.Org is The Web Site That Mashups Built. You name it, I put a third-party service on Xenite at one time or another: forums (through frames), Internet TV shows, Internet RPGs, Internet radio (I still miss my Xenite NetRock, NetCountry, and NetJazz services), scrolling/blinking/flashing news, comics, polls, and stuff I cannot even remember.

I love the creative environment one finds on the Web, but one day as I went looking for free content for Web sites, I realized that all the free content sources I was finding offered the same stuff, which had by this point pretty much become garbage. The Internet TV and radio services shut down or were sold; I installed UBB and later VBulletin so I could run forums off my own server (actually, I started with forum scripts from Matt's Script Archive); comic artists stopped updating their strips.

And science fiction and fantasy news was pretty much absorbed by SciFi.com. No one else could really turn it out fast enough. A few fan mega sites produced their own feeds -- and I once actually had the largest science fiction news resource on the Web -- but people got tired of writing news stories. They stopped updating their RSS feeds. The RSS-to-Javascript tools I used went offline.

Today, Xenite.Org relies more on original content than it ever did before because I simply got tired of yanking pages that no longer worked. You cannot rely on other people to keep the feeds coming, not unless you pay them. And I just don't think there are enough unique services for everyone to provide truly unique mashups.

So, when I see that Blogger now lets you draw images from Flickr, I think, "Okay, that's cool but still pretty tame." It would, in fact, be nicer if they could fix their image upload problem (because I haven't been able to upload any images).

And when I looked at Google AdSense's plug for Google Related Links, I wanted to say, Been There, Done That. Related Links aren't all they are cracked up to be. A lot of times, you get one cheesy video that has nothing to do with your topic. I know darned good and well there are a lot of Lord of the Rings videos available on Google video. Why do I get only 1? And why does only 1 headline show up in the News?

That is just so...lame. I'm supposed to impress people with this stuff? What would happen if I stack Related Links boxes on top of each other?

On the other hand, I noticed how Colin Colehour suggests using Google Base to upload genealogy records. Now, that could be kind of cool. My father sent me a family genealogy a few years ago that one of my uncle's sister's cousins's mother's brother's son's something or other did up. The history goes back a fair number of generations to some guy named Martinez who jumped off a ship near California.

But the last time I looked at Google Base, you could only post something for 30 days. What is the point, guys? I don't want to have to devote the rest of my life to logging in to Google every 30 days just to renew a genealogy listing. I'd feel so Lost doing that.

So, while Google has introduced some neat ideas, I don't think any of them really go far enough. Here are a few suggestions that I would recommend:

Google Base should let you download W3C-compliant HTML code that you can place on your own Web site (better yet, let Google Base push it to your Google Pages inventory). I realize people want to be found in Google Base, but after that first 30 days, a lot of us are going to say, "Eh. Not worth the trouble." It would be great if we could say, "At the end of 30 days, create this page at Google Page with this content."

Related Links should allow you to design a custom channel that displays 1-4 listings in the box where you can qualify the sources by domain, keyword, or something more detailed than just on-page content. Especially if your on-page content is a mashup (and my Lord of the Rings news page is now a mashup because -- surprise! -- I got tired of writing original content).

What Google really needs to do is create a Google Channels service where you can create a mashup-in-a-box combining content from all over the place. And allow people to make the channel distributable. I mean, let me add it to an inventory of predesigned channels that other people can subscribe to and add to their own Web pages. That's what was so great about Netscape's RSS-driven MyNetscape. They allowed us to create channels that people could add to their start pages. It was just so righteous.

I'd still like to feature up-and-coming comic artists on Xenite, but I'm tired of looking for them. Google, while you're adding services just to please me, let's see Google Comics, a combination between Google News Search (fed by preselected channels) and Google Base (timed content contributed by people who care enough about their work to jump through all the hoops). You might be able to use the Google Coop technology to produce it. I'm not sure. When I signed up for Google Coop and saw all the rigmarole I had to go through to produce a mashup, I immediately became depressed and closed the browser window.

It's got to be simple, smooth, and provide an immediate reward. By allowing people to distribute their content to other Web sites, the Google Mashup Express will stay fresh and innovative. It will blow the old free- and pay-syndication content services out of the water.

And when you've destroyed the competition, Google, you can monetize Google Channels. I hereby confer all rights to the trademark to Google, Inc. provided they deliver on my eensy, teensy little request and give me the service.

NOTE: Today is September 11, 2006. Five years ago, Osama Bin Laden proved he has no love for God or Man and no future in Paradise.

Who does Google trust now?

2006-09-08T21:02:00.000-07:00

What SEOs and Search Engines say about TrustRank and PageRank

Let me say up front that, so far as I am concerned, no one outside of Google is in a position to say definitively or authoritatively how Google determines trust. Nonetheless, many SEOs have been making very ignorant comments about Google and "trust" over the past 18 months or so. The problem began with everyone commenting on Google's listing TrustRank as a service mark. This was a curious situation because the expression TrustRank was coined by Yahoo!, who published a paper in conjunction with Stanford University introducing the TrustRank methodology for calculating PageRank more reliably.

Many of those SEOs have wrongly assumed (and stated repeatedly) that PageRank serves as the basis for Google's search results rankings. PageRank has apparently always been factored into the algorithm wherever possible, but Apostolous Gerasoulis of Ask has long claimed that Google never fully implemented PageRank anyway.

Matt Cutts has indicated that Google's internal PageRank drives their crawling priorities. I think this is probably for the main index only, but maybe it also drives the Supplemental Index crawling as well.

Google's apparent historical trust in sub-domains

It became apparent to me by early 2005 that Google had begun shifting its priorities in late 2004 (and perhaps earlier that year) to favor pages from older domains that I called Trusted Content Domains. I coined that expression to distinguish those domains from Spam Domains. Spam domains typically fall into one of two groups: 1-page doorway domains that redirect to primary content domains and domains that host a lot of worthless content.

I found that I could add content to an existing domain and see it rank well within a week to a few weeks, while people creating new domains were making no progress after several months. This was a marked change from the way my new-page content achieved rankings a year before. However, it very closely resembled the behavior of sub-domains coming off of primary domains going back to 2001 (see my comment on Aaron's post). I have complained in numerous public forums since 2001 that Google would automatically trust sub-domains. They never seemed to care, and a lot of sub-domain spam has been around for years because of that oversight.

In essence, Google has always seemed to confer without question to sub-domains the ability to achieve high rankings in search results. For technical reasons, I long resisted the temptation to hang sub-domains off Xenite.Org. I found that sub-directories often served my purposes, even though they took a little longer to establish relevance. However, though I am now starting to work with more sub-domains, I am concerned that Google may now be implementing serious sub-domain analysis and filtration. I may or may not inadvertently trip some filters simply through inexperience and experimentation.

How Google Determines Search Results

Because of SEOs' ridiculous infatuation with link-bombing based "optimization", the importance of relevance has long gone unheeded in the SEO community. Sergey Brin and Larry Page established that determining relevance was the core factor of their ranking methodology in their original paper about Google, but the inconvenient fact has been swept under the rug of ranking-through-link-spam.

In January 2006, Matt Cutts published an article in Google's newsletter for Librarians in which he recapped Google's basic ranking strategy. Matt naturally discussed the PageRank algorithm because it is so often referred to, but he emphasized that PageRank is not the key to ranking in Google's search results. In fact, Matt literally wrote that, "in order to present and score" results for a query, Google picks pages that "include the user's query somewhere" and then ranks "the matching pages in order of relevance".

The SEO community continues to look in the wrong direction

Despite this apocalyptic revelation, SEOs have continued to pound the podium in favor of link building. And I will admit to helping them pound the podium with all my link-building articles, although I have tried to point out that links are important for other reasons.

I write about link-building for one reason: since I know how to do it better than most SEOs, I felt it might help to establish my linking credentials in a community obsessed with links. Most of the more popular link schemes owe something to my research over the years anyway -- it's just that the young SEOs are too consumed with their snide tirades to do the research to find out where all their cherished strategies came from.

I didn't invent these linking schemes, but I helped test and prove their effectiveness back in the day when they could truly be efficient and effective. And, sad to say, I probably am one of the grand-daddies of link farming. But you can blame Inktomi for being so darned frustrating. Most of you have no idea of what it really means to have to rank on the basis of linkage. I do. I hope we never have to return to those kinds of search engines.

The consequences of all the bad SEO practices since 2001

When Adam Mathes coined the expression "Google bombing", he was only giving a bad name to a practice that actually went back to the days before Google. Adam noticed how effective the technique worked for bloggers, but spammers had been link bombing both Google and Inktomi for years. Well, after the media had their day with the new buzz word, a new generation of SEOs began building their business models on the foundation of link building.

After four years of thousands of SEOs blogging, writing articles, and sharing link-based ranking techniques in forums, FAQs, and eBooks, a large community of business decision-makers has been misled into believing that linkage is the key to ranking on Google. And what is truly sad is that it appears to be more true today than it was two years ago only because Google had to react to the massive onslought of manipulative linking that has mangled its relevance scoring.

All "white hat" SEOs who practice link-building are as guilty as all "black hat" SEOs and spammers of burning down the trees in our forest and destroying the environment in which we optimize. It will be years before SEOs take responsibility for their ill-considered practices. Black hats at least snicker at the idea of ethical optimization and shamelessly promote their Web sites in whatever way they can. They work on volume and build their networks and just adapt to the algorithm changes.

But the rest of the community has bogged itself down in a blind tradition that was a terrible solution to a non-existing problem in the first place. Now they are chained to the link-building treadmill because even the SEOs who realize there is more to search engine optimization have to deal with unrealistic client demands and expectations. The machine has lurched into high gear and tumbled out of control. Maybe a few of the operators notice they are no longer in charge, but most still mindlessly wade through SEO forums blathering about PR (Toolbar PageRank), "quality links", sending out reciprocal and 1-way link requests, and now TrustRank.

How Important Has Trust Become?

Because of SEO "best practices" based on link-building, Google has gradually gone into high anti-link building gear. Since early 2004 the so-called Sandbox Effect has been debated and tested and evaluated in six thousand directions. Consensus now seems to be settling on the idea that new domains are sandboxed because they lack links from Trusted Content Domains. I credit John Scott with being the first to offer the most reasonable explanation, though he now feels somewhat differently about what causes the effect (things do change).

Since mid-2005, Google has implemented filters against fake link directories, scraped content sites, and RSS-feed driven sites. When I warned Danny Sullivan about these kinds of sites in early 2005, he expressed complete and total ignorance of the problem. Swept up in the fake link directory blitz, however, were many "low quality" SEO directories -- directories set up by people for various reasons, including accruing PageRank, helping other sites build up linkage, and gaming Google.

Another problem that began to get attention from SEOs in late 2004, and which has gradually increased in severity, is the transfer of many legitimate content sites to the Supplemental Index. Only over the past few weeks have I found enough bits and pieces from Google to assemble a coherent idea of what the Supplemental Index may be.

With the rollout of Big Daddy in early 2006, Google exacerbated Webmaster frustrations by increasing main index crawling and decreasing supplemental index crawling. Suddenly, everyone started talking about trust as if they knew what was going on. Remember that I said at the beginning of this post that I don't believe anyone outside Google knows what is going on.

How can trust be algorithmically determined?

But several of us have tried to guess what is happening. Todd Mailcoat suggests that it's a trust filter based on Web site age, number and age of backlinks, and total "trustscore" of those backlinks. He adds: "Most trust criteria revolve around some dependence on age, which is actually a pretty good signal of quality". However, we know that Google ignores identified paid links among others, so "total number of backlinks" isn't helpful. Nor do I believe that age really matters as much as I once did.

Neither age of site nor age of links pointing to the site should really matter to how much a site can be trusted. A spammy link that sits around for 3 years is still a spammy link. A spammy site that sits around for 5 years is still a spammy site. I think Todd's third point is closer to the truth, and is really the only one required to explain what Google is doing.

But is Google scoring by trust or is it just trusting pages to confer PageRank and Link Anchor Text? In a follow up to his earlier Google Librarian article, Matt Cutts wrote "if more people trust your site, your site is more valuable" (implying that PageRank is used to help determine trustworthiness) and "we examine the content of neighboring pages, which can provide more clues as to whether the page we're looking at is trusted".

Another point Matt recently made was that the sudden appearance of hundreds of thousands of pages can trip a trust filter. That's a high threshold, but I'm sure it's that high for a reason.

Looking for trust in all the wrong neighborhoods

But what constitutes a "neighboring page" for a new domain? Any new page on an existing domain already has neighbors in its sibling pages (found in the same physical folder or directory) and cousins (found in other folders and directories on the same domain or sub-domain). New domains have to be placed into neighborhoods before they can have neighbors. Such neighborhoods are most likely only defined by linkage.

One simple possibility is that if a trusted "expert" or "hub" page links to a new domain, that expert/hub can be used to determine who the neighbors are. But even one expert's opinion isn't very informative. I think that Google looks for a variety of trusted expert opinions. These experts will include well-known human-edited directories with clear, definitive categories, but I think the expert votes also will come from some of the second-tier content sites. Any Web page that links to a group of related Web pages is usually considered to be an expert.

Until Google can form a collective opinion about where a new domain's "neighborhood" is, it isn't in much of a position to determine if that domain can be trusted. So, while many SEOs might be quick to say, "See? We do need to submit links to directories!" Maybe, but would you as a surfer want to trust a site only listed in directories? Why does no one else link to the site? You need more than one kind of expert opinion, in my opinion. Dan Thies suggested as much in late 2005 at the Highrankings Forum (and perhaps elsewhere).

"Well then," some hardcore reciprocators might say, "We just need to submit to directories and get reciprocal links from related pages."

Moving into the wrong neighborhood

But the problem is that Google looks for "excessive reciprocation". Some reciprocation is expected and tolerated. This is the World Wide Web, after all, where sites are expected to link to each other. But if you can only get links from directories and reciprocating sites, you're still not collecting independent opinions or votes of confidence from true authorities.

Authority pages has become another SEO buzzword, and I have seldom seen anyone in the SEO community use the expression in a way that conveyed a clear meaning to me. I am sure most people who speak of authority pages have a clear idea of what they mean, and can probably articulate that idea. But I have found no real consensus on what the SEO community collectively means.

I'll go with the traditional HITS definition: an authority page is linked to by many experts. But some experts are more trustworthy than others, and those experts are often linked to by many authority pages. It's all very circular, of course, but I think it's important that new domains be linked from authority pages in clear context. That is, a reciprocal link won't do the trick. You need to have content surrounding or adjoining the link that is relevannt to the link anchor text.

But let's back up a moment. Is it not possible that there are sham experts and authorities? Absolutely. So you need to ask if Google hasn't found a way to favor some neighborhoods over others. One potential trust-impacting factor is who you link to. Matt Cutts has been reluctant to explain why spammy-looking links on one page may be trouble and why similar appearing links on another page seem okay.

Neighborhoods must be bubbles of tightly connected Web sites, and the neighborhoods that are most trustworthy are probably linked to by many other neighborhoods. So now we're venturing into the realm of speculation with the concept of NeighborhoodRank. Does Google tag neighborhoods as being more or less trustworthy? If so, then it may be that an entire neighborhood has to gain trust before its member pages earn trust.

Why link baiting works

This may explain why Rand Fishkin of SEOMoz is able to boost sites past the Sandbox Effect so quickly. When he creates Link Bait, his sites draw linkage from both new neighborhoods and old neighborhoods, and the old neighborhoods undoubtedly include a lot of trusted neighborhoods. His Link Bait domains are therefore drawn into the better neighborhoods because of where they link to and from whence their inbound linkage comes.

In other words, successful Link Bait doesn't have to wait for its neighborhood to be approved for trust. It simply joins one or more already established good, trusted neighborhoods.

Why reciprocation sometimes fails

And that may explain why link reciprocation doesn't always work. Some people complain that after gaining several hundred reciprocal links, they still seem to be sandboxed. In evaluating the backlinks for many such sites, I often find they link out to and receve links from what I personally would deem to be low quality sites, many of which appear not to be trusted.

I have my own test for deducing whch sites may be trusted and which sites may not be. I don't disclose the test publicly because I don't know how accurate it is and I don't want to give away a possibly useful idea to people whom I don't want to help. My test is quick and simple, but even if it's on the right track I doubt it is 100% reliable. I am developing a couple of other tests to see if I can establish a consensus of results.

In the meantime, the continued emphasis on building links in quantity probably only maginifies the problem for most SEO'd Web sites. The more links the SEOs seek out from "tried and true" sources, probably the longer it takes to get sites to move past the Sandbox Effect. There will be differing degrees of success. Some SEOs most likely have very good sources of linkages. Most prbably do not.

Are Supplemental Index Pages 'bad neighborhoods'?

I don't believe so. I think these pages represent documents that have not yet earned trust, but that doesn't mean they are considered to be 'bad'. Matt suggested to one person on his blog that "the best way I know of to move sites from more supplemental to normal is to get high-quality links (don’t bother to get low-quality links just for links’ sake)".

I have more to say about Google's Supplemental Index at my SEO Web site.

Final Word

The bottom line is that we still don't know what Google is doing, but we all agree that they are now being strongly influenced by a need to distinguish which sites can be trusted from those that cannot be trusted. I think there are some highly implausible and convoluted theories being proposed by other people right now. The more complicated a proposed explanation becomes, the less likely it is to be correct. For now, I think Google is looking at aggregate linking relationships to determine where community trust really exists. It's very, very difficult to fake trust from a broad variety of sources.

Simply getting links from free directories, article submission sites, reciprocal links, and other popular link sources will probably gradually extend the length of time new sites require to earn trust if for no other reason than that they will only very slowly naturally attract links from trusted neighborhoods.

The real question comes down to this: if I am correct, or close to correct, in my analysis, how long will it take for spammers and SEOs to develop methodologies that effectively poison the "good" (trusted) neighborhoods and force Google to develop some filtration methodology?

I think maybe a year, perhaps 18 months. Until then, those SEOs who have inventories of trusted link sources will hoard their wealth and be very, very reluctant to share the gold. After all, the more people who know where to get the good links from, the less likely those link source will continue to be valuable.

Google shares the love...

2006-09-07T12:26:00.000-07:00

The more I read about Google's activities, the more impressed I have become with TSETSB. There are some things Google does that I don't approve of.

Two brief rants before we get to the Google raves

For example, I will never forgive them for Web Accelerator, which I continue to block from my network at the server level. Web Accelerator eats up bandwidth and Google has yet to offer an effective means of compensating Webmasters for needlessly wasted bandwidth. In Google's defense, I will say that their technology only incorporates an incredibly stupid standard that was proposed by people who should have known better than to come up with such a dumb idea. Maybe that's why it's so popular, I don't know. But I fear the day is coming when I'll be blocking all FireFox users from Xenite.Org. Unless they want to pay for subscriptions to our content.

I also complained about Google's removing the fetch date from their cache. I get so many hits from Googlebot in my server logs, figuring out when they actually fetched a file is not easy for me. And when other people ask me to do research on their sites, I am now almost completely blinded in one eye thanks to this ridiculous "improvement" from Google.

Google snuggles with the guv'mint

But enough whining and moaning. There are a lot of great things Google has done, is doing, that I can talk about. For example, I notice they are snuggling up with the U.S. Government these days. Adam Lasnik is teaching a class on search optimization to government Web designers at Washington University. And Google (Enterprise) has been named most influential commercial company providing technology to the Federal IT market. It was only a matter of time, I suppose.

But the ever impressive and highly innovative Google Book Search people (who, theoretically, should be on my list of demons because I have published books) have now announced that DIANE Publications has made its entire inventory of reprinted government publications available on Google Book Search. You know, we taxpayers paid for all that data collection and reporting, so it's about time we get access to it. This is actually a public service from Google that should help historians and people who are curious about what sort of publications the government has spent their money on in the past.

Lesson for Business: Share what you do!

What can smaller businesses learn from Google Book Search and Google Enterprise? I'd say that if you have partnerships with larger entities where your services or products are playing a significant role, you should be writing about those relationships on your corporate blogs. Put feature articles on your Web site. Mention your hallmark accmoplishments in your company history page. Tell people what you are doing for others, so they can get an idea of what you may be able to do for them.

After hours with Googlers, innovation, and invention

Innovation, of course, doesn't have to come from the corporate production process. One Googler offers a tip for organizing temporarily necessary cell phone numbers. Leave it to someone associated with search to think of prefixing names in a phone list.

Another Googler provides a fantastic report on Maker Faire, where innovation comes to life. The report will take a week for anyone to evaluate, but it's loaded with details, pictures, and video. Oh, my!

Significant revelations from Google

But now we're getting down to today's good stuff.

Google revives Tesseract OCR

First up, Google Code recently announced that Google had revived Hewlett-Packard's OCR technology (HP retired Tesseract in the mid-1990s). Think this is how Google has been scanning all those books? It doesn't matter, because as I pondered over the meaning of this post for the umpteenth time, today it hit me: Google may eventually be able to read all those graphics people use on their front pages. You know what I mean: the huge image files that say, "Michael Martinez is the best SEO in the world and you really should be paying him to help you rank at Google".

How many SEOs have complained about having to work around those Greeting Card images? Well, prognosticating what Google will do with its technology is not very productive, but if they are not thinking about how to scan Web greeting images and masthead graphics, they should be. Because there are just too many people who don't understand that a search engine cannot index the text embedded in a .GIF or .JPG.

Vanessa explains SiteLinks

The Webmaster Central Blog explains one of those curious SERP features that have puzzled, bemused, and bedazzled SEOs for years: SiteLinks (love the name, btw). SiteLinks are those sets of tightly compacted deep links that occasionally are included in a site's listing.

Google provides four levels of recognition for a Web site:

A simple listing for a single page in relation to the user's query.

Two listed pages, one indented under the first, in relation to to the user's query.

Two listed pages as above, but with an additional tag offering "More pages URL"

One or two listed pages as above, but with a compact list of SiteLinks providing quick access to deeper content

SEOs have lusted after those impressive SiteLinks results ever since they first started appearing. My most important site, Xenite.Org, has so far only achieved level three recognition despite many deep links and deep referrals. A lot of my pages come up in Google SERPs. But it takes more than what I've got so far to hit level four recognition.

NOTE: Some people might argue that having pictures from your site featured above search results, such as for Lucy Lawless, is a fifth level of recognition.

In any event, Google says that SiteLinks are completely automated. Maybe they are, but if any SEOs can figure out how to trigger their generation in SERPs, I think those SEOs will make even more money than before. Frankly, I haven't really tried to figure out the process.

And now, for the gold: Sharding

Do you know what shards are? I only have the vaguest idea, myself. I've watched a number of videos of Googlers making presentations. I've read some technical stuff. But I've never seen a shard in action. Google's database is so large it cannot all be contained on one server. Google reportedly uses up to 1,000 PCs to resolve any query. The database is spread out across some or all of those PCs in what Google calls "shards".

Last month, a Google went to a BarCamp and made a presentation called Scaling Data On The Cheap. Yup, he talked about shards.

Slide shows don't tell you a great deal when you cannot hear what the speaker has to say. But we can infer a few (possibly very incorrect) ideas from the slides. For example, it appears from one slide that a table could be replicated in multiple shards, split across multiple shards, or comprise a single shard by itself.

Google's original architecture (most likely no longer in use, at least going back to the January 2006 Big Daddy update, if not earlier) used many tables. There would have to be one or more master tables just to tell the various programs where all the other tables are. The paper says they had identified about 14 million words. Each word would have to have its own index. Rare words (occuring in the fewest documents) would have the smallest tables.

I can envision some programmatic advantages to replicating rare word tables across multiple shards, pairing some rare words with others in specific shards. And obviously large tables for the most common words would probably have to be split across multiple shards.

There is probably only minimal value to an SEO in knowing and understanding how shards actually work, but the slide show implies a great deal of redundancy has been built into Google's system architecture. It's like they have a lot of floppy database thingees that they lay partially across each other like blankets.

Well, it's food for thought, but I've already spent too much time on this post.

Getting down to Google Base icks...

2006-09-06T00:13:00.000-07:00

Google's AdWords Blog actually shares some interesting information. I don't mean that to sound like a bad thing, but since I don't run AdWords campaigns I don't really read the blog. Shame on me.

So, in Why is the location under my ad?, they explain that if you target your ads by location, then users who are identified by the Google system as coming from that area will be told the ad is relevant to their community. I like that. Can't imagine why anyone would complain, but sometimes business people have a very different set of expectations from me.

In their Get your products into our search results with Google Base post, they share the following advice:

Your site may already be included in our crawl index, but we want to ensure that you also know how you can supplement these results with Google Base - you can submit the products or services that you offer directly to Google Base making them eligible to show on Google.com when a user searches on a relevant query.

What the heck? Seeing as I'm now promoting my SEO Consulting services on a full-time basis, I thought I'd give the system a try.

Unfortunately, the user interface burped. I created a few attributes for my ad, picking from their list suggested attributes. When I clicked on PUBLISH, the system came back and said there was a problem. They lost the label for two of the attributes and combined their data into one unnamed field.

I didn't feel like trying again, so I edited the surviving data, put in a new label, and clicked on PUBLISH again. This time the ad went through safely and I'm good to go for 30 days.

I appreciate the Googlers' giving me advice on how to promote my consulting services, but as a programmer with many years' experience I couldn't help but cringe when I saw the bug. I hate it when I find bugs in my own software after it's been deployed. That's just one of the risks programmers face, but it's still annoying.

Browsing further through the blog, I noticed their Printable coupons for local businesses post. OH..MY..GOD.

Why hasn't anyone in the SEO community made a big fuss over this feature? I know some people who need to take advantage of this service.

Hm. I wonder if I can do that....

Anyway, the AdWords blog has turned out to be very useful and interesting to me just in a few minutes' time. That ain't bad for clicking on a previously unvisited link.

Well, in other useful Google blogging, Vanessa Fox discusses better details about when Googlebot last visited a page. She says that Google Cache will now reflect when Googlebot last sought information about a page, rather than when it was actually downloaded.

Um...that's not very helpful to me. I can see how some Webmasters may be pleased with knowing that Googlebot stopped by on September 1, but it won't explain to them that they are looking at a page copy from April 26. Let me explain why this can be a problem.

Googlebot comes by on April 25, fetches my page, and then I update it on June 12. Googlebot dutifully grabs the page on June 12 and then my server crashes. I restore from a backup made on June 11 and my server will think the page hasn't changed when Googlebot comes back on June 15.

Now, ideally, I want my June 12 version of the page. But for reasons beyond my control I cannot reproduce that page until, say, August 15. If Google dutifully indexes and caches the page in a matter of days, they are out of sync with my restored Web page.

Does this scenario happen? Well, server crashes happen all the time. It's anyone's guess as to how backups are restored and how the servers figure out whether to send a code 304 (Not Modified) or not. But it's a hole in the methodology and the blog doesn't address it to allay my fears and concerns.

I can also tweak my server and screw up its ability to send a code 304 at the right time. What if I accidentally configure my server to send a code 304 every time? Now Google's cache is telling me it visited the page on August 18 but I'm still seeing the restored June 11 backup. What's up with that? After August 15, I think I should be seeing my August 15 update, but because I've misconfigured my server, Google says it visited the page on August 18 and grabbed the pre-August copy from the restored June 11 backup (in truth, it grabbed nothing but I don't know that).

My point is that most Webmasters don't read the Google blogs and they are not going to understand what they are seeing with these dates.

In my opinion, the reported date needs to be the date the file was pulled. If Google really feels anyone needs to see from Google's side when Googlebot last dropped by, the ideal thing to do is show both dates (in my humble opinion).

Sorry, Vanessa, but this latest improvement is an "ick" in my book.

Introducing...Google User (or not)

2006-09-05T06:48:00.000-07:00

Seems like Google is trying to bend over backwards to prove its services are really all about the user experience.

A few days ago, they invited Philipp Lenssen to write about '55 Ways To Have Fun With Google'. In one fell swoop, they pretty much promoted everything about their business through a custom-written user testimonial. Will Philipp sell a few more copies of his book? Probably.

The Google Enterprise Blog featured a Map of Whole Foods. Talk about some major exposure. Okay, maybe a lot of people don't read the Enterprise Blog, but it's the flagship of the Google star fleet (how could I let an opportunity for a pun like that pass me by?).

The Whole Foods app entry should drive some curious traffic to the company's Web site. Maybe a lot of those people will be interested in organic foods. I don't know. But it's exposure that is hard to get in today's search-dominated Web.

Google Book Search, possibly the most innovative blog in the Google stable, has announced that you can now add Google Booksearch to your site. Folks, this is a significant tool that many hobbyists will latch on to. Business sites will eventually figure out ways to use it, too. I've already got some ideas rolling around my head. I just need time to site down and play with it ("site down" is not a typo).

An earlier blog post from Google Books a couple of weeks ago also has me thinking. They announced Authors@Google. You know, anyone with a video camera can now create a featured speakers program that is hosted by YouTube, Google Video, and similar services. Just stick your company logo on a wall somewhere, stand in for a minute to introduce your guest speaker, and then let him or her plug a book, business, or concept.

Did I say that Google Book Search is the most innovative of the Google blogs? Let me put this as delicately as possible: if you're an SEO and you don't read this blog, you're an idiot. There, I've said it. Many SEOs think I believe all SEOs are idiots. Well, that's not true. Just SEOs who think PageRank converges to an average of 1 and SEOs who don't have sense enough to read Google Book Search. You don't stay ahead of the pack by running with the crowd. Get out there in front and take some chances. Read things the other people don't read.

Speaking of Google Book Search, Philipp Lenssen's guest post on the official Google blog got me to thinking about Google's Public domain treasures, where you can download public domain books. A savvy Web marketer would publish a book and make it freely available for download from Google Books. It should not be long before we see books promoting "Buy my services!" on every page becoming available on Google Books. If only I were as smarmy as some of the other Web marketing gurus out there.

When Google renamed Site Maps as Webmaster Central, I noticed a lot of snickering among SEOs because it just didn't seem like a Webmaster Central type station. Nonetheless, Vanessa Fox is posting some great content on the Webmaster Central blog. Her article on how accented characters and interface languages impact search is a must read for anyone dealing with international language sites (and custom language sites, assuming you want to optimized for constructed languages that use accented characters).

A lot of recent Google blog posts have emphasized the user experience and how users can benefit from Google's services. What we can take away from this sampling of posts is that any business with a service or product can enhance its visibility and traffic by providing insightful, innovative, and intriguing tips and suggestions on how to utilize those services and products.

And for those of you still living in the SEO dark ages: such content produces a lot of linkage.

Remember: it's all about the user experience. Make that a good experience, and the users will love you.

More Google tips from Matt Cutts

2006-09-01T07:21:00.000-07:00

Matt Cutts went on a tear this week and posted several interesting items on his blog about Google and other search engines. In Video: Datacenter comments Matt listed the major points he covers in his latest video. (Philipp Lenssen has transcribed the video and Matt offered some clarifications.) What SEOs should take away from the video includes:

Data center IP addresses can point to more than one data center

Google redesigned its supplemental index earlier this year to use a different architecture from the main index

The site: command only presents an estimate of what Google has indexed

Another software infrastructure update that affects crawling for the main index is being gradually rolled out

In the comments section for his post, Matt elaborates on some other issues:

To Google, there is no distinction between "internal" and "external" backlinks

Finding pages in the supplemental index probably only indicates a lack of PageRank and links for many people

The main index is smaller than the supplemental index

If you see differing estimates with the site: command, the lower numbers are more likely correct

In Handling noindex meta tags Matt describes a case study he did. He found that if you include noindex in your robots meta tag, both Ask and Google will exclude the page from their indexes completely. MSN will index the URL but nothing else. Yahoo! ignores the instruction.

In Scoble visiting the Plex Matt notes that a URL with "cns!" followed by a long string of numbers, letters, and characters may look like a session ID to Google and other search engines. He also reiterates the helpfulness of repeating keywords from the title element in the page's URL, a point he has made in previous discussions. In the comments, Matt elaborates by saying, "including the keyword in the url just gives another chance for that keyword to match the user’s query in some way".

I've incorporated some of Matt's comments in these posts and others into new sections on my (new) SEO Consulting site. Look at the SEO Fundamentals, SEO Theories, and Search Engine Facts pages.

Google Page killed my Analytics

2006-08-29T06:39:00.000-07:00

I was curious about whether it was now possible to sign up for Google Pages. After poking around, I discovered that if you have a Google Mail account, you can get a Google Page account. To get a Google Mail account you have to have a cell phone.

I have a cell phone.

So I gave Google my cell phone number and got my Google Mail account. I gave Google my Google Mail account and got a Google Page account.

Then, just to make sure everything was still good with the rest of my Google activities, I tried to log into my Google Analytics account. My old login and password were not accepted.

That was not good.

So then I decided to log into my Google Groups account. Google explained that all my subscriptions had been transferred to my new Google Mail account (I didn't recall them asking me if I wanted my other Google accounts to be affected). Google now gave me the option of not transferring those subscriptions, and I took that option.

At which point Google informed me that I would have to recreate the old account in order to preserve the subscriptions. So I recreated the old account, verified that my Groups subscriptions were preserved, breathed a sigh of relief, and then remembered that I had problems with Google Analytics.

Immediately upon logging into Analytics, I was informed by Google that all my old data was gone, but that it might be restored if I nicely requested reactivation of the old account.

I've been tracking Google Analytics' performance since the beginning of the year in a year-long evaluation of the service. I have their code installed on many pages. Naturally, I sent off the request to have the old account reactivated and am still waiting to hear back from them.

But now 8 months' data is lost to me, perhaps forever, and Google didn't even so much as put a big warning box in my face when I signed up for the Google Mail account to advise me that I was about to (almost irrevocably) alter my many relationships with Google.

I think a somewhat less streamlined approach to consolidating users' Google experiences would be in order.

Matt Cutts to SEOs: It's the content, stupid!

2006-08-24T15:05:00.000-07:00

Matt is too nice a guy (or too professional) to be insulting and derisive in his blog or other online/public comments to the SEO community. But he has dealt the self-deluding SEO community a long-needed intellectual bloody nose by showing them that one doesn't need to rely upon links to rank well for expressions in Google.

In his August 21, 2006 blog post offering SEO advice, Matt demonstrated how one can easily rank for a popular expression ("SEO") while explaining how he used a previous blog entry to target a longer expression.

Delivering a 1-2 punch to the generally ignorant SEO community on the power of content hasn't gone unnoticed. In comments posted as followups to his blog, and on numerous blogs elsewhere, SEOs have been rationalizing how Matt is bending his own rules and suddenly revealing that "content ranks again" (tip: it always ranked, except where it was overwhelmed by massive numbers of links).

Just because SEOs have trained themselves only to rank by endlessly (and generally needlessly) building links upon links has never meant that Google was scoring only by links, mainly by links, or wantonly by links. In fact, the overemphasis on linkage has long been a self-defeating strategy for many SEOs, who have devoted much of the past three years complaining about how it's becoming more difficult to garner high rankings quickly through linkage.

This is the power of momentum in ideology: once nonsense takes on the authority of fact, the fiction outweighs all sensibility and reason. It's highly doubtful that Matt's SEO 101 lesson (develop content that is relevant to a query first and worry about linkage later) will part the waters, but it should at least make a splash that is heard around the world for a brief moment in time.

The momentity of the task is made more clear by the naive comments posted by Greg Boser (aka "Webguerilla") in his Amish Gokarts and Mini Bike Furniture post on August 23, 2006. Within a day, dozens if not hundreds of SEO blogs around the world have picked up Boser's inaccurate analysis and hailed it as another marvel of great SEO reverse engineering. Boser's conclusion that a host serving two domains from the same IP address is responsible for a Go Kart site ranking well for "Amish furniture" fails to take into consideration two links that Boser clearly didn't know how to find.

When one of the A-listers of SEOdom fails to pinpoint two or three obvious and easy-to-find links that clearly demonstrate why the Google algorithm would mistake a site for being relevant to "Amish furniture", it's time to ask if these people are really worth the money they are being paid by good businesses for their advice and consulting services.

Search engine optimization is still very much a smudgy art. Hopefully, search engineers like Matt Cutts will continue to emphasize the importance of looking at all the factors that have been openly documented for at least 8 years.

The SEO world has blathered on about inbound links and PageRank, completely blinding itself to the importance of outbound links and on-page content. Frankly, this lesson has come about five years too late.

Let's hope it doesn't take five more years for the next lesson to rattle the cages and foundations of the myths that SEOs have built their reputations on.

Magic healing powders

2006-08-23T15:12:00.000-07:00

Wrong blog. If you followed a link here about "magic healing powders", that should have led to this magic healing powders entry, not the page you're now looking at (which I only created when I found the mixed-up URL).

Sorry about that, Chief.

Check out Google Says... for comments on Google and Googler announcements, discussions, whatever and check out The Semi-Official Blog of Michael Martinez for my more diverse thoughts and bloggings.

Google sends Googlebot to obedience school...

2006-08-23T14:59:00.000-07:00

Vanessa Fox gives a detailed tutorial on how to work with Googlebot. One of the little gems buried in her list of do's is the following:

What should I do if Googlebot is crawling my site too much?
You can contact us -- we'll work with you to make sure we don't overwhelm your server's bandwidth. We're experimenting with a feature in our webmaster tools for you to provide input on your crawl rate, and have gotten great feedback so far, so we hope to offer it to everyone soon.

While many of you may wonder who would complain about Googlebot coming around too often, anyone with a very active forum or other large dynamic site has probably felt the pain of being deep-crawled by Google and Inktomi.

Deep crawls bring servers down to their knees. Your user experience degrades in a matter of minutes or hours. Getting the robots to back off is a painstaking process that may require hours' or days' worth of patience. Once you've gone through the experience, you generally don't want to go through it again. But if you actually get deep-crawled once, chances are pretty good you'll be deep-crawled again.

Google gets my applause for acknowledging the tribulations Webmasters endure from being crawled. It's great to see that they may make a more responsive automated tool available to us to help slow the speed of deep-crawls. But that's still not quite far enough in my book.

Google really needs to let Webmasters opt in to bandwidth-hogging activity, rather than force us to opt out. While Googlebot is not always a nuisance, Google's Web Accelerator remains on my network's banned IP list (a very short list) because I just cannot afford to let a lot of gung ho Google users draw down my server's resources.

You're making progress, Google. Please don't stop here. Bring the process home by giving the Webmasters more control over how you utilize their resources.

Google can't find the big SF conventions?

2006-08-23T06:02:00.000-07:00

Google will be partying at WorldCon, a small convention in California. If they were seriously interested in hitting the major SF Geek spots, they'd have headed east toward Atlanta for Dragon*Con.

Well, either the Google recruiting budget is a little sparse or else they are just embarking upon the SF fan experience and have a lot to learn about where to find the major Geekfests. Googlers, here's a tip: just Google for...er, search for largest sf convention in America.

Works like a charm, dudes.

Google closes off free, public access to scholarly literature

2006-08-22T09:47:00.000-07:00

The official Google Blog today makes exploring the scholarly neighborhood sound like a walk in the park. They have improved the service by providing links to related documents.

What they don't tell you, however, is that the search service now only links to front-end citation pages for article archives that charge membership fees or download fees.

Many of these scholarly articles have been freely indexed on the Web in the form of .PDF files and .HTML pages. If they are still there, Google has apparently now helped to bury them deeply back in the "invisible web", that segment of the World Wide Web that surfers mostly cannot reach through search engines.

Since the interface for Google Scholar has just been redesigned, one can suppose that maybe an entirely new crawl is required to repopulate the index. However, relying on fee-based article archives is hardly the best way to bring humanity's knowledge closer to people's finger tips.

How big is your sitemap?

2006-08-18T14:19:00.000-07:00

Although some people continue to report losing rankings immediately after uploading a sitemap to Google, it appears that Google Sitemaps is picking up steam and rolling along.

Googler Amanda points that you can include up to 50,000 URLs in a sitemap no larger than 10 megabytes if you have a lot of content that needs to be crawled. If you have more than 50,000 URLs, don't panic. She mentions that you can upload multiple sitemaps and create an index for them.

Cool.

It would be interesting to see Google respond to the various reports of lost rankings outside the favored SEO forums. They need to make an authoritative, definitive statement (even if they end up correcting themselves later on) about the issue, because many people claim to have yanked their sitemaps from the Google system over the past few months in order to restore rankings.

Technically, one incident proves nothing. That is, if you need to prove cause-and-effect, you should

1) document your rankings (multiple queries is good) with screen captures

2) upload a sitemap

3) document your lost rankings with screen captures

4) remove the sitemap

5) document your restored rankings with screen captures

6) upload the sitemap again

7) document your once again lost rankings with screen captures

8) remove the sitemap

9) document your once again restored rankings with screen captures

If you can present that kind of evidence, you will have gone a long way toward showing cause-and-effect. But you're not done, yet.

It is possible there is an error in your sitemap. What sort of error? I don't know. It could be an undocumented aspect that Google's engineers haven't taken into consideration. In software design, it usually only takes 1 misplaced or unexpected character -- a single byte of data -- to completely render six months or two years' worth of work into stuttering gibberish. Such anomalies are common and usually are easy to fix, but they only get fixed when you get defensive programmers to stop being defensive and look at your clear-cut evidence.

Another possible reason might have something to do with your robots.txt file. Google does, after all, use different crawler programs. It may be there is something in one of them that chokes on certain things in robots.txt.

I recently learned that because I was blocking certain Google IP addresses (for their Web Accelerator tool) I was unable to validate in Webmaster Center (what used to be Sitemaps). If you are cloaking, you may have a similar issue. Uploading a sitemap on a cloaked site may not be the smartest thing to do. It's just a suggestion, and one I am not in a position to test.

But these are a few things people with lost rankings (that they feel they lose because they submitted to Google Sitemaps) can look at: size of map, number of URLs, construction/syntax of the map, construction/syntax of the robots.txt, and whether a site is cloaked.

Google Base solves a massive, common problem in two blog posts

2006-08-15T14:08:00.000-07:00

One of the many common questions I see asked in SEO forums and online marketing groups boils down to: "How do I get Google to index thousands of product listings?"

The Google Base Blog has just solved that problem in two fell swoops.

On August 9, Sundar Subbarayan described in "Taking care of custom needs" how they took two feeds from a retailer with thousands of products and multiple store locations to generate a massive online catalogue for the retailer. The product feed contained lists of store IDs indicating where the products are available. Essentially, Google Base facilitated a classic table join (I'm talking about database tables, not HTML tables) to produce a virtual table.

Yesterday (August 14), Clint Guerrero mentioned in "Behind the Scenes: Life on the Google Base Support Team" that "more attributes we have, the better we can match search queries to your content. Attributes also enable us to include your items in refinement searches."

Jennifer Hyman explained how attributes can help in greater detail in her August 8 post, "Putting your attributes to work".

The potential for abuse, as with all great ideas and services, exists and I certainly hope people don't misuse Google Base's Attributes. But clearly if you need to manage a large online inventory listing, Google has provided some very useful technology. Google not only provides an extensive list of attributes to use in Google Base, you can define your own.

By branding your own queries through advertising, publicity, and other channels, you can direct targeted traffic to your Google Base-listed products and services. These tips directly address a very common need, and they should help cut across many misconceptions about what it takes to get indexed and to become visible to Google searchers.

Google on links at SES San Jose 2006

2006-08-11T15:09:00.000-07:00

Over at Spider-Food I wrote that the whole article just needs to be encased in glass and put up on everyone's wall in summarizing my impression of S.E. Roundtable's report on the Search Engine Q&A On Links.

This marks, I think, the first time that Adam Lasnik (affectionately called MiniMatt -- love the name), Matt Cutts' new assistant, has represented Google in front of the SEO world in a formal capacity.

Adam's opening remarks are summarized thus:

We are all interested in having webmasters make links that are useful for their users. It is not a numbers game, he said. He said the optimal number of links is 42, of course he is joking. It is not a numbers game. It is about making your links relevant. A garden site with links to mortgages, is not relevant. Do your links pass the "smell test" or the "common sense test." He then said if all the links say the same thing about you, then something is a bit sketchy.

I love that: "It's not a number game." It never has been, but thousands of SEOs around the world try to fit that square peg into the round hole of search engine optimization every day. Unfortunately, they've been spoon-fed bad information by the most popular SEO forums: Search Engine Watch, WebmasterWorld, SEOChat, and others that are high traffic, high visibility sites. These forums have been home to some of the lamest SEO gurus in the world for many, many years. The people who really know the score seldom speak up and when they do they are often ignored or shouted down.

Now, no one is right all the time on any subject. Some of the people whose opinions I've long respected say things occasionally that make me cringe. And I am challenged on an hourly basis to back up what I say on just about any topic, so the rule of thumb (among the SEO CrapMasters and many good SEOs) seems to be that they usually don't agree with anything I say.

Still, I think the answers people got from the search engine reps on this panel were eye-openers for the crowd. The only problem is, every time the voices of authority shoot down the B.S. that permeates SEO thinking, the SEOs immediately go on the defensive and accuse the search engine reps of being liars. I have no doubt that if anyone points to this Q&A session in various SEO forums, there will be hemming, hawing, and people looking down as they say, "Wellll, that's really not the case."

Right.

Like, we're supposed to get the facts from people who don't know any better than a drunken homeless man which way is up in the search engine world.

People often ask, "Michael, why do you bother reading the SEO forums if you so disapprove of what they say?"

There are three reasons for everyone interested in search engine placement to read active SEO forums.

Search engine representatives speak up in a couple of them (they should speak out in more forums, actually)
The crescendo of whining that erupts every time a major update occurs is the best indicator of an update process on the Internet
You occasionally see links to valuable articles, new services, and other things

Just because most SEOs don't know their heads from a hole in the ground doesn't mean they don't occasionally say something worthwhile. Even I occasionally say something worthwhile. I just put my foot in my mouth more often than most of you, so I look like I average more successful hits than many people but the truth is that I'm just doing it on volume.

No one who works outside a search engine is qualified to be telling people what the search engines are really doing. I mean no one. Not Michael Martinez. Not Danny Sullivan. Not Shari Thurow. Not Dan Thies. Not Eddie Lopez the Houston Salsa dance instructor (who, so far as I know, has no interest in this field).

I think Rand Fishkin is a creative genius and he'll probably dominate this industry in about 10 years if he doesn't derail himself in some catastrophic way. But Rand goes off the deep end sometimes, and like the rest of us he's shooting in the dark.

So every time a search engine representative makes a definitive statement about how search engines work, I sit up and take notice. I have to, because inevitably I'll find myself involved in an online discussion where some offended SEO who doesn't appreciate what I have to say will insist, "Prove that. Show us where X said Q-Z-Y."

Offended people often attempt to be clever by demanding that you produce specific language. You cannot be right about anything if you cannot back up what you say with unanticipated words, in their illogical thinking. That's taking an emotional approach to disagreements. Frankly, I couldn't care less what people think of me personally. If they are going to promote themselves as involved and active in a community that engages with all levels of the business world, I'm going to hold them to some very high standards of performance. Those are the same standards of performance that the business community has held me to through three decades of technical service.

So the point of this blog is that you don't need to agree with anything I say about the search engines and what they may be doing. What you do need to do is let go of your ego, stop insisting you have a point when all you're doing is blowing smoke, and pay attention when the search engine reps speak. They have all the financial incentive in the marketplace to help us place good content in their results. Lying to the SEO community and misleading the SEO community are not successful strategies.

I am sure the search engines retain strong reservations about the SEO industry in general because it's so closely tied to search engine spamming. That close correlation will never go away, since spammers inevitably take whatever works well and beat it to death.

Nonetheless, when a Googler speaks, I listen. The SEO community would do well to listen more to the Googlers, Yahoo!s, MSNers, and Askers than to their whiny friends who demand proof after proof and then pretend it doesn't exist after you link to it a dozen times.

If SEOs want to consider themselves to be professional, they had better earn the recognition by acting professional. So far, a lot of them have failed to make the grade in my book.

How they treat the technical revelations from search engine reps like Matt Cutts, Adam Lasnik, and their peers both at Google and other search engines will go a long way toward revealing just how much SEOs really know about what they are doing. The usual pattern is: applaud these guys when they speak (or post something), and then go back to repeating the same debunked crap almost immediately.

Those who don't learn from the search engines are doomed to be the targets of my endless rantings against idiot SEOs. And this is why I think Google, Ask, MSN, and Yahoo! should all give serious consideration to certifying search professionals. They are the only organizations really in a position to know who is right and who is wrong, and what the acceptable standards of performance should be.

Google teaches optimization...sort of

2006-08-10T14:44:00.000-07:00

Inside AdSense has been sharing page optimization secrets throughout the month of August. Their tips are focused, specific, and in some cases very detailed. They've even created an AdSense Help Group where you can share tips and ask questions.

It looks like Google is starting to get more directly involved in SEO education after all.

What the AdSense team is not teaching you is how to rank highly in Google's results, but it appears to me that some of their tips leave back doors open. That is, I don't think they fully appreciate just how vulnerable their algorithms can be to certain tactics they haven't yet publicly denounced.

While I applaud the AdSense team for being open and forthcoming (Google does have a financial incentive to do this, after all), they will probably get stung a time or two. It's almost guaranteed, and I hope they are allowing for that. It would be a shame to see Google pull back on this kind of advice because it really does offer them a bully pulpit for teaching most people how to do things that are generally acceptable to Google.

I was going to outline some of the vulnerabilities I see in the tips, but I had two reservations. First, I haven't tested them all, and it would be rather foolish for me to say, "Here is a vulnerability" when, in fact, it's already been blocked. Secondly, I don't particularly care to teach people how to game the system.

Content should be king (in my opinion), and I was glad to see this article posted. I cannot help but wonder if Google isn't in the early stages of developing technology to enable multi-windowed pages. I'm not talking about frames. Rather, I'm talking about creating virtual windows in a static page that have a mixture of static and dynamic content.

You can embed small gadgets now, such as ad boxes, scrolling headlines, etc. But the gadgets operate independently of each other. Is Google working on tools that will let you coordinate 2-5 gadgets in one corner of your page, 2-5 gadgets in another corner, etc.?

They probably won't call such a methodology virtual windowing (sounds too much like Microsoft). Maybe they'll call it zone management or something like that.

We'll just have to wait and see.

Google Book Search shows how to use blogs for marketing

2006-08-09T11:48:00.000-07:00

A lot of SEOs and self-promoting business operators still don't appreciate the full power of blogging, as it applies to promoting your core products and services. The Inside Google Book Search blog provides a very clear example that would serve many industries as a great template for what to do with corporate blogs.

Google Book Search is under a great deal of pressure. There are legal challenges to the service and authors are not clear on what benefit it brings to them. Nonetheless, Google's product management team are releasing a steady stream of posts that demonstrate to people how the service can best be used.

American history buffs and poker players may appreciate Aerielle Reinstein's Dead Man's Hand anecdote complete with links to sources. To be honest, I never knew Wild Bill Hickock had been shot in the back of the head.

Bethany Poole writes about early 19th century proposals for information management. One expert at the time estimated there were only 2 million books in the entire world. We have libraries with more than 2 million books now.

Adam Mathes writes about Aesop's fables. What's cool about his post is that you can look at a book from 1885 without having to dig through musty piles of old books in estate sales and used book stores.

This is exactly the way business blogs should be managed: give people useful information that demonstrates how your business resources can help them. That's all there is to it.

And my apologies to SEOMike at Spider-food. I just told him earlier today that I would not be writing about SEO-specific topics here. Still, it was too good an opportunity to pass up. I think Google deserves a little credit for the things I agree with in this blog, given how critical I've been lately.

The flaws in Google's Report on Third Party Click Fraud Auditing

2006-08-08T13:45:00.000-07:00

Google is naturally feeling defensive about all the criticisms directed at their click fraud detection methodologies. Part of the issue is that, in order to protect advertisers' interests, Google must keep their detection methodologies as secret as possible to prevent fraudulent click generators from taking advantage of potential flaws in their algorithm.

On the other hand, Google is still looking at the issue in a very misguided, naive fashion. They are attempting to place counter-blame against the blame laid on them. Their AdWords Blog touts their Report on Third-Party Click Fraud Auditing as "troubling findings".

Well, the scenarios that Google's engineers propose for explaining what they call "fictitious clicks" are certainly plausible. However, they fail to make the case that their scenarios are the only plausible explanations for these clicks. So Google is in no better position than the third-party auditors, which means that advertisors are left in the middle, asking who is right. The most definitive argument Google presents in its report is the statement:

Correctly determining that the latter events are not
caused by a Google ad click and are the result of subsequent user browsing behavior requires a more complex analysis. For example, advertisers could possibly analyze their logs to realize a user came from Google, went deeper in their site, and their subsequent request for the homepage is likely the result of user hitting the back button on their browser.

"likely"? Likely based on what statistics of user behavior? I can tell you that I sometimes hit the back button and sometimes I reload a page manually by trimming the URLs, and sometimes I go back to a search engine and rerun the search (yes, to find the same site again).

But click fraud is supposed to be big business for some people. If you're going to write a script to click on ads for yourself, you may get as sophisticated and convoluted as the technology permits or you may be very simplistic or you may fall somewhere in-between. Without access to the clicking technology itself, both Google and the third-party auditors are guessing at what is happening.

Has Google proven that fictitious clicks can occur? Yes. Have they proven that all server logs containing the data they analyze are documenting fictitious clicks? No.

They go on to question the validity of AdWatcher's deductive reasoning:

Therefore, in addition to inflating click counts similar to the previous page reload sequence, AdWatcher also attributes ad clicks on other ad networks such as Yahoo, and ad clicks on other advertisers in the number of fraudulent clicks that it reports to an advertiser. So it is not uncommon for a single Google ad click to be portrayed as tens of fraudulent events.

Okay, what the report has identified is an arbitrary decision by AdWatcher to label a collection of clicks across advertising networks as fraudulent (or potentially fraudulent) if they exceed a threshold. Google objects that AdWatcher is misreporting all those clicks as clicks only on Google's ads.

While this is a legitimate concern, they fail to document how AdWatcher actually accounts for the suspicious activity in their reports. Appendix B includes an AdWatcher case study. Google makes a great case for the third party auditors over-reporting click activity when compared to Google's logs. But there is a disconnect between Google's earlier claim and the data they present in the case study.

That is, Google says, "We reported and charged for X clicks but the service reported X+Y clicks that we never sent through". They have made the case for sloppy accounting on the third party auditors' part. That is unquestionable. But they aren't backing up all their assertions with a clear path from point A to point B.

Have they made the case for the accuracy of their own click fraud detection? No.

What would have helped this report would have been relevant excerpts from advertiser server logs (even with IP addresses and keywords blotted out). It would also have helped for Google to reveal data from their own logs.

Instead, they summarize data without showing that their own accounting is valid. This kind of sloppy record keeping doesn't make a convincing case for the core question: is Google accurately identifying and discarding a significant percentage of fraudulent clicks?

We still don't know.

What does 60 per cent really say about Google?

2006-08-03T10:36:00.000-07:00

Bill Tancer published the July 2006 search volume breakdown showing that Google processed an estimated 60.2% of all U.S. searches for the month of July 2006. People cite these rising statistics frequently, but I never see any in-depth analysis of the activity behind the numbers. Perhaps I have to pay someone $1500 or more to see the raw data. That's not going to happen.

Given how much search activity is generated on Google by people checking their PageRank, their rankings, their visibility in Google ("Googling themselves"), competitor rankings, coverage of their content by Google, and robots it should be no surprise that Google receives a tremendous amount of traffic.

But how much of that traffic is really useful? How much of Yahoo!'s traffic is really useful? How much of MSN and Ask's traffic are useful?

My feeling is that Google has a lower overall percentage of Human Need-based Traffic. By Human Need-based Traffic, I mean queries where someone actually wants to find something useful for their personal benefit. Maybe they are researching a future purchase, or seeking an online community to join, or looking for interesting news and gossip, or maybe they want to buy something.

What Human Need-based Traffic does not include are all the positioning-centric queries that online marketers generate for statistical purposes, all the automated queries performed by robots for the purpose of scraping results or generating advertising clicks, and vanity queries.

Need-based traffic is really what should be measured. Is that what we are seeing in these statistics, though? I think that Google still dominates Need-based searches by a wide margin, but not by as wide a margin as it dominates overall searches.

As MSN and Ask become more visible, they may be subjected to more positioning-centric queries than in the past. If that proves to be the case, it should shift their overall market share. There may or may not be a correlation to advertising revenues. It would be interesting to study where the positioning-centric queries come from.

Google school documentary on the Web

2006-08-03T06:25:00.000-07:00

Google employee Reza Behforooz points to an Australian documentary about Google called Behind.The.Screen.

The show lasts an hour but includes interviews with Google Superstars and lots of footage of the Googleplex. The production quality is about what American students would expect from school movies.

It's only a matter of time before Hollywood starts optioning the Google story....

How do you say that again?

2006-08-02T10:39:00.000-07:00

Back in April, Google Research announced their beta Arabic-English translation tools. I performed a simple translation test, one which exemplifies just how difficult it is for people to use online translation tools. Let me share an anecdote with you first before I reveal the test and my results.

A few years ago, I had a very popular Web site. I called it Parma Endorion. Originally created in the fall of 1996, Parma Endorion was just a collection of a handful of essays I wrote about randomly selected topics concerning J.R.R. Tolkien's Middle-earth. From 1996 to 1998, I received hundreds of emails from teachers, librarians, and students around the world asking me how they could print out the essays (I had deliberately made this difficult to do). It finally dawned on me that I should stop being so intellectually proprietous and let people print the essays.

So in 1998 I redesigned the site to work more like a book (parma is the Elvish word for "book" in Tolkien's invented languages). Well, my email exploded with thank yous for a while. And then something wonderful happened. People started linking to Parma Endorion all over the place. Problem was, new research was showing me that the essays needed serious updating. And my readers wanted a sequel to Visualizing Middle-earth I didn't have time to write. So in 2001 I arranged with Matt Tinaglia to update Parma Endorion and offer the third edition as a free eBook. All this is pretty well documented.

As the original essays had been translated into Polish and Italian, I thought it would be cool to translate the eBook. I contacted various overseas Tolkien groups and asked for help. I used a well-known online translation tool to write my letters of invitation. I translated the letters sentence-by-sentence, translating them back to English, repeating the process as often as necessary -- changing words where the translations didn't work -- until I found some consistency.

It's a brutal technique but it works for the most part. However, I just didn't pay close enough attention to the Spanish language translations. I can actually read Spanish to some degree. I used to read the Miami Herald's Spanish edition every day, so I really had no excuse for what happened. But I didn't notice that the tool had translated "fans" (as in devoted readers of Tolkien's books) to "ventiladores" (ventillators -- those rotating things that push air around). Well, when the Spanish ventiladores stopped laughing, one of them wrote back and said, "Yes, it's obvious you need help, and we'll be glad to help you."

Thus was born the Spanish translation of Parma Endorion.

So, what is the significance of all that? English has a fairly standardized spelling system. We have differences between the United States and areas of the British Commonwealth. For example, we write "color" and they write "colour". But for the most part English is very standardized. One area of notable exception, however, is the rendering of certain foreign languages into English. Arabic names in particular have multiple renderings in English. Take the terrorist organization Hezbollah. Or is that Hizballa? Or was that supposed to be Hizbullah? You can see the variation in spellings just by switching news sources in your Web browser.

So what does Google with these kinds of variations in spelling? I decided to type a phrase into the English to Arabic tool using a variation of Hizballah's name. The Arabic to English tool returned the exact English meaning of my phrase with a different spelling of "Hizbullah".

I wondered what their authoritative source for that spelling might be. I suppose there may be a book of translation standards running around, but even the Israeli news media cannot agree on how to render the name into English. The Jerusalem Post uses Hizbullah (like Google) and Haaretz uses Hezbollah. So what does Google do to normalize English spellings of non-English words?

While it may seem to some people that I'm getting lost in the details, the SEO world should pay some attention to what Google does with its translation technologies. Spellings are only one aspect of the challenges that face translators. Idiom -- the way we form phrases -- causes an even bigger headache. Today we say "I'm down with that" to mean "that's cool by me" which used to be "I'm okay with that" which replaced "I most heartily agree" which was subsequent to "I approve in all aspects".

When you think about how the Google tool translates Web pages, which may or may not use non-standard idiom, or obsolete idiom, think about how phrase normalization will become very important for Google. If they can accurately render whole passages of text into foreign languages (better than the older tool we're all so familiar with), Google will have taken online translation to a new level.

Furthermore, the significance of idiomatic translation is that a core relevance standard can be established. Think of a universal language that underlies all of our human languages. Linguists have been seeking the means of tying all human languages together for decades, perhaps centuries. They are not really close to doing that, but once they achieve a Unified Human Language Theory, they'll be able to offer translators new techniques and tools for determining what unusual passages may mean.

"may mean" is in itself significant. Language doesn't simply rely upon words and spelling and expressions. For example, nearly every human language -- if not indeed all of them -- incorporates metaphor to some degree. That is, we can use the phrase "hatching of the ugly duckling" to refer to the birth of something other than a duck. If you're a native Turkish language user and you have to translate a paper that uses "hatching of the ugly duckling", how do you determine what that expression is really referring to?

One of my Tolkien essays, "Is your canon on the loose", was translated into Hebrew a couple of years ago. The translator could not replicate the pun in my title ("canon" refers to the authoritative body of texts used for Tolkien research, but the title is styled on the popular idiomatic expression, "He is a loose cannon" -- "canon" and "cannon" are pronounced exactly the same way). My translator sensed the connection but did not fully appreciate it, and after I explained how the joke worked, he said, "We have no similar expression in Hebrew".

After giving the matter some thought, he chose to retitle the essay (with my approval, as well as my permission, after conferring with me and getting my opinion) "Choir of a thousand voices" (which is a much closer rendering of the actual Hebrew title than he could come to my original meaning). It was an appropriate choice for a twisted metaphor, as the core meaning for both expressions is quite the same thing. "Is your canon on the loose" refers to the fact that Tolkien canon discussions are very fluid -- I freely admit to changing contexts and canons on a frequent, mind-boggling basis. I have to because no two people use the same parameters.

So online translation tools are going to hit the same walls that human translators hit, and how those tools make their choices will be very important to search engine optimization specialists. Why is that? Because even though it's not presently possible for search engines to truly practice semantic indexing, that is exactly what they hope to achieve some day.

Semantic indexing would allow a search engine to capture a user's query in any language, any jargon, any idiomatic context and find all relevant documents -- in any language, any jargon, any idiomatic context. Isn't placing the world's collective knowledge at your fingertips one of Google's stated objectives? Hey, whether they can or cannot do it really doesn't matter right now. They are trying to do it.

Hopefully, they'll take whatever lessons they learn from these translation projects and apply them to working with user queries and page indexing.

But beware what you ask for. Today, the real estate for Web page design and optimization is wide open. If you cannot dominate one expression, you can dominate another, similar expression and then brand that similar expression into your targeted market. Teach them to search for the phrases you dominate and you blow your competition out of the water while he is still gloating over his number 1 ranking for the phrases you cannot touch.

If Google creates the Universal Translation Tool, they'll be able to substitute one expression for another in resolving queries. One might hope they would allow for an exact find search anyway (how else would one find an exact passage one wants to retrieve?). But if that day ever comes, optimizing for specific expressions will take a back seat to optimizing for concepts.

Concept optimization is in its barest infancy right now. Our methodologies are clumsy and rely mostly on brute force. Things will remain that way until the search engines become more sophisticated in language analysis and translation. When that happens, the rules will become more clear.

Until then, we'll have to keep an eye on things, standing watch over the camp, monitoring their progress, staying apace with their technological developments, matching their innovations with our improvisations.

We have to stay in touch on the issue.

Got that?

The Other Google Blog, and yet more things...

2006-08-01T10:42:00.000-07:00

I don't actually have the googleblog.blogspot.com address committed to memory. I usually just take a stab at typing in what I think the URL should be and if I don't see what I want I search on Google for the Google blog. It (so far) always comes up first.

Well, today I accidentally visited google-blog.blogspot.com and thought, "Hm. I guess Google doesn't mind if people hyphenate their name into blogs after all".

The other Google blog is an unofficial blog about Google posted by a Librarian named Susan Herzog, an "Information Literacry Librarian @ Eastern Connecticut State University". That's a bigger jawbreaker than anything in Khuzdul, as Sam Gamgee might be tempted to say. I can only guess what an Information Literacy Librarian might be -- sounds like something the U.S. military would dream up to describe a Webmaster.

Still, she provides some very good resources for people who want to do research about...Google. As one might expect, there are many entries concerning Google Print. I can see how librarians of all varieties would be interested in Google Print. And one of the headlines in Susan's blog reads, "What about authors?"

As a published author, I have been asked what I think about Google Print. My reaction is mixed. As someone who cannot afford to buy every book on Earth, it's tempting to have so many of them at my finger tips. On the other hand, I'm not sure how many people realize that you can, actually, print out the full contents of every book encased behind Google Print's protections. The result would look very ugly, but you can do it.

Which means that people can print Visualizing Middle-earth in its entirety, despite the fact that it's labeled as a "Limited Preview" book. Now, I'm not going to share the tedious details on how to print it all out, but if I can figure out how to do it, so can at least 7 other people. The rest of you probably don't care to try.

Why do I allow Google to index my book? Because I'm ambivalent about the whole matter. I might as well see what happens. I've earned a nice amount of money off of Visualizing Middle-earth (which is cream for me, considering I had been paid to write most of the essays by Suite101). It comes up in the top five or ten for a variety of searches such as "elves in Middle-earth", "tolkien middle-earth", "middle-earth movies", "lord of the rings movies", etc.

And I did absolutely nothing to optimize for those searches. I have no idea of how one would be able to optimize for Google Print. But the day may come when books are written with search engines like Google Print in mind. Does that frighten you? It shouldn't. For decades, some authors have written books that they hoped would be picked up for film or television adaptation. Such books are designed to make the transition easily. Does the concept work? I doubt it. The film and television industry tend to go after classic books as much as possible, in my experience.

But books have also been written for specialized markets for decades as well. There are book packagers who see a need in a special niche and they go out and hire authors, artists, editors, whomever they need to produce the exact type of book required. You might buy one of these books at the checkout stand of your supermarket. Your child might buy one of these books in a school book fair. You have no way of knowing if the book you buy was the result of a package deal. I was approached a few years ago by Chris Zavisa, who has worked with Stephen King and Dean R. Koontz, to write the main text for a Middle-earth book. That's all I can say about it, but Chris introduced me to the world of book packaging.

So the production of books for services like Google Print is, in my opinion, something that will happen. It's only a matter of time until someone figures out how to (possibly) make some money off of it. How would I do it? Maybe I'd embed a lot of permanent advertising in the book, and see that each page ranks highly for a variety of searches.

But how do you optimize for Google Print? I don't know. Only Google knows (at this stage). So it would be a bit of a crap shoot, which is (I am sure) how Google wants it. But how long will it be before Google monetizes Google Print? They run ads in the margins, but technologically it should be possible for Google to increase their revenues from Google Print in other ways.

Frankly, I think they should provide a subscription book service, where you can read (and bookmark) books online. The authors would, naturally, get a royalty for your access. Much like RIAA and MPAA, I suspect that mainstream publishers would be appalled at such an idea. After all, if done fairly and right, it would cut them out of the middle and get the money directly to the authors: many of whom wait a long, long time to see money for their work.

And all that came to me because I'm too lazy to memorize google.blogspot.com....

Should Search Engines Teach SEO?

2006-07-27T06:35:00.000-07:00

Matt Cutts recently asked his blog readers if he should offer some SEO-101 (basics/fundamentals) posts or go into the deeper nether regions of the Force. Naturally, he got a mixed bag of reactions.

Personally, I think Matt should start with the basics. Not just because it would be fun to see long-time SEO pundits disagreeing with him on minor points, but because before you teach the advanced stuff you should make sure your students understand where you're coming from.

Some of the things Matt puts on his blog are propaganda posts, such as his lectures about using rel=nofollow. The NoFollow attribute is Matt's baby, so you have to expect him to advocate its use -- anyone would be expected to go to bat for their ideas.

But sometimes Matt drops significant hints that are just absolutely missed by the SEO pundits. They dwell on stupid, meaningless stuff like PageRank, the implications of "excessive reciprocal linking", why such-and-such site isn't being outed, etc. You know, no automated system is perfect, and I believe Google relies mostly on automation to keep its search results clean. Yes, yes, they penalize and ban sites so there is clearly human intervention in some places, but most of it is automated. The software is occasionally going to miss something. People occasionally miss something.

It would be interesting to see what search engine employees like Matt and Laura Lippay of Yahoo! (who used to be on the SEO industry side) have to say about the best methods of optimization. They would strive to teach people how to optimize both fairly and effectively. They would speak with authority that typical SEO gurus (including me) simply lack.

There are several search engine optimization organications around these days, and at least two of them offer some sort of certification. While the people involved in these groups are widely respected, they don't have the proper credentials in search science to really be certifying anyone. A better certification system would rely upon a blend of traditional IR principles and commercial search engine placement practices.

And while many people in the SEO industry feel that the search engines would proselytize and use their access to SEO students to advocate "favored" practices that would, in fact, assist their engines in improving results, what's the harm in that? It's not like those students wouldn't be able to find the FAQs, tutorials, and forums that teach the Dark Side principles, that offer speculative advice, and that simply go off into the wild blue yonder prattling about PageRank converging to an average of 1.

Search engines have been looking over the fence for a long time, since before Google existed. They have quietly hired SEO professionals as "consultants" and examined various techniques and principles without saying much in public. Google employees openly attend SEO industry events and party with the Black Hat SEOs and give interviews and do all the main promotional stuff, but they haven't really done much to help Google lay a foundation of trust and understanding in the SEO community.

Educating SEOs in reliable methodologies that meet search engine guidelines would go a long way toward establishing professional standards. It would also help validate or calibrate the certifications being offered in the industry, because you know those certification curricula would be adjusted to at least examine what the search engines have to say about acceptale SEO practices.

But it would also help SEOs interact more directly with search engines. There is a dirty side to the long history of SEO-search service interactions that doesn't get talked about much. So far as I know, Google never played that game, but other search services have. In the past, people learned not to share too many secrets openly in certain SEO communities because community insiders took what they learned to their paying clients (search services) and ... well ... you can see where that leads.

I think Google, Yahoo!, MSN, and Ask all owe it to themselves and their users to discuss what they consider to be proper optimization in a more formal environment. I'm not saying I would want to pay Google $2000 for the privilege of sitting in the Googleplex for a week, but if it came down to that, then it would be better than the haphazard "what should I write about next?" from Matt Cutts.

Matt is a great resource for everyone, but he can't certify SEO professionals, and in my opinion, no one can do it properly. The need for certification has oft been discussed, proposed, and mostly sidelined despite the efforts being made. SEO certification needs to be standardized and made available to everyone at a cost-effective level. I can even envision 2-tier SEO certification, because project planning in itself is a major investment of time and resources.

Canvassing Google employee blogs

2006-07-26T11:40:00.000-07:00

It's been a slow news month for Google employee blogs, which I don't bookmark because they are legion and most -- unlike Matt Cutts rarely say much about their employer. Non-disclosure issues aside, I think Google employees show remarkable restraint given just how much speculative commentary is provided across the Web (and Swedish researchers might conclude I post about 10% of that commentary, but that's another story).

There was, however, a small brouhaha over Google's authentication service when one of their employees, Ben Laurie, took the blogging/journalism community to task for (in his personal opinion) misrepresenting or miscomparing the service to Microsoft's upcoming Live features.

I think the most significant fact to come out of this exchange is Ben's statement that "Google doesn't announce what it's going to do, only what it's already done." I'm not sure how accurate that statement is (it was not sanctioned by Google and is Ben's personal expression). After all, if Google releases a beta tool, is that something they've already done, something they are doing, something they will be doing, or all three? It's a bit complicated.

Take the Google Web Toolkit, for example. You build AJAX applications in Java and then convert them to Javascript. This is a beta tool, but is it essentially finished, or is it just a foreshadow of what Google will do with Webmaster tools in the future?

Google has apparently made a huge investment in Java technology. According to another Google employee (Crazybob), Google powers Gmail, Adwords, and Blogger with Java. Now, I didn't know that. I've been criticizing Web-based Java apps for years because they tend to run so slow. I guess maybe the reason has more to do with available resources than anything else, since I only occasionally cringe at the slow response of Blogger's server.

Gregor Hohpe (who hopes to follow in the footsteps of Crazybob, among others) casually mentions that Java and integration are very important to Google. In fact, Google sits on the JCP Executive Committee. So they must really like Java.

Wonder how well they get along with Sun Micrsystems? It's hard to say. Or will there be a merger? Hm....

I guess Google doesn't have much to say on the subject right now, but perhaps they will after the deed is done (if indeed done it ever will be).

Google disables "links to this post" - Why?

2006-07-26T11:08:00.000-07:00

I've noticed that the official Google blog no longer shows who is linking to their posts. Why is that? Too much link spam?

Google's click-fraud prevention techniques are 'reasonable'

2006-07-24T15:58:00.000-07:00

So says Alexander Tuzhilin, the independent 'expert' who reviews Google's fraud-detection systems for a court. It's not clear to me, after reading this document (which includes the author's bio), what qualifies him to review Google's click-fraud detection methodologies. He does not claim to have any prior experience or exposure to the management or generation of invalid clicks (as Google describes them).

Nonetheless, his conclusion is that Google makes a reasonable attempt to detect and neutralize invalid clicks. Googlers are understandably happy to be so vindicated in an official document. However, the fact that the author quotes Wikipedia further underscores his essential naivete. He provides no reservation about the votility of Wiki entries (although any comparison of a legal analysis from 100 years ago would be expected to make reference to contemporary dictionaries and source materials, he provided no publication date for the Wiki reference).

I think Google makes a better presentation in its filed objections to a proposed $90 million settlement, especially where they point out that the proposed settlement implies fraud exceeding their total revenues to date has occurred. If there are any valid scientific claims being made in this case, neither side has done a very good job of providing them.

Dr. Tuzhilin's analysis even includes a self-admitted unscientific Zipf graph, where he analyzes the long tail of invalid clicks. While there is a certain logic to what he proposes, he offers no evidence to support his contention that the behavior he is analyzing conforms to his proposed model. That would be equivalent to Einstein saying, "Well, I think Time changes near Mass, but I have no mathematical model to show how this works." It has taken scientists decades to accumulate confirmable observations that support the extensive math accompanying Einstein's theory, so maybe I'm being a little harsh in comparing the long tail model to Relativity.

However, Dr. Tuzhilin only examined the issue from Google's perspective. His report indicates nothing about any attempts to contact people who specialize in, or rely-upon invalid click generation. Such people exist, they have been in business since before Google was founded, and based on my own conversations with some members of that shadow industry, they were already using (8 years ago) advanced methodologies which Google's vague measures appear to be incapable of detecting.

Dr. Tuzhilin suggests that Google has not implemented any data mining techniques in its filter technology. Google, if I were you, I'd assign about a dozen people to catch up on that major deficiency right now. You really have no idea of what you are up against. You need to look at the history of those IP addresses that are clicking on your ads and search results. In my opinion, based on what I have read across the Web, I think Google probably does a very good job of attempting to detect click fraud. Probably no one is better at it by now.

I'll agree that their methods are reasonable, based on the available data.

But are they effective on a large enough scale to ensure advertisers that a majority of the invalid clicks are captured? There are two areas where that question must be answered: in the aggregate and in the specific. That is, overall, the effectiveness of the program may be acceptable ("acceptable" may require something more than a reasonable effort in some people's opinions). But some specific campaigns may be targeted for abuse that is slipping by the filters.

What Dr. Tuzhilin's report underscores is the fact that Google does not know how effective its filters truly are.

Who wrote Alex Chiles' essay?

2006-07-21T11:41:00.000-07:00

On July 1, The Google Blog patted itself on the back over an essay posted to GooglePages by a 12-year-old girl. Google Is The Center OF My Internet World is a well-written essay about the various features of Google that Alex likes.

It's so well-written that she reportedly got 106 out of 100 points, according to the Google Blog.

Curiously, Alex's other papers demonstrate a much more 12-year-old writing style. For example, compare Ties Shoes Bad, Bad, Bad! with the Google essay.

If you buy tie shoes you will spend a lot of money. You will spend an estimate of $452 in your lifetime buying 166 pairs of shoelaces! You need to buy new shoelaces for many reasons! Your cat might think it was a piece of string and chew it all up. Then you’d have a chewed up piece of string....

That could be something written by a 12-year-old.

Nine out of ten times when I use my computer I find myself interacting with Google’s products including Search, Froogle, Google News, Gmail, Google Earth, and the Google Toolbar. What do I like about these products? They make finding different types of information extremely easy. Google has different search engines that I use to find all the information I need, whether it’s a website, new article or an item for sale....

This could be something I might write. The sentences are longer and more complex and they make use of the conjunction whether, a word you don't often find 12-year-olds using frequently.

Google’s user interface (human computer interactions) is clear, easy to use and understandable....

I don't remember writing many parenthetical offsets when I was 12 years old, much less parenthetical offsets that make use of technical jargonese like "human computer interactions".

In addition to searching websites, Google also has news and product search. Google News makes finding articles extremely easy. Articles from all the popular news websites are gathered together and then made searchable. This puts all the news you could ever want in one place, at Google News. I use Google News mostly for current events homework and other research. Similar to Google News, Froogle gathers products for sale into one searchable place, providing “smart shopping”. ...

This paragraph looks like it may incorporate some original short-sentence writing with more mature full-sentence writing. Note the use of the offset expressions such as ", at Google News" (capitalized) and "Similar to Google News,".

And how many 12-year-old girls write about "searchable places"?

So, let's assume for the sake of discussion that, yes, an adult helped a 12-year-old score 106 out of 100 points for an essay. How many parents have helped their children with homework?

And there seem to be a lot of guys named "Bill Chiles", so it's impossible to know if Mr. Chiles is the Microsoft employee, the owner of an offshore oil contracting firm, or some other Bill Chiles. I'm pretty sure it's not the Bill Chiles who was sued in 1836, although perhaps he knew Nicholas Flamel and nicked some longevity potion before Nicholas gave up the Sorceror's Stone (aka Philosopher's Stone to some Harry Potter fans).

The point is, why was this essay given so much assistance when others apparently were not? Is this a bit of propaganda that fell into Google's lap, or is there something else to be said about it?

Google restores ad-free search: at a cost to us

2006-07-20T15:02:00.000-07:00

Google is testing a new Accessible Search interface that incorporates an accessibility scoring (and filtering) into ordering search results. So far, I've noticed that framed sites don't seem to do too well, but table layouts are not necessarily the kiss of death. At least, I've maintained some top rankings with table layouts but lost rankings with frames.

T.V. Raman says on Google's Blog that

In its current version, Google Accessible Search looks at a number of signals by examining the HTML markup found on a web page. It tends to favor pages that degrade gracefully--that is, pages with few visual distractions, and pages that are likely to render well with images turned off. Google Accessible Search is built on Google Co-op's technology, which improves search results based on specialized interests.

The scary part of this revelation is that they are using Google Co-Op to help order the results. He said "technology", not necessarily "content".

People are already striving to manipulate Google Co-Op (and it didn't take long for spam to show up on Google Pages, either -- their anti-manipulation strategies seem to be about as secure and pre-emptive as a Microsoft operating system).

What I think is good about the Accessible Search is the fact that it helps people get away from the Flash-designed sites, the framed sites (yes, I have a few, but they are largely experimental), and sites that burden the browser with long rendering times.

I do have at least one picture-heavy site that ranked pretty well for its search expression, but the topic is not immensely popular.

Accessible Search will probably be less prone to manipulation for a while for two reasons: first, the criteria have not been disclosed in anywhere near the degree that primary search criteria have been; secondly, the target audience is relatively small.

But if people switch over to the Accessible interface to get away from ads (I am seriously considering doing that myself), Google may find itself transported back to 2000. I doubt most people would make the switch, but frankly I'm sick of ad-laden search results. If Google could give us an option to turn them off in their primary search interface, I would use it in a heartbeat.

Google says they don't let click-fraud happen

2006-07-19T14:53:00.000-07:00

Shuman Ghosemajumder wrote on the Google Blog that a blogger who used selective citation wrote a "post made for an interesting read, but was unfortunately misleading".

The fact is that Google strives to detect every invalid click that passes through its system, and to prevent those clicks from ever reaching an advertiser's account. And Eric and many others at Google have discussed the problem of invalid clicks publicly many times -- on our quarterly earnings calls, at our Press Day, and in other places, such as blogs. Anyone who has followed Google knows that Eric, and others at Google, have stated several times that Google fights invalid clicks, that we've devoted significant resources to manage it, and that we take it very seriously.

I have no doubt that Google, Yahoo!, and MSN go to great lengths to detect click-fraud and deny the click pirates their plunder. But clickage is easily simulated by a variety of means and there are a fair number of people who openly claim to make money through click-fraud. I mean, there are forums out there were click-pirates exchange ideas and talk about how they create SpamAd pages (often referred to in SEO circles as MFAs -- made-for-ads) and how they have software click on the ads.

Click-fraud has been around for much longer than Google and Goto (now Yahoo!'s Internet Advertising service) have been selling pay-per-click ads. The earliest click-fraud I ever saw was conducted by people trying to hit the top 100 list on HitCounter's Web site. Those top 100 sites supposedly earned a lot more traffic from curious people who browsed the list.

Other early click pirates concentrated on boosting their banner ad revenues by generating massive numbers of fraudulent click-throughs to their banner farms. Some of them set up 2 or 3 pages that bounced visitors back and forth. Others simply wrote script that hammered the banner servers from multiple IP addresses (yes, if you control the server, this is possible).

Affiliate link farmers sometimes did this, too. Affiliate link programs usually include language in their terms of service forbidding such activity.

Click-fraud has evolved to a much more sophisticated phenomenon these days. While I am sure Google places an impressive array of resources in their battle against click pirates, they really are outnumbered and, in my opinion, they are being constantly outmaneuvered.

The old DirectHit search engine ranked Web sites on the basis of who clicked through their listings. Some people actually set up networks of servers, where the machines were in separate locations -- to mask their connectivity for traceroute requests, which ran software that simulated click-throughs on multiple listings. They selectively targeted both client listings and less relevant listings in order to bury competitor sites.

Click-fraud may still be used against Yahoo!, which tracks click-throughs in its search results. Click-tracking is not an efficient way of tracking user activity because it's so easy to create fraudulent clicks.

There have been calls for a pay-per-action advertising model, but it's not clear to me that such a model would long survive the determined efforts of click pirates to generate revenues. After all, it's relatively easy to write software that fills out forms. Link spammers have done that for years.

Pay-per-action advertising may have to incorporate captchas and other anti-spam techniques in order to be reliable -- but then, will users really want to do business with sites that migrate away from simple action selling?

Google didn't create the problem with click-fraud, but they certainly have accelerated the issue by making it easier for click pirates to steal money. It's going to take more from Google than simple assurances that they are doing something about click-fraud.

Looking for Google in all the strangest places

2006-07-19T14:51:00.000-07:00

Just looking around, you'll see everyone talks about Google, but they largely ignore what Google says. This blog will occasionally look at what Google actually says about given topics and comment.