Switching to new SEO Theory blog

I've long been wondering if I should not stop focusing on Google (with this blog) and expand the scope of my SEO blogging to cover all the major search engines. In fact, at least one Googler has suggested in my comments here that would be preferable.

So, while I'll still have many things to say about Google in the future (including, I hope, many positive things), I'll be saying them as part of a much broader commentary on search engines. Accordingly, I have set up a new blog on SEO Theory and will subsequently share my search engine-related thoughts there.

And I thought I should explain my situation a little better. In September I was offered and accepted a position as Director of Search Strategies with an Internet Marketing firm here in Seattle. We discussed my online visibility and how that might impact relationships with clients, or just my job performance in general. At the time it was agreed that I should become less active in the online SEO community.

Since then, I've been told it's okay to discuss SEO principles in general, but obviously with the expectation that I stay focused on my job. So I still won't be nearly as active and visible in the forums as I once was. But I plan to be more active on SEO Theory than I have been on Google Says ... over the past few months. I'll leave this blog in place for as long as the service allows it, since there are inbound links to the articles.

Thanks to all of you who have stopped by to share your thoughts with me. I hope you'll find the new SEO Theory blog to be at least as valuable and interesting.

SEO Discussion Search Tool and Google Woes

I have been creating custom search engines with Google's little Custom Search Engine tool. All that prevents me from turning them out en masse is the amount of time I have to spend evaluating Web sites. I look at hundreds of sites whenever I create one of these search engines.

The latest one is mostly for my own personal benefit because I'm sick and tired of sifting through listings I cannot see the real content for without having to register for memberships. That bugs me. I don't have time to buy my way into every crowded venue that may occasionally generate some useful information.

In fact, I treasure SE Roundtable because they often save me the trouble of having to find interesting forum discussions. And then encapsulate the important points pretty well, too.

In that vein, it would be nice if Threadwatch were more like the original site Nick Wilsdon created. Nick didn't always appreciate having me around (or maybe he did in a link baity way) because I tend to disagree with most SEOs' conventional wisdom (usually for good reason), but he had a great resource. Nowadays, Threadwatch is mostly a rant-center, albeit an important one as long as its signal-to-noise ratio remains relatively high.

So, I set up an SEO Discussion Search engine at my SEO Web site (which, for all intents and purposes, is in an imposed stage of dormancy since I am now the Director of Search Strategies for an Internet Marketing firm -- those pesky non-compete contracts keep me a little idle in the evenings).

Anyway, I scoured the Web for interesting SEO blogs and search engines. I wanted SEO communities that were open to public scrutiny, relatively active, and/or extremely useful. By that I mean I deliberately included a few sites that don't get much traffic (or at least not much comment) but which still produce a lot of worthwhile information. Bill Slawski's SEO By the Sea blog is a must-read for anyone who likes to prognosticate about where search technology may take us -- and that is my full-time job, now.

I left out some of the more popular SEO blogs because I know someone out there has already indexed them in an SEO blog CSE and because I didn't feel those blogs really contribute much useful information for search engine optimization. For example, Danny Sullivan is extremely popular, but Daggle.com is not loaded with deep insights in search. I did include Searchengineland because it will soon be loaded with search news.

But the purose of this CSE is not really to help people find the latest search news. Rather, I am constantly searching forums and blogs for specific things I know I have read somewhere at some time. I just cannot remember where or when. By narrowing the index to a handful of sites, I reasoned, I may have a pretty good chance of finding what I am looking for.

And since I am sure other people share that occasional frustration, I decided to expand the list of included blogs and forums to make sure the search engine has a pretty solid coverage. I tried not to exclude anyone's site on the basis of personal bias, but frankly some sites are so in-your-face with ads or nonsense or vitrolic ramblings I just don't see any value in them. So a few very well-known, very popular sites joined the other excluded very well-known, very popular sites solely because I just don't see any value in them.

I don't ever search those sites for anything useful. Take that however you want to.

Nonetheless, in setting up this new search engine, I have noticed some more issues I'd like to see resolved or at least clarified. For example, if I specify a sub-directory on a large content domain, how much of that domain will actually be included? Google made it easier to include sub-domains in an index but I'm not sure of what is actually being indexed if I just specify part of a domain.

And while Google also made it possible for people to include subscribed links to their custom search engines, what I would like is the ability to override Google's filtering for the CSE.

You see, one of the very best SEO forums for years has been Spider-Food, launched by J.K. Bowman about six years ago. J.K. is no longer as active in SEO as he once was, but he stays in touch with a core group of old buds (including me). A few years ago, Spider-Food was penalized by Google for using hidden divs -- J.K. was pretty good at both the White Hat and Black Hat stuff before anyone used such nicely worded cliches to distinguish between people who followed the search engine guidelines and people who exploited the algorithmic holes.

So because Spider-food had thousands of inbound links, J.K. pretty much ignored the Google ban. But when he stopped actively participating in the forum himself, people began dropping off. J.K. always said I had a lot to do with keeping the forums active. I usually publish my most original research there, for example. But in my opinion, J.K. is and will always be the life and heart of Spider-Food. He had a very no-nonsense approach to SEO and he was especially good at the under-the-hood SEO that few people today really appreciate.

Most of J.K.'s advice was sound and ethical, and much of it would still apply today. And for over a year J.K. has been promising to remodel Spider-Food, clean it up, and ask for reinclusion. He's just a bit too much of a perfectionist, I think, as well as dragged down by other demands on his time.

Well, Google has for years indexed the Spider-food forums and I've been able to find the threads I needed to get to through Google. Not any more. I don't know if the delisting is permanent or temporary. I'm not sure of why it happened, although I know that spammers were hitting the forums pretty hard the past few weeks. J.K. finally took steps to prevent the robots from dropping any more links. But maybe he acted too late.

I don't know.

All I know is that a great resource has been delisted. Since Google just rolled out some sort of update a couple of weeks ago, and since they usually recrawl the Web after they finish an update, I'm hoping to see Spider-food come back into the index. I know it's penalized, but as long as I can do site searches, that's fine by me.

But let people sit up and take notice, because Matt Cutts did warn the SEO community a few months ago that some very serious changes were on the way. This past week at SES Chicago Rand Fishkin of SEOmoz was quietly told clean up the outbound links in your profiles or suffer some consequences. I have no doubt the Googler who conveyed that warning was doing so out of a legitimate concern to help a widely valued resource from losing search engine value.

Nonetheless, it comes across as an act of bullying. While I have always maintained that Google has every right to do whatever it pleases with its search engine, Webmasters have a right to do what they please with their Web sites. Before there were search engines there were Web sites and even without search engines there are still Web sites. But as the lines of communication open up between Webmasters and search engines, pre-emptive warnings about impending algorithmic doom only confirm what conspiracy theorists have said for years: Google acts like it owns the Web.

In fact, without the Web there is no Google, and Google cannot honestly reach into every Web property and contact the right person. For example, if they were to try to send me a warning email as they send such emails to some Webmasters, it would never reach me. I've disabled all the traditioncal admin accounts because of email spam. Many other Webmasters have, too.

I'll know if Xenite.Org hits the skids only after the fact. Shame on me for linking to sites I think are valuable but which Google may not.

Now, in Rand's case, he was apparently told that some really undesirable sites were being linked to through SEOmoz. I don't link to sites like that. I suppose I'm not in any danger from that kind of algorithmic assessment. And I've been gradually closing off directories where I don't feel Google needs to be going over the past year anyway. Old URls that no longer exist except to redirect people following old links don't really need to be indexed by the search engines. Nonetheless, my redirects have always led me to wonder when or if the axe will fall.

Frankly, if a Googler were to say to me, "Dude, fix your site or get axed," I'd say, "It's your search engine but my site."

Google only sends me a fraction of the traffic I receive. I, on the other hand, provide Google with a lot of great content. It's more their loss than mine. Besides, I could easily enough build up content on other domains that would tell people where to find my orphaned domain. I've been promoting Web sites far longer than Google has been around.

But what is the happy medium? After all, I see the Google warning to SEOmoz as a friendly interventive action intended to benefit the entire Web community. The problem is that not everyone will get such friendly warnings, and the ominous clanging of shuttered windows and doors in various blogs and forums reflects the essentially suspicious attitude of many people toward Google.

Which leads me back to Spider-food.

I want to include their threads in my custom search engine. I'm not asking Googe to lift the ban completely. I just want access to good content -- content that I believe does not violate Google's guidelines. The forums are hosted on a sub-domain.

After years of my openly complaining about how Google has treated sub-domains as if they are independent domains, has Google changed the status quo? Or did the spam robots that hammered an innocent forum get it delisted?

Or am I simply jumping the gun and the forums will be recrawled and reindexed soon?

Well, I have to get some sleep. Thanks for listening to my rant.

And I hope that the CSE I set up does, actually, help other people in the SEO community. Even the folks who didn't make the cut. I was just trying to create a tool with a different value from the one I had already read about.

Google's Supplemental Pages and other neat things

When I signed off a couple of months ago, I did say I would probably occasionally post something new. I've been working as the Director of Search Strategies for a Seattle-based Internet Marketing firm, and one of my responsibilities is to keep an eye on the SEO community.

While my company is not particularly interested in what most SEO theorists have to say, I do need to stay aware of the community's concerns and current theories.

I have to admit that I was surprised to see hardly anyone react to the apparent update Google rolled out this week. It was breifly mentioned on SERoundtable and maybe a couple of other places, and that was it. Well, I suppose most people were either pre-occupied with the holidays or else they have simply gotten used to see the results change.

But I've noticed an increasing amount of discussion about Google's Supplemental Pages, and to be honest I find the answers being given to the usual questions somewhat inadequate. I feel the inadequacy is due more to boredom than anything else. Forum regulars are simply tired of speculating about the Supplemental Index, so they are now handing out brief, trite responses.

With one exception, who shall go unnamed, although I pretty much consider the guy to be an idiot, that is. The exception suggested one of the most outlandish ideas I've seen from him in quite a while. Well, after rolling my eyes I thought, "I could write a better answer than that in my sleep".

So I challenged myself to do it (while awake, though). And all that is to say that I have now posted a more in-depth discussion of Google's Supplemental Pages on my SEO site. And while I am at it, I might as well point out that I have slightly reorganized the site, moving this new page and a couple others into a new SEO Information section.

The bottom line: if you want to move a page from Supplemental Results to the Main Index, point a few good links at it. The links can come from anywhere, including your own pages, as long as the linking pages are not in the Supplemental Index. So far, that has worked for me, even if I'm wrong to suggest that the links must come from pages in the Main Index.

I met some Googlers at the Seattle MindCamp a couple of weeks ago. I wasn't sure what to say to them. When they asked me what I do, all I could think of was, "I manipulate your search results", which is entirely true. But since they manipulate their own search results (I mean that in a good way), I figured it was a safe response.

So, I hope everyone in the U.S. (or from the U.S.) who celebrates our Thanksgiving holiday has had a good holiday weekend so far, and I wish you all the best.

I may post something here again before the end of the year, but I cannot be sure of that.

Farewell for now...

Things have moved so quickly for me since the end of August. I have more projects to complete than I have time to work on them.

This week I accepted a position with an SEO-related firm that is doing the kind of work I enjoy and have been seeking out. They specifically like my theoretical approach, although they keep their clients' best interests at heart (no so-called "black hat" SEO).

Unfortunately, I won't be able to maintain this blog. I'll leave it up for posterity's sake, and perhaps to make an occasional (very occasional) comment in the future.

I have been gratified by the interest in my work and my ideas expressed by so many of you.

Thnak you all.

Interviewed by Sootle...

The Sootle Web Directory has been publishing a series of interviews with SEOs. They asked me for an interview and I am officially SEO Interview Part Five.

Thanks to Darren at Sootle Web for the interest. Other interviewees so far have beedn Bill Slawski (number four), Michael Grey (number three), Dave Davies (Beanstalk - number two), and Joe Balestrino (number one).

Each interviewee is asked six SEO questions.

Great series, Darren! (And not just because I'm included -- you have some top-notch names in the list.)

The uneveness of Google blogging

It's tough when you put all your eggs into one basket. I've occasionally wondered if I should not have called this blog "The Search Engines Say..." because Google doesn't always say anything noteworthy from my point of view. I don't want to do a roundup of every announcement that Google makes, either officially or unofficially. But Google being Google, it seems worthwhile to me to say something about their ponderances and pronouncements on occasion.

I have, however, noticed a pattern that demonstrates just how large and cumbersome Google is becoming. Some of their blogs are updated frequently and some of them are updated about every time a remake of "King Kong" is produced.

The main Google blog sometimes carries posts that don't seem very relevant to me. Such as today's Congratulations, Luis von Ahn post. Alan Estace, Google's VP of Engineering, claims that Google has built upon von Ahn's work, and they extend worthwhile recognition to him. But one does not get a full sense of how much people at Google feel they owe to von Ahn.

Maybe I should say I don't get that sense because I'm so used to reading gushing fan site copy that when someone sings a simple verse of praise, I react with a jaded, "That's it?" It's a worthy note, but doesn't shed much light on Google's mysterious ways to an algorithm chaser like me.

Nonetheless, at least the official blog gets updated on a frequent basis. That's important. It keeps many people coming back. There are other Google blogs that don't update very frequently. Surprisingly, one of those infrequently updated blogs is Blogger Buzz. Their last post as of this writing is September 11's Flickr Support for Blogger in Beta. Seems to me they could find something more to say for Blogger users. I'm not very good about digging into manuals and FAQs any more. I'd rather read about all the cool stuff I should be doing with my blog in contemporary posts.

The Google Analytics blog posts an update about once a week. That's almost enough to keep me coming back, but I have to admit that there are weeks where I forget to do that. So, I did see last week's Spotlight on: How to read the ROI column but I didn't really get excited. I actually liked the Tip: Tracking 404 Pages post more because I do have a lot of 404 issues and I should be paying closer attention to them. Analytics might be a better tool for that than sifting through my error log.

Google Base is not as consistent as Google Analytics in updating their blog. The Open House tip they shared yesterday is actually very intriguing. Can the highly competitive real estate industry benefit that much from Google Base? I may have to go out and recruit a few real estate sites just to test it. contact me if you want help promoting a real estate site. I'll give it some thought.

Normally, I shy away from real estate, as do so many other people. It's a tough, tough field. I do better with promoting insurance Web sites. But I digress. Google Base's blog offers helpful tips on how business site operaors can maximize their visibility in Google, and I wish they offered just a little more frequent information. 2 posts a week would be good for me, even if some of them repeat previous topics or just recap information from the past 3 months.

In fact, I think a quarterly recap of topics would be a good idea for all the Google blogs.

Pacing is a real issue for these blogs. For example, Google Checkout had a flurry of posts last week but has been dry ever since. That's not a good thing.

Google Code updates its blog quite often, but it took me a while to realize they did because their home page looks more like an information portal. Did you know that they have posted a Summer of Code update? BTW guys -- your permalink page layout doesn't look so good in my browser. The solution is not to tell me to switch browsers. Still, one of the great things about this post is that they point you to Drupal's project list, where you can download and test new software.

Speaking of coding, there is now a Google Data APIs blog. Leslie, I hope you and your team see what I am getting at here. Consistency is more important for building a readership than waiting for that whambang product release or update that absolutely has to be mentioned. Pace yourselves with the new blog. I would aim for 1-2 posts per week.

Google Enterprise does a good job of posting to their blog frequently. New version of Google Search Appliance! is one of those whambang product release announcements I referred to in the previous paragraph. But they posted about the Xythos OneBox and Webinar on Sunday and did a post on Friday, and so on. They keep the information flowing and I look forward to seeing what comes next from that blog.

The Google Maps blog is another of their 1-2 times a month updaters. How many people noticed that they posted an announcement about New Satellite Imagery and Performance on Friday, September 15? This post will give that post its first Permalink. Their last post occurred on September 6. Rip Van Winkle would not be likely to check in on a blog so seldom updated.

I have pretty much given up on Google Reader's blog. It hasn't had a post since August 3. Is that all there is to say? No tips on great feeds to find, or ways to find feeds, or how to manage your feeds? Come on, guys, surely you use your own tool, right? Share something.

Same for you, Google Scholar Blog. I'm tired of seeing the N-Gram headline. The joke was funny the first time I read it on August 3 and cute the next time I read it. Now I want more.

I don't understand the Google Video Blog, so I don't even read it -- oh, wait. You can't read it. That's the issue.

Google Web Toolkit blog almost had a September update. Only 11 more blogging days left in the months, folks.

I'm not sure I want to link to Google AdSense blog's September 19 post because I think I'm going to do this. It just sounds too cool to pass up. But it will depend on how complicated it really becomes in the implementation.

Alas! My most favorite of Google Blogs, Inside Google Book Search, has not updated since September 12. What happened, guys? You had a great thing going! I hope you didn't get banned from Blogger or something. That would just be so unfair. Inside Google Book Search is the most innovative of the Google blogs. They actually show people how to use their product 9 different ways from Molly. It's a great propaganda tool for a controversial service because it really does show the value the service provides.

The Google Desktop Blog is another disappointment. No posts since September 5.

And just as I was about to complain that Webmaster Central's blog isn't updating enough, Amanda posted their very cool and informative Debugging Blocked URLs tips. Because of Webmaster Central's name, this blog has the potential to become the most closely read blog of the group, as far as search engine optimizers and Web site promoters are concerned. That's the power of branding. But the power of expectation will impel the Webmaster Central staff to be more active on the blog if they want to keep their audience happy.

The real reason for this roundup is that Google hasn't given me much to write about lately. Sure, Matt Cutts wrote about Changes in URL queries yesterday, but frankly I found that topic to be a little boring. It's not really controversial. We see Google make changes to its interface every day. Changing a query syntax once in a while is no big thing, and I can easily accept the reasoning Matt offers.

In fact, this recent change will probably make my URL reference research a little easier. Unlike some people in the SEO industry, I have sense enough not to expect Yahoo! to tell me which links Google has indexed.

That whole concept still scares me, because those people get paid to do research. That's like looking at the Democratic Party's mailing ilst to see who contributes to the Republican Party.

So, Google, here are your orders: Post more stuff on your blogs. Say something. Give me a purpose, because I really don't know how to change the name of this blog to "The Search Engines Say...".

Using Google To Help Others

Google announced a United Negro College Fund scholarship program last week. I think our higher education system is way too expensive and am always happy to see more scholarships appear.

But in thinking about what other types of scholarship programs I could imagine Google funding (say, for Native Americans, Hispanic Americans, unwed mothers, orphans, and victims of drunk drivers -- not necessarily in that order), it occurred to me that there is one important segment of American society that never gets any attention: experienced workers whose skills are no longer required. I'm not just talking about unskiled laborers. I'm talking about college graduates.

Last year, Bill Gates whined about a shortage of skilled IT workers in the United States. Don't be fooled by the low percentages for unemployed IT workers mentioned in the article. In 2001, 500,000 American IT workers permanently lost their jobs. Estimates published since then indicate that as many as 400,000 of those workers have been unable to return to the IT field.

Many of those jobs went to India because the cost of hiring Indian programmers is considerably less than the cost of hiring American programmers. Now, I've been to India in a professional capacity and the programmers I met there struck me as being competent, capable programmers. Nonetheless, there is a growing backlash among Americans against the outsourcing of jobs to India. To his credit, Mr. Gates doesn't want to outsource jobs to India -- he wants to bring Indians to America on H1-B work visas and pay them American salaries.

In the meantime, approximately 400,000 former American tech workers find it very difficult to get jobs because they are "overqualified and will leave at the first good opportunity". Try submitting a resume that shows 25+ years' experience (including management positions) in the IT field to an unrelated industry and you'll see what the problem entails. There is a great deal of technical experience and skill languishing in this country because CEOs like Bill Gates are too cheap to invest in teaching skilled programmers how to work in a new language.

Google and other large corporations could give thought to tapping an immense resource of experienced talent that is currently not being exploited.

But enough about scholarships. Google AdWords' Blog has launched the first in a series of blog tips about how to create effective video ads. I don't think many companies have thought about what it would take to sell video ad space on their Web sites, but I've often wished I had the resources to put together some randomly available videos fo Xenite.Org. In about 3 years, these tips may look naive and misguided to experienced hands, but for now they are all that business advertisers have to work with in this emerging field.

If you operate a large multimedia site, think about how embedding cross-promotiona video ads in your own content can help you improve your on-site conversion ratios.

In August, Inside Booksearch published Finding Gems In Your Library. I have now seen this feature in action. I mainain a discussion group about Middle-earth and occasionally post news stories to the list. Tonight I shared a story about Amanda McKittrick Ros, an author whom J.R.R. Tolkien and his friends (the Inklings) read for amusement. I wanted to see if Google Books had any of her works (they don't, although they do have my first book Visualizing Middle-earth).

When I scroll down the first results page for Amanda McKittrick Ros, however, I see three searches of library catalogues. I shared that search with people on the Endor list and I think this marks the first time I have recommended Google Books to people. Don't assume I'm now wholly in favor of the service, but it would be crazy to pretend it is not there.

Google Enterprise says Xythos has developed a Google OneBox module without having to use a Google Search Appliance. If they share enough of their technology in their October 18 Webinar, other developers may be able to follow their lead.

And, finally, Google Webmaster Central reports that the login function for Public Search Service has been temporarily disabled. It's interesting that Google says they are not aware of any malicious exploits of the search function. But I have to wonder if this update was inspired by SEOMoz's 20 government links in 20 minutes post. There are Googlers who read SEOMoz. If they noticed that Public Search was being exploited, SEOMoz may have just helped Google close what could have become a serious loophole.

If you've never looked into what Google Webmaster Central offers, check out their resources. They make the SiteMaps Site Status tool available for free without requiring a login. Many people who don't trust the service may want to reconsider using this tool.

While it's true that Google most likely saves the queries, I doubt they use them to tag sites that are not compliant with Webmaster guidelines. They aready have filters to do that.