endellion.me.uk Search Engines
When I first bought a modem, the internet did not exist. A bit later on, the internet did exist, but URL's were like illegal drugs: crumpled paper, pressed into your palm at street corners, with "interesting" addresses on it. Suddenly there were search engines, and I remember feeling ripped off because none of my paper URL's were on it. Maybe I just didn't know how to search properly. HotBot was my favourite for a long time, and then there was Google.
When I started putting this little website on, it was ignored by all (even the people who had been given the address on the crumpled piece of paper...). The Googlebot came to index one day, but did only two pages and left again. It was to be another year before it came back. Nowadays it seems to be one of my fans. Not a day goes by without Google indexing some more. But the weird thing is, they never ever go to the new pages. Not until they're quite old pages anyway.
Which leads me to wonder, how does it work? I'm not bothered about page ranks and the such here, as I have never submitted my site or knowingly done anything to "promote" the site in any way. After all, I don't care at all, I'm running this as an experiment in Linux and Apache, not to sell anything or please anyone or whatever. (correction, 3 years on: how times have changed...)
Meanwhile, all the other search engines and some other things that pretend to be such, and some archiving sites and other stuff have also been, for example MSN search. This came into the logs sounding as foreboding as the rest of the Microsoft Outfit. You get this link to the MSN bot -- and it tells you that after being indexed, your site will be judged for all sorts, like lay-out and aesthetics. I must have missed the mark because the vast majority of visitors come here through Google. You did, didn't you?
So how does it work? Why have I got Google on my back every day? Why does Google mainly do the pages they have already got? Why does Google never do the whole site in one go? Why do I get indexed by Microsoft if they don't intend to publish the results (or does noone use their search?)
Now here's the weird bit
Visitor Statistics:
98% of the visitors to this website have clicked on a link from within Google Web Search (from many different countries, including the translation service. I shiver to think that some people may think there is something so very worthwhile here that they need to have a translation. And I shiver some more thinking what my writing is going to look like machine-translated into Croatian or Chinese).
Wireless Access Point Visitors' Use of Search Engines
20% Google
All the rest use some searchybar thing ("mysearchbar" for instance) that I would class well nigh a virus. Or AOL search. Or Yahoo search ("powered by Google").
Personally I use Google.
That is hardly an endorsement, but at least it comes up with some results now and again. Try search for "Service Pack 2" with MSN search. It's just not there, which is sort of odd I think. (Correction: I've just tried that and it was there. Shame it wasn't when I needed it last year.)
OK so this is the situation. My internet thieves are just not using Google. And they search for things like, "celebrity" and "moviestar" and "porn" (and "dansguardian" and "why can't I get onto bebo"). The visitors I get here search for the names of binaries and IP addresses and the such. Does this mean that braindead searchers are more likely not to use Google?