Page 1 of 1

Google [Bot], MSN [Bot] Etc. What's It Mean?

Posted: Thu Jul 12, 2007 3:59 am
by Terry_Glover
Board Index Page wrote:
Who is online
In total there are 16 users online :: 6 registered, 1 hidden and 9 guests (This data is based on users active over the past 5 minutes)
Most users ever online was 118 on Mon Jan 24, 2005 8:51 pm

Registered users: Google [Bot], MSN [Bot], MSNbot Media, Terry_Glover, Wowbagger, Yahoo [Bot]
Legend: Administrators, Global moderators
This "Who Is Online" info. appears at the bottom of the "Board Index" page.

What is Google [Bot], MSN [Bot], MSNbot Media etc... These appear to be counted as "Registered Users" who are logged in.

What does it all mean? :-?

Terry

Re: Google [Bot], MSN [Bot] Etc. What's It Mean?

Posted: Thu Jul 12, 2007 4:33 am
by Bruce1807
It's googles,yahoo and MSN search engine indexing pages for their search engines.

Re: Google [Bot], MSN [Bot] Etc. What's It Mean?

Posted: Thu Jul 12, 2007 6:34 am
by Wowbagger
So the search spiders are registered users? That seems a bit odd - did they register themselves, or did the site operators register them?

IF they registered themselves, I would take that as a sign that the site registration procedure needs to be tightened up to keep (spam) bots out.

Re: Google [Bot], MSN [Bot] Etc. What's It Mean?

Posted: Thu Jul 12, 2007 7:09 am
by alex
It's actually a "feature" of the new board system. This allows me to give permissions to the bots group to visit different forums and such. That way, Google can index the forums like a registered user, or I can keep it out of certain forums if I so choose.

I can also track when they come, where they go if I so choose. I haven't even gotten as far as to play with the feature...

-Alex

Re: Google [Bot], MSN [Bot] Etc. What's It Mean?

Posted: Thu Jul 12, 2007 7:28 am
by tvsjr
Wowbagger wrote:So the search spiders are registered users? That seems a bit odd - did they register themselves, or did the site operators register them?

IF they registered themselves, I would take that as a sign that the site registration procedure needs to be tightened up to keep (spam) bots out.
There are ways to verify the various well-behaved spiders out there (for instance, for Googlebot, do a reverse lookup on the IP-->name, do a forward lookup on the name, make sure everything looks right, and verify the domain is *.googlebot.com). These ID procedures are embedded into the phpBB code.

Re: Google [Bot], MSN [Bot] Etc. What's It Mean?

Posted: Sun Jul 15, 2007 2:28 pm
by kc7gr
Fair warning: Google search bots are not always benign. Before I all but accused Google of a DoS attack, one of their bots was hammering (and I do mean HAMMERING) my FTP site with a ton of anonymous login/logout attempts daily. There was no crawling of directories, no indexing whatsoever -- Just the stupid 'bot logging in and immediately logging out.

The full story may be found here.

http://www.bluefeathertech.com/googleabuse.html

Keep the peace(es).