Does Googlebot use inference when crawling?

Does Googlebot use inference when crawling?

Share

Email of your friends...

Added on Wednesday 09 September 2009 13:24:34
by GoogleWebmasterHelp
5236 Views

4.5 / 5
17 rates

Leon from the UK asks:

"Does GoogleBot use inference when spidering - having crawled site.com/article/page1.htm and /page2.htm, can it guess at the existence of a /page3.htm and crawl it? Or does it stick entirely to what it finds via the link graph and/or Sitemaps/feeds?"

This video is part of a "Grab Bag" series in which Matt Cutts, head of Google's webspam team, answers questions from webmasters. We're not currently taking new video questions, so your best bet for getting an answer about webmaster-related search issues is to head to our help forum: http://www.google.com/support/forum/p/Webmasters?hl=en

facebook

Comment(s)

Write your comment here...

100reyven

hkjb
Tuesday 25 October 2011 23:58:40
fferrero2008

Exactly. The title of the video is "Does Googlebot use inference when crawling?" and he clearly said "Yes." @almightyvegeta87: Do you expect to say how Google works? Hmmm... Maybe Microsoft should save millons and millons investing money on Bing and just came here and drop a simple question: "Hey Matt, how does Google work?".
Saturday 12 September 2009 20:50:28
viniweb

The question is if Google uses inference. He said "yes" and gave examples. I think that's a good answer.
Thursday 10 September 2009 16:18:42
viniweb

I have some websites that use the scheme "domain/title/id", where the "id" parameter is unique between pages, but the "title" may not be (there are cases where this is not uncommon, btw). With the approach Google uses, an URL can become ambiguous if there is more than a page with the same title. I think web crawlers should respect the link structure created by the webmaster, unless of course you're investigating duplicated content or whatever.
Thursday 10 September 2009 16:16:10
almightyvegeta87

Well he didn't answer another question again. He did mentioned Google uses inference but cleverly moved off-topic not telling how ! He talked more about how Google drops some parameters while crawling, how they are able to crawl new things like forms, drop down menus ... etc etc We expect more info Matt Cutts, you are the only public speaker and now you too have stopped video responses.
Thursday 10 September 2009 12:30:29