Distributed crawling of hyperlinked documents
US8266134B1 · kind B1 · utility
3Cited by
14References
17Claims
0Family size
Assignee
Inventors
Key dates
| Filing date | Oct 24, 2007 |
| Grant date | Sep 11, 2012 |
| Priority date | — |
| Expiry date | Sep 15, 2030 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F16/951
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Techniques for crawling hyperlinked documents are provided. Hyperlinked documents to be crawled are grouped by host and the host to be crawled next is selected according to a stall time of the host. The stall time can indicate the earliest time that the host should be crawled and the stall times can be a predetermined amount of time, vary by host and be adjusted according to actual retrieval times from the host.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.