Synchronizing crawler with notification source
US6424966B1 · kind B1 · utility
Assignee
Inventors
Key dates
| Filing date | Jun 30, 1998 |
| Grant date | Jul 23, 2002 |
| Priority date | — |
| Expiry date | Jun 30, 2018 |
Classification
- Technology area (CPC Y)Emerging Cross-Sectional Technologies
- CPC primaryY10S707/99933
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A method and system for the processing and maintenance of electronic information retrieved from electronic documents stored on a computer network. The gatherer program of the present invention employs a crawler to crawl a portion of the computer network to retrieve electronic documents found during the crawl and that meet a set of crawl restriction rules. Some or all of the data contained in the copies of electronic documents is then stored in a data store such as an index. The invention keeps the data in the data store current by accepting notifications of when a previously retrieved document has changed. The notifications are sent by a notification source that monitors a space containing the previously retrieved documents for changes occurring after the document was last retrieved by the gatherer program. Because the document is being monitored for changes by the notification source, the gatherer program only needs to retrieve the document again when the gatherer program has been notified that the document has changed. If the notification source experiences a discontinuity, such as a system shutdown, the notification source requests that the gatherer perform an initialization cra…
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.