System and method for dynamically limiting robot access to server data
US6662230B1 · kind B1 · utility
Assignee
Inventors
Key dates
| Filing date | Oct 20, 1999 |
| Grant date | Dec 9, 2003 |
| Priority date | — |
| Expiry date | Oct 20, 2019 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F2221/2101
- WIPO fieldDigital communication
- WIPO sectorElectrical engineering
Abstract
A method for automatically limiting access of a client computer to data objects accessed through a server computer dynamically prevents robots or webcrawlers from obtaining too much of the server database and from dramatically reducing server performance. The method includes the steps of receiving a request for a data object, recording a log entry for the request, calculating client request values, and refusing the request if a client request value exceeds one of a set of corresponding predefined maximum request values. Each log entry contains a client identifier, timestamp, and at least one data object identifier for the request. The client request values preferably include a request frequency, which is compared with a predefined maximum request frequency, and a cumulative data request, which is compared with a data access threshold. If the client is refused access, the client identifier is added to a deny list, and future requests from the client are automatically denied. The calculated cumulative data request may be for a single client, or it may be for all clients, in order to detect a robot that is divided among multiple client identifiers. The cumulative data request check ma…
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.