Method and system for preventing web crawling detection
US7953868B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Jan 31, 2007 |
| Grant date | May 31, 2011 |
| Priority date | — |
| Expiry date | Jul 24, 2029 |
Classification
- Technology area (CPC H)Electricity
- CPC primaryH04L61/2503
- WIPO fieldDigital communication
- WIPO sectorElectrical engineering
Abstract
A method and system for preventing a detection of web crawling. A randomizing HTTP proxy server receives a first request from a web crawler to scan a website and forwards the first request to a randomly selected first proxy computer. The first proxy computer utilizes a first network address translation (NAT)-enabled router to forward the first request to the website. A NAT algorithm associates a first source Internet Protocol (IP) address with the first request. The randomizing HTTP proxy server receives a second web crawler-initiated request to scan the website and forwards the second request to a randomly selected second proxy computer. The second proxy computer utilizes a second NAT-enabled router to forward the second request to the website. The NAT algorithm associates a second source IP address with the second request. The web server identifies the first and second source IP addresses as being different.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.