Method and system for automatically obtaining web page content in the presence of redirects
US8789177B1 · kind B1 · utility
Assignee
Inventors
Key dates
| Filing date | Apr 11, 2011 |
| Grant date | Jul 22, 2014 |
| Priority date | — |
| Expiry date | Oct 27, 2031 |
Classification
- Technology area (CPC H)Electricity
- CPC primaryH04L67/02
- WIPO fieldDigital communication
- WIPO sectorElectrical engineering
Abstract
A method and apparatus for automatically obtaining web page content in the presence of redirects whereby an incoming message is received and analyzed to determine if the message contains any URLs. Any detected URLs are then extracted and activated to analyze the contents of the web page linked to by the URL. The HTTP response headers and content sent from a web page server in response to the browser HTTP requests to activate the URL link are analyzed to determine if the response includes a redirect to a new, or destination, URL, and associated web page, i.e., to determine if the detected URLs result in redirects. If the HTTP response indicates a redirect, a URL redirect analysis process is initiated that includes a set of redirect processing procedures that are selectively applied depending on the type of redirect encountered, and each redirect is automatically followed. For chains of redirects, the process is recursive, i.e., is repeated automatically for each redirect, from the beginning, and as if the new (destination) URL is itself an original URL.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.