Systems and methods for tokenizing and interpreting uniform resource locators
US8001106B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Jun 27, 2008 |
| Grant date | Aug 16, 2011 |
| Priority date | — |
| Expiry date | Oct 28, 2029 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F16/955
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Aspects include methods, computer readable storing instructions for such methods, and systems for processing text strings such as URLs that comprise patterns of parameters and values for such parameters, delimited in a site-specific manner. Such aspects provide for accepting a number of text strings that are expected to have a common delimiting strategy, then deeply tokenizing those text strings to arrive at a set of tokens from which are selected anchor tokens used to form patterns having the anchor tokens separated by wildcard portions for recursive processing. The patterns formed can be mapped to a tree of nodes. Information concerning relationships between nodes and between tokens within a given node, as well as other heuristics concerning which tokens are parameters and which are values can be used as observed events for producing probabilities that certain tokens are parameters or values, using a dynamic programming algorithm, such as a Viterbi algorithm.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.