Systems and methods for intelligent content filtering and persistence
US11163840B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | May 24, 2018 |
| Grant date | Nov 2, 2021 |
| Priority date | — |
| Expiry date | Dec 29, 2038 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F40/30
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A source content processor receives content from a crawler and calls a text mining engine. The text mining engine mines the content and provides metadata about the content. The source content processor applies a source content filtering rule to the content utilizing the metadata from the text mining engine. The source content filtering rule is previously built based on at least one of a named entity, a category, or a sentiment. The source content processor determines whether to persist the content according to a result from applying the source content filtering rule to the content and either stores the content in a data store or deletes the contents from the data ingestion pipeline such that the content is not persisted anywhere. Embodiments disclosed herein can significantly reduce the amount of irrelevant content through the data ingestion pipeline, prior to data persistence.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.