Text deduplication method and apparatus, and storage medium
US11379422B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Jun 14, 2019 |
| Grant date | Jul 5, 2022 |
| Priority date | — |
| Expiry date | Jan 7, 2041 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F40/30
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A text deduplication method and apparatus, and a storage medium are provided. The method includes: obtaining a text set, the text set including a plurality of to-be-deduplicated texts; capturing, for each to-be-deduplicated text, a corresponding subtext string from the to-be-deduplicated text; and determining, in the text set, to-be-deduplicated texts having a same subtext string, to obtain text subsets. Each subtext string corresponds to a text subset, and each text subset includes one or more to-be-deduplicated texts that have the corresponding subtext string. The method also includes performing text deduplication processing on the text subset corresponding to each subtext string, to obtain a deduplicated text set corresponding to each subtext string; and obtaining, according to the deduplicated text set corresponding to each subtext string, a result text set of the text set after the deduplication.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.