Utilizing a joint-learning self-distillation framework for improving text sequential labeling machine-learning models
US11537950B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Oct 14, 2020 |
| Grant date | Dec 27, 2022 |
| Priority date | — |
| Expiry date | Mar 26, 2041 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06N7/01
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
This disclosure describes one or more implementations of a text sequence labeling system that accurately and efficiently utilize a joint-learning self-distillation approach to improve text sequence labeling machine-learning models. For example, in various implementations, the text sequence labeling system trains a text sequence labeling machine-learning teacher model to generate text sequence labels. The text sequence labeling system then creates and trains a text sequence labeling machine-learning student model utilizing the training and the output of the teacher model. Upon the student model achieving improved results over the teacher model, the text sequence labeling system re-initializes the teacher model with the learned model parameters of the student model and repeats the above joint-learning self-distillation framework. The text sequence labeling system then utilizes a trained text sequence labeling model to generate text sequence labels from input documents.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.