Systems and methods for mutual information based self-supervised learning
US12198060B2 · kind B2 · utility
0Cited by
0References
20Claims
0Family size
Assignee
Inventors
Key dates
| Filing date | Aug 28, 2020 |
| Grant date | Jan 14, 2025 |
| Priority date | — |
| Expiry date | Sep 4, 2042 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06N3/0895
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Embodiments described herein combine both masked reconstruction and predictive coding. Specifically, unlike contrastive learning, the mutual information between past states and future states are directly estimated. The context information can also be directly captured via shifted masked reconstruction—unlike standard masked reconstruction, the target reconstructed observations are shifted slightly towards the future to incorporate more predictability. The estimated mutual information and shifted masked reconstruction loss can then be combined as the loss function to update the neural model.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.