Patent · US Active

Systems and methods for refining pre-trained language models with improved gender fairness

US12073178B2 · kind B2 · utility

1Cited by

1References

20Claims

0Family size

Assignee

Salesforce, Inc. · US

Inventors

Zahra Fatemi · Chicago, US
Caiming Xiong · Menlo Park, US
Wenhao Liu · Dongguan, CN
Chen Xing · Sunnyvale, US

Key dates

Filing date	Jan 27, 2022
Grant date	Aug 27, 2024
Priority date	—
Expiry date	Feb 20, 2043

Classification

Technology area (CPC G)Physics
CPC primaryG06N3/09
WIPO fieldComputer technology
WIPO sectorElectrical engineering

Abstract

Embodiments are directed to a training framework for reducing gender bias in a pre-trained language model. To reduce gender bias a gender neutral dataset is generated. Next, parameters of the pre-trained language model are frozen and do not change during a subsequent training phase. As all the pre-trained parameters are frozen, forgetting of information from the original training data is minimized. New parameters are added to the language model. The new parameters may be associated with gender related terms, such as profession names. In a subsequent training phase the new parameters of the language model are trained using a gender neutral dataset.

Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.