Generation of training data for machine learning based models for named entity recognition for natural language processing

US12001798B2 · kind B2 · utility

Assignee

• Salesforce, Inc. · US

Inventors

• Jingyuan Liu · Jingwei, CN
• Abhishek Sharma · Mountain View, US
• Suhail Barot · Urbana, US
• Gurkirat Singh · Elk Grove, US
• Mridul Gupta · San Mateo, US
• Shiva Kumar Pentyala · Mountain View, US
• Ankit Chadha · San Jose, US

Key dates

• Filing date: Mar 15, 2021
• Grant date: Jun 4, 2024
• Expiry date: Dec 28, 2041

Classification

• Technology area (CPC H): Electricity
• CPC primary: H04L51/02
• WIPO field: Computer technology
• WIPO sector: Electrical engineering

Abstract

A system performs named entity recognition for performing natural language processing, for example, for conversation engines. The system uses context information in named entity recognition. The system includes the context of a sentence during model training and execution. The system generates high quality contextual data for training NER models. The system utilizes labeled and unlabeled contextual data for training NER models. The system provides NER models for execution in production environments. The system uses heuristics to determine whether to use a context-based NER model or a simple NER model that does not use context information. This allows the system to use simple NER models when the likelihood of improving the accuracy of prediction based on context is low.