Machine learning runtime library for neural network acceleration

US11694066B2 · kind B2 · utility

Assignee

• XILINX, INC. · US

Inventors

• Aaron Ng · San Jose, US
• Jindrich Zejda · Saratoga, US
• Elliott Delaye · San Jose, US
• Xiao Teng · Cupertino, US
• Sonal Santan · San Jose, US
• Soren T. Soe · San Jose, US
• Ashish Sirasao · San Jose, US
• Ehsan Ghasemi · San Jose, US
• Sean Settle · San Jose, US

Key dates

• Filing date: Oct 17, 2017
• Grant date: Jul 4, 2023
• Expiry date: Nov 14, 2040

Classification

• Technology area (CPC G): Physics
• CPC primary: G06V10/955
• WIPO field: Computer technology
• WIPO sector: Electrical engineering

Abstract

Embodiments herein describe techniques for interfacing a neural network application with a neural network accelerator using a library. The neural network application may execute on a host computing system while the neural network accelerator executes on a massively parallel hardware system, e.g., a FPGA. The library operates a pipeline for submitting the tasks received from the neural network application to the neural network accelerator. In one embodiment, the pipeline includes a pre-processing stage, an FPGA execution stage, and a post-processing stage which each correspond to different threads. When receiving a task from the neural network application, the library generates a packet that includes the information required for the different stages in the pipeline to perform the tasks. Because the stages correspond to different threads, the library can process multiple packets in parallel which can increase the utilization of the neural network accelerator on the hardware system.