Code synthesis model evaluation harnessing real-world code repositories
US12174734B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | May 17, 2022 |
| Grant date | Dec 24, 2024 |
| Priority date | — |
| Expiry date | Feb 15, 2043 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F8/30
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A computer system identifies a code synthesis model and a set of software repositories, each comprising corresponding source code and corresponding tests for testing the corresponding source code. The computer system tests a plurality of generated source code predictions using the set of software repositories. The testing includes, for each software repository in the set of software repositories, identifying a portion of the corresponding source code of the software repository that is covered by the corresponding tests of the software repository, using the code synthesis model to generate a source code prediction for the portion of the corresponding source code, and using the corresponding tests of the software repository to test the source code prediction that was generated using the code synthesis model. Based on testing the plurality of generated source code predictions using the set of software repositories, the computer system generates an evaluation of the code synthesis model.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.