Automated locating of GUI elements during testing using multidimensional indices
US12332770B2 · kind B2 · utility
Assignee
Inventors
Key dates
| Filing date | Jun 28, 2021 |
| Grant date | Jun 17, 2025 |
| Priority date | — |
| Expiry date | Dec 16, 2041 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06T2207/20081
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
Disclosed are techniques for automated locating of user interface elements during graphical user interface testing. When a graphical user interface (GUI) is received for testing, images of the GUI are inputted to a machine learning algorithm, where image processing techniques are applied to identify groups of user interface elements and their constituent elements. Multi-dimensional index values are assigned to groups and elements corresponding to their positions within the GUI. Automated testing of the user interface elements of the GUI is performed by locating the user interface elements by their index values. If an element is not found, a scrolling technique is applied to generate an expanded virtual GUI of one or more groups of user interface elements, and the machine learning algorithm refreshes the index values using the expanded virtual GUI.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.