Method and system to compensate for inaccuracy associated with processing values with finite precision
US8037115B1 · kind B1 · utility
Assignee
Inventors
Key dates
| Filing date | Jan 17, 2007 |
| Grant date | Oct 11, 2011 |
| Priority date | — |
| Expiry date | Aug 3, 2030 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F7/49942
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
A method and system to compensate for inaccuracy associated with processing values with finite precision includes a process for selecting a display value whereby an initial value is provided in a first numbering system. The initial value is then converted into an equivalent stored value in a second numbering system. Then a display value in the first numbering system is determined and selected such that the selected display value includes the least number of significant digits that convert from the first numbering system to the second numbering system exactly as the stored value. The selected display value in the first numbering system is then used for display and/or further processing when the stored value in the second numbering system is recalled.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.