Systems and methods for processing machine learning algorithms in a MapReduce environment

US8612368B2 · kind B2 · utility

Assignee

• International Business Machines Corporation · US

Inventors

• Douglas R. Burdick · San Jose, US
• Amol Ghoting · Pomona, US
• Rajasekar Krishnamurthy · San Jose, US
• Edwin Peter Dawson Pednault · Somers, US
• Berthold Reinwald · San Jose, US
• Vikas Sindhwani · Hastings-on-Hudson, US
• Shirish Tatikonda · Santa Clara, US
• Yuanyuan Tian · San Jose, US
• Shivakumar Vaithyanathan · San Jose, US

Key dates

• Filing date: Mar 1, 2011
• Grant date: Dec 17, 2013
• Expiry date: Feb 16, 2032

Classification

• Technology area (CPC G): Physics
• CPC primary: G06F8/427
• WIPO field: Computer technology
• WIPO sector: Electrical engineering

Abstract

Systems and methods for processing Machine Learning (ML) algorithms in a MapReduce environment are described. In one embodiment of a method, the method includes receiving a ML algorithm to be executed in the MapReduce environment. The method further includes parsing the ML algorithm into a plurality of statement blocks in a sequence, wherein each statement block comprises a plurality of basic operations (hops). The method also includes automatically determining an execution plan for each statement block, wherein at least one of the execution plans comprises one or more low-level operations (lops). The method further includes implementing the execution plans in the sequence of the plurality of the statement blocks.