Method and apparatus of executing collaborative job for spark faced to multiple K8s clusters
US11954525B1 · kind B1 · utility
Assignee
Inventors
Key dates
| Filing date | Apr 13, 2023 |
| Grant date | Apr 9, 2024 |
| Priority date | — |
| Expiry date | Apr 13, 2043 |
Classification
- Technology area (CPC G)Physics
- CPC primaryG06F2209/505
- WIPO fieldComputer technology
- WIPO sectorElectrical engineering
Abstract
The present disclosure discloses Spark collaborative computing, job method and apparatus for multiple K8s clusters, and addresses the problem that most of the current multiple K8s clusters adopt the model of federated clusters, and Spark's own method of scheduling and optimization cannot be implemented across domains, by implementing a cross-domain job method, setting the multiple K8s clusters as a master cluster and a slave cluster, with the master cluster being responsible for creating Spark's The master cluster is responsible for creating Spark's Driver container and Pods, and the slave cluster is responsible for creating Spark's Executor container and Pods. After the containers are created, a direct tunnel is established between the master cluster and the slave cluster by aggregating address information and access credentials through the Collaboration Center, and the containers in the slave cluster register with Driver and continuously send heartbeat messages through the tunnel.
Source: USPTO / EPO open patent data. Objective bibliographic and citation counts.