Cloud Dataproc is a fully managed cloud service for running Apache Spark and Hadoop clusters in a simpler, and more cost-efficient manner, by reducing operational hours, and you paying only for the resources used.
Now, Google Cloud brings Spark as a service to the Kubernetes container, and ditching the virtual machine-based Hadoop clusters, with other non-Spark analytics engines support coming in the future. While the open source container orchestration platform, Kubernetes has been a big deal in the Cloud industry, which cluster computing has become increasingly important in big data processing.
Google is launching the alpha of Cloud Dataproc to Kubernetes as an important step for the Cloud service to serve as a hybrid cloud model.
The overriding idea, however is for enterprise customers to have the ability to run Apache Spark on Google Kubernetes Engine (GKE) clusters, with products such as Anthos making GKE available virtually anywhere, customers will be able to take Cloud Dataproc to their data centers as well.
Google Cloud Dataproc coming to Kubernetes is significant as it provides customers with single control plane for both deployment and managing of Apache Spark on Google Kubernetes Engine on public cloud or on-premises environment.
This is bringing enterprise-grade support, management, and security to Apache Spark jobs on Kubernetes, which is also the first of many objectives, including to simplify infrastructure complexities for data scientists around the world.
No comments