HoK is Hadoop on Kubernetes, It helps you to deploy Hadoop stack on Kubernetes

Kubernetes is now proven technology to deploy and distribute modules quickly and efficiently. Many cloud vendors are now offering Hadoop as a service. Companies are moving towards the model where they want to provision an instance of service on the fly and use it for analytics. Usually, it takes weeks to provision a production-ready Hadoop cluster. This platform is container-native platform serves as the Backbone for all other analytical services. This provides the user to spawn Hadoop cluster using the self-serve portal, which helps them to onboard the team quickly and efficiently so their developers can start using the cluster as soon as they join the team

This Helm Chart requires Helm 2.

Depending of the version of the HoKStack, it supports the following platforms:

The Hok need saperet namespace which can be team-name or value-stream anything unique. It deploys all the Hadoop components in statefulsets. To install the HokStack via Helm run the following command:

$ helm repo add hok https://raw.githubusercontent.com/hokstack/hok-helm/master/repos/stable

To update simply update your helm repositories and check the latest version

$ helm repo update

You can then check for the latest version by searching your Helm repositories for the HokStack

$ helm search hok 

You can then check for the latest version by searching your Helm repositories for the HokStack

$ helm install hok/hokstack --name hok-team1 --set teamname=team1 --namespace team1

You can then check for the latest version by searching your Helm repositories for the HokStack

$ helm install hok/hokstack --name hok-team2 --set teamname=team2 --set metacontroller.crds.create=false --namespace team2

Remove OneAgent custom resources and clean-up all remaining OneAgent Operator specific objects:

$ helm remove <release name> -n <namespace>

By default it will install following statefulsets.

• postgres
• ambariserver
• masternode
• datanode 3 replica
• edgenode
• dante-proxy
• kdcserver optional
• metacontroller

• HDFS
• Yarn
• Mapreduce
• Tez
• Hive
• Oozie
• ZooKeeper
• Kafka
• Spark
• Spark2

hdfsyarn.json contains the ambari blueprints, add componenets in the host_groups to add more HDP componenets.

In the stack, you will find dante-proxy pod is running its SOCKS5 proxy running in container using that you can access the Ambari UI and other Hadoop UIs.

Get the NodePort of dante-proxy using below command

$ kubectl get svc dante-proxy

Enter this port and Node IP address to the web-browser in proxy section, preferably Firefox as nowadays Chrome manages my orgnisations.

Acces cluster using the edgenode port you can login into that using

kubectl exec -it edgenode-0 bash

Aslo SSH is enabled on the edgenode-0 it can be accessed

The following table lists the configurable parameters of the HELM chart and their default values.

Global values

Full and up-to-date documentation can be found in the comments of the values.yaml file.

HokStack is maintained by:

• Rohit Sharma (@rohitrsh)
• Shubhomoy Biswas (@shubhmoy)

Initially inspired from https://www.cloudera.com/tutorials/getting-started-with-hdp-sandbox.html

Currently we work on the code in our free time, any assistance is highly appreciated. Please read CONTRIBUTING.md for details on our code of conduct, and the process for submitting pull requests.

HoKStack is under Apache 2.0 license. See LICENSE.md for details.