Quick Start: Kubernetes Test Cluster w/ Antrea CNI

Recently, VMware announced an open source Kubernetes Networking project called Antrea. This project uses Open vSwitch (more here) as the data plane for a compatible Container Network Interface (CNI). To run Kubernetes (k8s) clusters, it is required that you provide a CNI to allow for pod to pod communication. It is assumed the hosts (physical or VMs) making up the cluster are already networked together.

In this post, I’d like to go over setting up a single node k8s cluster using Kubeadm on an Ubuntu 18.04 and Antrea CNI (with latest versions). For me, this is an easy way to spin up a cluster to mess around with or do some quick testing. A couple other ways that I’ve used and love are KinD (here) and simply enabling it in Docker for Desktop (probably the easiest way for most).

To start, you’ll need a single Ubuntu 18.04 machine. I’ve done this on AWS, and using VMware Workstation on my laptop and it’s worked well on both. The recommendation is to make sure you have 2 vCPU and 2 Gb RAM. (and if you use the script below, the install will fail without these resources)

To prepare the Ubuntu machine for k8s we need to install Docker (original Docker doc):

sudo apt update
sudo apt install -y apt-transport-https ca-certificates curl software-properties-common
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu bionic stable"
sudo apt update
apt-cache policy docker-ce
sudo apt install -y docker-ce

sudo usermod -aG docker $USER 

In the commands above, Docker was added to the apt repository then downloaded and installed. Then we added our current user to the Docker group so we don’t have to use sudo with all the Docker commands. If someone knows differently, please let me know, but it has always required a restart for me for that to take effect. Which we will do after we download the rest of the required k8s system components, cli and kubelet.

sudo apt-get update && sudo apt-get install -y apt-transport-https curl
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
cat <<EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF
sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

sudo reboot

With that, we should have everything needed to continue. Before we begin the Kubeadm bootstrap, we need to ensure swap is turned off because it will cause us problems if we don’t.

#turn off swap
sudo swapoff -a 

#initialize master cluster
sudo kubeadm init --pod-network-cidr=10.244.0.0/16 

#Remove taint from master to use that node
kubectl taint nodes --all node-role.kubernetes.io/master-

#get cluster credentials and copy them into our kube-config file
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

#apply antrea CNI network latest version
sudo kubectl apply -f https://raw.githubusercontent.com/vmware-tanzu/antrea/master/build/yamls/antrea.yml

In the code above, we turn off swap for this machine and then pull down the Kubeadm images that are used to create our cluster components. This includes our API server, etcd database, controller manager, and scheduler.

After that we are initializing our cluster with the kubeadm init --cidr= command. The network address range we pass in here will be used for our pods and controlled by Antrea, which we are installing in the next command. **Important to grab the token given by Kubeadm if you want to grow your cluster with additional worker nodes!

After that, we are simply removing the taint from the master node so that we can run our pod/container workloads on the same node. By default, a taint is applied to the master so that workloads do not interfere with the operation of our control plane….obviously the right thing to do when it matters!

Make it faster for me: https://github.com/fitz0017/k8s

I have the script broken down into 2 parts, because I can’t get Docker to run properly without a full reboot. To run the scripts, log into your Ubuntu machine that has 2 CPU and at least 2Gb RAM and:

git clone https://github.com/fitz0017/k8s.git
source k8s/install_k8s_1.sh

At this point you may need to input your sudo password, and select ‘yes’ when asking if you will allow for system services to be restarted, if doing this on Workstation.

When that completes, you should see a full reboot of your Ubuntu machine. So log yourself back in and:

source k8s/install_k8s.sh 

This will kick off the initialization of the cluster and application of the Antrea CNI. Again, make sure to copy the discovery token output at the end of initialization if you want to grow this cluster.

From here, when ssh’d into that machine. You have access to a k8s cluster for testing and learning! Please tell me if you notice any problems or give feedback in the comments.

Good luck!

One thought on “Quick Start: Kubernetes Test Cluster w/ Antrea CNI”

Leave a Reply

Your email address will not be published. Required fields are marked *