Three Ways to Easily Manage Many Kubernetes Clusters

What is a Kubernetes cluster?

A Kubernetes cluster is a set of nodes with a certain amount of CPU and RAM that run and manage your application workload in a resilient way. Kubernetes is ideal in case of worker node disruption. There is no need to panic; Kubernetes got you back and will keep your application up and running.

Why would you want to manage many Kubernetes clusters?

It is widespread to have two separate clusters for staging and production; if you don’t already know why we have an article about it but why could you need many Kubernetes clusters? Here are a few reasons:

Location

If you want your apps to be close to your clients to reduce latency, you will probably need several clusters in several geographical zones.

Having multiple clusters can also be a requirement due to data regulation depending on the business, e.g. financial companies in Hong Kong can’t move the data out of the country.

Scalability

There are two main reasons why you might need several clusters for scalability:

Anticipation: Even if you can (theoretically) reach 15 000 nodes on your cluster Kube before needing more than one cluster, you don’t want to wait until the day that this happens to dispatch everything because it will be very time consuming, so it is a good practice to do it from day one to save some headache in the long run.

Complexity: The more workload you will have on one cluster, the more complex it will be to manage, and even if the maximum nodes that you can have in a cluster Kube are (theoretically) 15 000, a lot of companies are having issues when they reach more than 1 000 nodes, that’s the experience of OpenAI that scaled their Kubernetes to 7 500 nodes and had several problems listed in this article or PayPal who scaled to over 4 000 nodes and 200k pods which also talked about their experience in this article.

Security

In this article, we’ve covered the case where you need to create a single-tenant application and physically isolate your applications for each customer. It’s a valid and widespread use case for companies working in the healthcare industry. Suppose each Kubernetes cluster is for one single user, then you can have thousands of clusters to manage.

Fun

Alright, I don’t know if it can be fun to have many Kubernetes clusters. But imagine all the fancy things that you can do. What about making a Kubernetes battleship where every ship is a Kubernetes cluster that you have to destroy? 😅

How to manage many Kubernetes clusters with Qovery

With Qovery's DevOps automation tool, you don’t have one but three different ways to manage many clusters so let’s dig into it and see which one is the best.

1. Web interface

This one will be pretty painful as we will manually create our clusters one by one with our lovely Qovery console.

Once you have created your organization, head to the organization settings, select “clusters", and click “add a cluster”.

You can then enter a name for your first cluster, set your credentials, add the features you want and click on “create”. You should then see your cluster, and it’s now time to click on the three dots on the right of your cluster and select “install”.

While your first cluster is deploying, you can do the same with the second one, then the third one, and repeat the process until you have the number needed 😅

Yes, it is long and fastidious, but it’s a way to do it; now, don’t panic; we have two much better ways to achieve this goal with Qovery.

2. API (bash)

Qovery provides a fluent web API with complete documentation that you can find at: api-doc.qovery.com.

Bash allows you to create as many clusters as you want easily; the line of code is the same every time except that you need to change the name of your cluster and the region if you wish to have a different region for each cluster. The example below shows you three clusters: a “prod cluster”, a “staging cluster”, and a “dev cluster”, all created with three lines of code only.

curl -H 'Content-type: application/json' -X POST -d '{"name": "prod cluster", "cloud_provider": "AWS", "region": "us-east-1"}' -H "Authorization: Token $token" "/cluster"
curl -H 'Content-type: application/json' -X POST -d '{"name": "staging cluster", "cloud_provider": "AWS", "region": "us-east-1"}' -H "Authorization: Token $token" "/cluster"
curl -H 'Content-type: application/json' -X POST -d '{"name": "dev cluster", "cloud_provider": "AWS", "region": "us-east-1"}' -H "Authorization: Token $token" "/cluster"

3. Terraform provider

If you are a big fan of Qovery already, you might know that the Qovery terraform provider is now available, and if you want to know everything about it, I have the right article for you.

How can we use this Terraform provider to manage many Kubernetes clusters?

As a video is worth a thousand words, here is a video showing how to create a cluster (and even more) with Terraform.

Now to create several ones, you need to create several “resource” with all your different clusters in your Terraform file like showcase just below with a "prod_cluster", "staging_cluster", and "dev_cluster".

terraform {
  required_providers {
    qovery = {
      source  = "qovery/qovery"
    }
  }
}

provider "qovery" {
  # read to get your token - 
  # execute command `qovery token`
  token = var.qovery_access_token
}

resource "qovery_aws_credentials" "your_aws_creds" {
  organization_id   = var.organization_id
  name              = "my AWS creds"
  access_key_id     = var.aws_access_key_id
  secret_access_key = var.aws_secret_access_key
}

resource "qovery_cluster" "prod_cluster" {
  organization_id = var.organization_id
  credentials_id  = qovery_aws_credentials.your_aws_creds.id
  name            = "prod cluster"
  description     = "this is a production cluster created with Terraform"
  cloud_provider  = "AWS"
  region          = "us-east-1"
  cpu             = 2000
  memory          = 8192

  depends_on = [
    qovery_aws_credentials.your_aws_creds
  ]
}

resource "qovery_cluster" "staging_cluster" {
  organization_id = var.organization_id
  credentials_id  = qovery_aws_credentials.your_aws_creds.id
  name            = "staging cluster"
  description     = "this is a staging cluster created with Terraform"
  cloud_provider  = "AWS"
  region          = "us-east-1"
  cpu             = 2000
  memory          = 8192

  depends_on = [
    qovery_aws_credentials.your_aws_creds
  ]
}

resource "qovery_cluster" "dev_cluster" {
  organization_id = var.organization_id
  credentials_id  = qovery_aws_credentials.your_aws_creds.id
  name            = "dev cluster"
  description     = "this is a dev cluster created with Terraform"
  cloud_provider  = "AWS"
  region          = "us-east-1"
  cpu             = 2000
  memory          = 8192
  
  depends_on = [
    qovery_aws_credentials.your_aws_creds
  ]
}

Wrapping up

Whether you need many Kubernetes clusters for scalability or location reasons or just two to separate your production and your staging, Qovery got your back with the multi-cluster feature!

With just a few lines of code, you can easily manage as many clusters as you need and use your precious time to focus on what matters the most, the success of your product.