Advanced Cluster Setup

System Requirements

Root and Cluster orchestrators:

  • Docker + Docker Compose v2
  • 5GB of Disk
  • 500MB of RAM
  • ARM64 or AMD64 architecture

Worker Nodes:

  • Linux-based distro with iptables compatibility
  • 50MB of space
  • 100MB RAM
  • ARM64 or AMD64 architecture

Deploy Multiple Clusters

Having one root orchestrator and one cluster orchestrator on one device is a great way to start using Oakestra, but the true power of the system lies in it’s federated architecture.

This guide will walk you through deploying stand-alone components, allowing you to deploy multiple cluster orchestrators across multiple devices. These clusters will be managed by a single root orchestrator.

Stand-alone Root Orchestrator

arm
x86
arm
Root Orchestrator
Worker
Worker
Worker
Worker
Cluster Orchestrator
Cluster Orchestrator
arm
x86
arm
Root Orchestrator
Worker
Worker
Worker
Worker
Cluster Orchestrator
Cluster Orchestrator

First, let’s deploy a stand-alone root orchestrator. This component will manage your clusters and provide an interface for users to interact with the Oakestra setup by providing an API and a dashboard.

curl -sfL oakestra.io/install-root.sh | sh - 

This script will download the required files to the directory ~/oakestra/root_orchestrator. From there it will build the root orchestrator.

Network Configuration

If you run into a restricted network (e.g., on a cloud VM) you need to configure the firewall rules and the NetManager component accordingly. Please refer to: Firewall Setup

Stand-alone Cluster Orchestrator

arm
x86
arm
Root Orchestrator
Worker
Worker
Worker
Worker
Cluster Orchestrator
Cluster Orchestrator
arm
x86
arm
Root Orchestrator
Worker
Worker
Worker
Worker
Cluster Orchestrator
Cluster Orchestrator

Next, we can deploy a stand-alone cluster orchestrator. This component will manage the nodes by delegating applications, creating subnetworks and facilitating communication. Additionally the cluster orchestrator sends aggregated reports to the root orchestrator.

curl -sfL oakestra.io/install-cluster.sh | sh - 

This script will download the required files to the directory ~/oakestra/cluster_orchestrator. From there it will walk you through configuring the cluster. Once the setup is complete, it will build the cluster orchestrator and register with the root orchestrator.

You can register as many cluster orchestrators with the root orchestrator as you would like. Repeat the above command on a new device and configure the orchestrator with a unique Cluster Name and Cluster Location.

Registering Nodes

Check out how to register worker nodes with a cluster

Custom Deployments

Oakestra also allows you to customize your system deployment to best suit your needs.

Oakestra Addons

Check out addons for even more customization options!

Configure the Orchestrators

The root and cluster orchestrator can be configured using environment variables. If they have not been set, they are automatically set when using the above scripts. They can also be manually configured, e.g. when building using the docker compose files.

Root Orchestrator Environment Variables

  • SYSTEM_MANAGER_URL: Specify how the root orchestrator can be reached (URL or IP)
# Example configuration for root orchestrator
export SYSTEM_MANAGER_URL=192.158.18.104

Cluster Orchestrator Environment Variables

  • SYSTEM_MANAGER_URL: Specify how the root orchestrator can be reached (URL or IP)
  • CLUSTER_NAME: Specify the name of the cluster. Make sure this is unique for every additional cluster
  • CLUSTER_LOCATION: (optional) Specify the location of the cluster in the format <latitude>, <longitude>, <radius>
# Example configuration for cluster orchestrator
export SYSTEM_MANAGER_URL=192.158.18.104
export CLUSTER_NAME=My_Cluster
export CLUSTER_LOCATION=48.26275365157924,11.668627302307236,100

Watch out!

The root orchestrator has to be reachable by the cluster orchestrator. When not on the same network the root orchestrator URL has to be a public address!

Choose a Different Branch

By default these scripts will compose the services of the main oakestra branch, the latest stable release. However this can be changed by setting the environment variable OAKESTRA_BRANCH before running the startup script. This allows you to experiment with some unreleased features.

export OAKESTRA_BRANCH=develop

This will allow you to use the services from the alpha Oakestra release.

Note

Oakestra has many features which have not yet been released. You can check out what’s in the pipeline by taking a look at some of the active branches here.

Compose Overrides

Since Oakestra uses docker-compose to build the components, we can use overrides to fine-tune our build environment.

To use the override files, specify the them in a comma-seperated list by setting the OVERRIDE_FILES env variable before running the startup script.

export OVERRIDE_FILES=override-alpha-versions.yaml
Click to see overview of Root Orchestrator overrides
  • override-addons.yml: Eanble the addons engine and marketplace
  • override-no-dashboard.yml: Do not deploy the dashboard
  • override-no-network.yml: Exclude network components
  • override-ipv6-enabled.yml: Enable IPv6 for container deployments
  • override-no-observe.yml: Disable the observability stack
Click to see overview of Cluster Orchestrator overrides