Advanced Cluster Setup
System Requirements
Root and Cluster orchestrators:
- Docker + Docker Compose v2
- 5GB of Disk
- 500MB of RAM
- ARM64 or AMD64 architecture
Worker Nodes:
- Linux-based distro with
iptables
compatibility - 50MB of space
- 100MB RAM
- ARM64 or AMD64 architecture
Deploy Multiple Clusters
Having one root orchestrator and one cluster orchestrator on one device is a great way to start using Oakestra, but the true power of the system lies in it’s federated architecture.
This guide will walk you through deploying stand-alone components, allowing you to deploy multiple cluster orchestrators across multiple devices. These clusters will be managed by a single root orchestrator.
Stand-alone Root Orchestrator
First, let’s deploy a stand-alone root orchestrator. This component will manage your clusters and provide an interface for users to interact with the Oakestra setup by providing an API and a dashboard.
curl -sfL oakestra.io/install-root.sh | sh -
This script will download the required files to the directory ~/oakestra/root_orchestrator
. From there it will build the root orchestrator.
Network Configuration
If you run into a restricted network (e.g., on a cloud VM) you need to configure the firewall rules and the NetManager component accordingly. Please refer to: Firewall Setup
Stand-alone Cluster Orchestrator
Next, we can deploy a stand-alone cluster orchestrator. This component will manage the nodes by delegating applications, creating subnetworks and facilitating communication. Additionally the cluster orchestrator sends aggregated reports to the root orchestrator.
curl -sfL oakestra.io/install-cluster.sh | sh -
This script will download the required files to the directory ~/oakestra/cluster_orchestrator
. From there it will walk you through
configuring the cluster. Once the setup is complete, it will build the cluster orchestrator and register with the root orchestrator.
You can register as many cluster orchestrators with the root orchestrator as you would like. Repeat the above command on a new device and configure the orchestrator with a unique Cluster Name
and Cluster Location
.
Registering Nodes
Check out how to register worker nodes with a cluster
Custom Deployments
Oakestra also allows you to customize your system deployment to best suit your needs.
Oakestra Addons
Check out addons for even more customization options!
Configure the Orchestrators
The root and cluster orchestrator can be configured using environment variables. If they have not been set, they are automatically set when using the above scripts. They can also be manually configured, e.g. when building using the docker compose files.
Root Orchestrator Environment Variables
SYSTEM_MANAGER_URL
: Specify how the root orchestrator can be reached (URL or IP)
# Example configuration for root orchestrator
export SYSTEM_MANAGER_URL=192.158.18.104
Cluster Orchestrator Environment Variables
SYSTEM_MANAGER_URL
: Specify how the root orchestrator can be reached (URL or IP)CLUSTER_NAME
: Specify the name of the cluster. Make sure this is unique for every additional clusterCLUSTER_LOCATION
: (optional) Specify the location of the cluster in the format<latitude>, <longitude>, <radius>
# Example configuration for cluster orchestrator
export SYSTEM_MANAGER_URL=192.158.18.104
export CLUSTER_NAME=My_Cluster
export CLUSTER_LOCATION=48.26275365157924,11.668627302307236,100
Watch out!
The root orchestrator has to be reachable by the cluster orchestrator. When not on the same network the root orchestrator URL has to be a public address!
Choose a Different Branch
By default these scripts will compose the services of the main
oakestra branch, the latest stable release. However this can be changed by setting the environment variable OAKESTRA_BRANCH
before running the startup script.
This allows you to experiment with some unreleased features.
export OAKESTRA_BRANCH=develop
This will allow you to use the services from the alpha Oakestra release.
Note
Oakestra has many features which have not yet been released. You can check out what’s in the pipeline by taking a look at some of the active branches here.
Compose Overrides
Since Oakestra uses docker-compose to build the components, we can use overrides to fine-tune our build environment.
To use the override files, specify the them in a comma-seperated list by setting the OVERRIDE_FILES
env variable before running the startup script.
export OVERRIDE_FILES=override-alpha-versions.yaml
Click to see overview of Root Orchestrator overrides
override-addons.yml
: Eanble the addons engine and marketplaceoverride-no-dashboard.yml
: Do not deploy the dashboardoverride-no-network.yml
: Exclude network componentsoverride-ipv6-enabled.yml
: Enable IPv6 for container deploymentsoverride-no-observe.yml
: Disable the observability stack
Click to see overview of Cluster Orchestrator overrides
override-ipv6-enabled.yml
: Enable IPv6 for container deploymentsoverride-no-observe.yml
: Disable the observability stackoverride-mosquitto-auth.yml
: Enable MQTT Authentication