Rapid iterative development with Tilt

Overview

This document describes how to use kind and Tilt for a simplified workflow that offers easy deployments and rapid iterative builds.

Prerequisites

Docker v19.03 or newer
kind v0.9 or newer (other clusters can be used if preload_images_for_kind is set to false)
kustomize
Tilt v0.30.8 or newer
envsubst or similar to handle clusterctl var replacement
Clone the Cluster API repository locally
Clone the cluster-api-provider-ibmcloud repository you want to deploy locally as well

Create a kind cluster

First, make sure you have a kind cluster and that your KUBECONFIG is set up correctly:

kind create cluster

This local cluster will be running all the cluster api controllers and become the management cluster which then can be used to spin up workload clusters on IBM Cloud.

Create a tilt-settings.yaml file

Next, create a tilt-settings.yaml file and place it in your local copy of cluster-api. Here is an example:

Example tilt-settings.yaml for CAPI-IBM clusters:

Make sure to replace the parameter IBMCLOUD_API_KEY with a valid API key.

default_registry: "gcr.io/you-project-name-here"
provider_repos:
- ../cluster-api-provider-ibmcloud
enable_providers:
- ibmcloud
- kubeadm-bootstrap
- kubeadm-control-plane
kustomize_substitutions:
  IBMCLOUD_API_KEY: "XXXXXXXXXXXXXXXXXX"

Add following extra_args to log PowerVS REST API Requests/Responses

extra_args:
  ibmcloud:
    - '-v=5'

1. Configuration to deploy PowerVS workload cluster with external cloud controller manager

To deploy workload cluster with PowerVS cloud controller manager which is currently in experimental stage, Set POWERVS_PROVIDER_ID_FORMAT to v2 and enable cluster resourceset feature gate under kustomize_substitutions. Currently, ClusterResourceset is experimental feature so we need to enable the feature gate by setting EXP_CLUSTER_RESOURCE_SET variable under kustomize_substitutions.

default_registry: "gcr.io/you-project-name-here"
provider_repos:
- ../cluster-api-provider-ibmcloud
enable_providers:
- ibmcloud
- kubeadm-bootstrap
- kubeadm-control-plane
kustomize_substitutions:
  IBMCLOUD_API_KEY: "XXXXXXXXXXXXXXXXXX"
  POWERVS_PROVIDER_ID_FORMAT: "v2"
  EXP_CLUSTER_RESOURCE_SET: "true"

2. Configuration to deploy VPC workload cluster with external cloud controller manager

To deploy workload cluster with cloud controller manager which is currently in experimental stage, Set PROVIDER_ID_FORMAT to v2 and enable cluster resourceset feature gate under kustomize_substitutions. Currently, ClusterResourceset is experimental feature so we need to enable the feature gate by setting EXP_CLUSTER_RESOURCE_SET variable under kustomize_substitutions.

default_registry: "gcr.io/you-project-name-here"
provider_repos:
- ../cluster-api-provider-ibmcloud
enable_providers:
- ibmcloud
- kubeadm-bootstrap
- kubeadm-control-plane
kustomize_substitutions:
  IBMCLOUD_API_KEY: "XXXXXXXXXXXXXXXXXX"
  PROVIDER_ID_FORMAT: "v2"
  EXP_CLUSTER_RESOURCE_SET: "true"

3. Configuration to deploy workload cluster from ClusterClass template

To deploy workload cluster with clusterclass-template under kustomize_substitutions set POWERVS_PROVIDER_ID_FORMAT to v2. Currently, both ClusterClass and ClusterResourceset are experimental feature so we need to enable the feature gate by setting EXP_CLUSTER_RESOURCE_SET, CLUSTER_TOPOLOGY variable under kustomize_substitutions.

default_registry: "gcr.io/you-project-name-here"
provider_repos:
- ../cluster-api-provider-ibmcloud
enable_providers:
- ibmcloud
- kubeadm-bootstrap
- kubeadm-control-plane
kustomize_substitutions:
  IBMCLOUD_API_KEY: "XXXXXXXXXXXXXXXXXX"
  POWERVS_PROVIDER_ID_FORMAT: "v2"
  EXP_CLUSTER_RESOURCE_SET: "true"
  CLUSTER_TOPOLOGY: "true"

4. Configuration to deploy workload cluster with Custom Service Endpoint

To deploy workload cluster with Custom Service Endpoint, Set SERVICE_ENDPOINT environmental variable in semi-colon separated format: ${ServiceRegion}:${ServiceID1}=${URL1},${ServiceID2}=${URL2...}

default_registry: "gcr.io/you-project-name-here"
provider_repos:
- ../cluster-api-provider-ibmcloud
enable_providers:
- ibmcloud
- kubeadm-bootstrap
- kubeadm-control-plane
kustomize_substitutions:
  IBMCLOUD_API_KEY: "XXXXXXXXXXXXXXXXXX"
  SERVICE_ENDPOINT: "us-south:vpc=https://us-south-stage01.iaasdev.cloud.ibm.com,powervs=https://dal.power-iaas.test.cloud.ibm.com,rc=https://resource-controller.test.cloud.ibm.com"
  IBMCLOUD_AUTH_URL: "https://iam.test.cloud.ibm.com"

NOTE: For information about all the fields that can be used in the tilt-settings.yaml file, check them here.

Run Tilt

To launch your development environment, run:

tilt up

Kind cluster becomes a management cluster after this point, check the pods running on the kind cluster by running kubectl get pods -A.

Create workload clusters

To provision your workload cluster, check the Creating a cluster section for VPC and PowerVS.

After deploying it, check the tilt logs and wait for the clusters to be created.

Clean up

Before deleting the kind cluster, make sure you delete all the workload clusters.

kubectl delete cluster <clustername>
tilt up (ctrl-c)
kind delete cluster

Kubernetes Cluster API Provider IBM Cloud