An operator for managing BuildKit instances on Kubernetes.
First, deploy one or more BuildkitTemplate resources that define the configuration and scheduling for BuildKit instances:
apiVersion: buildkit.seatgeek.io/v1alpha1
kind: BuildkitTemplate
metadata:
name: buildkit-arm64
namespace: some-ns
spec:
# This is a simplified example; many other spec fields are available.
rootless: true
port: 1234
buildkitdToml: |
[log]
format = "json"
[worker.oci]
enabled = true
max-parallelism = 3
cniPoolSize = 16
[worker.containerd]
enabled = false
resources:
default:
requests:
cpu: 500m
memory: 8Gi
limits:
memory: 8Gi
maximum:
cpu: 8
memory: 16Gi
scheduling:
nodeSelector:
kubernetes.io/arch: arm64
kubernetes.io/os: linux
tolerations:
- key: dedicated
operator: Equal
value: buildkit
effect: NoScheduleThen create any number of Buildkit resources that reference the templates:
apiVersion: buildkit.seatgeek.io/v1alpha1
kind: Buildkit
metadata:
name: buildkit-arm64-instance
namespace: some-ns
spec:
template: buildkit-arm64The operator will then deploy a BuildKit pod for each Buildkit resource, setting the TCP connection URL into the Buildkit resource's status:
# ...
status:
endpoint: tcp://10.1.2.3:1234Use the .status.endpoint field to connect to the BuildKit instance. When you're done, delete the Buildkit resource and the associated pod will be cleaned up automatically.
The easiest way to install the buildkit-operator is using the Helm chart:
# Install the latest stable release
helm install buildkit-operator \
oci://ghcr.io/seatgeek/charts/buildkit-operator \
--namespace buildkit-system \
--create-namespaceInstall latest from main branch (bleeding edge):
# Note: Chart versions for main branch use format 0.0.0-main-<commit-sha>
# You can find the exact version in GitHub Actions or use --devel flag
helm install buildkit-operator \
oci://ghcr.io/seatgeek/charts/buildkit-operator \
--namespace buildkit-system \
--create-namespace \
--set image.tag=main \
--develInstall a specific version:
helm install buildkit-operator \
oci://ghcr.io/seatgeek/charts/buildkit-operator \
--version 1.0.0 \
--namespace buildkit-system \
--create-namespaceTest a pull request:
helm install buildkit-operator \
oci://ghcr.io/seatgeek/charts/buildkit-operator \
--namespace buildkit-system \
--create-namespace \
--set image.tag=pr-123Container images are available at:
ghcr.io/seatgeek/buildkit-operator:latest- Latest stable release (tagged releases only)ghcr.io/seatgeek/buildkit-operator:main- Latest from main branchghcr.io/seatgeek/buildkit-operator:v1.0.0- Specific version releasesghcr.io/seatgeek/buildkit-operator:pr-123- Pull request builds
All images support both linux/amd64 and linux/arm64 architectures.
Note: Development versions from the main branch use commit-based chart versions (e.g., 0.0.0-main-abc12345) for better traceability and correlation with the source code.
helm uninstall buildkit-operator --namespace buildkit-systemYou will need kind and frpc, which can be installed on macOS with brew:
brew install kind frpc(Re)start your local kind cluster and run the operator:
make recreate
make runIf you'd rather run the operator from your IDE with a debugger attached, run these commands instead:
make recreate
make start_webhook_reverse_proxy # Keep this running in the background until you're done debuggingThen have your IDE run ./cmd/operator in debug mode with the following program arguments set: --kubeconfig ./kind/kubeconfig --kubecontext kind-buildkit
You can interact with the cluster either of these ways:
kubectl --kubeconfig ./kind/kubeconfig [command]k9s --kubeconfig ./kind/kubeconfig --write
Congratulations! You are now running buildkit-operator locally.