This folder contains the Terraform configuration to spawn the build bots.
Before deploying anything new, use
terraform plan to check that you're only modifying the parts that you intended to.
To set up your local machine to deploy changes to the cluster follow these steps:
llvm-zorg/buildbot/google/gcloud_config.shto configure the Google Cloud SDK.
export GOOGLE_APPLICATION_CREDENTIALS=~/.config/gcloud/legacy_credentials/<your email>/adc.json export GOOGLE_CLOUD_KEYFILE_JSON=$GOOGLE_APPLICATION_CREDENTIALSNote: Some documentation recommends
GOOGLE_CREDENTIALSfor this, however this does not work for accessing the Google Cloud Storage backend in terraform. You need to set both variables, as they are required by different tools
When deploying this cluster to a completely new Google Cloud project, these manual steps are required:
gsutil versioning set on gs://<bucket name>
To keep secrets a secret, they MUST not be stored in version control. The right place on kubernetes is a “secret”. To create a kubernetes secret for the agent token:
kubectl create secret generic buildbot-token-mlir-nvidia --from-file=token=<file name>
The file in
<file name> then must contain the password of the buildbot worker in plain text. In the
Deployment of a container, the secret is defined as a special type of volume and mounted in the specified path. During runtime the secret can then be read from that file.
An example: The secret
buildbot-token-mlir-nvidia is defined (as above) in Kubernetes. In the deployment
mlir-nvidia it is used as a volume of type
secret and then mounted at
/secrets. During the runtime of the docker container, the script run.sh reads the secret from the file
/secrets/token and uses it to create the worker configuration.
Terraform does not support deployments on GCP using a GPU at the moment. So we need to deploy such cases using plain Kubernetes configuration files. See this issue for more details. The buildbot mlir-nvidia is configured in
deployment-mlir-nvidia.yaml in this folder.
For all non-GPU cases add a
main.tf. The contents is identical to the the Kubernetes file, just the markup is different.
Kubernetes files are also declarative, so you can re-deploy them when you made a change. They can be deployed with:
kubectl apply -f myfile.yaml