0b56574a5c
Bumps [DeterminateSystems/update-flake-lock](https://github.com/determinatesystems/update-flake-lock) from 22 to 23. - [Release notes](https://github.com/determinatesystems/update-flake-lock/releases) - [Commits](https://github.com/determinatesystems/update-flake-lock/compare/v22...v23) --- updated-dependencies: - dependency-name: DeterminateSystems/update-flake-lock dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <support@github.com> |
||
---|---|---|
.github | ||
bin | ||
buildbot_effects | ||
buildbot_nix | ||
examples | ||
nix | ||
.gitignore | ||
.mergify.yml | ||
default.nix | ||
flake.lock | ||
flake.nix | ||
pyproject.toml | ||
README.md |
Buildbot-nix
Buildbot-nix is a NixOS module designed to integrate Buildbot, a continuous integration (CI) framework, into the Nix ecosystem. This module is under active development, and while it's generally stable and widely used, please be aware that some APIs may change over time.
Getting Started with Buildbot Setup
To set up Buildbot using Buildbot-nix, you can start by exploring the provided examples:
- Check out the basic setup in example.
- Learn about configuring the Buildbot master in master module.
- Understand how to set up a Buildbot worker in worker module.
Additionally, you can find real-world examples at the end of this document.
Buildbot masters and workers can be deployed either on the same machine or on separate machines. To support multiple architectures, configure them as nix remote builders. For a practical NixOS example, see this remote builder configuration.
Using Buildbot in Your Project
Buildbot-nix automatically triggers builds for your project under these conditions:
- When a pull request is opened.
- When a commit is pushed to the default git branch.
It does this by evaluating the .#checks
attribute of your project's flake in
parallel. Each attribute found results in a separate build step. You can test
these builds locally using nix flake check -L
or
nix-fast-build.
If you need to build other parts of your flake, such as packages or NixOS
machines, you should re-export these into the .#checks
output. Here are two
examples to guide you:
- Using flake-parts.
- A plain flake example.
Authentication backend
At the moment all projects are visible without authentication.
For some actions a login is required. This login can either be based on GitHub
or on Gitea (more logins may follow). The backend is set by the
services.buildbot-nix.master.authBackend
NixOS option ("gitea"/"github",
"github" by default).
We have the following two roles:
- Admins:
- The list of admin usernames is hard-coded in the NixOS configuration.
- admins can reload the project list
- Organisation member:
- All member of the organisation where this repository is located
- They can restart builds
Integration with GitHub
GitHub App
This is the preferred option to setup buildbot-nix for GitHub.
To integrate with GitHub using app authentication:
- GitHub App:
- Create a new GitHub app by navigating to
https://github.com/settings/apps/new
for single-user installations orhttps://github.com/organizations/<org>/settings/apps/new
for organisations where<org>
is the name of your GitHub organizaction. - GitHub App Name: "buildbox-nix "
- Homepage URL:
https://buildbot.<your-domain>
- Callback URL:
https://buildbot.<your-domain>/auth/login
. - Disable the Webhook
- Repository Permissions:
- Contents: Read-only
- Commit statuses: Read and write
- Metadata: Read-only
- Webhooks: Read and write
- Create a new GitHub app by navigating to
- GitHub App private key: Get the app private key and app ID from GitHub,
configure using the buildbot-nix NixOS module.
- Set
services.buildbot-nix.master.github.authType.app.id = <your-github-id>;
- Set
services.buildbot-nix.master.github.authType.app.secretKeyFile = "/path/to.pem";
- Set
- Install App: Install the app for an organization or specific user.
- Refresh GitHub Projects: Currently buildbot-nix doesn't respond to changes (new repositories or installations) automatically, it is therefore necessary to manually trigger a reload or wait for the next periodic reload.
Token Auth
To integrate with GitHub using legacy token authentication:
- GitHub Token: Obtain a GitHub token with
admin:repo_hook
andrepo
permissions. For GitHub organizations, it's advisable to create a separate GitHub user for managing repository webhooks.
Optional when using GitHub login
- GitHub App: Set up a GitHub app for Buildbot to enable GitHub user authentication on the Buildbot dashboard. (can be the same as for GitHub App auth)
- OAuth Credentials: After installing the app, generate OAuth credentials
and configure them in the buildbot-nix NixOS module. Set the callback url to
https://<your-domain>/auth/login
.
Afterwards add the configured github topic to every project that should build with buildbot-nix. Notice that the buildbot user needs to have admin access to this repository because it needs to install a webhook.
Integration with Gitea
To integrate with Gitea
- Gitea Token Obtain a Gitea token with the following permissions
write:repository
andwrite:user
permission. For Gitea organizations, it's advisable to create a separate Gitea user. Buildbot-nix will use this token to automatically setup a webhook in the repository. - Gitea App: (optional). This is optional, when using GitHub as the
authentication backend for buildbot. Set up a OAuth2 app for Buildbot in the
Applications section. This can be done in the global "Site adminstration"
settings (only available for admins) or in a Gitea organisation or in your
personal settings. As redirect url set
https://buildbot.your-buildbot-domain.com/auth/login
, wherebuildbot.your-buildbot-domain.com
should be replaced with the actual domain that your buildbot is running on.
Afterwards add the configured gitea topic to every project that should build with buildbot-nix. Notice that the buildbot user needs to have repository write access to this repository because it needs to install a webhook in the repository.
Binary caches
To access the build results on other machines there are two options at the moment
Local binary cache (harmonia)
You can set up a binary cache on your buildbot-worker machine to make its nix store accessible from other machines. Check out the README of the project, for an example configuration
Cachix
Buildbot-nix also supports pushing packages to cachix. Check out the comment out example configuration in our repository.
Attic
Buildbot-nix does not have native support for pushing packages to attic yet. However it's possible to integrate run a systemd service as described in this example configuration. The systemd service watches for changes in the local buildbot-nix store and uploads the contents to the attic cache.
Real-World Deployments
See Buildbot-nix in action in these deployments:
The following instances run on GitHub:
- Nix-community infra: Configuration | Instance
- Mic92's dotfiles: Configuration | Instance
- Technical University Munich: Configuration | Instance
- Numtide: Instance
- Ngi0: Instance
The following instances integrated with Gitea:
- Clan infra: Configuration | Instance
Get in touch
We have a matrix channel at buildbot-nix.