auxolotl/labs

Fork 4

first "trivial" builders #10

New issue

Open

opened 2024-09-28 13:30:51 +00:00 by austreelis · 5 comments

austreelis commented

2024-09-28 13:30:51 +00:00

Contributor

I think equivalents to nixpkgs.writeText and nixpkgs.buildEnv could be added to either foundation or tidepool (or a new experiment if deemed better). This would enable mudenv to avoid relying on nixpkgs for such basic tools (the only dependencies should be bash and coreutils), and it would already unlock some other uses (like dotfiles management).

I can work on a pr once the details are figured out, mainly:

are all the building blocks already here: yes
where should such builders go: most probably tidepool
which builders should be implemented at first: writeTextFile, runCommand (with optional cc), concatTextFile, symlinkJoin.
how should their API look like

I think equivalents to `nixpkgs.writeText` and `nixpkgs.buildEnv` could be added to either `foundation` or `tidepool` (or a new experiment if deemed better). This would enable [mudenv](https://codeberg.org/austreelis/mudenv) to avoid relying on nixpkgs for such basic tools (the only dependencies should be `bash` and `coreutils`), and it would already unlock some other uses (like dotfiles management). I can work on a pr once the details are figured out, mainly: - [x] are all the building blocks already here: yes - [x] where should such builders go: most probably tidepool - [x] which builders should be implemented at first: `writeTextFile`, `runCommand` (with optional `cc`), `concatTextFile`, `symlinkJoin`. - [ ] how should their API look like

coded commented

2024-09-28 14:01:34 +00:00

Owner

could be added to either foundation or tidepool (or a new experiment if deemed better).

I guess it depends on whether or not we want something like nixpkgs lib or if we just want all of the pkg builders in the normal pkg set. I personally don't lean one way or the other.

> could be added to either foundation or tidepool (or a new experiment if deemed better). I guess it depends on whether or not we want something like nixpkgs lib or if we just want all of the pkg builders in the normal pkg set. I personally don't lean one way or the other.

coded commented

2024-09-28 14:16:34 +00:00

Owner

Though thinking about it. I think we have a basic builder in tidepool already, so it should probably go alongside that.

Though thinking about it. I think we have a [basic builder in tidepool](https://git.auxolotl.org/auxolotl/labs/src/branch/main/tidepool/src/builders/basic.nix) already, so it should probably go alongside that.

coded commented

2024-09-28 14:28:20 +00:00

Owner

As for the rest of the points:

are all the building blocks already here
We do have bash and coreutils packaged and x-compiled to x86

which builders should be implemented at first
Probably writeText, buildEnv, writeScript (and writeShellScript), runCommand, and the concat set (TextFile, Text, Script)

how should their API look like
I'm not sure I follow to much with this question, do you mean where they should be located under config.builders, what their inputs should look like, etc?

As for the rest of the points: > are all the building blocks already here We do have `bash` and `coreutils` packaged and x-compiled to x86 > which builders should be implemented at first Probably `writeText`, `buildEnv`, `writeScript` (and `writeShellScript`), `runCommand`, and the concat set (TextFile, Text, Script) > how should their API look like I'm not sure I follow to much with this question, do you mean where they should be located under config.builders, what their inputs should look like, etc?

austreelis commented

2024-09-28 17:28:39 +00:00

Author

Contributor

I'm not sure I follow to much with this question, do you mean where they should be located under config.builders, what their inputs should look like, etc?

Yes and yes. If they should be a builder like tidepool's basic, should text scripts be packages too ? This means that config options for widely different builders may need to be shared:

{
  config
}:
let
  inherit (config) builders;
in
{
  config.packages.aux.my-script.versions.latest =
    { ... }:
    {
      config = {
        name = "my-script";
        builder = builders.text;
        # New options for text files that any builder can use
        # Now any packages can specify these options, which makes little sense
        executable = true;
        destination = "bin/quantum-sort";
        text = ''
          if ! sorted $@; then
            reboot now
            exit 0
          fi
          echo $@
        '';
      };
    };
}

I'm not sure what kind of error happens (if any) in case the builder is mistakenly set to builders.basic for something that was supposed to be a text script, or vice-versa, but I'm not sure I'm a fan of this extremely loose coupling between builders and package definitions.

Ideally I'd want something like:

{
  config
}:
{
  # Not a package, a text file
  # I kept the version aliases but not sure that would be needed
  config.textFiles.aux.my-script.versions.latest =
    { ... }:
    {
      config = {
        name = "my-script";
        # don't specify a builder, it's chosen on the basis that it's a script
        # builder = ...
        # Text files-specific options that no other builder can use
        executable = true;
        destination = "bin/quantum-sort";
        text = ''
          if ! sorted $@; then
            reboot now
            exit 0
          fi
          echo $@
        '';
      };
    };
}

That kind of challenges the whole "builders" design in tidepool, and that's not the point of that issue, so it may be ok to just add a builder, some package options, and think about the grand scheme of tidepool's design later.

TL;DR: Should trivial builders just piggyback on the existing package submodule and "just be another builder", or be an entire separate business in order to have a potentially cleaner API.

Probably writeText, buildEnv, writeScript (and writeShellScript), runCommand, and the concat set (TextFile, Text, Script)

I agree. I think that I'd start with just writeTextFile (from which all the write{Shell,}{Text,Script,}{File,Dir,} can be derived), runCommand (with optional cc), concatTextFile (from which all others can be derived) and symlinkJoin (which is really just the buildEnv primitive and could be extended later).

I'd prefer to focus on getting basic but extensible building blocks.

> I'm not sure I follow to much with this question, do you mean where they should be located under config.builders, what their inputs should look like, etc? Yes and yes. If they should be a builder like tidepool's basic, should text scripts be packages too ? This means that config options for widely different builders may need to be shared: ```nix { config }: let inherit (config) builders; in { config.packages.aux.my-script.versions.latest = { ... }: { config = { name = "my-script"; builder = builders.text; # New options for text files that any builder can use # Now any packages can specify these options, which makes little sense executable = true; destination = "bin/quantum-sort"; text = '' if ! sorted $@; then reboot now exit 0 fi echo $@ ''; }; }; } ``` I'm not sure what kind of error happens (if any) in case the builder is mistakenly set to `builders.basic` for something that was supposed to be a text script, or vice-versa, but I'm not sure I'm a fan of this extremely loose coupling between builders and package definitions. Ideally I'd want something like: ```nix { config }: { # Not a package, a text file # I kept the version aliases but not sure that would be needed config.textFiles.aux.my-script.versions.latest = { ... }: { config = { name = "my-script"; # don't specify a builder, it's chosen on the basis that it's a script # builder = ... # Text files-specific options that no other builder can use executable = true; destination = "bin/quantum-sort"; text = '' if ! sorted $@; then reboot now exit 0 fi echo $@ ''; }; }; } ``` That kind of challenges the whole "builders" design in tidepool, and that's not the point of that issue, so it may be ok to just add a builder, some package options, and think about the grand scheme of tidepool's design later. **TL;DR:** Should trivial builders just piggyback on the existing package submodule and "just be another builder", or be an entire separate business in order to have a potentially cleaner API. --- > Probably `writeText`, `buildEnv`, `writeScript` (and `writeShellScript`), `runCommand`, and the concat set (TextFile, Text, Script) I agree. I think that I'd start with just `writeTextFile` (from which all the `write{Shell,}{Text,Script,}{File,Dir,}` can be derived), `runCommand` (with optional `cc`), `concatTextFile` (from which all others can be derived) and `symlinkJoin` (which is really just the `buildEnv` primitive and could be extended later). I'd prefer to focus on getting basic but extensible building blocks.

coded commented

2024-09-29 01:11:25 +00:00

Owner

I agree. I think that I'd start with just writeTextFile (from which all the write{Shell,}{Text,Script,}{File,Dir,} can be derived), runCommand (with optional cc), concatTextFile (from which all others can be derived) and symlinkJoin (which is really just the buildEnv primitive and could be extended later).

100% in agreement, the ones I listed are fairly easy to implement being extensions and could likely be done all in one go but your set makes more sense for a starting point.

TL;DR: Should trivial builders just piggyback on the existing package submodule and "just be another builder", or be an entire separate business in order to have a potentially cleaner API.

I'm of the mind it should be separate. Having config level categories would be quite nice. Your .textFiles is a good example, to go along with that we'd probably also want .commands and .envs

> I agree. I think that I'd start with just writeTextFile (from which all the write{Shell,}{Text,Script,}{File,Dir,} can be derived), runCommand (with optional cc), concatTextFile (from which all others can be derived) and symlinkJoin (which is really just the buildEnv primitive and could be extended later). 100% in agreement, the ones I listed are fairly easy to implement being extensions and could likely be done all in one go but your set makes more sense for a starting point. > TL;DR: Should trivial builders just piggyback on the existing package submodule and "just be another builder", or be an entire separate business in order to have a potentially cleaner API. I'm of the mind it should be separate. Having config level categories would be quite nice. Your `.textFiles` is a good example, to go along with that we'd probably also want `.commands` and `.envs`