auxolotl/labs
26
12
Fork 16
Code Issues 6 Pull requests 9 Projects Releases Packages Wiki Activity Actions

WIP: basic capabilities #56

Draft
vlinkz wants to merge 3 commits from vlinkz/labs:capabilitiespoc into main
pull from: vlinkz/labs:capabilitiespoc
merge into: auxolotl:main
Conversation 5 Commits 3 Files changed 6 +184 -44
vlinkz commented 2025-11-18 08:30:02 +00:00
Owner
Copy link

init unpack, configure, and make capabilities.
I'm debating making higher level capabilities that import other capabilities, eg makecc would include unpack, configure, & make.

What do we think about including capabilities like

  includes = with capabilities; [
    unpack
    make
    configure
  ];

versus having them all imported with enable options

capabilities.unpack.enable = true;
capabilities.make = {
  enable = true;
  # other options
};

?

init unpack, configure, and make capabilities. I'm debating making higher level capabilities that import other capabilities, eg makecc would include unpack, configure, & make. What do we think about including capabilities like ```nix includes = with capabilities; [ unpack make configure ]; ``` versus having them all imported with enable options ```nix capabilities.unpack.enable = true; capabilities.make = { enable = true; # other options }; ``` ?
vlinkz added 3 commits 2025-11-18 08:30:05 +00:00
jakehamilton commented 2025-11-18 21:27:48 +00:00
Owner
Copy link

We should handle capabilities being defined like we do dependencies. They should be specified as an attribute set so that the values can be overridden, extended, set to null, etc.

We should handle capabilities being defined like we do dependencies. They should be specified as an attribute set so that the values can be overridden, extended, set to null, etc.
👀 1
RossSmyth requested changes 2025-12-03 19:49:29 +00:00
RossSmyth left a comment
Contributor
Copy link

This is great! My initial thoughts.

Also my notes about structuredAttrs, the structuredAttrs API is here:

Lines 840 to 845 in f511f6d
# TOTHINK: Currently this only validates for Bash variables.
# In the future we could/should allow users to have JSON exports
# as well, which have relaxes requirements compared to Bash.
#
# Either relaxing this var, or adding another var.
vars = lib.options.create {

This is great! My initial thoughts. Also my notes about `structuredAttrs`, the structuredAttrs API is here: https://git.auxolotl.org/auxolotl/labs/src/commit/f511f6d4eb1c13ce39f0cf70c9e3bb8f07ea98c8/tidepool/src/lib/types.nix#L840-L845
tidepool/src/capabilities/configure.nix
@ -0,0 +43,4 @@
);
in
{
configureConfigure = lib.dag.entry.between [ "unpack" ] [ "configure" ] ''
RossSmyth commented 2025-12-03 19:38:58 +00:00
Contributor
Copy link

It is a bit unfortunate that they cannot directly set the configure phase

It is a bit unfortunate that they cannot directly set the `configure` phase
tidepool/src/capabilities/configure.nix
@ -0,0 +44,4 @@
in
{
configureConfigure = lib.dag.entry.between [ "unpack" ] [ "configure" ] ''
bash ./configure \
RossSmyth commented 2025-12-03 19:34:37 +00:00
Contributor
Copy link

We shouldn't call bash ./configure directly because there is more software than you may realize that use Python, Perl, Scheme, or something else for their configure scripts. I would prefer to just use the shebangs, but Aux does not have a way to patch shebangs (yet), so I would instead just put a TODO.

We shouldn't call `bash ./configure` directly because there is more software than you may realize that use Python, Perl, Scheme, or something else for their configure scripts. I would prefer to just use the shebangs, but Aux does not have a way to patch shebangs (yet), so I would instead just put a TODO.
tidepool/src/capabilities/configure.nix
@ -0,0 +45,4 @@
{
configureConfigure = lib.dag.entry.between [ "unpack" ] [ "configure" ] ''
bash ./configure \
${configureFlags}
RossSmyth commented 2025-12-03 19:36:10 +00:00
Contributor
Copy link

One thing I'd like to devise is a way to pass arguments down the to derivation call so that we can use structuredAttrs for configure flags and such, as they means Aux doesn't need to worry about escaping and spaces, as it will Just Work since Nix does the proper escaping for us.

If you don't want to, can just put a TODO there.

One thing I'd like to devise is a way to pass arguments down the to derivation call so that we can use `structuredAttrs` for configure flags and such, as they means Aux doesn't need to worry about escaping and spaces, as it will Just Work since Nix does the proper escaping for us. If you don't want to, can just put a TODO there.
tidepool/src/capabilities/make.nix
@ -0,0 +32,4 @@
phases =
let
makeFlags = builtins.concatStringsSep " " config.make.flags;
RossSmyth commented 2025-12-03 19:38:03 +00:00
Contributor
Copy link

Same thing, TODO for structuredAttrs if you want, so we don't need to concat things with spaces naively.

As a minor improvement quoting each argument is probably fine.

Same thing, TODO for `structuredAttrs` if you want, so we don't need to concat things with spaces naively. As a minor improvement quoting each argument is probably fine.
tidepool/src/capabilities/unpack.nix
@ -0,0 +8,4 @@
;
in
{
options.unpack = {
RossSmyth commented 2025-12-03 19:47:08 +00:00
Contributor
Copy link

Should add an option for the command to run for unpacking, defaulting to tar xfz so that other formats can be unpacked (zip, deb, ...)

Should add an option for the command to run for unpacking, defaulting to `tar xfz` so that other formats can be unpacked (zip, deb, ...)
tidepool/src/capabilities/unpack.nix
@ -0,0 +33,4 @@
xz = packages.foundation.xz.versions."5.4.3-stage1-passthrough";
};
phases = {
RossSmyth commented 2025-12-03 19:39:38 +00:00
Contributor
Copy link

Same thing about structuredAttrs here as well.

Same thing about `structuredAttrs` here as well.
tidepool/src/capabilities/unpack.nix
@ -0,0 +35,4 @@
phases = {
unpackUnpack = lib.dag.entry.before [ "unpack" ] ''
before=$(ls -d */ 2>/dev/null | sort)
RossSmyth commented 2025-12-03 19:45:27 +00:00
Contributor
Copy link

I would recommend using find instead as ls is really only for human output, and ls will often adjust names for better human consumption (if they are not alphanumeric).

Also the glob will fail if there are names with dashes in it, so it should be something like ls -d -- */, or else they will becomes arguments to ls

I would recommend using `find` instead as `ls` is really only for human output, and `ls` will often adjust names for better human consumption (if they are not alphanumeric). Also the glob will fail if there are names with dashes in it, so it should be something like `ls -d -- */`, or else they will becomes arguments to `ls`
tidepool/src/packages/foundation/bash/versions/5.2.15-stage1.nix
@ -83,42 +79,19 @@ in
};
};
unpack.bootstrap = true;
RossSmyth commented 2025-12-03 19:47:34 +00:00
Contributor
Copy link

This looks great!

This looks great!
RossSmyth commented 2025-12-04 20:19:11 +00:00
Contributor
Copy link

They should be specified as an attribute set so that the values can be overridden, extended, set to null, etc.

Also I will not that I am not really for making every part of a definition overrideable. Because that implicitly means that every part of a definition is part of the public API, and users will assume that arbitrarily reaching into package definitions' guts and rearranging it is supported, and could lead to major ossification. When in reality each definition is generally only made to build a specific version of a specific package in a specific configuration.

So we'll have to think very carefully about what is actually exposed.

> They should be specified as an attribute set so that the values can be overridden, extended, set to null, etc. Also I will not that I am not really for making every part of a definition overrideable. Because that implicitly means that every part of a definition is part of the public API, and users will assume that arbitrarily reaching into package definitions' guts and rearranging it is supported, and could lead to major ossification. When in reality each definition is generally only made to build a specific version of a specific package in a specific configuration. So we'll have to think very carefully about what is actually exposed.
jakehamilton commented 2025-12-04 23:28:14 +00:00
Owner
Copy link

@RossSmyth wrote in #56 (comment):

They should be specified as an attribute set so that the values can be overridden, extended, set to null, etc.

Also I will not that I am not really for making every part of a definition overrideable. Because that implicitly means that every part of a definition is part of the public API, and users will assume that arbitrarily reaching into package definitions' guts and rearranging it is supported, and could lead to major ossification. When in reality each definition is generally only made to build a specific version of a specific package in a specific configuration.

So we'll have to think very carefully about what is actually exposed.

One extremely painful part of Nixpkgs / NixOS is the inability to modify things that weren't explicitly designed to support it. This is part of the reason that Aux Lib added Portable Submodules and Tidepool is so reactive. If a user wants to override a dependency entirely or just set a single setting on a dependency, then we should let them.

@RossSmyth wrote in https://git.auxolotl.org/auxolotl/labs/pulls/56#issuecomment-1545: > > They should be specified as an attribute set so that the values can be overridden, extended, set to null, etc. > > Also I will not that I am not really for making every part of a definition overrideable. Because that implicitly means that every part of a definition is part of the public API, and users will assume that arbitrarily reaching into package definitions' guts and rearranging it is supported, and could lead to major ossification. When in reality each definition is generally only made to build a specific version of a specific package in a specific configuration. > > So we'll have to think very carefully about what is actually exposed. One extremely painful part of Nixpkgs / NixOS is the inability to modify things that weren't explicitly designed to support it. This is part of the reason that Aux Lib added Portable Submodules and Tidepool is so reactive. If a user wants to override a dependency entirely or just set a single setting on a dependency, then we should let them.
RossSmyth commented 2025-12-05 02:49:10 +00:00
Contributor
Copy link

The problem with nixpkgs is that nothing really was designed for overriding. So now as fairly extensive problem within Nixpkgs is that some package will change something, then break some other package that was overriding it by accident, or someone out-of-tree. A common pattern is doing something like foo.overrideAttrs { src = otherVersion; }, but as I discussed above, packages are not generic builders. So when a package gets bumped, they risk breaking other packages by accident.

Now since Aux has versions built-in, that is less of an issue, but I also imagine that not all version will forever live in-tree to forever be maintained. So either they will receive patches, or the stable alias will change and then someone who is overriding it will break.

The problem with nixpkgs is that nothing really was designed for overriding. So now as fairly extensive problem within Nixpkgs is that some package will change something, then break some other package that was overriding it by accident, or someone out-of-tree. A common pattern is doing something like `foo.overrideAttrs { src = otherVersion; }`, but as I discussed above, packages are not generic builders. So when a package gets bumped, they risk breaking other packages by accident. Now since Aux has versions built-in, that is less of an issue, but I also imagine that not all version will forever live in-tree to forever be maintained. So either they will receive patches, or the stable alias will change and then someone who is overriding it will break.
This pull request is marked as a work in progress.
This branch is out-of-date with the base branch
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u capabilitiespoc:vlinkz-capabilitiespoc
git checkout vlinkz-capabilitiespoc

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git checkout main
git merge --no-ff vlinkz-capabilitiespoc
git checkout vlinkz-capabilitiespoc
git rebase main
git checkout main
git merge --ff-only vlinkz-capabilitiespoc
git checkout vlinkz-capabilitiespoc
git rebase main
git checkout main
git merge --no-ff vlinkz-capabilitiespoc
git checkout main
git merge --squash vlinkz-capabilitiespoc
git checkout main
git merge --ff-only vlinkz-capabilitiespoc
git checkout main
git merge vlinkz-capabilitiespoc
git push origin main
Sign in to join this conversation.
Reviewers
No reviewers
RossSmyth
Labels
Clear labels
  
Compat
Breaking
Breaking change that won't be backward compatible

  
Kind/Bug

Something is not working

  
Kind/Documentation

Documentation changes

  
Kind/Enhancement

Improve existing functionality

  
Kind/Feature

New functionality

  
Kind/Security

This is security issue

  
Kind/Testing

Issue or pull request related to testing

  
Priority
Critical
The priority is critical

  
Priority
High
The priority is high

  
Priority
Low
The priority is low

  
Priority
Medium
The priority is medium

  
Reviewed
Confirmed
Issue has been confirmed

  
Reviewed
Duplicate
This issue or pull request already exists

  
Reviewed
Invalid
Invalid issue

  
Reviewed
Won't Fix
This issue won't be fixed

  
Status
Abandoned
Somebody has started to work on this but abandoned work

  
Status
Blocked
Something is blocking this issue or pull request

  
Status
Need More Info
Feedback is required to reproduce issue or to continue work

No labels
Compat
Breaking
Kind/Bug
Kind/Documentation
Kind/Enhancement
Kind/Feature
Kind/Security
Kind/Testing
Priority
Critical
Priority
High
Priority
Low
Priority
Medium
Reviewed
Confirmed
Reviewed
Duplicate
Reviewed
Invalid
Reviewed
Won't Fix
Status
Abandoned
Status
Blocked
Status
Need More Info
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: auxolotl/labs#56
No description provided.
Delete branch "vlinkz/labs:capabilitiespoc"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?