auxolotl/labs
26
12
Fork 16
Code Issues 6 Pull requests 9 Projects Releases Packages Wiki Activity Actions

feat: custom targets #42

Open
vlinkz wants to merge 1 commit from vlinkz/labs:customtargets into main
pull from: vlinkz/labs:customtargets
merge into: auxolotl:main
Conversation 14 Commits 1 Files changed 30 +261 -151
vlinkz commented 2025-10-17 04:10:36 +00:00
Owner
Copy link

Allow for passing in custom targets. Along with this feature I added support for cross-compiling to use musl rather than glibc. In order to cross compile something targeting musl, you can run

nix-build --expr 'let tidepool = import ./tidepool; extraTargets = { config }: { config.systems.extraTargets = { x86_64-linux-musl = { system = "x86_64-unknown-linux-musl"; }; }; }; in (tidepool.extend { modules = [ extraTargets ]; }).config.packages.foundation.gcc.latest.packages.x86_64-linux.x86_64-linux-musl.package'

extraTargets are essentially anything that lib.systems.withBuildInfo can consume to turn into a full target system. While support is not implemented yet, you can imagine doing something like:

let
  tidepool = import ./tidepool;
  targets =
    { config }:
    {
      config.systems.extraTargets = {
        "my-custom-target" = {
          system = "x86_64-linux-musl";
          isStatic = true;
          gcc.arch = "x86-64-v4";
        };
      };
    };
in
tidepool.extend { modules = [ targets ]; }

On the package side, we can check these platform options and use the appropriate tool/configuration.

Allow for passing in custom targets. Along with this feature I added support for cross-compiling to use musl rather than glibc. In order to cross compile something targeting musl, you can run ``` nix-build --expr 'let tidepool = import ./tidepool; extraTargets = { config }: { config.systems.extraTargets = { x86_64-linux-musl = { system = "x86_64-unknown-linux-musl"; }; }; }; in (tidepool.extend { modules = [ extraTargets ]; }).config.packages.foundation.gcc.latest.packages.x86_64-linux.x86_64-linux-musl.package' ``` `extraTargets` are essentially anything that `lib.systems.withBuildInfo` can consume to turn into a full target system. While support is not implemented yet, you can imagine doing something like: ```nix let tidepool = import ./tidepool; targets = { config }: { config.systems.extraTargets = { "my-custom-target" = { system = "x86_64-linux-musl"; isStatic = true; gcc.arch = "x86-64-v4"; }; }; }; in tidepool.extend { modules = [ targets ]; } ``` On the package side, we can check these platform options and use the appropriate tool/configuration.
vlinkz added 1 commit 2025-10-17 04:10:38 +00:00
jakehamilton commented 2025-10-17 07:34:01 +00:00
Owner
Copy link

I really don't like the arg pattern. Why would we use this instead of writing a Nix expression (in an file) and building that instead?

I really don't like the `arg` pattern. Why would we use this instead of writing a Nix expression (in an file) and building that instead?
vlinkz commented 2025-10-17 16:21:26 +00:00
Author
Owner
Copy link

@jakehamilton wrote in #42 (comment):

I really don't like the arg pattern. Why would we use this instead of writing a Nix expression (in an file) and building that instead?

That was just to have a quick one-liner example. But I imagine actual consumers of tidepool would define extraTargets in a file when importing (similarly to how nixpkgs import defines overlays, system, etc).

@jakehamilton wrote in https://git.auxolotl.org/auxolotl/labs/pulls/42#issuecomment-1092: > I really don't like the `arg` pattern. Why would we use this instead of writing a Nix expression (in an file) and building that instead? That was just to have a quick one-liner example. But I imagine actual consumers of tidepool would define extraTargets in a file when importing (similarly to how nixpkgs import defines overlays, system, etc).
jakehamilton commented 2025-10-17 20:52:45 +00:00
Owner
Copy link

@vlinkz why not use the module system them?

@vlinkz why not use the module system them?
vlinkz commented 2025-10-17 22:10:38 +00:00
Author
Owner
Copy link

@jakehamilton wrote in #42 (comment):

@vlinkz why not use the module system them?

It does, you can set config.systems.extraTargets. Setting module option in lib is ok right?

Line 33 in 22da4d7
systems.extraTargets = (

@jakehamilton wrote in https://git.auxolotl.org/auxolotl/labs/pulls/42#issuecomment-1094: > @vlinkz why not use the module system them? It does, you can set `config.systems.extraTargets`. Setting module option in lib is ok right? https://git.auxolotl.org/auxolotl/labs/src/commit/22da4d71969b2bc2fab918e54e450630de47e9fc/tidepool/src/lib/systems.nix#L33
jakehamilton commented 2025-10-17 23:41:37 +00:00
Owner
Copy link

Right, so why is Tidepool now a function that requires taking an argument if we would rather set things on the module? It seems like this makes it less friendly/more complex to import.

Right, so why is Tidepool now a function that requires taking an argument if we would rather set things on the module? It seems like this makes it less friendly/more complex to import.
❤️ 1
vlinkz commented 2025-10-17 23:44:55 +00:00
Author
Owner
Copy link

Hm good point, in that I was trying to make it easier to test, but we could make a separate file for such use cases and leave default.nix as is. Lemme try that

Hm good point, in that I was trying to make it easier to test, but we could make a separate file for such use cases and leave default.nix as is. Lemme try that
vlinkz force-pushed customtargets from 22da4d7196 to 0d8ad0c11b 2025-10-18 00:49:52 +00:00 Compare
vlinkz commented 2025-10-18 00:56:16 +00:00
Author
Owner
Copy link

@jakehamilton wrote in #42 (comment):

Right, so why is Tidepool now a function that requires taking an argument if we would rather set things on the module? It seems like this makes it less friendly/more complex to import.

Updated to no longer use arg. Added updated examples for building packages with custom targets

@jakehamilton wrote in https://git.auxolotl.org/auxolotl/labs/pulls/42#issuecomment-1096: > Right, so why is Tidepool now a function that requires taking an argument if we would rather set things on the module? It seems like this makes it less friendly/more complex to import. Updated to no longer use arg. Added updated examples for building packages with custom targets
👍 1
jakehamilton reviewed 2025-10-18 02:49:24 +00:00
tidepool/src/lib/systems.nix Outdated
@ -2329,3 +2347,2 @@
doubles = {
resolved = builtins.map lib.systems.from.string lib.systems.doubles.all;
targets = {
jakehamilton commented 2025-10-18 02:49:24 +00:00
Owner
Copy link

Is this name accurate? The previous made it clear that this was a collection of platform doubles (arch + os), but this one seems ambiguous.

Is this name accurate? The previous made it clear that this was a collection of platform doubles (arch + os), but this one seems ambiguous.
vlinkz commented 2025-10-18 03:17:10 +00:00
Author
Owner
Copy link

the reason I elected to change it here was as once we add custom targets, they won't necessarily conform to arch+os. The change makes it more obvious that in order to get the double, you should (lib.systems.withBuildInfo target).double

the reason I elected to change it here was as once we add custom targets, they won't necessarily conform to arch+os. The change makes it more obvious that in order to get the double, you should `(lib.systems.withBuildInfo target).double`
jakehamilton reviewed 2025-10-18 02:49:59 +00:00
tidepool/src/lib/systems.nix Outdated
@ -1754,3 +1767,3 @@
value:
let
parts = lib.strings.split "-" value;
value' =
jakehamilton commented 2025-10-18 02:49:59 +00:00
Owner
Copy link

Let's avoid names using ' where possible. I find it mostly causes confusion.

Let's avoid names using `'` where possible. I find it mostly causes confusion.
👍 1
vlinkz marked this conversation as resolved
jakehamilton reviewed 2025-10-18 02:50:45 +00:00
tidepool/src/lib/systems.nix Outdated
@ -33,3 +44,4 @@
lib.systems = {
match = builtins.mapAttrs (lib.fp.const matchAnyAttrs) lib.systems.patterns;
extraTargets = config.systems.extraTargets;
jakehamilton commented 2025-10-18 02:50:45 +00:00
Owner
Copy link

Why not have a set for systems.targets that contains all of the defined targets and let users add to it directly?

Why not have a set for `systems.targets` that contains all of the defined targets and let users add to it directly?
👀 1
vlinkz commented 2025-10-18 04:39:43 +00:00
Author
Owner
Copy link

This one is tricky, as currently implemented, the user can enter a string parsable by lib.systems.from.string, or an attrset with custom options like { system = "x86_64-linux"; isStatic = true; linker = "mold"; }. The current list of targets is just a list of strings that is then converted into lib.systems.types.platformWithBuildInfo. If we exposed a list that the user appends to, they would need to do the lib.systems.withBuildInfo conversion manually on their side.

Edit: Oh also this line specifically was just leftover from testing, getting rid of it

This one is tricky, as currently implemented, the user can enter a string parsable by `lib.systems.from.string`, or an attrset with custom options like `{ system = "x86_64-linux"; isStatic = true; linker = "mold"; }`. The current list of targets is just a list of strings that is then converted into `lib.systems.types.platformWithBuildInfo`. If we exposed a list that the user appends to, they would need to do the `lib.systems.withBuildInfo` conversion manually on their side. Edit: Oh also this line specifically was just leftover from testing, getting rid of it
jakehamilton reviewed 2025-10-18 02:53:05 +00:00
tidepool/src/packages/foundation/musl/versions/1.2.4-stage1.nix Outdated
@ -13,2 +13,4 @@
in
{
options = {
dynamicLinker = lib.options.create {
jakehamilton commented 2025-10-18 02:53:05 +00:00
Owner
Copy link

Thoughts on making this linker? We probably also want to consider static linking as an option in the future.

Thoughts on making this `linker`? We probably also want to consider static linking as an option in the future.
vlinkz commented 2025-10-18 03:21:36 +00:00
Author
Owner
Copy link

The original idea was to disambiguate between the compile time linker (tools like bfd ld, mold, etc), and the systems dynamic linker loader (ld-linux-x86-64.so.2, libc.so, etc). Once we support alternative linkers there would probably be a naming collision

The original idea was to disambiguate between the compile time linker (tools like bfd `ld`, `mold`, etc), and the systems dynamic linker loader (`ld-linux-x86-64.so.2`, `libc.so`, etc). Once we support alternative linkers there would probably be a naming collision
vlinkz force-pushed customtargets from 0d8ad0c11b to e1ecf35f40 2025-10-18 04:44:39 +00:00 Compare
vlinkz changed title from wip: custom targets to feat: custom targets 2025-10-19 23:51:14 +00:00
srd424 commented 2025-10-20 09:47:48 +00:00
Member
Copy link

Hydra is happy: https://hydra.aux-cache.dev/jobset/aux-prs/labs-pr42 (it only builds bash though as an example, I haven't figured out a sane way to evaluate and build the whole tidepool set!)

Hydra is happy: https://hydra.aux-cache.dev/jobset/aux-prs/labs-pr42 (it only builds bash though as an example, I haven't figured out a sane way to evaluate and build the whole tidepool set!)
RossSmyth approved these changes 2025-10-21 03:00:15 +00:00
RossSmyth left a comment
Contributor
Copy link

Looks good to me. Only concern I might have is the conflation of "target" and "target":

  1. Target: When building GCC the platform/system that the built compiler will emit machine code for
  2. Target: A platform/system
Looks good to me. Only concern I might have is the conflation of "target" and "target": 1. Target: When building GCC the platform/system that the built compiler will emit machine code for 2. Target: A platform/system
srd424 commented 2025-12-25 18:08:16 +00:00
Member
Copy link

Hydra is still mostly happy, we need a rebase to pick up !53.

Hydra is still mostly happy, we need a rebase to pick up !53.
This pull request has changes conflicting with the target branch.
  • tidepool/src/builders/foundation/basic.nix
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u customtargets:vlinkz-customtargets
git checkout vlinkz-customtargets

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git checkout main
git merge --no-ff vlinkz-customtargets
git checkout vlinkz-customtargets
git rebase main
git checkout main
git merge --ff-only vlinkz-customtargets
git checkout vlinkz-customtargets
git rebase main
git checkout main
git merge --no-ff vlinkz-customtargets
git checkout main
git merge --squash vlinkz-customtargets
git checkout main
git merge --ff-only vlinkz-customtargets
git checkout main
git merge vlinkz-customtargets
git push origin main
Sign in to join this conversation.
Reviewers
No reviewers
RossSmyth
Labels
Clear labels
  
Compat
Breaking
Breaking change that won't be backward compatible

  
Kind/Bug

Something is not working

  
Kind/Documentation

Documentation changes

  
Kind/Enhancement

Improve existing functionality

  
Kind/Feature

New functionality

  
Kind/Security

This is security issue

  
Kind/Testing

Issue or pull request related to testing

  
Priority
Critical
The priority is critical

  
Priority
High
The priority is high

  
Priority
Low
The priority is low

  
Priority
Medium
The priority is medium

  
Reviewed
Confirmed
Issue has been confirmed

  
Reviewed
Duplicate
This issue or pull request already exists

  
Reviewed
Invalid
Invalid issue

  
Reviewed
Won't Fix
This issue won't be fixed

  
Status
Abandoned
Somebody has started to work on this but abandoned work

  
Status
Blocked
Something is blocking this issue or pull request

  
Status
Need More Info
Feedback is required to reproduce issue or to continue work

No labels
Compat
Breaking
Kind/Bug
Kind/Documentation
Kind/Enhancement
Kind/Feature
Kind/Security
Kind/Testing
Priority
Critical
Priority
High
Priority
Low
Priority
Medium
Reviewed
Confirmed
Reviewed
Duplicate
Reviewed
Invalid
Reviewed
Won't Fix
Status
Abandoned
Status
Blocked
Status
Need More Info
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
4 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: auxolotl/labs#42
No description provided.
Delete branch "vlinkz/labs:customtargets"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?