auxolotl/labs
26
12
Fork 16
Code Issues 6 Pull requests 10 Projects Releases Packages Wiki Activity Actions

WIP: Make bootstrapping platform configurable #48

Draft
aleksi wants to merge 7 commits from aleksi/labs:wip-config-bootstrap-system into main
pull from: aleksi/labs:wip-config-bootstrap-system
merge into: auxolotl:main
Conversation 2 Commits 7 Files changed 74 +243 -296
aleksi commented 2025-10-29 12:20:09 +00:00
First-time contributor
Copy link

Not for merge just yet; aux/foundation currently supports only i686-linux for now, but this PR presents a demo of how tidepool could leverage different bootstrap platforms. I'm working on supporting riscv64-linux in the bootstrap, and @srd424 is working on x86_64-linux as well.

In case you are wondering why I separated bootstrap and build here: The separation may be useful in cases where the bootstrap does not support the desired build platform, yet where a (potentially emulated) bootstrap can cross-compile an userland to run on the final build platform. For example, the user might have a very slow i686-linux machine and a very fast aarch64-linux machine, and so they want to bootstrap natively on the slow machine and then perform most compiling on their aarch64-linux. Please let me know what your thoughts on this approach are!

In any case, this PR allows the user to do e.g.

  nix build -f . packages.foundation.bash.latest.packages.\
     i686-linux.x86_64-linux.package

to use a riscv64-linux bootstrap, then compile a tidepool foundation running on i686-linux targeting x86_64-linux, and then use this to cross-compile bash to run on x86_64-linux. I also tested the plain x86_64-linux->x86_64-linux case and it seemed to work good as well.

I was especially not sure about whether my way of configuring the platforms here is semantically correct. Also, builders/foundation/basic.nix assumes that the bash to be used should either come from aux/foundation stage2 (if build == bootstrap) or should be bootstrap->build-cross compiled. I'm not sure if this is what we want, or if we should rather build up another layer to ultimately build->build-compile a bash which is then used to build anything that wants build->host.

Sorry about the verbose wall of text here, please let me know if anything is unclear as well.

Not for merge just yet; aux/foundation currently supports only i686-linux for now, but this PR presents a demo of how tidepool could leverage different bootstrap platforms. I'm working on supporting riscv64-linux in the bootstrap, and @srd424 is working on x86_64-linux as well. In case you are wondering why I separated bootstrap and build here: The separation may be useful in cases where the bootstrap does not support the desired build platform, yet where a (potentially emulated) bootstrap can cross-compile an userland to run on the final build platform. For example, the user might have a very slow `i686-linux` machine and a very fast `aarch64-linux` machine, and so they want to bootstrap natively on the slow machine and then perform most compiling on their `aarch64-linux`. Please let me know what your thoughts on this approach are! In any case, this PR allows the user to do e.g. ``` nix build -f . packages.foundation.bash.latest.packages.\ i686-linux.x86_64-linux.package ``` to use a `riscv64-linux` bootstrap, then compile a tidepool foundation running on `i686-linux` targeting `x86_64-linux`, and then use this to cross-compile bash to run on `x86_64-linux`. I also tested the plain `x86_64-linux->x86_64-linux` case and it seemed to work good as well. I was especially not sure about whether my way of configuring the platforms here is semantically correct. Also, builders/foundation/basic.nix assumes that the bash to be used should either come from aux/foundation stage2 (if build == bootstrap) or should be bootstrap->build-cross compiled. I'm not sure if this is what we want, or if we should rather build up another layer to ultimately build->build-compile a bash which is then used to build anything that wants build->host. Sorry about the verbose wall of text here, please let me know if anything is unclear as well.
aleksi added 7 commits 2025-10-29 12:20:12 +00:00 
This patch is a little dubious and I'm not sure I understand the
purpose of stage1-passthrough completely.
Without this flags, bash might hit errors with old-style C function
prototypes. It can also be fixed by upgrading bash.
Not for merge just yet. This allows the user to do e.g.

  nix build -f . packages.foundation.bash.latest.packages.\
     i686-linux.x86_64-linux.package

to bootstrap a riscv64-linux aux/foundation, then compile an i686-linux
tidepool foundation targeting x86_64-linux, and then use this to
cross-compile bash for x86_64-linux.
aleksi force-pushed wip-config-bootstrap-system from e86cf9b0d4 to 372bd96ff3 2025-10-29 12:22:29 +00:00 Compare
aleksi reviewed 2025-10-29 12:26:36 +00:00
tidepool/default.nix Outdated
@ -19,0 +26,4 @@
};
platform.bootstrap = lib.options.create {
type = lib.types.string;
default.value = "i686-linux";
aleksi commented 2025-10-29 12:26:36 +00:00
Author
First-time contributor
Copy link

Once more platforms are supported in bootstrap, this default could have a smarter automatic selection, preferring a platform that can run natively on the specific final build platform if possible.

Once more platforms are supported in bootstrap, this default could have a smarter automatic selection, preferring a platform that can run natively on the specific final build platform if possible.
RossSmyth reviewed 2025-10-29 14:19:27 +00:00
tidepool/default.nix Outdated
@ -22,1 +33,3 @@
type = lib.types.attrs.of lib.types.derivation;
config = {
internal.packages.foundation = foundation;
platform.bootstrap = "riscv64-linux";
RossSmyth commented 2025-10-29 14:19:13 +00:00
Contributor
Copy link

is this a good value to set?

is this a good value to set?
aleksi commented 2025-10-29 17:56:51 +00:00
Author
First-time contributor
Copy link

Nah, this assignment is really here only for the sake of testing. Perhaps instead there could be something akin to .packages.x86_64-linux... to allow choosing the bootstrap platform from the CLI in a convenient way.

Nah, this assignment is really here only for the sake of testing. Perhaps instead there could be something akin to `.packages.x86_64-linux...` to allow choosing the bootstrap platform from the CLI in a convenient way.
tidepool/src/packages/foundation/glibc/versions/2.38-bootstrap.nix Outdated
@ -66,0 +73,4 @@
"ld-linux-riscv64-lp64d.so.1"
else
"ld-linux.so.2"
);
RossSmyth commented 2025-10-29 14:16:01 +00:00
Contributor
Copy link

I would recommend something like this instead:

linkerMap = {
  x86_64-linux = "ld-linux-x86-64.so.2";
  aarch64-linux = "ld-linux-aarch64.s0.1";
  ...
};
dynamicLinker = linkerMap.${global.platform.bootstrap} or "ld-linux.so.2";

as it make the mapping clear, and the default clear.

I would recommend something like this instead: ```nix linkerMap = { x86_64-linux = "ld-linux-x86-64.so.2"; aarch64-linux = "ld-linux-aarch64.s0.1"; ... }; dynamicLinker = linkerMap.${global.platform.bootstrap} or "ld-linux.so.2"; ``` as it make the mapping clear, and the default clear.
aleksi commented 2025-10-29 17:57:28 +00:00
Author
First-time contributor
Copy link

Ah, that makes a lot of sense! I'll be updating it shortly.

Ah, that makes a lot of sense! I'll be updating it shortly.
RossSmyth commented 2025-11-02 21:15:52 +00:00
Contributor
Copy link

Oh yeah, another thing that I forgot about when reviewing, but have now remembered, is that this is only true on glibc platforms. So on musl this would be different. I would give it a think about how to design it so it is relatively painless for musl or other libcs, even if it's not explicitly done here.

Oh yeah, another thing that I forgot about when reviewing, but have now remembered, is that this is only true on glibc platforms. So on musl this would be different. I would give it a think about how to design it so it is relatively painless for musl or other libcs, even if it's not explicitly done here.
aleksi commented 2025-11-03 13:38:06 +00:00
Author
First-time contributor
Copy link

How about we add a similar option to the musl package, and then require that any libc implementation package must expose such an option? That is, if I've understood the concern correctly 😅

How about we add a similar option to the musl package, and then require that any libc implementation package must expose such an option? That is, if I've understood the concern correctly 😅
RossSmyth commented 2025-12-04 15:07:21 +00:00
Contributor
Copy link

Yeah that sounds good. Basically on musl the linker has a different name (ld-musl-x86_64.so.1 and such)

Yeah that sounds good. Basically on musl the linker has a different name (`ld-musl-x86_64.so.1` and such)
aleksi force-pushed wip-config-bootstrap-system from 372bd96ff3 to 01f43626ef 2025-10-29 18:20:28 +00:00 Compare
This pull request has changes conflicting with the target branch.
  • tidepool/src/builders/foundation/basic.nix
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u wip-config-bootstrap-system:aleksi-wip-config-bootstrap-system
git checkout aleksi-wip-config-bootstrap-system

Merge

Merge the changes and update on Forgejo.

Warning: The "Autodetect manual merge" setting is not enabled for this repository, you will have to mark this pull request as manually merged afterwards.

git checkout main
git merge --no-ff aleksi-wip-config-bootstrap-system
git checkout aleksi-wip-config-bootstrap-system
git rebase main
git checkout main
git merge --ff-only aleksi-wip-config-bootstrap-system
git checkout aleksi-wip-config-bootstrap-system
git rebase main
git checkout main
git merge --no-ff aleksi-wip-config-bootstrap-system
git checkout main
git merge --squash aleksi-wip-config-bootstrap-system
git checkout main
git merge --ff-only aleksi-wip-config-bootstrap-system
git checkout main
git merge aleksi-wip-config-bootstrap-system
git push origin main
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
  
Compat
Breaking
Breaking change that won't be backward compatible

  
Kind/Bug

Something is not working

  
Kind/Documentation

Documentation changes

  
Kind/Enhancement

Improve existing functionality

  
Kind/Feature

New functionality

  
Kind/Security

This is security issue

  
Kind/Testing

Issue or pull request related to testing

  
Priority
Critical
The priority is critical

  
Priority
High
The priority is high

  
Priority
Low
The priority is low

  
Priority
Medium
The priority is medium

  
Reviewed
Confirmed
Issue has been confirmed

  
Reviewed
Duplicate
This issue or pull request already exists

  
Reviewed
Invalid
Invalid issue

  
Reviewed
Won't Fix
This issue won't be fixed

  
Status
Abandoned
Somebody has started to work on this but abandoned work

  
Status
Blocked
Something is blocking this issue or pull request

  
Status
Need More Info
Feedback is required to reproduce issue or to continue work

No labels
Compat
Breaking
Kind/Bug
Kind/Documentation
Kind/Enhancement
Kind/Feature
Kind/Security
Kind/Testing
Priority
Critical
Priority
High
Priority
Low
Priority
Medium
Reviewed
Confirmed
Reviewed
Duplicate
Reviewed
Invalid
Reviewed
Won't Fix
Status
Abandoned
Status
Blocked
Status
Need More Info
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: auxolotl/labs#48
No description provided.
Delete branch "aleksi/labs:wip-config-bootstrap-system"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?