refactor: eval performant target #40

Merged

vlinkz merged 1 commit from vlinkz/labs:targetperf into main

2025-10-15 03:23:26 +00:00

vlinkz commented

2025-10-13 00:46:02 +00:00

Owner

Using .extend function causes nix to not cache the evaluation of a package (a.extend {} != a.extend {} ). In effect this meant that every time we used

	          gcc-cross = packages.foundation.gcc.versions."13.2.0-stage2".extend {
            settings.target = config.platform.host;
          };
          binutils-cross = packages.foundation.binutils.versions."2.41-stage1".extend {
            settings.target = config.platform.host;
          };

we were performing full re-evaluations of binutils and gcc. To make this worse, since binutils and gcc depend on other binutils and gcc (sometimes with this .extend), the amount of evaluations would start to blow up.
Before this patch evaluating packages.foundation.gcc.latest.packages.x86_64-linux.x86_64-linux.package would evaluate binutils stage1 139 times, after this patch (and before target patch) it's 4.

Using `.extend` function causes nix to not cache the evaluation of a package (`a.extend {} != a.extend {}` ). In effect this meant that every time we used ``` gcc-cross = packages.foundation.gcc.versions."13.2.0-stage2".extend { settings.target = config.platform.host; }; binutils-cross = packages.foundation.binutils.versions."2.41-stage1".extend { settings.target = config.platform.host; }; ``` we were performing full re-evaluations of binutils and gcc. To make this worse, since binutils and gcc depend on other binutils and gcc (sometimes with this `.extend`), the amount of evaluations would start to blow up. Before this patch evaluating `packages.foundation.gcc.latest.packages.x86_64-linux.x86_64-linux.package` would evaluate `binutils` stage1 139 times, after this patch (and before target patch) it's 4.

vlinkz added 1 commit 2025-10-13 00:46:03 +00:00

refactor: eval performant target c4c94f40e4

srd424 commented

2025-10-13 09:44:11 +00:00

Member

Looks good on hydra: https://hydra.aux-cache.dev/jobset/aux-prs/labs-pr40

5225225 commented

2025-10-13 20:07:02 +00:00

First-time contributor

ah, so that's why my script to "evaluate all packages(that i can build(platform.build==i686 or x86_64))" was taking absolutely forever

with this patch cherrypicked it goes from "too long to bother re-running to check(an hour?) and OOMing my system" to a minute(which isn't bad for 800 or so packages)

ah, so that's why my script to "evaluate all packages(that i can build(`platform.build==i686 or x86_64`))" was taking absolutely forever with this patch cherrypicked it goes from "too long to bother re-running to check(an hour?) and OOMing my system" to a minute(which isn't bad for 800 or so packages)

srd424 commented

2025-10-13 22:13:56 +00:00

Member

Oh I think I hit that, too. It's why I'm only building bash from tidepool at the moment on hydra. I guess this means I have to revisit the wider package set?

vlinkz force-pushed targetperf from c4c94f40e4 to ec017a1b7a

2025-10-13 22:43:43 +00:00

Compare

vlinkz commented

2025-10-15 03:23:12 +00:00

Author

Owner

Going to merge this now to save people the evaluation pain

vlinkz merged commit e2f6de0b11 into main

2025-10-15 03:23:26 +00:00

vlinkz referenced this pull request from a commit

2025-10-15 03:23:28 +00:00