Move lib
into a separate repository #7
No reviewers
Labels
No labels
Compat
Breaking
Kind/Bug
Kind/Documentation
Kind/Enhancement
Kind/Feature
Kind/Security
Kind/Testing
Priority
Critical
Priority
High
Priority
Low
Priority
Medium
Reviewed
Confirmed
Reviewed
Duplicate
Reviewed
Invalid
Reviewed
Won't Fix
Status
Abandoned
Status
Blocked
Status
Need More Info
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: auxolotl/core#7
Loading…
Reference in a new issue
No description provided.
Delete branch "tcmal/main"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
This PR moves lib out of this repository, and into a separate one here. This is meant to make maintenance easier by allowing us to develop the two separately.
It also removes the dependency on
nixpkgs
, and some usages of maintainers/teams that were stopping evaluation. Probably these should come from thecommunity
repo instead.I think at this point it would also be helpful to run
nixfmt
on the whole tree and start enforcing that, but I guess there's not really a good time to do that.I noticed that now https://github.com/auxolotl/lib has nixpkgs as an input, and as such nixpkgs is present within core's
flake.lock
file. Once we switch over to using core as the source fornix
, this would create a circular dependency. Would it be possible to have a separate flake for tests? Or perhaps resolve the dependence via this: flake.md.I think we could also have the
lib
flake export a function that takes a *pkgs instance and returns a derivation (but this makes running the tests a little weirder). Not sure which is best.Either way, will need to hold off on this until we have another way of getting
nixfmt
andnixdoc
, which are both used in the lib flake.As I mentioned here https://github.com/auxolotl/core/pull/2#discussion_r1598736880, that is the plan. But lets save that for a later PR.
Hopefully you can remove the upstream decency when https://github.com/auxolotl/core/pull/8 is merged. Then we can do
inputs.auxlib.inputs.core.follows = self;
or something like that?I'm not entirely convinced this is a good idea. What's the use case for taking lib out of core? The two are going to be very closely intertwined with each other (especially if lib is going to do anything relating to stringing together derivations a la
callPackage
), and putting it in another repo means it's one more place we have to manage issues and PRs - ideally, we'd like to keep that as low as possible while still giving SIGs the autonomy they need. Considering Core would likely be the ones maintaining lib, I'm not sure I see a pressing need to split them out at the moment. One might argue that it allows lib to be consumed independently of everything else, but if it's going to depend on Core anyway, that's kind of a moot point.I generally find the same amount of issues across multiple repos easier to handle than all in one repo. Synchronising changes is obviously harder, but this would be an issue even if
lib
was here, so long aslib
is consumed by sig repos.The set of functions in there right now is currently just the same as
nixpkgs
- there might be things we don't want to provide or need to rethink. If there are library functions that are actually dependent on the package set (ie trivial builders) then I agree they probably should be in here.My understanding is that
callPackage
deals with dependency injection and overrides, which we probably need to rethink for a multi-repo structure regardless. However we deal with it, this is a slightly different concern than packaging imo so it makes sense for it to not be here.Right now this is basically a limitation of flakes, since
core
is only really a dev dependency. Having thought about it more, I thinklib
should probably have a different flake inside it for docs and tests, unless there's some better way to handle this I'm not aware of (possibly just doing tests outside of flakes?).What about storing the tests in a separate branch with its own flake that depends on
lib
andcore
?Sure, or
tests
and docs` could just be in a subdirectory. Probably best to open an issue in auxolotl/libKeeping repo count down isn't just for maintainers - having an additional repo to track issues in adds additional confusion to users when they go to eg. report an issue ("is this a bug in Core or lib?"). There's also the extra overhead of being another repo to checkout, with the possibility of grabbing multiple different copies of it depending on the state of lockfiles. I'm not sure I'm seeing much benefit to outweigh the downsides here.
I'm weary of doing this - I'm not a fan of branches in one repo having completely divergent histories, it doesn't lend well to code discovery (since users have to be aware the branch exists to know that code is even there). Plus, it becomes impossible to update the tests and the actual code at the same time in one PR. It's also an issue that just wouldn't exist if lib was a part of the Core repo, which IMO furthers the case for not splitting.
To me they seem pretty easily separated, but I guess I would think that lol.
Agreed.
I think right now we don't have consensus on what exactly core is meant to be, so I think we should come back to this during/after the upcoming meeting.
I think it will be helpful on the maintaince side.
import <auxlib>
and I could go to the repo it was defined in, that would me much more approachable in terms of figuring out what code is actually doing.lib
andcore
later, but the reverse isn't necessarily true. So if we change our mind later its no big deal so long as they're split now.Even outside of maintaince though:
I have tried this a number of times for myself thinking it was a good idea. I've had a pretty bad time every time.
The only time it was a mediocre experience was when it was an "inherited" model, with one base branch and the others just always auto-merging that base branch into themselves. But even then it was a difficult mental model needing to not make a change on a branch, go to base, make the change, then back to the feature branch, and merge base. I very frequently forgot and had to redo commits on base.
Very much not an experience I would want to try with a team of people.
For issues, I agree @Sorixelle one place is nice. I had problems reporting "nix" issues and them telling me "no that's a nixpkgs issue" and I didnt understand the difference at the time. I think we should consider disabling issues on both lib and core, or locking them to contributors only. Then all issues can be reported through either an aux or auxpkgs repo. For lib and core, if its only contributors making issues, I think the separation of lib and core would be helpful.
For PR's I think having it as an additional place for PR's is very much a good thing. A core PR and a lib PR should have different standards and CI tests.
I'll throw my 2 cents on this here since I probably can't meet.
I think core is a useful separation for keeping messy bootstrapping isolated, for keeping a small mental model for onboarding new contributors trying to understand aux, for staying small to prevent security issues, and for ensuring we don't accidentally bloat the dependencies of the nix/aux cli (e.g. we shouldn't rope in nodejs just to get semver parsing). To keep it as small as possible, and to have an objective definition to avoid hurt feelings, I'm in favor of it being defined as the minimum packages necessary to build nix CLI. With the one additional job of making those packages (for building nix CLI) look as similar as possible across platforms (distinguishing platform-specific core from platform-agnostic core).
I say this specifically in contrast to the subjective idea that core is "commonly used build tools for all packages". Which I think, in the long run, will be the cause of many debates and hurt feelings if someone's favorite library isn't considered "core". It also isn't necessarily a helpful "soft" defintion for keeping the messy bootstrapping code isolated, since some commonly used stuff doesn't have messy bootstrapping (e.g. it would be just as easy to maintain a commonly-used package outside of core). If you guys end up wanting this kind of "broad" core definition despite this, I would at least advocate for an objective definition, like "is a dependency of at least 40% of packages", to avoid hurt-feelings and subjective debates later.
Checkout
From your project repository, check out a new branch and test the changes.Merge
Merge the changes and update on Forgejo.