feat: add portable submodule type #11

Open

jakehamilton wants to merge 1 commit from feat/portable-submodules into main

jakehamilton commented

2024-10-08 19:26:55 +00:00

Owner

This change may not seem like a lot, but the implications of having such a type are pretty monumental. This is a generic, extracted version of the Package type in Tidepool. With this helper it is now possible to assign whole submodule definitions to other locations in the configuration, bringing along all options and avoiding issues with otherwise preevaluated defaults.

A primary issue this fixed is one where you have two options of the same submodule type. Imagine you want to assign the value of config.a to config.b. Well, config.a has been evaluated and results in an attribute set full of its final config, including default values. So when you assign config.b = config.a these old default values then become non-default overrides, making computed options no longer work. However, by re-evaluating the set of submodules (the config from a and the submodule definition itself) we are able to recompute these dynamic pieces!

Assignment

Notably, this type is pretty flexible in how it lets users interact with it. There are two different ways that the value may be set:

Raw configuration

A user can operate on this submodule just like any other, setting values like the following:

{
  config.a.myValue = 42;
}

The attribute set assigned to config.a is then transformed into a full module definition, containing the user's configuration:

{
 config.myValue = 42;
}

This module is then evaluated with all of the other definitions at this location and added to the submodule's __modules__ list to propagate. The resulting value is the full evaluation of all these submodules:

{
  __modules__ = [ <the-module-with-my-value> ];
  config = {
    myValue = 42;
  };
}

Portable reuse

An existing portable submodule value can be assigned to another location of the same type:

{
  config.b = config.a; # where `config.a` is a portable submodule
}

The submodule's __modules__ attribute will be detected and used directly. These modules are combined with the existing modules at this location and are evaluated just like before, producing the final value:

{
  __modules__ = [ <config-a-module> ];
  config = {
    # whatever was in `config.a`...
  };
}

Extension

It is quite common that you will want to use most of the values of a submodule, but you want to also modify a portion of it. Without any additional helper, this would require the use of lib.modules.merge to leverage the portable submodule type's merging behavior. However, this can be a bit cumbersome and we can remove the boilerplate by providing an extend helper. This function takes a module (or direct configuration) and returns a newly evaluated portable with the __modules__ list updated.

{
  config.a = {
    myValue = 42;
    myOtherValue = 43;
  };
  config.b = config.a.extend {
    myOtherValue = lib.modules.overrides.force 99;
  };
}

This results in the following value for config.b:

{
  myValue = 42;
  myOtherValue = 99;
}

Extension allows you to assign and operate on the values in one place rather than having to provide a list of definitions to merge.

This change may not seem like a lot, but the implications of having such a type are pretty monumental. This is a generic, extracted version of the [Package type in Tidepool](https://git.auxolotl.org/auxolotl/labs/src/commit/866b8902c975a1aaec547445976dd39d60def4ab/tidepool/src/lib/types.nix#L110). With this helper it is now possible to assign whole submodule definitions to other locations in the configuration, bringing along all options and avoiding issues with otherwise preevaluated defaults. A primary issue this fixed is one where you have two options of the same submodule type. Imagine you want to assign the value of `config.a` to `config.b`. Well, `config.a` has been evaluated and results in an attribute set full of its final config, including default values. So when you assign `config.b = config.a` these old default values then become non-default overrides, making computed options no longer work. However, by re-evaluating the set of submodules (the config from `a` and the submodule definition itself) we are able to recompute these dynamic pieces! ## Assignment Notably, this type is pretty flexible in how it lets users interact with it. There are two different ways that the value may be set: 1. Raw configuration A user can operate on this submodule just like any other, setting values like the following: ```nix { config.a.myValue = 42; } ``` The attribute set assigned to `config.a` is then transformed into a full module definition, containing the user's configuration: ```nix { config.myValue = 42; } ``` This module is then evaluated with all of the other definitions at this location and added to the submodule's `__modules__` list to propagate. The resulting value is the full evaluation of all these submodules: ```nix { __modules__ = [ <the-module-with-my-value> ]; config = { myValue = 42; }; } ``` 2. Portable reuse An existing portable submodule value can be assigned to another location of the same type: ```nix { config.b = config.a; # where `config.a` is a portable submodule } ``` The submodule's `__modules__` attribute will be detected and used directly. These modules are combined with the existing modules at this location and are evaluated just like before, producing the final value: ```nix { __modules__ = [ <config-a-module> ]; config = { # whatever was in `config.a`... }; } ``` ## Extension It is quite common that you will want to use most of the values of a submodule, but you want to also modify a portion of it. Without any additional helper, this would require the use of `lib.modules.merge` to leverage the portable submodule type's merging behavior. However, this can be a bit cumbersome and we can remove the boilerplate by providing an `extend` helper. This function takes a module (or direct configuration) and returns a newly evaluated portable with the `__modules__` list updated. ```nix { config.a = { myValue = 42; myOtherValue = 43; }; config.b = config.a.extend { myOtherValue = lib.modules.overrides.force 99; }; } ``` This results in the following value for `config.b`: ```nix { myValue = 42; myOtherValue = 99; } ``` Extension allows you to assign and operate on the values in one place rather than having to provide a list of definitions to merge.

❤️ 1 🎉 3

jakehamilton added 1 commit 2024-10-08 19:26:55 +00:00

feat: add portable submodule type

buildbot/nix-eval Build done.

Details

a75b76e86e

austreelis commented

2024-10-09 08:58:55 +00:00

Contributor

Love this ! I got hit by this when fleshing out mudenv (though I eventually took another approach).

Any reason why to keep them separate from lib.types.submodule/lib.types.submodules.of ? Assuming portable submodules are strictly more powerful, I don't see why we'd want to keep "non-portable" submodules around - unless I'm wrong and there's trade-offs between both.

Love this ! I got hit by this when fleshing out mudenv (though I eventually took another approach). Any reason why to keep them separate from `lib.types.submodule`/`lib.types.submodules.of` ? Assuming portable submodules are strictly more powerful, I don't see why we'd want to keep "non-portable" submodules around - unless I'm wrong and there's trade-offs between both.