Commit graph

55 commits

Author SHA1 Message Date
Naïm Favier 1128c8fb69
Bundle pandoc filters (#670)
Continuation of https://github.com/NixOS/nixos-search/pull/669
2023-07-03 22:03:00 +02:00
Naïm Favier 67bac7fe6d
Bundle link-manpages.nix (#669)
It's being removed from nixpkgs as part of the dedocbookification
process. Eventually we'll want to use nixos-render-docs to render
options.
2023-07-03 20:29:32 +02:00
Naïm Favier f31c2a395e
Render descriptions as Markdown (#667)
* Render descriptions as Markdown

Following https://github.com/NixOS/nixpkgs/pull/237557, DocBook
descriptions are not supported any more.

We cannot use options.json any more because it still has DocBook
descriptions in 22.11, so read the options from the modules directly.

* Bump VERSION
2023-06-28 17:44:44 +02:00
Naïm Favier 496623d4f9
flake-info: don't render definition lists (#627)
* flake-info: don't render definition lists

Nothing uses them in nixpkgs and they mess up descriptions that have a
line starting with a tilde.

* Bump VERSION
2023-02-22 14:28:44 +01:00
Naïm Favier e5a15ec2e3
Auto-link bare URIs in MarkDown (#621) 2023-02-16 10:53:47 +01:00
Naïm Favier dff4ba7132
Search programs provided by a package (#610)
* Search programs provided by a package

Use the `programs.sqlite` database provided with nixpkgs channels to
populate a `package_programs` field so that searches for e.g. `make` return
`gnumake` with a higher priority.

* Bump VERSION

* frontend: show programs
2023-02-07 16:31:17 +01:00
Naïm Favier 32097fc62e
flake-info: abort early if index exists (#609)
Make the exports lazy using a closure, so that we can abort early if
the index exists before doing the expensive work.

Raise an error if neither `--json` nor `--push` is specified.

Flakes are staying strict for now because determining the index name is
too tightly coupled with the actual exports.
2023-02-07 16:30:51 +01:00
Naïm Favier eb9fa64c33
flake-info: render longDescription for flake packages (#607) 2023-02-05 10:24:26 +01:00
Naïm Favier 651ef6fc62
flake-info: update manpage filter (#597)
Update the nixpkgs input and use the new link-manpages filter to avoid
duplicated links.

Co-authored-by: Rok Garbas <rok@garbas.si>
2023-01-27 18:50:29 +01:00
Naïm Favier 25e8dbdb94
flake-info: remove broken NIXOS_CHANNELS check (#586)
Fixing it would be undesirable.

Co-authored-by: Rok Garbas <rok@garbas.si>
2023-01-27 18:49:26 +01:00
Naïm Favier 9e5ff1ceec
flake-info: support platform patterns (#602)
...for some value of "support". For now, just skip them without causing
an error. Later, maybe figure out a way to display them properly.

An example of a pattern is `lib.systems.inspect.patterns.isGnu`.

Co-authored-by: Rok Garbas <rok@garbas.si>
2023-01-27 18:37:52 +01:00
Naïm Favier d04f341a5d
Use path_hierarchy tokenizer for attr paths (#584) 2022-12-23 13:39:45 +01:00
Naïm Favier 4bea45b150
Remove reverse fields (#579) 2022-12-06 23:58:51 +01:00
Naïm Favier 3a12920489
Take meta.badPlatforms into account (#580) 2022-12-06 23:30:58 +01:00
Naïm Favier 66162e9896
Give more license to the license field (#572) 2022-12-02 13:46:35 +01:00
Naïm Favier ba0b05b998
Render Markdown in option docs for flakes and longDescription (#539) 2022-10-28 15:49:15 +02:00
Naïm Favier 4776108b4b
Allow modulesPath in NixOS modules (#536) 2022-09-08 16:29:53 +02:00
Naïm Favier 9e64e2f6c3
Ignore _module options for flakes (#528)
Having the documentation for `_module.args` in the nixpkgs options is useful,
but there's no need to have a copy of it per flake module.
2022-09-08 10:45:01 +02:00
Yannik Sander 7d1c1046ba
Allow elastic feature to be disabled (#529) 2022-08-30 10:13:08 +02:00
Naïm Favier b8518f6d48
flake-info: fix compilation warnings (#503)
Removes some dead code, including the unimplemented `--elastic-user`,
`--elastic-pw` and `--gc` flags. Builds and tests with no warnings.
2022-08-29 17:17:29 +02:00
Philip Munksgaard 9e5f095369
Add support for sourcehut links (#517)
* Add support for SourceHut flakes

* Update flake-info/README.md

Co-authored-by: Naïm Favier <n@monade.li>

Co-authored-by: Naïm Favier <n@monade.li>
2022-08-23 14:42:06 +02:00
Naïm Camille Favier fd53fda132
flake-info: remove --read-only (#504) 2022-07-24 13:07:31 +02:00
Naïm Camille Favier c43ed8c85f
Disallow IFD in flakes (#502)
I think it is surprising that `flake-info` tries to build things in
order to index flakes. Hence, forbid IFD.

`--read-only` is for performance, see https://github.com/NixOS/nix/pull/6323
2022-07-21 12:27:21 +02:00
Naïm Camille Favier 18e5da679b
Don't panic during export (#501) 2022-07-21 12:20:52 +02:00
Naïm Camille Favier 75e8457105
flake-info: Stop relying on user's NIX_PATH for flake extraction (#487) 2022-06-06 12:42:40 +02:00
Naïm Camille Favier d0ea61ad4e
Rely on upstream for extracting nixpkgs options (#460)
* flake-info: rely on upstream for extracting nixpkgs options

Use the `options.json` generated in nixpkgs to be more in sync with the
manual.

Also fixes usage of `nixpkgs-archive` with local paths.

* Bump VERSION
2022-05-19 12:57:42 +02:00
Rok Garbas eea6cd3ffb
Feat/dynamic import (#473)
* Make the channel importer versions dynamic

* few things fromt he list

1. add nixos-org-configurations as input to flake.nix
2. evaluate channels.nix file and export channels via environment variable.
   that environment variable (lets call it NIXOS_CHANNELS) should be present
   during the build and inside the nix shell. the content of the variable can
   be JSON.
3. we pickup the NIXOS_CHANNELS environment variable in
   frontend/webpack.config.js and pass it further to webpack process, just
   like we do with ELASTICSEARCH_MAPPING_SCHEMA_VERSION.
4. we forward NIXOS_CHANNELS to Elm via frontend/src/index.js as an Elm
   application flag. Just like we do with other variables there.

* Decode nixosChannels in Elm

* Use nixosChannels that came via application flag

* read nixos channels in github action

* defaultNixOSChannel should be calculated

* add two pointers where the check should be added

* pass nixosChannels to flake-info and remove title, rather calculate it

* Add NixosChannels struct validation and validation Error

* Read NIXOS_CHANNEL variable

* Check channel

* Add channel struct to fix parsing NIXOS_CHANNELS

* Use `eachDefaultSystem` instead of listing them manually

* Add individual dev shells for frontend and backend

* Update .github/workflows/import-to-elasticsearch.yml

Co-authored-by: Naïm Favier <n@monade.li>

* use both development environments by default (as it was)

but still provide devShells for each of the subprojects

* pkgs.lib → lib everywhere

and define lib = nixpkgs.lib before the call to eachDefaultSystem
Also, version = lib.fileContents ./VERSION;

* Update flake.nix

Co-authored-by: Naïm Favier <n@monade.li>

* typo

* bumping version to test the changes to import-to-elasticsearch github action

* some invisibile characters needed to be removed

* typo

* will this work

* typo

* forgot the checkout step

* add some debugging

* typo

* read NIXOS_CHANNELS from environment not via argument

* fix for the NIXOS_CHANNELS variable

Co-authored-by: Janne Heß <janne@hess.ooo>
Co-authored-by: ysndr <me@ysndr.de>
Co-authored-by: Naïm Favier <n@monade.li>
2022-04-24 23:48:01 +02:00
Naïm Favier ff22728d9a
Fix display of null and missing defaults and examples (#457)
* flake-info: fix import of `null` defaults and examples

Currently they're being treated as missing fields, and hence displayed
as "Not given" on the frontend.

* Bump VERSION

* frontend/Options: don't show missing fields

Co-authored-by: Rok Garbas <rok@garbas.si>
2022-04-07 23:57:35 +02:00
Naïm Favier 5798bb000e
Show default outputs (#452)
* Show default outputs

Following https://github.com/NixOS/nix/pull/6242, we now have access to
default outputs so we can single them out in the frontend.

We can also drop `--out-path`, which speeds up import quite a lot and
doesn't require increasing swap space or allowing broken/unfree
packages anymore.

* Use nixUnstable in CI

* Bump VERSION
2022-03-27 15:52:54 +02:00
Naïm Favier b2e0ce28fc
flake-info: enforce minimal nix version (#451)
Co-authored-by: Yannik Sander <me@ysndr.de>
2022-03-18 21:57:28 +01:00
Naïm Favier 6e2650b6d3
flake-info: don't allow options to evaluate disallowed packages (#449)
Options shouldn't evaluate packages in documentation attributes.
These are errors that should be fixed in nixpkgs, which
https://github.com/NixOS/nixpkgs/pull/136909 did.
2022-03-17 22:31:30 +01:00
Yannik Sander d2ca8bb5c9
Import Nixpkgs from path (#445) 2022-03-17 08:14:36 +01:00
Naïm Favier 7be68b6dc0
Show package outputs (#419)
As mentioned in [^1] this PR includes a bump in swap size of the github action evaluating nixpkgs.
This is subject to change once a tentative change to Nix is merged.

^1: https://github.com/NixOS/nixos-search/pull/419#issuecomment-1065356169

—

* flake-info: use packages-config.nix straight from nixpkgs

No need to override anything anymore, see discussion at
https://github.com/NixOS/nixos-search/pull/343#issuecomment-1021147104

* flake-info: query package outputs

`package_outputs` is now set to the actual outputs of the derivation
instead of `meta.outputsToInstall`

Also updates nixpkgs to get Rust 1.57 which has HashMap.into_keys

* frontend: show package outputs

* Drop 21.05 channel

* Increase swap space in import-nixpkgs worker

* Bump VERSION

* frontend: improvements and refactoring

- move licenses to the end (without a line break)
- pluralise "Licenses:" conditionally
- change the homepage emoji from a house to a world
- replace backwards List.append pipelines with ++ chains
- avoid text nodes in <ul> outside of <li>
- improve rendering of maintainers

* frontend: sort package outputs
2022-03-16 10:43:20 +01:00
Yannik Sander 33da3a4a0c
Add group reports for flake-group check (#436)
* `nix flake {info -> metadata}`

* Fix error variant for io errors

* Enable backtrace support for anyhow

* Improve error printing

* Write error report file

* Format workflow file

* Use report file

* Set non-zero exit status if a group fails

* Do not use `local`

* Apply suggestions from review

* Move exit outside the loop

* Fix multi line output

* Fix var substitution

* Different work around for multi lines
2022-03-01 13:50:18 +01:00
Naïm Favier b4163eb8b2
Authenticate to the GitHub API in workflows (#425) 2022-02-05 15:24:16 +01:00
Naïm Favier ec4dbe03a4
Print underlying serialisation errors (#417) 2022-01-25 16:03:21 +01:00
Naïm Favier 7d80f60fd9
Pass --meta to nix-env (#421)
Since Nix 2.6 (https://github.com/NixOS/nix/pull/5883) nix-env doesn't
output the `meta` attribute unless this flag is passed.
2022-01-25 16:02:29 +01:00
Naïm Favier f31e580e52
Clean up workflows (#415)
* Clean up workflows

* Rename workflow files

* Temporarily enable workflows from PR

* Add nixpkgs channel to NIX_PATH

* Fix typo

* Fix index names for warmup

* Build verbosely

* Fix index names again

* Re-add jq

* Revert "Temporarily enable workflows from PR"

This reverts commit dd040e94645d9e61b80505471f57b1c0e35b1210.
2022-01-17 12:18:00 +01:00
erikarvstedt 096ea9e49c
Support flake attr nixosModule (#401)
* flake_info.nix: remove helper fn `default`

Nix has builtin syntax for fetching attrs with a default value.

* flake_info.nix: minor refactor

- Remove commented out code
- Simplify pkg filtering
- Break overlong line

* flake_info.nix: support flake attr `nixosModule`

* Support default modules in rust backend and elm

Co-authored-by: Yannik Sander <me@ysndr.de>
2022-01-06 15:42:31 +01:00
sterni 32700e3eee
Use upstream nixpkgs' packages-config.nix (#405)
The files in nixos-search and nixpkgs have already unnecessarily
diverged — more than they should given that the only difference is that
the search ignores unfree packages (while nixpkgs includes them for the
benefit of repology).

Since we update the NIX_PATH when calling nix-env in nixpkgs_info.rs, we
can simply import the packages-config.nix from nixpkgs using that
information and update it with our addition `allowUnfree = false`.

Fixes #337.
2022-01-03 13:45:05 +01:00
Naïm Favier 194ba6d4e0
Fix xrefs (#402) 2021-12-31 00:29:13 +01:00
Naïm Favier b6f1dc9667
Render DocBook strings containing < but not </ (#400)
The current check is too restrictive and will not match strings containing only self-closing tags, for example the description of `time.timeZone`.
2021-12-28 23:21:23 +01:00
Yannik Sander 3dfcf4fd45
Fix/explicit printing (#395)
* Extract pandoc function to own module

* Bump schema version

* Make string handling more explicit

* Update logging and pandoc crates

* Improve serializing error handling

* Serialize values as string (for elastic)

* Perform option doc parsing entirely in rust

* Show non pandoc'ed results as code elements

* Parse correct html string

* Change expected `option_type` type to String

* Allow treat string unparsable  as html like non-docbook strings

* Improve deserializing error reporting using serde_path_to_error

* Format code
2021-12-26 16:03:09 +01:00
Yannik Sander a4d8421312
Allow plain strings for nixpkgs maintainer fields (#393)
* Allow plain strings for nixpkgs maintainer fields

* apply fmt to changed files
2021-12-18 14:45:08 +01:00
Yannik Sander fa5fe2a61c
Remove noisy prints to stderr (#386)
* Make stderr less noisy

* Set and refer to pandoc filter path in derivation
2021-12-06 23:50:15 +01:00
Yannik Sander d0f0b84eea
Print pandoc error rather than panic-ing (#382)
* Print pandoc error rather than panic-ing

* Maintain current failing behaviour

* Update export.rs
2021-12-06 18:23:10 +01:00
Yannik Sander dbf8c2d3a4
Convert import::Kind enum to a clap compatible arg_enum (#365) 2021-11-23 13:58:55 +01:00
Yannik Sander f3571efdef
Translate example/default values from JSON to NIx (#357) 2021-11-11 20:29:44 +01:00
Naïm Favier b74147b671
Support literalExpression and literalDocBook (#368)
* Support literalExpression and literalDocBook

* Use consistent nixpkgs versions in get_nixpkgs_options
2021-10-15 10:20:18 +02:00
Yannik Sander 9e018c36da
Don't fail import of groups as long as some imports succeed (#358)
* Don't fail import of groups as long as some imports succeed
2021-09-30 14:24:35 -04:00