aux-search/flake-info
Rok Garbas eea6cd3ffb
Feat/dynamic import (#473)
* Make the channel importer versions dynamic

* few things fromt he list

1. add nixos-org-configurations as input to flake.nix
2. evaluate channels.nix file and export channels via environment variable.
   that environment variable (lets call it NIXOS_CHANNELS) should be present
   during the build and inside the nix shell. the content of the variable can
   be JSON.
3. we pickup the NIXOS_CHANNELS environment variable in
   frontend/webpack.config.js and pass it further to webpack process, just
   like we do with ELASTICSEARCH_MAPPING_SCHEMA_VERSION.
4. we forward NIXOS_CHANNELS to Elm via frontend/src/index.js as an Elm
   application flag. Just like we do with other variables there.

* Decode nixosChannels in Elm

* Use nixosChannels that came via application flag

* read nixos channels in github action

* defaultNixOSChannel should be calculated

* add two pointers where the check should be added

* pass nixosChannels to flake-info and remove title, rather calculate it

* Add NixosChannels struct validation and validation Error

* Read NIXOS_CHANNEL variable

* Check channel

* Add channel struct to fix parsing NIXOS_CHANNELS

* Use `eachDefaultSystem` instead of listing them manually

* Add individual dev shells for frontend and backend

* Update .github/workflows/import-to-elasticsearch.yml

Co-authored-by: Naïm Favier <n@monade.li>

* use both development environments by default (as it was)

but still provide devShells for each of the subprojects

* pkgs.lib → lib everywhere

and define lib = nixpkgs.lib before the call to eachDefaultSystem
Also, version = lib.fileContents ./VERSION;

* Update flake.nix

Co-authored-by: Naïm Favier <n@monade.li>

* typo

* bumping version to test the changes to import-to-elasticsearch github action

* some invisibile characters needed to be removed

* typo

* will this work

* typo

* forgot the checkout step

* add some debugging

* typo

* read NIXOS_CHANNELS from environment not via argument

* fix for the NIXOS_CHANNELS variable

Co-authored-by: Janne Heß <janne@hess.ooo>
Co-authored-by: ysndr <me@ysndr.de>
Co-authored-by: Naïm Favier <n@monade.li>
2022-04-24 23:48:01 +02:00
..
.cargo Show default outputs (#452) 2022-03-27 15:52:54 +02:00
examples Rewrite import script in Rust (#320) 2021-08-17 10:55:08 +02:00
src Feat/dynamic import (#473) 2022-04-24 23:48:01 +02:00
Cargo.lock flake-info: enforce minimal nix version (#451) 2022-03-18 21:57:28 +01:00
Cargo.toml flake-info: enforce minimal nix version (#451) 2022-03-18 21:57:28 +01:00
default.nix Feat/dynamic import (#473) 2022-04-24 23:48:01 +02:00
README.md Rewrite import script in Rust (#320) 2021-08-17 10:55:08 +02:00

Flake Info

A tool that fetches packages and apps from nix flakes.

Usage

flake-info 0.3.0
Extracts various information from a given flake

USAGE:
    flake-info [FLAGS] [OPTIONS] [extra]... <SUBCOMMAND>

FLAGS:
        --push       Push to Elasticsearch (Configure using FI_ES_* environment variables)
    -h, --help       Prints help information
        --json       Print ElasticSeach Compatible JSON output
    -V, --version    Prints version information

OPTIONS:
        --elastic-exists <elastic-exists>
            How to react to existing indices [env: FI_ES_EXISTS_STRATEGY=]  [default: abort]  [possible values: Abort,
            Ignore, Recreate]
        --elastic-index-name <elastic-index-name>            Name of the index to store results to [env: FI_ES_INDEX=]
    -p, --elastic-pw <elastic-pw>
            Elasticsearch password (unimplemented) [env: FI_ES_PASSWORD=]

        --elastic-schema-version <elastic-schema-version>
            Which schema version to associate with the operation [env: FI_ES_VERSION=]

        --elastic-url <elastic-url>
            Elasticsearch instance url [env: FI_ES_URL=]  [default: http://localhost:9200]

    -u, --elastic-user <elastic-user>                        Elasticsearch username (unimplemented) [env: FI_ES_USER=]
    -k, --kind <kind>
            Kind of data to extract (packages|options|apps|all) [default: all]


ARGS:
    <extra>...    Extra arguments that are passed to nix as it

SUBCOMMANDS:
    flake
    group
    help       Prints this message or the help of the given subcommand(s)
    nixpkgs

flake

Flakes can be imported using the flake subcommand

USAGE:
    flake-info flake [FLAGS] <flake>

FLAGS:
        --gc            Whether to gc the store after info or not
    -h, --help          Prints help information
        --temp-store    Whether to use a temporary store or not. Located at /tmp/flake-info-store
    -V, --version       Prints version information

ARGS:
    <flake>    Flake identifier passed to nix to gather information about

The <flake> argument should contain a valid reference to a flake. It accepts all formats nix accepts:

use git+ to checkout a git repository at use /local/absolute/path or ./relative/path to load a local source use gitlab://github:/ to shortcut gitlab or github repositories

Optionally, analyzing can be done in a temporary store enabled by the --temp-store option.

Example

$ flake-info flake github:ngi-nix/offen

nixpkgs

nixpkgs currently have to be imported in a different way. This is what the nixpkgs subcommand exists for.

It takes any valid git reference to the upstream nixos/nixpkgs repo as an argument and produces a complete output.

This operation may take a short while and produces lots of output

Example

$ flake-info nixpkgs nixos-21.05

group

to perform a bulk import grouping multiple inputs under the same name/index use the group command.

It expects a json file as input that contains references to flakes or nixpkgs. If those resources are on github or gitlab they can be extended with more meta information including pinning the commit hash/ref.

The second argument is the group name that is used to provide the index name.

Example

An example targets.json file can look like the following

[
    {
        "type": "git",
        "url": "./."
    },
    {
        "type": "git",
        "url": "github:fluffynukeit/adaspark"
    },
    {
        "type": "github",
        "owner": "ngi-nix",
        "repo": "offen",
        "hash": "4052febf151d60aa4352fa1960cf3ae088f600aa",
        "description": "Hier könnte Ihre Werbung stehen"
    }
]
$ flake-info group ./targets.json small-group

Elasticsearch

A number of flags is dedicated to pushing to elasticsearch.

    --elastic-exists <elastic-exists>
        How to react to existing indices [env: FI_ES_EXISTS_STRATEGY=]  [default: abort]
                                            [possible values: Abort, Ignore, Recreate]
    --elastic-index-name <elastic-index-name>
        Name of the index to store results to [env: FI_ES_INDEX=]
-p, --elastic-pw <elastic-pw>
        Elasticsearch password (unimplemented) [env: FI_ES_PASSWORD=]

    --elastic-schema-version <elastic-schema-version>
        Which schema version to associate with the operation [env: FI_ES_VERSION=]

    --elastic-url <elastic-url>
        Elasticsearch instance url [env: FI_ES_URL=]  [default: http://localhost:9200]

-u, --elastic-user <elastic-user>                        Elasticsearch username (unimplemented) [env: FI_ES_USER=]

Example

$ flake-info --push \
             --elastic-url http://localhost:5555 \
             --elastic-index-name latest-21-21.05
             --elastic-schema-version 21 group ./examples/ngi-nix.json ngi-nix

Installation

Preparations

This tool requires your system to have Nix installed!

You can install nix using this installer: https://nixos.org/guides/install-nix.html Also, see https://nixos.wiki/wiki/Nix_Installation_Guide if your system is special.

Preparations (Docker)

If you do not want to install nix on your system, using Docker is an alternative.

Enter the nixos/nix docker image and proceed

Setup nix flakes

Note that you also need to have nix flakes support.

Once you have nix installed run the following commands:

  1. $ nix-shell -I nixpkgs=channel:nixos-21.05 -p nixFlakes
    
    to enter a shell with the preview version of nix flakes installed.
  2. $ mkdir -p ~/.config/nix
    $ echo "experimental-features = nix-command flakes" > .config/nix/nix.conf
    
    to enable flake support

Installation, finally

This project is defined as a flake therefore you can build the tool using

$ nix build <project root>
or
$ nix build github:miszkur/github-search

Replace build with run if you want to run the tool directly.