Spark

`services.spark.confDir`

Spark configuration directory. Spark will use the configuration files (spark-defaults.conf, spark-env.sh, log4j.properties, etc) from this directory. Type: path

Default

"${package}/conf"

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/spark/default.nix

`services.spark.logDir`

Spark log directory. Type: path

Default

"/var/log/spark"

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/spark/default.nix

`services.spark.master.bind`

Address the spark master binds to. Type: string

Default

"127.0.0.1"

Example

"0.0.0.0"

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/spark/default.nix

`services.spark.master.enable`

Whether to enable Spark master service. Type: boolean

Default

false

Example

true

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/spark/default.nix

`services.spark.master.extraEnvironment`

Extra environment variables to pass to spark master. See spark-standalone documentation. Type: attribute set of string

Default

{ }

Example

{SPARK_MASTER_OPTS = "-Dspark.deploy.defaultCores=5";SPARK_MASTER_WEBUI_PORT = 8181;}

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/spark/default.nix

`services.spark.master.restartIfChanged`

Automatically restart master service on config change. This can be set to false to defer restarts on clusters running critical applications. Please consider the security implications of inadvertently running an older version, and the possibility of unexpected behavior caused by inconsistent versions across a cluster when disabling this option.

Type: boolean

Default

true

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/spark/default.nix

`services.spark.package`

The spark package to use. Type: package

Default

pkgs.spark

Example

`#!nix spark.overrideAttrs (super: rec { pname = "spark"; version = "2.4.4";

src = pkgs.fetchzip { url = "mirror://apache/spark/"${pname}-${version}/${pname}-${version}-bin-without-hadoop.tgz"; sha256 = "1a9w5k0207fysgpxx6db3a00fs5hdc2ncx99x4ccy2s0v5ndc66g"; }; }) `

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/spark/default.nix

`services.spark.worker.enable`

Whether to enable Spark worker service. Type: boolean

Default

false

Example

true

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/spark/default.nix

`services.spark.worker.extraEnvironment`

Extra environment variables to pass to spark worker. Type: attribute set of string

Default

{ }

Example

{SPARK_WORKER_CORES = 5;SPARK_WORKER_MEMORY = "2g";}

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/spark/default.nix

`services.spark.worker.master`

Address of the spark master. Type: string

Default

"127.0.0.1:7077"

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/spark/default.nix

`services.spark.worker.restartIfChanged`

Automatically restart worker service on config change. This can be set to false to defer restarts on clusters running critical applications. Please consider the security implications of inadvertently running an older version, and the possibility of unexpected behavior caused by inconsistent versions across a cluster when disabling this option.

Type: boolean

Default

true

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/spark/default.nix

`services.spark.worker.workDir`

Spark worker work dir. Type: path

Default

"/var/lib/spark"

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/spark/default.nix