Hadoop

`services.hadoop.containerExecutorCfg`

Yarn container-executor.cfg definition https://hadoop.apache.org/docs/r2.7.2/hadoop-yarn/hadoop-yarn-site/SecureContainer.html

Type: attribute set of anything

Default

{"feature.mount-cgroup.enabled" = 1;"feature.terminal.enabled" = 1;"min.user.id" = 1000;"yarn.nodemanager.linux-container-executor.group" = "hadoop";}

Example

options.services.hadoop.containerExecutorCfg.default // {"feature.terminal.enabled" = 0;}

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/default.nix

`services.hadoop.coreSite`

Hadoop core-site.xml definition https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/core-default.xml

Type: attribute set of anything

Default

{ }

Example

{"fs.defaultFS" = "hdfs://localhost";}

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/default.nix

`services.hadoop.extraConfDirs`

Directories containing additional config files to be added to HADOOP_CONF_DIR Type: list of path

Default

[ ]

Example

[./extraHDFSConfs./extraYARNConfs]

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/default.nix

`services.hadoop.gatewayRole.enable`

Whether to enable gateway role for deploying hadoop configs. Type: boolean

Default

false

Example

true

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/default.nix

`services.hadoop.gatewayRole.enableHbaseCli`

Whether to enable HBase CLI tools. Type: boolean

Default

false

Example

true

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hbase.nix

`services.hadoop.hbase.master.enable`

Whether to enable HBase master. Type: boolean

Default

false

Example

true

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hbase.nix

`services.hadoop.hbase.master.environment`

Environment variables passed to master. Type: attribute set of string

Default

{ }

Example

{HBASE_MASTER_OPTS = "-Dcom.sun.management.jmxremote.ssl=true";}

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hbase.nix

`services.hadoop.hbase.master.extraFlags`

Extra flags for the master service. Type: list of string

Default

[ ]

Example

[ "--backup" ]

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hbase.nix

`services.hadoop.hbase.master.initHDFS`

Whether to enable initialization of the hbase directory on HDFS. Type: boolean

Default

false

Example

true

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hbase.nix

`services.hadoop.hbase.master.openFirewall`

Open firewall ports for HBase master. Type: boolean

Default

false

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hbase.nix

`services.hadoop.hbase.master.restartIfChanged`

Restart master con config change. Type: boolean

Default

false

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hbase.nix

`services.hadoop.hbase.package`

The hbase package to use. Type: package

Default

pkgs.hbase

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hbase.nix

`services.hadoop.hbase.regionServer.enable`

Whether to enable HBase regionServer. Type: boolean

Default

false

Example

true

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hbase.nix

`services.hadoop.hbase.regionServer.environment`

Environment variables passed to regionServer. Type: attribute set of string

Default

{ }

Example

{HBASE_MASTER_OPTS = "-Dcom.sun.management.jmxremote.ssl=true";}

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hbase.nix

`services.hadoop.hbase.regionServer.extraFlags`

Extra flags for the regionServer service. Type: list of string

Default

[ ]

Example

[ "--backup" ]

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hbase.nix

`services.hadoop.hbase.regionServer.openFirewall`

Open firewall ports for HBase regionServer. Type: boolean

Default

false

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hbase.nix

`services.hadoop.hbase.regionServer.overrideHosts`

Remove /etc/hosts entries for "127.0.0.2" and "::1" defined in nixos/modules/config/networking.nix Regionservers must be able to resolve their hostnames to their IP addresses, through PTR records or /etc/hosts entries.

Type: boolean

Default

true

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hbase.nix

`services.hadoop.hbase.regionServer.restartIfChanged`

Restart regionServer con config change. Type: boolean

Default

false

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hbase.nix

`services.hadoop.hbase.rest.enable`

Whether to enable HBase rest. Type: boolean

Default

false

Example

true

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hbase.nix

`services.hadoop.hbase.rest.environment`

Environment variables passed to rest. Type: attribute set of string

Default

{ }

Example

{HBASE_MASTER_OPTS = "-Dcom.sun.management.jmxremote.ssl=true";}

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hbase.nix

`services.hadoop.hbase.rest.extraFlags`

Extra flags for the rest service. Type: list of string

Default

[ ]

Example

[ "--backup" ]

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hbase.nix

`services.hadoop.hbase.rest.infoPort`

web UI port Type: signed integer

Default

8085

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hbase.nix

`services.hadoop.hbase.rest.openFirewall`

Open firewall ports for HBase rest. Type: boolean

Default

false

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hbase.nix

`services.hadoop.hbase.rest.port`

RPC port Type: signed integer

Default

8080

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hbase.nix

`services.hadoop.hbase.rest.restartIfChanged`

Restart rest con config change. Type: boolean

Default

false

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hbase.nix

`services.hadoop.hbase.rootdir`

This option will set "hbase.rootdir" in hbase-site.xml and determine the directory shared by region servers and into which HBase persists. The URL should be 'fully-qualified' to include the filesystem scheme. If a core-site.xml is provided, the FS scheme defaults to the value of "fs.defaultFS".

Filesystems other than HDFS (like S3, QFS, Swift) are also supported.

Type: string

Default

"/hbase"

Example

"hdfs://nameservice1/hbase"

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hbase.nix

`services.hadoop.hbase.thrift.enable`

Whether to enable HBase thrift. Type: boolean

Default

false

Example

true

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hbase.nix

`services.hadoop.hbase.thrift.environment`

Environment variables passed to thrift. Type: attribute set of string

Default

{ }

Example

{HBASE_MASTER_OPTS = "-Dcom.sun.management.jmxremote.ssl=true";}

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hbase.nix

`services.hadoop.hbase.thrift.extraFlags`

Extra flags for the thrift service. Type: list of string

Default

[ ]

Example

[ "--backup" ]

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hbase.nix

`services.hadoop.hbase.thrift.infoPort`

web UI port Type: signed integer

Default

9095

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hbase.nix

`services.hadoop.hbase.thrift.openFirewall`

Open firewall ports for HBase thrift. Type: boolean

Default

false

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hbase.nix

`services.hadoop.hbase.thrift.port`

RPC port Type: signed integer

Default

9090

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hbase.nix

`services.hadoop.hbase.thrift.restartIfChanged`

Restart thrift con config change. Type: boolean

Default

false

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hbase.nix

`services.hadoop.hbase.zookeeperQuorum`

This option will set "hbase.zookeeper.quorum" in hbase-site.xml. Comma separated list of servers in the ZooKeeper ensemble.

Type: null or strings concatenated with ","

Default

null

Example

"zk1.internal,zk2.internal,zk3.internal"

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hbase.nix

`services.hadoop.hbaseSite`

Additional options and overrides for hbase-site.xml https://github.com/apache/hbase/blob/rel/2.4.11/hbase-common/src/main/resources/hbase-default.xml

Type: attribute set of anything

Default

{ }

Example

{"hbase.hregion.max.filesize" = 20*1024*1024*1024;"hbase.table.normalization.enabled" = "true";}

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hbase.nix

`services.hadoop.hbaseSiteDefault`

Default options for hbase-site.xml

Type: attribute set of anything

Default

{"hbase.cluster.distributed" = "true";"hbase.master.info.bindAddress" = "0.0.0.0";"hbase.master.ipc.address" = "0.0.0.0";"hbase.regionserver.info.bindAddress" = "0.0.0.0";"hbase.regionserver.ipc.address" = "0.0.0.0";}

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hbase.nix

`services.hadoop.hdfs.datanode.dataDirs`

Tier and path definitions for datanode storage. Type: null or (list of (submodule))

Default

null

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hdfs.nix

`services.hadoop.hdfs.datanode.dataDirs.*.path`

Determines where on the local filesystem a data node should store its blocks. Type: path

Example

["/var/lib/hadoop/hdfs/dn"]

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hdfs.nix

`services.hadoop.hdfs.datanode.dataDirs.*.type`

Storage types ([SSD]/[DISK]/[ARCHIVE]/[RAM_DISK]) for HDFS storage policies.

Type: one of "SSD", "DISK", "ARCHIVE", "RAM_DISK"

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hdfs.nix

`services.hadoop.hdfs.datanode.enable`

Whether to enable HDFS DataNode. Type: boolean

Default

false

Example

true

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hdfs.nix

`services.hadoop.hdfs.datanode.extraEnv`

Extra environment variables for HDFS DataNode Type: attribute set of string

Default

{ }

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hdfs.nix

`services.hadoop.hdfs.datanode.extraFlags`

Extra command line flags to pass to HDFS DataNode Type: list of string

Default

[ ]

Example

["-Dcom.sun.management.jmxremote""-Dcom.sun.management.jmxremote.port=8010"]

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hdfs.nix

`services.hadoop.hdfs.datanode.openFirewall`

Open firewall ports for HDFS DataNode. Type: boolean

Default

false

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hdfs.nix

`services.hadoop.hdfs.datanode.restartIfChanged`

Automatically restart the service on config change. This can be set to false to defer restarts on clusters running critical applications. Please consider the security implications of inadvertently running an older version, and the possibility of unexpected behavior caused by inconsistent versions across a cluster when disabling this option.

Type: boolean

Default

false

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hdfs.nix

`services.hadoop.hdfs.httpfs.enable`

Whether to enable HDFS JournalNode. Type: boolean

Default

false

Example

true

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hdfs.nix

`services.hadoop.hdfs.httpfs.extraEnv`

Extra environment variables for HDFS JournalNode Type: attribute set of string

Default

{ }

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hdfs.nix

`services.hadoop.hdfs.httpfs.extraFlags`

Extra command line flags to pass to HDFS JournalNode Type: list of string

Default

[ ]

Example

["-Dcom.sun.management.jmxremote""-Dcom.sun.management.jmxremote.port=8010"]

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hdfs.nix

`services.hadoop.hdfs.httpfs.openFirewall`

Open firewall ports for HDFS JournalNode. Type: boolean

Default

false

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hdfs.nix

`services.hadoop.hdfs.httpfs.restartIfChanged`

Automatically restart the service on config change. This can be set to false to defer restarts on clusters running critical applications. Please consider the security implications of inadvertently running an older version, and the possibility of unexpected behavior caused by inconsistent versions across a cluster when disabling this option.

Type: boolean

Default

false

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hdfs.nix

`services.hadoop.hdfs.httpfs.tempPath`

HTTPFS_TEMP path used by HTTPFS Type: path

Default

"/tmp/hadoop/httpfs"

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hdfs.nix

`services.hadoop.hdfs.journalnode.enable`

Whether to enable HDFS JournalNode. Type: boolean

Default

false

Example

true

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hdfs.nix

`services.hadoop.hdfs.journalnode.extraEnv`

Extra environment variables for HDFS JournalNode Type: attribute set of string

Default

{ }

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hdfs.nix

`services.hadoop.hdfs.journalnode.extraFlags`

Extra command line flags to pass to HDFS JournalNode Type: list of string

Default

[ ]

Example

["-Dcom.sun.management.jmxremote""-Dcom.sun.management.jmxremote.port=8010"]

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hdfs.nix

`services.hadoop.hdfs.journalnode.openFirewall`

Open firewall ports for HDFS JournalNode. Type: boolean

Default

false

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hdfs.nix

`services.hadoop.hdfs.journalnode.restartIfChanged`

Automatically restart the service on config change. This can be set to false to defer restarts on clusters running critical applications. Please consider the security implications of inadvertently running an older version, and the possibility of unexpected behavior caused by inconsistent versions across a cluster when disabling this option.

Type: boolean

Default

false

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hdfs.nix

`services.hadoop.hdfs.namenode.enable`

Whether to enable HDFS NameNode. Type: boolean

Default

false

Example

true

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hdfs.nix

`services.hadoop.hdfs.namenode.extraEnv`

Extra environment variables for HDFS NameNode Type: attribute set of string

Default

{ }

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hdfs.nix

`services.hadoop.hdfs.namenode.extraFlags`

Extra command line flags to pass to HDFS NameNode Type: list of string

Default

[ ]

Example

["-Dcom.sun.management.jmxremote""-Dcom.sun.management.jmxremote.port=8010"]

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hdfs.nix

`services.hadoop.hdfs.namenode.formatOnInit`

Format HDFS namenode on first start. This is useful for quickly spinning up ephemeral HDFS clusters with a single namenode. For HA clusters, initialization involves multiple steps across multiple nodes. Follow this guide to initialize an HA cluster manually: https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithQJM.html

Type: boolean

Default

false

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hdfs.nix

`services.hadoop.hdfs.namenode.openFirewall`

Open firewall ports for HDFS NameNode. Type: boolean

Default

false

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hdfs.nix

`services.hadoop.hdfs.namenode.restartIfChanged`

Automatically restart the service on config change. This can be set to false to defer restarts on clusters running critical applications. Please consider the security implications of inadvertently running an older version, and the possibility of unexpected behavior caused by inconsistent versions across a cluster when disabling this option.

Type: boolean

Default

false

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hdfs.nix

`services.hadoop.hdfs.zkfc.enable`

Whether to enable HDFS ZooKeeper failover controller. Type: boolean

Default

false

Example

true

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hdfs.nix

`services.hadoop.hdfs.zkfc.extraEnv`

Extra environment variables for HDFS ZooKeeper failover controller Type: attribute set of string

Default

{ }

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hdfs.nix

`services.hadoop.hdfs.zkfc.extraFlags`

Extra command line flags to pass to HDFS ZooKeeper failover controller Type: list of string

Default

[ ]

Example

["-Dcom.sun.management.jmxremote""-Dcom.sun.management.jmxremote.port=8010"]

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hdfs.nix

`services.hadoop.hdfs.zkfc.restartIfChanged`

Automatically restart the service on config change. This can be set to false to defer restarts on clusters running critical applications. Please consider the security implications of inadvertently running an older version, and the possibility of unexpected behavior caused by inconsistent versions across a cluster when disabling this option.

Type: boolean

Default

false

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/hdfs.nix

`services.hadoop.hdfsSite`

Additional options and overrides for hdfs-site.xml https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml

Type: attribute set of anything

Default

{ }

Example

{"dfs.nameservices" = "namenode1";}

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/default.nix

`services.hadoop.hdfsSiteDefault`

Default options for hdfs-site.xml

Type: attribute set of anything

Default

{"dfs.namenode.http-address" = "0.0.0.0:9870";"dfs.namenode.http-bind-host" = "0.0.0.0";"dfs.namenode.rpc-bind-host" = "0.0.0.0";"dfs.namenode.servicerpc-bind-host" = "0.0.0.0";}

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/default.nix

`services.hadoop.httpfsSite`

Hadoop httpfs-site.xml definition https://hadoop.apache.org/docs/current/hadoop-hdfs-httpfs/httpfs-default.html

Type: attribute set of anything

Default

{ }

Example

{"hadoop.http.max.threads" = 500;}

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/default.nix

`services.hadoop.log4jProperties`

log4j.properties file added to HADOOP_CONF_DIR Type: path

Default

"${config.services.hadoop.package}/etc/hadoop/log4j.properties"

Example

"${pkgs.hadoop}/etc/hadoop/log4j.properties";

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/default.nix

`services.hadoop.mapredSite`

Additional options and overrides for mapred-site.xml https://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml

Type: attribute set of anything

Default

{ }

Example

{"mapreduce.map.java.opts" = "-Xmx900m -XX:+UseParallelGC";}

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/default.nix

`services.hadoop.mapredSiteDefault`

Default options for mapred-site.xml

Type: attribute set of anything

Default

{"mapreduce.framework.name" = "yarn";"yarn.app.mapreduce.am.env" = "HADOOP_MAPRED_HOME=${config.services.hadoop.package}";"mapreduce.map.env" = "HADOOP_MAPRED_HOME=${config.services.hadoop.package}";"mapreduce.reduce.env" = "HADOOP_MAPRED_HOME=${config.services.hadoop.package}";}

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/default.nix

`services.hadoop.package`

The hadoop package to use. Type: package

Default

pkgs.hadoop

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/default.nix

`services.hadoop.yarn.nodemanager.addBinBash`

Add /bin/bash. This is needed by the linux container executor's launch script.

Type: boolean

Default

true

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/yarn.nix

`services.hadoop.yarn.nodemanager.enable`

Whether to enable Hadoop YARN NodeManager. Type: boolean

Default

false

Example

true

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/yarn.nix

`services.hadoop.yarn.nodemanager.extraEnv`

Extra environment variables Type: attribute set of string

Default

{ }

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/yarn.nix

`services.hadoop.yarn.nodemanager.extraFlags`

Extra command line flags to pass to the service Type: list of string

Default

[ ]

Example

["-Dcom.sun.management.jmxremote""-Dcom.sun.management.jmxremote.port=8010"]

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/yarn.nix

`services.hadoop.yarn.nodemanager.localDir`

List of directories to store localized files in. Type: null or (list of path)

Default

null

Example

["/var/lib/hadoop/yarn/nm"]

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/yarn.nix

`services.hadoop.yarn.nodemanager.openFirewall`

Open firewall ports for nodemanager. Because containers can listen on any ephemeral port, TCP ports 1024–65535 will be opened.

Type: boolean

Default

false

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/yarn.nix

`services.hadoop.yarn.nodemanager.resource.cpuVCores`

Number of vcores that can be allocated for containers. Type: null or (positive integer, meaning >0)

Default

null

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/yarn.nix

`services.hadoop.yarn.nodemanager.resource.maximumAllocationMB`

The maximum physical memory any container can be allocated. Type: null or (positive integer, meaning >0)

Default

null

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/yarn.nix

`services.hadoop.yarn.nodemanager.resource.maximumAllocationVCores`

The maximum virtual CPU cores any container can be allocated. Type: null or (positive integer, meaning >0)

Default

null

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/yarn.nix

`services.hadoop.yarn.nodemanager.resource.memoryMB`

Amount of physical memory, in MB, that can be allocated for containers. Type: null or (positive integer, meaning >0)

Default

null

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/yarn.nix

`services.hadoop.yarn.nodemanager.restartIfChanged`

Automatically restart the service on config change. This can be set to false to defer restarts on clusters running critical applications. Please consider the security implications of inadvertently running an older version, and the possibility of unexpected behavior caused by inconsistent versions across a cluster when disabling this option.

Type: boolean

Default

false

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/yarn.nix

`services.hadoop.yarn.nodemanager.useCGroups`

Use cgroups to enforce resource limits on containers

Type: boolean

Default

true

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/yarn.nix

`services.hadoop.yarn.resourcemanager.enable`

Whether to enable Hadoop YARN ResourceManager. Type: boolean

Default

false

Example

true

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/yarn.nix

`services.hadoop.yarn.resourcemanager.extraEnv`

Extra environment variables Type: attribute set of string

Default

{ }

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/yarn.nix

`services.hadoop.yarn.resourcemanager.extraFlags`

Extra command line flags to pass to the service Type: list of string

Default

[ ]

Example

["-Dcom.sun.management.jmxremote""-Dcom.sun.management.jmxremote.port=8010"]

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/yarn.nix

`services.hadoop.yarn.resourcemanager.openFirewall`

Open firewall ports for resourcemanager

Type: boolean

Default

false

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/yarn.nix

`services.hadoop.yarn.resourcemanager.restartIfChanged`

Automatically restart the service on config change. This can be set to false to defer restarts on clusters running critical applications. Please consider the security implications of inadvertently running an older version, and the possibility of unexpected behavior caused by inconsistent versions across a cluster when disabling this option.

Type: boolean

Default

false

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/yarn.nix

`services.hadoop.yarnSite`

Additional options and overrides for yarn-site.xml https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-common/yarn-default.xml

Type: attribute set of anything

Default

{ }

Example

{"yarn.resourcemanager.hostname" = "${config.networking.hostName}";}

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/default.nix

`services.hadoop.yarnSiteDefault`

Default options for yarn-site.xml

Type: attribute set of anything

Default

{"yarn.nodemanager.admin-env" = "PATH=$PATH";"yarn.nodemanager.aux-services" = "mapreduce_shuffle";"yarn.nodemanager.aux-services.mapreduce_shuffle.class" = "org.apache.hadoop.mapred.ShuffleHandler";"yarn.nodemanager.bind-host" = "0.0.0.0";"yarn.nodemanager.container-executor.class" = "org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor";"yarn.nodemanager.env-whitelist" = "JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,CLASSPATH_PREPEND_DISTCACHE,HADOOP_YARN_HOME,HADOOP_HOME,LANG,TZ";"yarn.nodemanager.linux-container-executor.group" = "hadoop";"yarn.nodemanager.linux-container-executor.path" = "/run/wrappers/yarn-nodemanager/bin/container-executor";"yarn.nodemanager.log-dirs" = "/var/log/hadoop/yarn/nodemanager";"yarn.resourcemanager.bind-host" = "0.0.0.0";"yarn.resourcemanager.scheduler.class" = "org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler";}

Declared by: https://github.com/nixos/nixpkgs/blob/master/nixos/modules/services/cluster/hadoop/default.nix