Configuration

Configuration files

When you launch a pipeline script, Nextflow detects configuration files from multiple sources and applies them in the following order (from lowest to highest priority):

  1. $NXF_HOME/config (defaults to $HOME/.nextflow/config)

  2. nextflow.config in the project directory

  3. nextflow.config in the launch directory

  4. Config files specified with -c <config-files>

Tip

You can use the -C <config-file> option to specify a fixed set of configuration files and ignore all other files.

Syntax

Nextflow configuration uses a similar syntax as Nextflow scripts. Configuration options must be set declaratively, but the value of a config option can be an arbitrary expression.

A config file may contain any number of assignments, blocks, and includes. You can add comments just like in scripts.

Assignments

A config assignment sets a config option to a value:

workDir = 'work'
docker.enabled = true
process.maxErrors = 10

The config option consists of an option name prefixed by any number of scopes. Scopes group related config options. See Configuration options for the full set of config options.

The value is typically a literal value, such as a number, boolean, or string. However, you can use any expression:

params.helper_file = "${projectDir}/assets/helper.txt"

Blocks

You can specify config options in the same scope as a block:

// dot syntax
docker.enabled = true
docker.runOptions = '-u $(id -u):$(id -g)'

// block syntax
docker {
    enabled = true
    runOptions = '-u $(id -u):$(id -g)'
}

As a result, deeply nested config options can be assigned in multiple ways. The following three assignments are equivalent:

executor.retry.maxAttempt = 5

executor {
    retry.maxAttempt = 5
}

executor {
    retry {
        maxAttempt = 5
    }
}

Includes

You can include configuration files in other configuration files with the includeConfig keyword:

process.executor = 'sge'
process.queue = 'long'
process.memory = '10G'

includeConfig 'path/extra.config'

Relative paths are resolved against the location of the including file.

Note

Config files should only be included at the top level or in a profile. This ensures the included config file is valid on its own and in the context in which it is included.

Constants and functions

The following constants are globally available in a Nextflow configuration file:

baseDir: Path

Deprecated since version 20.04.0.

Alias for projectDir.

launchDir: Path

The directory where the workflow was launched.

projectDir: Path

The directory where the main script is located.

secrets: Map<String,String>

Map of pipeline secrets. See Secrets for more information.

The following functions are globally available in a Nextflow configuration file:

env( name: String ) -> String

New in version 25.04.0.

Get the value of the environment variable with the specified name in the Nextflow launch environment.

Parameters

You can define pipeline parameters in config files using the params scope:

// dot syntax
params.max_cpus = 64
params.publish_mode = 'copy'

// block syntax
params {
    max_cpus = 64
    publish_mode = 'copy'
}

Note

When including a config file, the included config is evaluated with the parameters defined before the include. The included config cannot see parameters defined after the include.

You should declare parameters in the config file only when other config options use them. When you use a parameter in the script, you should declare it there and override it in config profiles as needed:

// main.nf
params.input = null

// nextflow.config
params {
    publish_mode = 'copy'
}

workflow.output.mode = params.publish_mode

profiles {
    test {
        params.input = "${projectDir}/test/input.txt"
    }
}

See Pipeline parameters for information about how Nextflow resolves pipeline parameters at runtime.

Process configuration

You can use the process scope to specify process directives separately from the pipeline code.

For example:

process {
    executor = 'sge'
    queue = 'long'
    clusterOptions = '-pe smp 10 -l virtual_free=64G,h_rt=30:00:00'
}

This configuration executes all processes using the SGE executor with the given settings.

Process selectors

You can use the withLabel selector to configure all processes annotated with a label directive:

process {
    withLabel: big_mem {
        cpus = 16
        memory = 64.GB
        queue = 'long'
    }
}

This configuration assigns 16 CPUs, 64 GB of memory, and the long queue to all processes with the big_mem label.

You can use the withName selector to configure a specific process by its name:

process {
    withName: hello {
        cpus = 4
        memory = 8.GB
        queue = 'short'
    }
}

The withName selector matches both:

  • Processes defined with that name

  • Processes included under that alias

When you include a process with an alias, selectors for the alias take priority over selectors for the original name. For example, if you define a process as hello and include it as sayHello, both withName: hello and withName: sayHello apply, with withName: sayHello taking priority.

Tip

You don’t need to enclose label and process names in quotes unless they contain special characters (-, !, etc.) or are keywords or built-in type identifiers.

Selector expressions

You can use regular expressions in process selectors to apply the same configuration to all processes matching the pattern:

process {
    withLabel: 'hello|bye' {
        cpus = 2
        memory = 4.GB
    }
}

This configuration requests 2 CPUs and 4 GB of memory for processes labeled as hello or bye.

You can negate a selector expression by prefixing it with the special character !:

process {
    withLabel: 'hello' { cpus = 2 }
    withLabel: '!hello' { cpus = 4 }
    withName: '!align.*' { queue = 'long' }
}

This configuration sets 2 CPUs for processes labeled as hello and 4 CPUs for all processes not labeled as hello. It also specifies the long queue for processes whose name does not start with align.

Selector priority

Nextflow applies process configuration settings in the following order (from lowest to highest priority):

  1. Process configuration settings (without a selector)

  2. Process directives in the process definition

  3. withLabel selectors matching any of the process labels

  4. withName selectors matching the process name

  5. withName selectors matching the process included alias

  6. withName selectors matching the process fully qualified name

For example:

process {
    cpus = 4
    withLabel: hello { cpus = 8 }
    withName: bye { cpus = 16 }
    withName: 'aloha:bye' { cpus = 32 }
}

This configuration:

  • Sets 4 CPUs for all processes (unless otherwise specified in their process definition)

  • Sets 8 CPUs for processes annotated with the hello label

  • Sets 16 CPUs for any process named bye (or imported as bye)

  • Sets 32 CPUs for any process named bye (or imported as bye) invoked by a workflow named aloha

Config profiles

Configuration files can define one or more profiles. A profile is a set of configuration settings that can be selected at runtime using the -profile command line option.

Configuration profiles are defined in the profiles scope. For example:

profiles {
    standard {
        process.executor = 'local'
    }

    cluster {
        process.executor = 'sge'
        process.queue = 'long'
        process.memory = '10GB'
    }

    cloud {
        process.executor = 'cirrus'
        process.container = 'cbcrg/imagex'
        docker.enabled = true
    }
}

This configuration defines three profiles: standard, cluster, and cloud. Each profile provides a different configuration for a given execution environment. When you do not specify a profile, Nextflow uses the standard profile by default.

You can enable configuration profiles at runtime as a comma-separated list:

nextflow run main.nf -profile standard,cloud

Nextflow applies config profiles in the order in which they were defined in the config, regardless of the order you specify them on the command line.

New in version 25.04.0: When using the strict syntax, Nextflow applies profiles in the order you specify them on the command line.

Workflow handlers

Deprecated since version 25.10.0: Use a trace observer in a plugin to add custom workflow handlers to a pipeline via configuration.

You can define workflow event handlers in the config file:

workflow.onComplete = {
    println "Pipeline complete"
    println "Command line: $workflow.commandLine"
}

workflow.onError = {
    println "Error: something went wrong"
}

This approach is useful for handling workflow events without modifying the pipeline code. See Workflow handlers for more information.