dbt-core’s API documentation

How to invoke dbt commands in python runtime

Right now the best way to invoke a command from python runtime is to use the dbtRunner we exposed

from dbt.cli.main import dbtRunner
cli_args = ['run', '--project-dir', 'jaffle_shop']

# initialize the dbt runner
dbt = dbtRunner()
# run the command
res, success = dbt.invoke(args)

You can also pass in pre constructed object into dbtRunner, and we will use those objects instead of loading up from the disk.

# preload profile and project
profile = load_profile(project_dir, {}, 'testing-postgres')
project = load_project(project_dir, False, profile, {})

# initialize the runner with pre-loaded profile and project
dbt = dbtRunner(profile=profile, project=project)
# run the command, this will use the pre-loaded profile and project instead of loading
res, success = dbt.invoke(cli_args)

For the full example code, you can refer to core/dbt/cli/example.py

API documentation

Command: build

defer

Type: boolean

If set, defer to the state variable for resolving unselected nodes.

exclude

Type: unknown

Specify the nodes to exclude.

fail_fast

Type: boolean

Stop execution on first failure.

favor_state

Type: boolean

If set, defer to the argument provided to the state flag for resolving unselected nodes, even if the node(s) exist as a database object in the current environment.

full_refresh

Type: boolean

If specified, dbt will drop incremental models and fully-recalculate the incremental table from the model definition.

indirect_selection

Type: choice: [‘eager’, ‘cautious’, ‘buildable’]

Select all tests that are adjacent to selected resources, even if they those resources have been explicitly selected.

profile

Type: string

Which profile to load. Overrides setting in dbt_project.yml.

profiles_dir

Type: path

Which directory to look in for the profiles.yml file. If not set, dbt will look in the current working directory first, then HOME/.dbt/

project_dir

Type: path

Which directory to look in for the dbt_project.yml file. Default is the current working directory and its parents.

resource_types

Type: unknown

TODO: No current help text

select

Type: unknown

Specify the nodes to include.

selector

Type: string

The selector name to use, as defined in selectors.yml

show

Type: boolean

Show a sample of the loaded data in the terminal

state

Type: path

If set, use the given directory as the source for json files to compare with this project.

store_failures

Type: boolean

Store test results (failing rows) in the database

target

Type: string

Which target to load for the given profile

target_path

Type: path

Configure the ‘target-path’. Only applies this setting for the current run. Overrides the ‘DBT_TARGET_PATH’ if it is set.

threads

Type: int

Specify number of threads to use while executing models. Overrides settings in profiles.yml.

vars

Type: YAML

Supply variables to the project. This argument overrides variables defined in your dbt_project.yml file. This argument should be a YAML string, eg. ‘{my_variable: my_value}’

version_check

Type: boolean

Ensure dbt’s version matches the one specified in the dbt_project.yml file (‘require-dbt-version’)

Command: clean

profile

Type: string

Which profile to load. Overrides setting in dbt_project.yml.

profiles_dir

Type: path

Which directory to look in for the profiles.yml file. If not set, dbt will look in the current working directory first, then HOME/.dbt/

project_dir

Type: path

Which directory to look in for the dbt_project.yml file. Default is the current working directory and its parents.

target

Type: string

Which target to load for the given profile

vars

Type: YAML

Supply variables to the project. This argument overrides variables defined in your dbt_project.yml file. This argument should be a YAML string, eg. ‘{my_variable: my_value}’

Command: compile

defer

Type: boolean

If set, defer to the state variable for resolving unselected nodes.

exclude

Type: unknown

Specify the nodes to exclude.

favor_state

Type: boolean

If set, defer to the argument provided to the state flag for resolving unselected nodes, even if the node(s) exist as a database object in the current environment.

full_refresh

Type: boolean

If specified, dbt will drop incremental models and fully-recalculate the incremental table from the model definition.

introspect

Type: boolean

Whether to scaffold introspective queries as part of compilation

parse_only

Type: boolean

TODO: No help text currently available

profile

Type: string

Which profile to load. Overrides setting in dbt_project.yml.

profiles_dir

Type: path

Which directory to look in for the profiles.yml file. If not set, dbt will look in the current working directory first, then HOME/.dbt/

project_dir

Type: path

Which directory to look in for the dbt_project.yml file. Default is the current working directory and its parents.

select

Type: unknown

Specify the nodes to include.

selector

Type: string

The selector name to use, as defined in selectors.yml

state

Type: path

If set, use the given directory as the source for json files to compare with this project.

target

Type: string

Which target to load for the given profile

target_path

Type: path

Configure the ‘target-path’. Only applies this setting for the current run. Overrides the ‘DBT_TARGET_PATH’ if it is set.

threads

Type: int

Specify number of threads to use while executing models. Overrides settings in profiles.yml.

vars

Type: YAML

Supply variables to the project. This argument overrides variables defined in your dbt_project.yml file. This argument should be a YAML string, eg. ‘{my_variable: my_value}’

version_check

Type: boolean

Ensure dbt’s version matches the one specified in the dbt_project.yml file (‘require-dbt-version’)

Command: debug

config_dir

Type: string

If specified, DBT will show path information for this project

profile

Type: string

Which profile to load. Overrides setting in dbt_project.yml.

profiles_dir

Type: path

Which directory to look in for the profiles.yml file. If not set, dbt will look in the current working directory first, then HOME/.dbt/

project_dir

Type: path

Which directory to look in for the dbt_project.yml file. Default is the current working directory and its parents.

target

Type: string

Which target to load for the given profile

vars

Type: YAML

Supply variables to the project. This argument overrides variables defined in your dbt_project.yml file. This argument should be a YAML string, eg. ‘{my_variable: my_value}’

version_check

Type: boolean

Ensure dbt’s version matches the one specified in the dbt_project.yml file (‘require-dbt-version’)

Command: deps

profile

Type: string

Which profile to load. Overrides setting in dbt_project.yml.

profiles_dir

Type: path

Which directory to look in for the profiles.yml file. If not set, dbt will look in the current working directory first, then HOME/.dbt/

project_dir

Type: path

Which directory to look in for the dbt_project.yml file. Default is the current working directory and its parents.

target

Type: string

Which target to load for the given profile

vars

Type: YAML

Supply variables to the project. This argument overrides variables defined in your dbt_project.yml file. This argument should be a YAML string, eg. ‘{my_variable: my_value}’

Command: docs

Command: init

project_name

Type: string

profile

Type: string

Which profile to load. Overrides setting in dbt_project.yml.

profiles_dir

Type: path

Which directory to look in for the profiles.yml file. If not set, dbt will look in the current working directory first, then HOME/.dbt/

project_dir

Type: path

Which directory to look in for the dbt_project.yml file. Default is the current working directory and its parents.

skip_profile_setup

Type: boolean

Skip interactive profile setup.

target

Type: string

Which target to load for the given profile

vars

Type: YAML

Supply variables to the project. This argument overrides variables defined in your dbt_project.yml file. This argument should be a YAML string, eg. ‘{my_variable: my_value}’

Command: list

exclude

Type: unknown

Specify the nodes to exclude.

indirect_selection

Type: choice: [‘eager’, ‘cautious’, ‘buildable’]

Select all tests that are adjacent to selected resources, even if they those resources have been explicitly selected.

models

Type: unknown

Specify the nodes to include.

output

Type: choice: [‘json’, ‘name’, ‘path’, ‘selector’]

TODO: No current help text

output_keys

Type: unknown

Space-delimited listing of node properties to include as custom keys for JSON output (e.g. `–output json –output-keys name resource_type description`)

profile

Type: string

Which profile to load. Overrides setting in dbt_project.yml.

profiles_dir

Type: path

Which directory to look in for the profiles.yml file. If not set, dbt will look in the current working directory first, then HOME/.dbt/

project_dir

Type: path

Which directory to look in for the dbt_project.yml file. Default is the current working directory and its parents.

resource_types

Type: unknown

TODO: No current help text

select

Type: unknown

Specify the nodes to include.

selector

Type: string

The selector name to use, as defined in selectors.yml

state

Type: path

If set, use the given directory as the source for json files to compare with this project.

target

Type: string

Which target to load for the given profile

vars

Type: YAML

Supply variables to the project. This argument overrides variables defined in your dbt_project.yml file. This argument should be a YAML string, eg. ‘{my_variable: my_value}’

Command: list

exclude

Type: unknown

Specify the nodes to exclude.

indirect_selection

Type: choice: [‘eager’, ‘cautious’, ‘buildable’]

Select all tests that are adjacent to selected resources, even if they those resources have been explicitly selected.

models

Type: unknown

Specify the nodes to include.

output

Type: choice: [‘json’, ‘name’, ‘path’, ‘selector’]

TODO: No current help text

output_keys

Type: unknown

Space-delimited listing of node properties to include as custom keys for JSON output (e.g. `–output json –output-keys name resource_type description`)

profile

Type: string

Which profile to load. Overrides setting in dbt_project.yml.

profiles_dir

Type: path

Which directory to look in for the profiles.yml file. If not set, dbt will look in the current working directory first, then HOME/.dbt/

project_dir

Type: path

Which directory to look in for the dbt_project.yml file. Default is the current working directory and its parents.

resource_types

Type: unknown

TODO: No current help text

select

Type: unknown

Specify the nodes to include.

selector

Type: string

The selector name to use, as defined in selectors.yml

state

Type: path

If set, use the given directory as the source for json files to compare with this project.

target

Type: string

Which target to load for the given profile

vars

Type: YAML

Supply variables to the project. This argument overrides variables defined in your dbt_project.yml file. This argument should be a YAML string, eg. ‘{my_variable: my_value}’

Command: parse

compile

Type: boolean

TODO: No help text currently available

profile

Type: string

Which profile to load. Overrides setting in dbt_project.yml.

profiles_dir

Type: path

Which directory to look in for the profiles.yml file. If not set, dbt will look in the current working directory first, then HOME/.dbt/

project_dir

Type: path

Which directory to look in for the dbt_project.yml file. Default is the current working directory and its parents.

target

Type: string

Which target to load for the given profile

target_path

Type: path

Configure the ‘target-path’. Only applies this setting for the current run. Overrides the ‘DBT_TARGET_PATH’ if it is set.

threads

Type: int

Specify number of threads to use while executing models. Overrides settings in profiles.yml.

vars

Type: YAML

Supply variables to the project. This argument overrides variables defined in your dbt_project.yml file. This argument should be a YAML string, eg. ‘{my_variable: my_value}’

version_check

Type: boolean

Ensure dbt’s version matches the one specified in the dbt_project.yml file (‘require-dbt-version’)

write_manifest

Type: boolean

TODO: No help text currently available

Command: run

defer

Type: boolean

If set, defer to the state variable for resolving unselected nodes.

favor_state

Type: boolean

If set, defer to the argument provided to the state flag for resolving unselected nodes, even if the node(s) exist as a database object in the current environment.

exclude

Type: unknown

Specify the nodes to exclude.

fail_fast

Type: boolean

Stop execution on first failure.

full_refresh

Type: boolean

If specified, dbt will drop incremental models and fully-recalculate the incremental table from the model definition.

profile

Type: string

Which profile to load. Overrides setting in dbt_project.yml.

profiles_dir

Type: path

Which directory to look in for the profiles.yml file. If not set, dbt will look in the current working directory first, then HOME/.dbt/

project_dir

Type: path

Which directory to look in for the dbt_project.yml file. Default is the current working directory and its parents.

select

Type: unknown

Specify the nodes to include.

selector

Type: string

The selector name to use, as defined in selectors.yml

state

Type: path

If set, use the given directory as the source for json files to compare with this project.

target

Type: string

Which target to load for the given profile

target_path

Type: path

Configure the ‘target-path’. Only applies this setting for the current run. Overrides the ‘DBT_TARGET_PATH’ if it is set.

threads

Type: int

Specify number of threads to use while executing models. Overrides settings in profiles.yml.

vars

Type: YAML

Supply variables to the project. This argument overrides variables defined in your dbt_project.yml file. This argument should be a YAML string, eg. ‘{my_variable: my_value}’

version_check

Type: boolean

Ensure dbt’s version matches the one specified in the dbt_project.yml file (‘require-dbt-version’)

Command: run_operation

macro

Type: string

args

Type: YAML

Supply arguments to the macro. This dictionary will be mapped to the keyword arguments defined in the selected macro. This argument should be a YAML string, eg. ‘{my_variable: my_value}’

profile

Type: string

Which profile to load. Overrides setting in dbt_project.yml.

profiles_dir

Type: path

Which directory to look in for the profiles.yml file. If not set, dbt will look in the current working directory first, then HOME/.dbt/

project_dir

Type: path

Which directory to look in for the dbt_project.yml file. Default is the current working directory and its parents.

target

Type: string

Which target to load for the given profile

vars

Type: YAML

Supply variables to the project. This argument overrides variables defined in your dbt_project.yml file. This argument should be a YAML string, eg. ‘{my_variable: my_value}’

Command: seed

exclude

Type: unknown

Specify the nodes to exclude.

full_refresh

Type: boolean

If specified, dbt will drop incremental models and fully-recalculate the incremental table from the model definition.

profile

Type: string

Which profile to load. Overrides setting in dbt_project.yml.

profiles_dir

Type: path

Which directory to look in for the profiles.yml file. If not set, dbt will look in the current working directory first, then HOME/.dbt/

project_dir

Type: path

Which directory to look in for the dbt_project.yml file. Default is the current working directory and its parents.

select

Type: unknown

Specify the nodes to include.

selector

Type: string

The selector name to use, as defined in selectors.yml

show

Type: boolean

Show a sample of the loaded data in the terminal

state

Type: path

If set, use the given directory as the source for json files to compare with this project.

target

Type: string

Which target to load for the given profile

target_path

Type: path

Configure the ‘target-path’. Only applies this setting for the current run. Overrides the ‘DBT_TARGET_PATH’ if it is set.

threads

Type: int

Specify number of threads to use while executing models. Overrides settings in profiles.yml.

vars

Type: YAML

Supply variables to the project. This argument overrides variables defined in your dbt_project.yml file. This argument should be a YAML string, eg. ‘{my_variable: my_value}’

version_check

Type: boolean

Ensure dbt’s version matches the one specified in the dbt_project.yml file (‘require-dbt-version’)

Command: snapshot

defer

Type: boolean

If set, defer to the state variable for resolving unselected nodes.

exclude

Type: unknown

Specify the nodes to exclude.

favor_state

Type: boolean

If set, defer to the argument provided to the state flag for resolving unselected nodes, even if the node(s) exist as a database object in the current environment.

profile

Type: string

Which profile to load. Overrides setting in dbt_project.yml.

profiles_dir

Type: path

Which directory to look in for the profiles.yml file. If not set, dbt will look in the current working directory first, then HOME/.dbt/

project_dir

Type: path

Which directory to look in for the dbt_project.yml file. Default is the current working directory and its parents.

select

Type: unknown

Specify the nodes to include.

selector

Type: string

The selector name to use, as defined in selectors.yml

state

Type: path

If set, use the given directory as the source for json files to compare with this project.

target

Type: string

Which target to load for the given profile

threads

Type: int

Specify number of threads to use while executing models. Overrides settings in profiles.yml.

vars

Type: YAML

Supply variables to the project. This argument overrides variables defined in your dbt_project.yml file. This argument should be a YAML string, eg. ‘{my_variable: my_value}’

Command: source

Command: test

defer

Type: boolean

If set, defer to the state variable for resolving unselected nodes.

exclude

Type: unknown

Specify the nodes to exclude.

fail_fast

Type: boolean

Stop execution on first failure.

favor_state

Type: boolean

If set, defer to the argument provided to the state flag for resolving unselected nodes, even if the node(s) exist as a database object in the current environment.

indirect_selection

Type: choice: [‘eager’, ‘cautious’, ‘buildable’]

Select all tests that are adjacent to selected resources, even if they those resources have been explicitly selected.

profile

Type: string

Which profile to load. Overrides setting in dbt_project.yml.

profiles_dir

Type: path

Which directory to look in for the profiles.yml file. If not set, dbt will look in the current working directory first, then HOME/.dbt/

project_dir

Type: path

Which directory to look in for the dbt_project.yml file. Default is the current working directory and its parents.

select

Type: unknown

Specify the nodes to include.

selector

Type: string

The selector name to use, as defined in selectors.yml

state

Type: path

If set, use the given directory as the source for json files to compare with this project.

store_failures

Type: boolean

Store test results (failing rows) in the database

target

Type: string

Which target to load for the given profile

target_path

Type: path

Configure the ‘target-path’. Only applies this setting for the current run. Overrides the ‘DBT_TARGET_PATH’ if it is set.

threads

Type: int

Specify number of threads to use while executing models. Overrides settings in profiles.yml.

vars

Type: YAML

Supply variables to the project. This argument overrides variables defined in your dbt_project.yml file. This argument should be a YAML string, eg. ‘{my_variable: my_value}’

version_check

Type: boolean

Ensure dbt’s version matches the one specified in the dbt_project.yml file (‘require-dbt-version’)