dbt ls (list) Command: Usage & Examples

Introduction

dbt is a transformative tool in the world of analytics, allowing data teams to transform, test, and document their data workflows. One of its powerful commands is dbt ls, which provides a way to list resources in your dbt project. This tutorial will guide you through its usage, key arguments, and practical examples.

dbt list (ls) Basic Usage

The dbt ls command, also known as dbt list, is designed to list resources in your dbt project. Unlike some other commands, it doesn't connect to the database or run any queries. It only reads your your connection profile to resolve target-specific logic.

dbt ls

Key Arguments

Resource Type (--resource-type)

This argument allows you to specify the type of resource you want to list, such as models, sources, seeds, or snapshots.

dbt ls --resource-type model

Selection (--select and --models)

Use these arguments to filter the nodes or models you want to list. The shorthand for --select is -s and the shorthand for --models is -m.

For instance, if you have a sales model and want to list only those, you'd use:

dbt ls --select intermediate.*

Exclusion (--exclude)

If you want to list all models except for a specific model or group of models, you can use the --exclude argument. The command below will return all models except for the ones that have the v2 tag.

dbt ls --resource-type model --exclude tag:v2

Named Selectors (--selector)

For complex projects, you might have predefined selectors in a selectors.yml file. Use this argument to list resources based on those selectors.

dbt ls --selector my_selector

Advanced Output Options

Output Format (--output)

By default, the output is in a readable format. However, for integration with other tools, you might need a JSON output:

dbt ls --output json

Output Keys (--output-keys)

When using JSON output, you can specify which keys you want to include:

dbt ls --output json --output-keys name resource_type

Practical dbt ls Examples

Listing Specific Models for a Retail Business

Suppose you have models related to inventory and want to list all of them:

dbt ls --select inventory.*

Listing Tests for Financial Reporting

For businesses in the finance sector, ensuring data integrity is crucial. List all tests related to financial data by filtering on the tag financial that we’ve applied on all tests in that category:

dbt ls --select tag:financial --resource-type test

Listing Incremental Models in E-commerce

In e-commerce, data like user activity might be stored incrementally. To list such models:

dbt ls --select config.materialized:incremental

Output for Integration with Business Dashboards

If you're integrating with a business dashboard tool and need a JSON output:

$ dbt ls --select sales.* --output json

Common Pitfalls and Tips

• Always ensure that the models you're trying to list are not disabled.
• Familiarize yourself with the connection profile and target-specific logic to avoid unexpected results.

Conclusion

The dbt ls command is a versatile tool in the dbt suite, allowing for efficient resource listing and management. By mastering its arguments and understanding its use cases, you can streamline your data workflow and improve your project's organization.