To build and run Airflow DAGs locally and interact with the Astronomer API
Astro CLI is a command-line interface tool designed to simplify the creation, testing, and deployment of Airflow DAGs (Directed Acyclic Graphs) for data pipeline workflows. It provides seamless integration with the Astronomer API, enabling users to manage and monitor their pipelines efficiently.
Key Features:
Local execution of DAGs for rapid development and testing.
Integration with the Astronomer platform for cloud-based deployment and management.
Streamlined dependency management for Airflow plugins and packages.
Debugging tools to identify and resolve issues within DAGs.
Support for multiple Airflow versions, ensuring compatibility across environments.
Audience & Benefit:
Ideal for data engineers, pipeline developers, and teams leveraging Apache Airflow, Astro CLI accelerates the development lifecycle by enabling local testing before deployment. It reduces errors and ensures consistent performance across local and cloud environments, ultimately improving workflow reliability.
Astro CLI can be installed via winget, making it easy to integrate into your existing development setup.
README
Astro CLI
The Astro CLI is a command-line interface for data orchestration. It allows you to get started with Apache Airflow quickly and it can be used with all Astronomer products.
Usage
astro [command]
Core commands
login Log in to the Astro CLI
logout Log out of the Astro CLI
dev init Initialize an Astro project in an empty local directory
dev start Build your Astro project into a Docker image and spin up a local Docker container for each Airflow component
dev stop Pause all Docker containers running your local Airflow environment
dev restart Stop your Airflow environment, rebuild your Astro project into a Docker image, and restart your Airflow environment with the new Docker image
deploy Deploy code to a Deployment on Astro
deployment Manage your Deployments running on Astronomer
Go to the Releases page of the Astro CLI GitHub repository, scroll to a CLI version, and then download the .exe file that matches the CPU architecture of your machine.
For example, to install v1.0.0 of the Astro CLI on a Windows machine with an AMD 64 architecture, download astro_1.0.0-converged_windows_amd64.exe.
Rename the file astro.exe.
Add the filepath for the directory containing the new astro.exe as a PATH environment variable. For example, if astro.exe is stored in C:\Users\username\astro.exe, you add C:\Users\username as your PATH environment variable. To learn more about configuring the PATH environment variable, see How do I set or change the PATH system variable?.
Restart your machine.
Windows with winget
Starting with Astro CLI version 1.6, you can use the Windows Package Manager winget command-line tool to install the Astro CLI.
Open Windows PowerShell as an administrator and then run the following command:
winget install -e --id Astronomer.Astro
Linux
curl -sSL install.astronomer.io | sudo bash -s
Specific version
Mac
To install a specific version of the Astro CLI, specify the version you want to install at the end of the command:
brew install astro@
Windows
Delete the existing astro.exe file on your machine.
Go to the Releases page of the Astro CLI GitHub repository, scroll to a CLI version, and then download the .exe file that matches the CPU architecture of your machine.
For example, to install v1.0.0 of the Astro CLI on a Windows machine with an AMD 64 architecture, download astro_1.0.0-converged_windows_amd64.exe.
Rename the file astro.exe.
Add the filepath for the directory containing the new astro.exe as a PATH environment variable. For example, if astro.exe is stored in C:\Users\username\astro.exe, you add C:\Users\username as your PATH environment variable. To learn more about configuring the PATH environment variable, see How do I set or change the PATH system variable?.
Restart your machine.
Windows with winget
Starting with Astro CLI version 1.6, you can use the Windows Package Manager winget command-line tool to install a specific version of the Astro CLI.
To install a specific version of the Astro CLI, specify the version you want to install at the end of the command. For example, running the following command in Windows PowerShell as an administrator installs Astro CLI version 1.6:
winget install -e --id Astronomer.Astro -v 1.6.0
Linux
To install a specific version of the CLI, specify the version number as a flag at the end of the command. For example, to install v1.1.0 of the CLI, you would run:
> If you receive a mkdir error during installation, download and run the godownloader script locally using:
>
>sh >$ cat godownloader.sh | bash -s -- -b /usr/local/bin >
Troubleshoot installation issues
If you encounter issues when installing the Astro CLI:
Make sure the Docker Desktop WSL 2 backend is installed and configured correctly. See Docker Desktop WSL 2 backend.
Enable Hyper-V on the Docker and Linux Containers and run the Astro CLI natively. For additional information, see the Docker docs troubleshooting guide.
Get started
Create a project
$ mkdir hello-astro && cd hello-astro
$ astro dev init
Install the binary and Confirm the install worked:
DAGs can go in the dags folder, custom Airflow plugins in plugins, python packages needed can go in requirements.txt, and OS level packages can go in packages.txt.
Run astro dev start to start a local version of airflow on your machine. This will spin up a few locally running docker containers - one for the airflow scheduler, one for the webserver, and one for postgres.
(Run astro dev ps to verify).
Versions
Astro CLI versions are released regularly and use semantic versioning. Backwards compatibility between versions cannot be guaranteed. Compatibility is only guaranteed between matching minor versions of the platform and the Astro CLI. For example, Astro CLI 0.9.0` is guaranteed to be compatible with houston-api v0.9.x, but not houston-api v0.10.x.
Astronomer ships major, minor, and patch releases of the Astro CLI in the following format:
{MAJOR_RELEASE}.{MINOR_RELEASE}.{PATCH_RELEASE}
All Astro CLI releases prior to 1.0.0 are considered beta.
Change Log
Change log between each version can be found on the releases page
Debug
The Astro CLI includes a debug flag that allows you to view queries and internal logs. To enable it, you can pass --verbosity=debug in your commands, or you can add the following entry to your ~/.astro/config.yaml file:
verbosity: debug
Adding this entry to your ~/.astro/config.yaml file turns on debugging for all requests until you change it to info, or you remove it from the file.
If you're unable to resolve your issue after reviewing the documentation, you can post a question on the Astronomer web forum or you can contact Astronomer support.
