Skip to main content

Prepare production style runs

Production environments typically reuse pre-built runtimes to avoid the startup cost of packaging dependencies on each run. The demo shows this by pre-deploying the pixi environment and pointing the resource configuration at it.

pixi run deploy-prod-docker  # builds and uploads the pixi environment

Inspect the generated metadata and persist the target path:

deployment_path="$(jq -er '.deployment_path' deployment_metadata.json)"
echo "DAGSTER_PROD_ENV_PATH=${deployment_path}" > .env.prod
export DAGSTER_PROD_ENV_PATH="${deployment_path}"

Then start Dagster in production mode:

# for local integration testing -> see the docker mode in how to contriubte for explanation
# pixi run start-prod-docker

# for the real HPC system
pixi run start-production-supercomputer