Run a dedicated scheduler worker
Run scheduled deliveries on a dedicated worker so heavy exports can't starve the API
🛠 This page is for engineering teams self-hosting their own Qyra instance. If you want to set up scheduled deliveries, go to the Scheduled deliveries guide.
By default, the Qyra backend runs scheduled jobs in-process. In production, run a dedicated scheduler worker so a heavy dashboard export can't starve the API. Run one through the Helm chart's built-in worker, or as a standalone process you start yourself.
Helm chart
Set scheduler.enabled: true and the chart deploys a dedicated scheduler worker:
scheduler:
enabled: true
replicas: 1 # scale up if deliveries queue at peak times
concurrency: 3 # jobs per replica (sets SCHEDULER_CONCURRENCY)
resources:
requests:
cpu: 500m
memory: 1425Mi
ephemeral-storage: 1GiWhen scheduler.enabled: true, the chart automatically sets SCHEDULER_ENABLED: "false" on the backend — don't set it yourself. Tune delivery throughput with SCHEDULER_JOB_TIMEOUT, SCHEDULER_SCREENSHOT_TIMEOUT, and scheduler.tasks.include / scheduler.tasks.exclude to shard task types across worker groups. See the scheduler environment variables for the full reference.
Standalone worker
For infrastructure that doesn't use the Helm chart, run the scheduler worker as a separate process.
1. Disable the scheduler worker in the main server
Set the following environment variable value:
SCHEDULER_ENABLED=false2. Run the standalone scheduler worker
Command to start scheduler worker:
yarn workspace backend schedulerNote that it expects the same environment variables as the main server.
Example docker-compose.yml
version: "3.8"
x-environment: &commonEnvironment
- PGHOST=${PGHOST:-db}
- PGPORT=${PGPORT:-5432}
- PGUSER=${PGUSER:-postgres}
- PGPASSWORD=${PGPASSWORD}
- PGDATABASE=${PGDATABASE:-postgres}
- SECURE_COOKIES=${SECURE_COOKIES:-false}
- TRUST_PROXY=${TRUST_PROXY:-false}
- QYRA_SECRET=${QYRA_SECRET}
- PORT=${PORT:-8080}
- SITE_URL=${SITE_URL}
- EMAIL_SMTP_HOST=${EMAIL_SMTP_HOST}
- EMAIL_SMTP_PORT=${EMAIL_SMTP_PORT}
- EMAIL_SMTP_SECURE=${EMAIL_SMTP_SECURE}
- EMAIL_SMTP_USER=${EMAIL_SMTP_USER}
- EMAIL_SMTP_PASSWORD=${EMAIL_SMTP_PASSWORD}
- EMAIL_SMTP_ALLOW_INVALID_CERT=${EMAIL_SMTP_ALLOW_INVALID_CERT}
- EMAIL_SMTP_SENDER_NAME=${EMAIL_SMTP_SENDER_NAME}
- EMAIL_SMTP_SENDER_EMAIL=${EMAIL_SMTP_SENDER_EMAIL}
- HEADLESS_BROWSER_HOST=headless-browser
- HEADLESS_BROWSER_PORT=3000
services:
headless-browser:
image: browserless/chrome
restart: always
ports:
- "3001:3000"
db:
image: postgres
restart: always
environment:
POSTGRES_PASSWORD: ${PGPASSWORD}
POSTGRES_USER: ${PGUSER:-postgres}
POSTGRES_DB: ${PGDATABASE:-postgres}
volumes:
- db-data:/var/lib/postgresql/data
qyra:
image: quanvio/qyra:latest
depends_on:
- db
environment:
<<: *commonEnvironment
SCHEDULER_ENABLED: 'false'
volumes:
- "${DBT_PROJECT_DIR}:/usr/app/dbt"
ports:
- ${PORT:-8080}:${PORT:-8080}
scheduler:
image: quanvio/qyra:latest
entrypoint: ["yarn", "workspace", "backend", "scheduler"]
depends_on:
- db
environment: *commonEnvironment
volumes:
db-data: