Multi-Environment Deployment from S3

You want one flAPI container image, three running environments — dev, staging, prod — each picking up its own flapi.yaml and SQL templates from S3. When somebody merges to main, the prod templates in S3 get refreshed; the running pods pull the new config on their next reload. No image rebuild, no rolling deployment for a configuration-only change.

This recipe wires that up end-to-end against AWS S3. The same pattern works for GCS and Azure Blob — only the URI scheme and credentials change.

Architecture

                      git push to main
                              |
                              v
                    +-------------------+
                    |   GitHub Action   |   syncs config/ -> s3://my-flapi-config/prod/
                    +-------------------+
                              |
                              v
                +---------------------------+
                |   s3://my-flapi-config/   |
                |     dev/                  |
                |     staging/              |
                |     prod/                 |
                +---------------------------+
                      ^         ^         ^
                      |         |         |
              flapi --config s3://...   (per environment)
                      ^         ^         ^
                      |         |         |
                    [dev]   [staging]   [prod]   pods

The flAPI binary loads YAML and SQL files through DuckDB's virtual filesystem (VFS), so any URI DuckDB can read, flAPI can read.

S3 Layout

s3://my-flapi-config/
  dev/
    flapi.yaml
    sqls/
      customers.yaml
      customers.sql
  staging/
    flapi.yaml
    sqls/
      customers.yaml
      customers.sql
  prod/
    flapi.yaml
    sqls/
      customers.yaml
      customers.sql

The split is fully physical — each environment has its own folder. There is no {{env.STAGE}} magic; the deployment knows which folder to load by passing the right --config URI.

Deployment Invocations

The same image, three different startup commands:

# dev
flapi --config s3://my-flapi-config/dev/flapi.yaml

# staging
flapi --config s3://my-flapi-config/staging/flapi.yaml

# prod
flapi --config s3://my-flapi-config/prod/flapi.yaml

If you also expose the Config Service REST API for hot reloads, add --config-service:

flapi --config s3://my-flapi-config/prod/flapi.yaml \
      --config-service \
      --config-service-token "$CONFIG_SERVICE_TOKEN"

See the Config Service REST API for the reload endpoints.

The flapi.yaml in S3

One example for prod. template.path is set to a sibling prefix in the same bucket so that endpoint YAMLs and SQL files don't need full URIs.

# s3://my-flapi-config/prod/flapi.yaml
project-name: products-api-prod
project-description: Production deployment, config from S3

template:
  path: s3://my-flapi-config/prod/sqls/
  environment-whitelist:
    - '^AWS_.*'
    - '^FLAPI_.*'

connections:
  products-data:
    init: |
      INSTALL httpfs;
      LOAD httpfs;
      SET s3_region='us-east-1';
    properties:
      base_path: s3://my-data/parquet/
      path: s3://my-data/parquet/products.parquet

duckdb:
  access_mode: READ_WRITE
  threads: 8

ducklake:
  enabled: true
  alias: cache
  metadata-path: ./data/cache.ducklake
  data-path: ./data/cache
  retention:
    keep-last-snapshots: 30
    max-snapshot-age: 14d
  scheduler:
    enabled: true
    scan-interval: 5m

storage:
  cache:
    enabled: true
    ttl: 300
    max_size: 50MB

The dev and staging variants are byte-identical except for the bucket prefix, connection paths, and retention numbers.

Required Environment Variables

flAPI talks to S3 through DuckDB's httpfs extension, which reads credentials from the standard AWS environment:

Variable	Required?	Notes
AWS_ACCESS_KEY_ID	yes (off-AWS)	Provided automatically on EC2/ECS/EKS via IAM role
AWS_SECRET_ACCESS_KEY	yes (off-AWS)	Same as above
AWS_REGION	yes	e.g. us-east-1
AWS_SESSION_TOKEN	optional	For STS / assumed-role credentials
AWS_ENDPOINT_URL	optional	For S3-compatible storage (MinIO, LocalStack)

When the pod runs inside AWS with an attached IAM role, no environment variables are needed — the role is picked up by the AWS SDK chain automatically.

Scheme Whitelisting

By default flAPI's path validator only allows two URI schemes:

• file:// (and plain local paths)
• https://

This is enforced by PathValidator::Config::allowed_schemes in src/include/path_validator.hpp, which is the gate that every config and template path passes through before it's resolved.

To use s3:// for --config you have to opt in by letting flAPI know that S3 is a real, configured backend. The pattern is to declare a connection whose init: block installs and loads the httpfs extension:

connections:
  cloud-data:
    init: |
      INSTALL httpfs;
      LOAD httpfs;
      SET s3_region='us-east-1';
    properties:
      base_path: s3://my-flapi-config/prod/

Once httpfs is loaded and an S3-backed connection exists, s3:// becomes available for --config, template.path, and any template-source. The same applies to gs:// and az:// — install/load the right extension in a connection's init block.

If you skip this and pass --config s3://... cold, flAPI will reject the path with URL scheme not allowed: s3 from PathValidator::ValidateRemotePath.

IAM Permissions

The pod's IAM role (or static credentials) needs read access to the bucket prefix. Minimum policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ReadFlapiConfig",
      "Effect": "Allow",
      "Action": [
        "s3:GetObject"
      ],
      "Resource": "arn:aws:s3:::my-flapi-config/*"
    },
    {
      "Sid": "ListFlapiConfigBucket",
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket"
      ],
      "Resource": "arn:aws:s3:::my-flapi-config"
    }
  ]
}

s3:GetObject is needed for every YAML and SQL file read. s3:ListBucket is needed for template.path directory enumeration. Scope the resource ARN to a single environment prefix (e.g. my-flapi-config/prod/*) for stricter isolation.

If your data sources also live on S3, add a second statement covering the data bucket.

GitOps Workflow

The idea: the canonical configuration lives in git. A merge to main syncs the relevant subfolder to S3, and the running pods pick the change up on their next config reload.

Repository layout:

my-flapi-configs/
  dev/
    flapi.yaml
    sqls/...
  staging/
    flapi.yaml
    sqls/...
  prod/
    flapi.yaml
    sqls/...
  .github/
    workflows/
      sync-config.yml

GitHub Actions workflow

# .github/workflows/sync-config.yml
name: Sync flAPI config to S3

on:
  push:
    branches: [main]
    paths:
      - 'dev/**'
      - 'staging/**'
      - 'prod/**'

permissions:
  id-token: write
  contents: read

jobs:
  sync:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        env: [dev, staging, prod]
    steps:
      - uses: actions/checkout@v4

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::123456789012:role/flapi-config-sync
          aws-region: us-east-1

      - name: Sync ${{ matrix.env }} to S3
        run: |
          aws s3 sync ./${{ matrix.env }}/ \
            s3://my-flapi-config/${{ matrix.env }}/ \
            --delete \
            --exclude '.git*'

Optional add-ons for production hygiene:

• Run a flapi --validate-config s3://my-flapi-config/staging/flapi.yaml step before syncing prod, to catch broken YAML.
• After the sync, post a notification or call the running pod's /api/v1/_config/refresh endpoint (via the Config Service API) so changes take effect within seconds rather than at the next scheduled reload.

Hot Reload

flAPI does not poll S3 on every request. Remote files are cached in an LRU per the storage.cache block (default TTL 300s) — that's the VFS cache documented in the Cloud Storage guide. Two ways to push config out faster:

1. Lower the TTL on storage.cache.ttl if you want passive reload, at the cost of more S3 reads.
2. Call the Config Service for active reload. With --config-service enabled, the running pod exposes endpoints like POST /api/v1/_config/refresh and per-endpoint reload routes that re-read templates from S3 immediately. See the Config Service REST API.

The CI workflow above can issue an authenticated POST to the prod pods at the end of the sync step to make the rollout effectively instantaneous.

Kubernetes Deployment Snippet

A trimmed prod Deployment. The same manifest deployed three times (or via Kustomize/Helm) covers all environments — only the FLAPI_CONFIG_S3_URL ConfigMap key changes.

apiVersion: v1
kind: ConfigMap
metadata:
  name: flapi-env-prod
  namespace: flapi
data:
  FLAPI_CONFIG_S3_URL: s3://my-flapi-config/prod/flapi.yaml
  AWS_REGION: us-east-1
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: flapi-prod
  namespace: flapi
spec:
  replicas: 3
  selector:
    matchLabels:
      app: flapi
      env: prod
  template:
    metadata:
      labels:
        app: flapi
        env: prod
    spec:
      serviceAccountName: flapi-prod  # IRSA-bound to the IAM role above
      containers:
        - name: flapi
          image: ghcr.io/datazoode/flapi:latest
          args:
            - --config
            - $(FLAPI_CONFIG_S3_URL)
            - --config-service
          envFrom:
            - configMapRef:
                name: flapi-env-prod
          env:
            - name: CONFIG_SERVICE_TOKEN
              valueFrom:
                secretKeyRef:
                  name: flapi-prod-secrets
                  key: config-service-token
          ports:
            - containerPort: 8080
              name: http
          readinessProbe:
            httpGet:
              path: /api/v1/_config/health
              port: http
            initialDelaySeconds: 5
            periodSeconds: 10
          resources:
            requests:
              cpu: 200m
              memory: 512Mi
            limits:
              cpu: 2
              memory: 4Gi

Notes:

• serviceAccountName: flapi-prod is bound to the IAM role with the policy from the previous section via IRSA (IAM Roles for Service Accounts). No AWS_ACCESS_KEY_ID is set on the pod.
• args references the env var with shell-style $(VAR) substitution — Kubernetes resolves it before exec.
• The readiness probe hits the Config Service health endpoint, which reports VFS status (the bucket has to be reachable for the pod to be ready).
• Three replicas are intentional: a config rollout via the GitHub Action plus a POST /_config/refresh propagates to all three pods.

Testing the Setup Locally

Before deploying, smoke-test the same invocation against MinIO or LocalStack:

# Start LocalStack with S3
docker run -d --name localstack -p 4566:4566 localstack/localstack

# Set the S3-compatible endpoint
export AWS_ACCESS_KEY_ID=test
export AWS_SECRET_ACCESS_KEY=test
export AWS_REGION=us-east-1
export AWS_ENDPOINT_URL=http://localhost:4566

# Upload one env
aws --endpoint-url $AWS_ENDPOINT_URL s3 mb s3://my-flapi-config
aws --endpoint-url $AWS_ENDPOINT_URL s3 sync ./prod/ s3://my-flapi-config/prod/

# Run flAPI against it
flapi --config s3://my-flapi-config/prod/flapi.yaml

If you see URL scheme not allowed: s3, your flapi.yaml is missing a connection that installs httpfs — see Scheme Whitelisting.

Common Pitfalls

• Forgetting s3:ListBucket. Reading individual files works, but template.path: s3://.../sqls/ directory enumeration fails silently.
• Mixing buckets across environments. If prod accidentally points template.path at the dev bucket, the difference will be invisible until a prod-only template gets edited. Keep prefixes physically separate and lock down IAM per environment.
• Region drift. Pods running in eu-west-1 reading from a us-east-1 bucket will work but with measurable latency on every cache miss. Co-locate the config bucket with the running cluster, or lean on a longer storage.cache.ttl.
• Forgetting to call reload. Pods will eventually pick up new config when the LRU TTL elapses, but if you need a guaranteed rollout, finish your CI pipeline with a Config Service refresh call.

Next Steps

• Cloud Storage and VFS — full reference for all supported URI schemes (S3, GCS, Azure, HTTPS) and credential sources.
• Config Service REST API — endpoints for runtime reloads, manual cache refresh, and health checks.
• Deployment guide — broader deployment topologies including Docker, ConfigMaps, Cloud Run, and Lambda.