YAML Syntax & Structure

flAPI extends standard YAML with powerful features for modular configuration, environment variables, and dynamic includes. This guide covers the extended syntax, best practices for structuring endpoint definitions, and path resolution rules.

Extended YAML Features

flAPI's YAML parser supports standard YAML plus these extensions:

1. Include Syntax

Include external YAML files for modular configuration:

# Main configuration
connections: {{include from 'config/connections.yaml'}}

# With relative paths
security: {{include from './config/security.yaml'}}

# Nested includes work too
endpoints: {{include from 'endpoints/all.yaml'}}

Rules:

Paths are relative to the including file
Circular includes are detected and prevented
Includes are resolved at load time
Syntax: {{include from 'path/to/file.yaml'}}

2. Environment Variables

Reference environment variables in your configuration:

# Basic syntax
database:
  host: ${DB_HOST}
  password: ${DB_PASSWORD}

# With default values
database:
  port: ${DB_PORT|5432}
  timeout: ${DB_TIMEOUT|30}

# In connection strings
connection_string: 'postgresql://${DB_USER}:${DB_PASSWORD}@${DB_HOST}/${DB_NAME}'

Rules:

Syntax: ${VAR_NAME} or ${VAR_NAME|default}
Must be whitelisted in environment-whitelist
Resolved at load time
Missing required vars cause errors

3. Template Variables

Access special variables in endpoint configurations:

# Connection properties
template-source: query.sql
connection:
  - my-data

# In SQL template, access:
# {{{conn.my-data.path}}}
# {{{conn.my-data.host}}}

Available variables:

params.* - Request parameters
conn.* - Connection properties
context.user.* - User context (auth)
cache.* - Cache properties
env.* - Whitelisted environment variables

Endpoint YAML Structure

Basic Endpoint

# sqls/customers.yaml
# ═══════════════════════════════════════════════════════════════
# CUSTOMERS API ENDPOINT
# Provides access to customer data with filtering by segment
# ═══════════════════════════════════════════════════════════════

# ───────────────────────────────────────────────────────────────
# ENDPOINT CONFIGURATION
# ───────────────────────────────────────────────────────────────
url-path: /customers/

# ───────────────────────────────────────────────────────────────
# REQUEST PARAMETERS
# ───────────────────────────────────────────────────────────────
request:
  - field-name: segment
    field-in: query
    description: Market segment to filter by (e.g., AUTOMOTIVE, BUILDING)
    required: false
    validators:
      - type: string
        enum: ['AUTOMOTIVE', 'BUILDING', 'FURNITURE', 'MACHINERY', 'HOUSEHOLD']
  
  - field-name: min_balance
    field-in: query
    description: Minimum account balance
    required: false
    validators:
      - type: number
        min: 0

# ───────────────────────────────────────────────────────────────
# SQL TEMPLATE
# ───────────────────────────────────────────────────────────────
template-source: customers.sql

# ───────────────────────────────────────────────────────────────
# DATA CONNECTION
# ───────────────────────────────────────────────────────────────
connection:
  - customers-parquet

# ───────────────────────────────────────────────────────────────
# CACHING (OPTIONAL)
# ───────────────────────────────────────────────────────────────
cache:
  enabled: true
  table: customers_cache
  schema: analytics
  schedule: 60m
  template_file: customers_cache.sql

Complete Endpoint with All Features

# sqls/orders-advanced.yaml
# ═══════════════════════════════════════════════════════════════
# ORDERS API - ADVANCED CONFIGURATION
# Full-featured endpoint with caching, auth, rate limiting, and MCP
# ═══════════════════════════════════════════════════════════════

# ───────────────────────────────────────────────────────────────
# METADATA
# ───────────────────────────────────────────────────────────────
description: Retrieve and filter order data with comprehensive options
version: 2.1.0
owner: api-team
tags: [orders, sales, analytics]

# ───────────────────────────────────────────────────────────────
# ENDPOINT CONFIGURATION
# ───────────────────────────────────────────────────────────────
url-path: /orders/
methods: [GET]

# ───────────────────────────────────────────────────────────────
# REQUEST PARAMETERS
# ───────────────────────────────────────────────────────────────
request:
  # Query parameters
  - field-name: customer_id
    field-in: query
    description: Filter by customer ID
    required: false
    validators:
      - type: int
        min: 1
    example: 12345
  
  - field-name: status
    field-in: query
    description: Order status (pending, completed, cancelled)
    required: false
    validators:
      - type: string
        enum: [pending, completed, cancelled, refunded]
    default: completed
  
  - field-name: start_date
    field-in: query
    description: Start date (YYYY-MM-DD)
    required: false
    validators:
      - type: string
        pattern: '^\d{4}-\d{2}-\d{2}$'
  
  - field-name: end_date
    field-in: query
    description: End date (YYYY-MM-DD)
    required: false
    validators:
      - type: string
        pattern: '^\d{4}-\d{2}-\d{2}$'
  
  - field-name: limit
    field-in: query
    description: Number of results to return
    required: false
    validators:
      - type: int
        min: 1
        max: 1000
    default: 100
  
  - field-name: offset
    field-in: query
    description: Number of results to skip (for pagination)
    required: false
    validators:
      - type: int
        min: 0
    default: 0

# ───────────────────────────────────────────────────────────────
# SQL TEMPLATE
# ───────────────────────────────────────────────────────────────
template-source: orders/list.sql

# ───────────────────────────────────────────────────────────────
# DATA CONNECTION
# ───────────────────────────────────────────────────────────────
connection:
  - bigquery-warehouse
  - customers-cache  # Join with cached customer data

# ───────────────────────────────────────────────────────────────
# CACHING CONFIGURATION
# ───────────────────────────────────────────────────────────────
cache:
  enabled: true
  table: orders_cache
  schema: analytics
  schedule: 15m  # Refresh every 15 minutes
  strategy: merge
  merge_key: order_id
  updated_at_column: last_modified
  template_file: orders/cache.sql
  on_startup: true
  persist: true

# ───────────────────────────────────────────────────────────────
# AUTHENTICATION & AUTHORIZATION
# ───────────────────────────────────────────────────────────────
authentication:
  required: true
  methods: [jwt, api-key]
  roles: [user, admin]

# Row-level security: users see only their orders
security:
  row_level:
    enabled: true
    # In SQL template, use: {{#context.user.roles.admin}}...{{/context.user.roles.admin}}

# ───────────────────────────────────────────────────────────────
# RATE LIMITING
# ───────────────────────────────────────────────────────────────
rate-limit:
  enabled: true
  requests: 1000
  window: 60s
  per: user

# ───────────────────────────────────────────────────────────────
# MCP TOOL CONFIGURATION (for AI agents)
# ───────────────────────────────────────────────────────────────
mcp-tool:
  name: get_orders
  description: |
    Retrieve customer orders with flexible filtering options.
    Supports filtering by customer ID, order status, and date range.
    Returns up to 1000 orders per request with pagination support.
  
  examples:
    - description: Get recent completed orders
      parameters:
        status: completed
        limit: 10
    
    - description: Get orders for specific customer
      parameters:
        customer_id: 12345
        status: completed
    
    - description: Get orders in date range
      parameters:
        start_date: '2024-01-01'
        end_date: '2024-01-31'

# ───────────────────────────────────────────────────────────────
# RESPONSE CONFIGURATION
# ───────────────────────────────────────────────────────────────
response:
  format: json
  include_metadata: true
  pagination:
    enabled: true
    max_limit: 1000

# ───────────────────────────────────────────────────────────────
# MONITORING & LOGGING
# ───────────────────────────────────────────────────────────────
monitoring:
  enabled: true
  log_queries: true
  log_slow_queries: true
  slow_query_threshold: 1000  # milliseconds

MCP Tool Description

Basic MCP Tool

# sqls/analytics.yaml
url-path: /analytics/revenue/

# MCP tool for AI agents
mcp-tool:
  name: get_revenue_analytics
  description: Get revenue analytics for specified time period and region
  
request:
  - field-name: region
    field-in: query
    description: Geographic region (US, EU, APAC)
    required: false
    validators:
      - type: string
        enum: [US, EU, APAC, LATAM]

template-source: analytics_revenue.sql
connection:
  - bigquery-warehouse

Advanced MCP Tool with Examples

# sqls/customers-mcp.yaml
url-path: /customers/search/

# ───────────────────────────────────────────────────────────────
# MCP TOOL - DETAILED CONFIGURATION
# ───────────────────────────────────────────────────────────────
mcp-tool:
  name: search_customers
  
  description: |
    Search and retrieve customer information with comprehensive filtering.
    
    This tool provides access to customer data including:
    - Basic information (name, contact details)
    - Account status and balance
    - Market segment classification
    - Transaction history summary
    
    Use this tool when you need to:
    - Look up customer details by name or ID
    - Find customers in specific market segments
    - Identify high-value customers by balance
    - Analyze customer distribution by region
  
  # Provide usage examples for AI agents
  examples:
    - description: Find all automotive segment customers
      parameters:
        segment: AUTOMOTIVE
        limit: 50
      expected_use_case: Market analysis for automotive sector
    
    - description: Find high-value customers
      parameters:
        min_balance: 10000
        limit: 20
      expected_use_case: Identify VIP customers for special offers
    
    - description: Search customer by name
      parameters:
        search: "Acme Corp"
        limit: 10
      expected_use_case: Customer lookup for support
    
    - description: Get customers in specific region
      parameters:
        region: US
        segment: MACHINERY
        limit: 100
      expected_use_case: Regional sales analysis
  
  # Additional metadata for AI agents
  response_format:
    type: array
    items:
      - customer_id: integer
      - name: string
      - segment: string
      - balance: number
      - region: string
      - contact_email: string
  
  limitations:
    - Maximum 1000 results per request
    - Results are cached (refresh every 60 minutes)
    - Historical data limited to last 2 years

request:
  - field-name: segment
    field-in: query
    description: Market segment filter
    required: false
    validators:
      - type: string
        enum: [AUTOMOTIVE, BUILDING, FURNITURE, MACHINERY, HOUSEHOLD]
  
  - field-name: min_balance
    field-in: query
    description: Minimum account balance
    required: false
    validators:
      - type: number
        min: 0
  
  - field-name: region
    field-in: query
    description: Geographic region
    required: false
  
  - field-name: search
    field-in: query
    description: Search term for customer name
    required: false
    validators:
      - type: string
        min_length: 3
        max_length: 100
  
  - field-name: limit
    field-in: query
    description: Maximum number of results
    required: false
    validators:
      - type: int
        min: 1
        max: 1000
    default: 50

template-source: customers/search.sql
connection:
  - customers-cache

File Structure Best Practices

Recommended Directory Layout

project/
├── flapi.yaml                 # Main configuration
├── config/                    # Shared configuration modules
│   ├── connections.yaml       # Data source connections
│   ├── security.yaml          # Security settings
│   └── environments/          # Environment-specific configs
│       ├── development.yaml
│       ├── staging.yaml
│       └── production.yaml
│
├── sqls/                      # Endpoint definitions & SQL templates
│   ├── customers/             # Customer-related endpoints
│   │   ├── list.yaml          # GET /customers/
│   │   ├── list.sql
│   │   ├── detail.yaml        # GET /customers/{id}/
│   │   ├── detail.sql
│   │   ├── search.yaml        # GET /customers/search/
│   │   ├── search.sql
│   │   └── cache.sql          # Shared cache template
│   │
│   ├── orders/                # Order-related endpoints
│   │   ├── list.yaml
│   │   ├── list.sql
│   │   ├── create.yaml        # POST /orders/
│   │   ├── create.sql
│   │   ├── cache.sql
│   │   └── _includes/         # Shared SQL snippets
│   │       ├── filters.sql
│   │       └── joins.sql
│   │
│   ├── analytics/             # Analytics endpoints
│   │   ├── revenue.yaml
│   │   ├── revenue.sql
│   │   ├── customers.yaml
│   │   └── customers.sql
│   │
│   └── _shared/               # Shared across all endpoints
│       ├── authentication.yaml
│       └── rate-limits.yaml
│
└── data/                      # Local data files
    ├── customers.parquet
    └── products.parquet

Naming Conventions

# ✅ Good: Clear, descriptive names
# sqls/customers/list.yaml          → GET /customers/
# sqls/customers/detail.yaml        → GET /customers/{id}/
# sqls/customers/search.yaml        → GET /customers/search/
# sqls/orders/create.yaml           → POST /orders/
# sqls/analytics/revenue.yaml       → GET /analytics/revenue/

# ❌ Bad: Unclear, inconsistent names
# sqls/cust.yaml
# sqls/endpoint1.yaml
# sqls/api_2.yaml
# sqls/test.yaml

File Organization Strategies

Strategy 1: By Resource (Recommended)

sqls/
├── customers/          # All customer endpoints together
├── orders/            # All order endpoints together
└── products/          # All product endpoints together

Strategy 2: By Function

sqls/
├── read/              # All GET endpoints
├── write/             # All POST/PUT endpoints
└── analytics/         # All analytics endpoints

Strategy 3: By Domain

sqls/
├── sales/             # Sales domain
├── inventory/         # Inventory domain
└── finance/           # Finance domain

Path Resolution

Relative Path Resolution

Paths are resolved relative to the file containing the reference:

# flapi.yaml (in project root)
connections: {{include from 'config/connections.yaml'}}
# Resolves to: ./config/connections.yaml

# config/connections.yaml
bigquery: {{include from './bigquery.yaml'}}
# Resolves to: ./config/bigquery.yaml

bigquery2: {{include from 'bigquery.yaml'}}
# Also resolves to: ./config/bigquery.yaml

Template Source Path Resolution

# sqls/customers/list.yaml
template-source: list.sql
# Resolves to: sqls/customers/list.sql (same directory)

template-source: ./list.sql
# Also resolves to: sqls/customers/list.sql

template-source: ../shared/filters.sql
# Resolves to: sqls/shared/filters.sql (parent directory)

template-source: queries/list.sql
# Resolves to: sqls/customers/queries/list.sql (subdirectory)

Cache Template Path Resolution

# sqls/orders/list.yaml
cache:
  template_file: cache.sql
  # Resolves to: sqls/orders/cache.sql

cache:
  template_file: ../shared/cache.sql
  # Resolves to: sqls/shared/cache.sql

cache:
  template_file: ./caching/orders.sql
  # Resolves to: sqls/orders/caching/orders.sql

Absolute vs Relative Paths

# Relative paths (recommended)
template-source: customers.sql          # Relative to YAML file
connections: {{include from 'config/connections.yaml'}}

# Absolute paths (use sparingly)
template-source: /app/sqls/customers.sql
connections: {{include from '/app/config/connections.yaml'}}

# Home directory expansion
template-source: ~/projects/flapi/sqls/customers.sql

Path Resolution Order

Check if absolute path - Use as-is
Check if starts with ./ - Relative to current file directory
Check if starts with ../ - Relative to parent directory
Otherwise - Relative to current file directory

YAML Best Practices

1. Use Consistent Indentation

# ✅ Good: 2 spaces throughout
request:
  - field-name: status
    field-in: query
    validators:
      - type: string
        enum: [active, inactive]

# ❌ Bad: Mixed indentation
request:
  - field-name: status
      field-in: query
    validators:
        - type: string
          enum: [active, inactive]

2. Add Comments for Documentation

# ✅ Good: Explains purpose and usage
# Customer search endpoint
# Supports filtering by segment, region, and balance
# Returns cached data (refreshed hourly)
url-path: /customers/search/

# ❌ Bad: No explanation
url-path: /customers/search/

3. Use Descriptive Field Names

# ✅ Good: Clear and descriptive
- field-name: customer_segment
  description: Market segment classification
- field-name: minimum_account_balance
  description: Minimum balance in USD

# ❌ Bad: Cryptic abbreviations
- field-name: cust_seg
- field-name: min_bal

# ✅ Good: Logical grouping with separators
# ═══════════════════════════════════════════════════════════════
# ENDPOINT CONFIGURATION
# ═══════════════════════════════════════════════════════════════
url-path: /api/

# ═══════════════════════════════════════════════════════════════
# REQUEST PARAMETERS
# ═══════════════════════════════════════════════════════════════
request:
  - field-name: id

# ❌ Bad: Everything mixed together
url-path: /api/
request:
  - field-name: id
cache:
  enabled: true
authentication:
  required: true

5. Use Includes for Reusable Configuration

# ✅ Good: Shared configuration in separate files
authentication: {{include from '../_shared/jwt-auth.yaml'}}
rate-limit: {{include from '../_shared/rate-limit-standard.yaml'}}

# _shared/jwt-auth.yaml
required: true
methods: [jwt]
roles: [user, admin]

# ❌ Bad: Duplicate configuration in every file
authentication:
  required: true
  methods: [jwt]
  roles: [user, admin]

6. Validate Enum Values

# ✅ Good: Explicit enum validation
validators:
  - type: string
    enum: [pending, completed, cancelled]

# ❌ Bad: No validation (SQL injection risk)
# (no validators)

7. Provide Examples in MCP Descriptions

# ✅ Good: Clear examples for AI agents
mcp-tool:
  name: search_orders
  description: Search orders by status and date range
  examples:
    - description: Recent completed orders
      parameters:
        status: completed
        limit: 10

# ❌ Bad: No examples
mcp-tool:
  name: search_orders
  description: Search orders

8. Use Multi-line Strings for Long Descriptions

# ✅ Good: Readable multi-line description
description: |
  Retrieve customer orders with comprehensive filtering.
  Supports filtering by status, date range, and customer ID.
  Returns paginated results with up to 1000 orders per request.

# ❌ Bad: Long single line
description: Retrieve customer orders with comprehensive filtering. Supports filtering by status, date range, and customer ID. Returns paginated results with up to 1000 orders per request.

9. Version Your Configuration

# ✅ Good: Version tracking
version: 2.1.0
description: Orders API v2 - Added pagination support
changelog:
  - 2.1.0: Added pagination parameters
  - 2.0.0: Rewrote with caching support
  - 1.0.0: Initial release

# ❌ Bad: No version information

10. Keep Secrets Out of YAML

# ✅ Good: Use environment variables
database:
  password: ${DB_PASSWORD}
  api_key: ${API_KEY}

# ❌ Bad: Hardcoded secrets
database:
  password: my-secret-password
  api_key: sk-1234567890abcdef

Common Patterns

Pattern 1: Shared Authentication

_shared/authentication.yaml:

required: true
methods: [jwt]
roles: [user, admin]

Usage:

# sqls/orders/list.yaml
authentication: {{include from '../_shared/authentication.yaml'}}

Pattern 2: Standard Rate Limiting

_shared/rate-limits.yaml:

standard:
  enabled: true
  requests: 1000
  window: 60s
  per: user

strict:
  enabled: true
  requests: 100
  window: 60s
  per: user

Usage:

# sqls/sensitive/data.yaml
rate-limit: {{include from '../_shared/rate-limits.yaml'}}[strict]

Pattern 3: Environment-Specific Configs

# flapi.yaml
environment: ${FLAPI_ENV|development}
connections: {{include from 'config/environments/${FLAPI_ENV}/connections.yaml'}}

Validation & Testing

Validate YAML Syntax

# Validate single endpoint
$ flapii endpoints validate /customers/

✓ YAML syntax valid
✓ All required fields present
✓ Template file exists
✓ Connection exists
✓ Validators configured correctly

# Validate all endpoints
$ flapii endpoints validate --all

✓ Validated 24 endpoints
✓ All configurations valid

Test Path Resolution

# Show resolved paths
$ flapii config paths sqls/customers/list.yaml

Endpoint: /customers/
YAML file: sqls/customers/list.yaml
Template: sqls/customers/list.sql (resolved)
Cache template: sqls/customers/cache.sql (resolved)
Includes: none

Export Resolved Configuration

# See final configuration with all includes resolved
$ flapii config export sqls/customers/list.yaml

# Output shows fully resolved configuration
url-path: /customers/
template-source: sqls/customers/list.sql
authentication:
  required: true
  methods: [jwt]
connection:
  - customers-cache

Troubleshooting

YAML Syntax Errors

Error: YAML parse error on line 23: unexpected ':'

Fix: Check indentation and ensure consistent spacing

Include Not Found

Error: Include file not found: config/missing.yaml

Fix:
1. Check file exists: ls -la config/missing.yaml
2. Verify path is correct relative to including file
3. Check for typos in filename

Environment Variable Not Resolved

Error: Environment variable 'DB_HOST' not found

Fix:
1. Set the variable: export DB_HOST=localhost
2. Add to whitelist in flapi.yaml
3. Check spelling matches exactly

Path Resolution Issues

Error: Template file not found: sqls/customers/list.sql

Fix:
1. Check file exists in expected location
2. Verify template-source path is correct
3. Use ./list.sql for same directory
4. Use ../shared/list.sql for parent directory

Next Steps

Endpoints Overview: Complete endpoint configuration guide
SQL Templating: Learn Mustache template syntax
Parameters: Define request parameters
Validation: Validate input parameters
MCP Overview: MCP tool integration
Configuration Service: Advanced config management
CLI Tools: Validate and test configurations

Extended YAML Features​

1. Include Syntax​

2. Environment Variables​

3. Template Variables​

Endpoint YAML Structure​

Basic Endpoint​

Complete Endpoint with All Features​

MCP Tool Description​

Basic MCP Tool​

Advanced MCP Tool with Examples​

File Structure Best Practices​

Recommended Directory Layout​

Naming Conventions​

File Organization Strategies​

Path Resolution​

Relative Path Resolution​

Template Source Path Resolution​

Cache Template Path Resolution​

Absolute vs Relative Paths​

Path Resolution Order​

YAML Best Practices​

1. Use Consistent Indentation​

2. Add Comments for Documentation​

3. Use Descriptive Field Names​

4. Group Related Configuration​

5. Use Includes for Reusable Configuration​

6. Validate Enum Values​

7. Provide Examples in MCP Descriptions​

8. Use Multi-line Strings for Long Descriptions​

9. Version Your Configuration​

10. Keep Secrets Out of YAML​

Common Patterns​

Pattern 1: Shared Authentication​

Pattern 2: Standard Rate Limiting​

Pattern 3: Environment-Specific Configs​

Validation & Testing​

Validate YAML Syntax​

Test Path Resolution​

Export Resolved Configuration​

Troubleshooting​

YAML Syntax Errors​

Include Not Found​

Environment Variable Not Resolved​

Path Resolution Issues​

Next Steps​