Skip to content

Quick Start: PostgreSQL

Five commands to go from files in S3 to materialised dbt models in PostgreSQL -- bronze, silver, and gold layers, all generated and validated automatically. No cloud warehouse account required.

Prerequisites

  • skippr on PATH (Install)
  • Python venv with dbt-core and dbt-postgres
  • Authenticated via skippr user login (or SKIPPR_API_KEY for CI)
  • PostgreSQL running locally or remotely
  • PostgreSQL and AWS credentials in your environment:
bash
export POSTGRES_HOST="localhost"
export POSTGRES_USER="myuser"
export POSTGRES_PASSWORD="mypassword"
export AWS_ACCESS_KEY_ID="AKIA..."
export AWS_SECRET_ACCESS_KEY="..."

Need help with credentials? See PostgreSQL and S3.

Build the pipeline

bash
# 1. Create the project
mkdir my-workspace && cd my-workspace
skippr init s3-pipeline

# 2. Point at your warehouse
skippr connect warehouse postgres \
  --database analytics \
  --schema raw

# 3. Point at your source
skippr connect source s3 \
  --bucket my-data-bucket \
  --prefix raw/

# 4. Verify everything is wired up
skippr doctor

# 5. Run it
skippr run

That's it. skippr run discovers your file schemas, extracts the data, loads it into PostgreSQL, and generates a complete dbt project with silver and gold models -- compiled and materialised.

What you get

dbt models (ready to extend)

models/
├── schema.yml                   # source definitions
└── staging/
    ├── stg_raw_events.sql       # silver model
    └── stg_raw_sessions.sql     # silver model

PostgreSQL schemas (populated and queryable)

SchemaContents
rawBronze -- raw extracted data
s3_pipeline_silverSilver -- staged and cleansed
s3_pipeline_goldGold -- mart-ready models

Project config

yaml
# skippr.yaml
project: s3_pipeline

warehouse:
  kind: postgres
  database: analytics
  schema: raw

source:
  kind: s3
  s3_bucket: my-data-bucket
  s3_prefix: raw/

What's next

  • Run skippr run again -- it's incremental, only new and changed rows are synced.
  • The dbt project is yours. Add tests, snapshots, or custom gold models.
  • See How It Works for the full pipeline breakdown.
  • Ready for production? Swap to Snowflake or any of the supported warehouses with a single command.