Cloud Storage

By default, Columnstore table data is stored locally in a directory as parquet files with Iceberg or Delta Metadata.

You can also configure pg_mooncake to use an Object Store for storing both data and metadata. Currently, we support S3 and R2 buckets.

S3 Buckets

Use the following query to add your S3 credentials:

SELECT mooncake.create_secret('<name>', 'S3', '<key_id>', 
          '<secret>', '{"REGION": "<s3-region>"}');

R2 Buckets

For R2 buckets, use this query to add your credentials:

SELECT mooncake.create_secret('<name>', 'S3', '<key_id>', 
          '<secret>', '{"ENDPOINT":"<ACCOUNT_ID>.r2.cloudflarestorage.com/"}');

Set Default Bucket

After adding your credentials, set the default bucket for your Postgres endpoint:

SET mooncake.default_bucket = 's3://<bucket>';

By default, all columnstore tables created from this Postgres endpoint will be written to the specified bucket.

Notes

If you are using pg_mooncake on Neon, you must bring your own S3 bucket for now. We’re working to improve this developer experience.