> ## Documentation Index
> Fetch the complete documentation index at: https://docs.erdo.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Data Lakes

> Connect your parquet data lake on GCS or S3 and query it in place — no copy, no warehouse.

If your data already lives as **parquet files in object storage** (a "data
lake"), Erdo can query it **where it is**. You don't copy anything into Erdo, and
you don't need a warehouse like BigQuery or Snowflake in the middle — Erdo reads
the parquet directly with DuckDB over the object store.

This is the right connector when you have a lake like:

```
gs://your-bucket/warehouse/
  events/year=2026/month=06/day=25/*.parquet
  orders/year=2026/month=06/day=25/*.parquet
```

Each top-level folder (`events`, `orders`, …) becomes a **table** you can query.
[Hive-partitioned](https://duckdb.org/docs/data/partitioning/hive_partitioning)
columns like `year`/`month`/`day` are real, queryable columns too.

## What you'll need

<CardGroup cols={2}>
  <Card title="GCS" icon="google">
    Connect the same way as **BigQuery** — either:

    * **Sign in with Google** (OAuth, read-only storage scope), or
    * paste a **service-account JSON** for an account with read access to the
      bucket (`roles/storage.objectViewer`).

    No HMAC interoperability keys (which many orgs disable by policy).
  </Card>

  <Card title="S3 / S3-compatible" icon="aws">
    An **access key id + secret** with read access. Works with AWS S3 and
    S3-compatible stores (Cloudflare R2, MinIO, Backblaze B2, …) — supply the
    custom **endpoint** for those.
  </Card>
</CardGroup>

The credentials only need **read** access — the connector never writes to your lake.

## Connect it

<Steps>
  <Step title="Open Data → Connectors → Data Lake">
    Choose the **Data Lake** connector.
  </Step>

  <Step title="Enter your lake details">
    * **Provider** — `gcs` or `s3`
    * **Root** — e.g. `gs://your-bucket/warehouse` or `s3://your-bucket/warehouse`
    * **Credentials** — **Sign in with Google** or paste a **service-account JSON**
      (GCS); or an **access key + secret** (S3)
    * **Region / endpoint** — for S3 / S3-compatible stores (optional)
  </Step>

  <Step title="Pick your tables">
    Erdo lists the tables it finds under the root, with their columns. Select the
    ones you want to make queryable — they become [datasets](/data#upload-a-dataset)
    just like any other connected source.
  </Step>

  <Step title="Query and build">
    Ask an agent a question, or build a [page](/pages) on top. Agents query your
    lake with SQL and reason over the results — the parquet never leaves your bucket.
  </Step>
</Steps>

## How queries run

Erdo runs each query in a sandboxed DuckDB that reads your parquet over the
network using your credentials. A few things worth knowing:

* **In place, no copy.** Data stays in your bucket; Erdo streams only what a query
  needs (parquet column + partition pruning keeps reads small).
* **Scoped to your tables.** Agent-generated SQL can only read the tables you
  selected — it can't be pointed at arbitrary paths, even within the same bucket.
* **Partition-aware.** Filter on the Hive partition columns (`year`, `month`, …)
  and Erdo only scans the matching files.

<Note>
  **Lake vs. warehouse.** Connect a **data lake** when your source of truth is
  parquet files in object storage. If your data lives in a query engine
  (PostgreSQL, Snowflake, BigQuery, ClickHouse), connect that
  [database](/data#connect-a-source) instead — Erdo will query it natively.
</Note>