> ## Documentation Index
> Fetch the complete documentation index at: https://docs.ellomas.com/llms.txt
> Use this file to discover all available pages before exploring further.

# How It Works

> Architecture and pipeline of Seedling's data generation engine.

# How It Works

Seedling follows a four-stage pipeline: **Introspect → Plan → Generate → Write**.

## Pipeline

```mermaid theme={null}
graph LR
  A[Database Schema] --> B[Introspector]
  B --> C[schema.yaml]
  C --> D[PlanBuilder]
  D --> E[StreamGenerator]
  E --> F[Writer]
  F --> G[SQL / CSV / JSONL / DB]
```

## Stage 1: Introspect

Reads `information_schema` from a live Postgres or MySQL database and produces a structured `Schema` object containing:

* All tables and their columns with types
* Foreign key relationships
* Unique constraints, NOT NULL, CHECK, DEFAULT values
* Column comments (used as generator hints)

Output is written as `schema.yaml` (or JSON), which serves as the input for generation.

```bash theme={null}
seedling introspect --db postgres://localhost:5432/mydb --output schema.yaml
```

## Stage 2: Plan

The `PlanBuilder` performs a **topological sort** of tables by FK dependency (Kahn's algorithm). This ensures parent rows exist before child rows referencing them.

Each column is automatically assigned a generator based on its type and name:

| Column Type                      | Auto-detected Generator |
| -------------------------------- | ----------------------- |
| `serial` / `bigserial`           | Sequence                |
| `varchar(255)` with name "email" | `Email`                 |
| `varchar` with name "phone"      | `Phone`                 |
| `timestamptz`                    | `Now`                   |
| FK column                        | FK lookup from parent   |
| `varchar`                        | Random string           |

## Stage 3: Generate

The `StreamGenerator` iterates tables in dependency order:

1. For each table, generate N rows
2. Each row's columns are produced by their assigned generators
3. FK columns look up values from already-generated parent rows via `FKPool`
4. Unique constraints are tracked and enforced by `UniqueTracker`
5. Circular FK dependencies are split into multi-pass groups

## Stage 4: Write

Generated rows are streamed to a writer. Supported formats:

| Writer            | Description                             |
| ----------------- | --------------------------------------- |
| `SqlWriter`       | Batched `INSERT INTO ...` statements    |
| `CsvWriter`       | One file per table                      |
| `JsonLinesWriter` | One JSON object per row                 |
| `ParquetWriter`   | Tabular output                          |
| `DbWriter`        | Direct batched INSERT into database     |
| `CopyWriter`      | Postgres COPY protocol (max throughput) |

## Determinism

When a seed is provided (`--seed <int>`), all generators use a ChaCha8-based deterministic PRNG. Every column gets a derived sub-seed, ensuring:

* Same schema + seed + count = identical output
* Parallel generation is deterministic within a run
* No `crypto/rand` usage in deterministic mode

## Architecture Diagram

```mermaid theme={null}
graph TB
  subgraph "Seedling"
    I[Introspector]
    PB[PlanBuilder]
    SG[StreamGenerator]
    W[Writer]
    FP[FKPool]
    UT[UniqueTracker]
    PRNG[ChaCha8 PRNG]
  end

  DB[(Postgres / MySQL)] --> I
  I --> Y[schema.yaml]
  Y --> PB
  PB --> SG
  SG --> W
  SG --> FP
  SG --> UT
  PRNG --> SG
  W --> OUT[(Output)]
```