> ## Documentation Index
> Fetch the complete documentation index at: https://private-7c7dfe99-mintlify-8c05c8a2.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# explain() method

> View DataStore execution plans with the explain() method

The `explain()` method shows the execution plan for a DataStore query, helping you understand what operations will be performed and what SQL will be generated.

<h2 id="basic">
  Basic Usage
</h2>

```python theme={null}
from pathlib import Path
Path("sales.csv").write_text("""\
region,product,category,amount,quantity,price,date,order_id
East,Widget,Electronics,5200,10,120,2024-01-15,1001
West,Gadget,Electronics,800,5,160,2024-02-20,1002
East,Gizmo,Home,6500,3,100,2024-03-10,1003
North,Widget,Electronics,4500,6,150,2024-06-18,1004
West,Gadget,Electronics,2000,8,250,2024-09-14,1005
""")

from chdb import datastore as pd

ds = pd.read_csv("sales.csv")

query = (ds
    .filter(ds['amount'] > 1000)
    .groupby('region')
    .agg({'amount': ['sum', 'mean']})
    .sort('sum', ascending=False)
)

# View execution plan
query.explain()
```

<h2 id="syntax">
  Syntax
</h2>

```python theme={null}
explain(verbose=False) -> None
```

**Parameters:**

| Parameter | Type | Default | Description              |
| --------- | ---- | ------- | ------------------------ |
| `verbose` | bool | `False` | Show additional metadata |

<h2 id="output-format">
  Output Format
</h2>

<h3 id="standard">
  Standard Output
</h3>

```text theme={null}
================================================================================
Execution Plan (in execution order)
================================================================================

 [1] 📊 Data Source: file('sales.csv', 'csv')

Operations:
────────────────────────────────────────────────────────────────────────────────
    ️  Segment 1 [chDB] (from source): Operations 2-5
    ️  Note: SQL operations after Pandas ops use Python() table function

 [2] 🚀 [chDB] WHERE: "amount" > 1000
 [3] 🚀 [chDB] GROUP BY: region
 [4] 🚀 [chDB] AGGREGATE: sum(amount), avg(amount)
 [5] 🚀 [chDB] ORDER BY: sum DESC

────────────────────────────────────────────────────────────────────────────────
Final State: 📊 Pending (lazy, not yet executed)
             └─> Will execute when print(), .to_df(), .execute() is called

────────────────────────────────────────────────────────────────────────────────
Generated SQL Query:
────────────────────────────────────────────────────────────────────────────────

SELECT region, SUM(amount) AS sum, AVG(amount) AS mean
FROM file('sales.csv', 'csv')
WHERE "amount" > 1000
GROUP BY region
ORDER BY sum DESC

================================================================================
```

<h3 id="icons">
  Icons Legend
</h3>

| Icon | Meaning              |
| ---- | -------------------- |
| 📊   | Data source          |
| 🚀   | chDB (SQL) operation |
| 🐼   | pandas operation     |

<h3 id="verbose">
  Verbose Output
</h3>

```python theme={null}
query.explain(verbose=True)
```

Verbose mode shows additional details for each operation, including the full SQL query with internal row-order tracking mechanisms.

***

<h2 id="phases">
  Three Execution Phases
</h2>

The explain output shows operations in three phases:

<h3 id="phase-1">
  Phase 1: SQL Query Building (Lazy)
</h3>

Operations that compile to SQL:

```text theme={null}
  1. Source: file('sales.csv', 'CSVWithNames')
  2. Filter: amount > 1000      
  3. GroupBy: region
  4. Aggregate: sum(amount)
```

<h3 id="phase-2">
  Phase 2: Execution Point
</h3>

When a trigger occurs:

```text theme={null}
  5. Execute SQL -> DataFrame
     Trigger: to_df() called
```

<h3 id="phase-3">
  Phase 3: DataFrame Operations
</h3>

Operations after execution:

```text theme={null}
  6. [pandas] pivot_table(...)
  7. [pandas] apply(custom_func)
```

***

<h2 id="understanding">
  Understanding the Plan
</h2>

<h3 id="source">
  Source Information
</h3>

```text theme={null}
Source: file('sales.csv', 'CSVWithNames')
```

* `file()` - ClickHouse file() table function
* `'CSVWithNames'` - File format with header

Other source types:

```text theme={null}
Source: s3('bucket/data.parquet', ...)
Source: mysql('host', 'db', 'table', ...)
Source: __dataframe__  (pandas DataFrame input)
```

<h3 id="filter">
  Filter Operations
</h3>

```text theme={null}
Filter: amount > 1000 AND status = 'active'
```

Shows the WHERE clause that will be applied.

<h3 id="groupby">
  GroupBy and Aggregate
</h3>

```text theme={null}
GroupBy: region, category
Aggregate: sum(amount), avg(amount), count(id)
```

Shows GROUP BY columns and aggregation functions.

<h3 id="sort">
  Sort Operations
</h3>

```text theme={null}
Sort: sum DESC, region ASC
```

Shows ORDER BY clause.

<h3 id="limit">
  Limit Operations
</h3>

```text theme={null}
Limit: 10
Offset: 100
```

Shows LIMIT and OFFSET.

***

<h2 id="engine">
  Engine Information
</h2>

When using verbose mode, you can see which engine will be used:

```text theme={null}
Filter: amount > 1000
  - Engine: chdb
  - Pushdown: Yes

Apply: custom_function
  - Engine: pandas
  - Pushdown: No
```

<h3 id="pushdown">
  Pushdown
</h3>

* **Yes**: Operation will be executed at the data source (SQL)
* **No**: Operation requires pandas execution

***

<h2 id="examples">
  Examples
</h2>

<h3 id="example-simple">
  Simple Query
</h3>

```python theme={null}
from pathlib import Path
Path("data.csv").write_text("""\
name,age,city,salary,department
Alice,25,NYC,55000,Engineering
Bob,30,LA,65000,Product
Charlie,35,NYC,80000,Engineering
Diana,28,SF,70000,Design
Eve,42,NYC,95000,Product
""")

ds = pd.read_csv("data.csv")
ds.filter(ds['age'] > 25).explain()
```

```text theme={null}
================================================================================
Execution Plan (in execution order)
================================================================================

 [1] 📊 Data Source: file('data.csv', 'csv')

Operations:
────────────────────────────────────────────────────────────────────────────────
    ️  Segment 1 [chDB] (from source): Operations 2-2

 [2] 🚀 [chDB] WHERE: "age" > 25

────────────────────────────────────────────────────────────────────────────────
Generated SQL Query:
────────────────────────────────────────────────────────────────────────────────

SELECT * FROM file('data.csv', 'csv') WHERE "age" > 25

================================================================================
```

<h3 id="example-complex">
  Complex Aggregation
</h3>

```python theme={null}
query = (ds
    .filter(ds['date'] >= '2024-01-01')
    .filter(ds['amount'] > 100)
    .select('region', 'category', 'amount')
    .groupby('region', 'category')
    .agg({
        'amount': ['sum', 'mean', 'count']
    })
    .sort('sum', ascending=False)
    .limit(20)
)
query.explain()
```

```text theme={null}
================================================================================
Execution Plan (in execution order)
================================================================================

 [1] 📊 Data Source: file('sales.csv', 'csv')

Operations:
────────────────────────────────────────────────────────────────────────────────
    ️  Segment 1 [chDB] (from source): Operations 2-8

 [2] 🚀 [chDB] WHERE: "date" >= '2024-01-01'
 [3] 🚀 [chDB] WHERE: "amount" > 100
 [4] 🚀 [chDB] SELECT: region, category, amount
 [5] 🚀 [chDB] GROUP BY: region, category
 [6] 🚀 [chDB] AGGREGATE: sum(amount), avg(amount), count(amount)
 [7] 🚀 [chDB] ORDER BY: sum DESC
 [8] 🚀 [chDB] LIMIT: 20

────────────────────────────────────────────────────────────────────────────────
Generated SQL Query:
────────────────────────────────────────────────────────────────────────────────

SELECT region, category, 
       SUM(amount) AS sum, 
       AVG(amount) AS mean, 
       COUNT(amount) AS count
FROM file('sales.csv', 'csv')
WHERE "date" >= '2024-01-01' AND "amount" > 100
GROUP BY region, category
ORDER BY sum DESC
LIMIT 20

================================================================================
```

<h3 id="example-mixed">
  Mixed SQL and pandas
</h3>

When operations cannot be fully pushed to SQL, the plan shows multiple segments:

```python theme={null}
query = (ds
    .filter(ds['age'] > 25)           # SQL
    .groupby('city')                   # SQL
    .agg({'salary': 'mean'})           # SQL
    .apply(lambda x: x * 1.1)          # pandas (triggers segment split)
    .filter(ds['mean'] > 50000)        # SQL (new segment)
)
query.explain()
```

```text theme={null}
================================================================================
Execution Plan (in execution order)
================================================================================

 [1] 📊 Data Source: file('data.csv', 'csv')

Operations:
────────────────────────────────────────────────────────────────────────────────
    ️  Segment 1 [chDB] (from source): Operations 2-4
    ️  Segment 2 [Pandas] (on DataFrame): Operation 5
    ️  Segment 3 [chDB] (on DataFrame): Operation 6
    ️  Note: SQL operations after Pandas ops use Python() table function

 [2] 🚀 [chDB] WHERE: "age" > 25
 [3] 🚀 [chDB] GROUP BY: city
 [4] 🚀 [chDB] AGGREGATE: avg(salary)
 [5] 🐼 [Pandas] APPLY: lambda
 [6] 🚀 [chDB] WHERE: "mean" > 50000

================================================================================
```

***

<h2 id="debugging">
  Debugging with explain()
</h2>

<h3 id="debug-filter">
  Check Filter Logic
</h3>

```python theme={null}
# Verify your filter is correct
query = ds.filter((ds['age'] > 25) & (ds['city'] == 'NYC'))
query.explain()
# Output shows: Filter: age > 25 AND city = 'NYC'
```

<h3 id="debug-select">
  Verify Column Selection
</h3>

```python theme={null}
# Check column pruning
query = ds.select('name', 'age').filter(ds['age'] > 25)
query.explain()
# Output shows: SELECT name, age FROM ... WHERE age > 25
```

<h3 id="debug-agg">
  Understand Aggregation
</h3>

```python theme={null}
# Check aggregation functions
query = ds.groupby('dept').agg({'salary': ['sum', 'mean', 'std']})
query.explain()
# Output shows: SELECT dept, SUM(salary), AVG(salary), stddevPop(salary)
```

***

<h2 id="best-practices">
  Best Practices
</h2>

<h3 id="best-practice-1">
  1. Check Before Executing Large Queries
</h3>

```python theme={null}
# Always explain first for large data
query = ds.complex_pipeline()
query.explain()  # Check plan

# If plan looks correct
result = query.to_df()  # Execute
```

<h3 id="best-practice-2">
  2. Use Verbose for Debugging
</h3>

```python theme={null}
# When something seems wrong
query.explain(verbose=True)
# Shows engine selection and pushdown info
```

<h3 id="best-practice-3">
  3. Compare with to\_sql()
</h3>

```python theme={null}
# explain() shows the plan
query.explain()

# to_sql() shows just the SQL
print(query.to_sql())

# Both useful for different purposes
```

<h3 id="best-practice-4">
  4. Check Pushdown Status
</h3>

```python theme={null}
# Verbose mode shows if operations are pushed down
query.explain(verbose=True)

# If Pushdown: No, operation runs in pandas
# Consider restructuring query for better performance
```