Skip to content

SQL / data

DuckDB integration — SQL nodes over Polars frames. Requires the sql extra (pip install "golit[sql]"); DuckDB is imported lazily, never at framework import time. See the SQL nodes tutorial.

DuckDB integration — SQL nodes over Polars frames.

DuckDB is in-process and exchanges data with Polars zero-copy, so a node can run SQL over its upstream frames and hand a Polars frame downstream — memoized and rendered like any other frame. Use :func:sql for an eager SQL node:

from golit import sql

@app.reactive
def by_region(data):                      # `data` is an upstream Polars frame
    return sql(
        "SELECT region, sum(revenue) AS revenue "
        "FROM data GROUP BY region ORDER BY region",
        data=data,
    )

Raw duckdb.sql(...) relations returned from a node also work: Golit detects a DuckDBPyRelation and materializes it to Polars for rendering and hashing. DuckDB is an optional dependency (pip install "golit[sql]"); it is imported only inside :func:sql, never at framework import time.

load_extension 

load_extension(name: str) -> None

Install and load a DuckDB extension on this thread's connection, once.

DuckDB downloads an extension on first INSTALL (e.g. spatial for ST_* functions); we track what's already loaded per connection so repeated calls on the hot path are a no-op. Used by :func:golit.gis.spatial_sql.

Source code in python/golit/data.py 
48
49
50
51
52
53
54
55
56
57
58
59
60
def load_extension(name: str) -> None:
    """Install and load a DuckDB extension on this thread's connection, once.

    DuckDB downloads an extension on first ``INSTALL`` (e.g. ``spatial`` for ``ST_*``
    functions); we track what's already loaded per connection so repeated calls on the
    hot path are a no-op. Used by :func:`golit.gis.spatial_sql`."""
    con = _connection()
    loaded = _local.loaded
    if name in loaded:
        return
    con.install_extension(name)
    con.load_extension(name)
    loaded.add(name)

is_duckdb_relation 

is_duckdb_relation(value: Any) -> bool

Whether value is a DuckDBPyRelation (without importing duckdb).

The class lives in the _duckdb pybind11 module on current builds (and duckdb on older ones), so match either by substring.

Source code in python/golit/data.py 
63
64
65
66
67
68
69
def is_duckdb_relation(value: Any) -> bool:
    """Whether ``value`` is a ``DuckDBPyRelation`` (without importing duckdb).

    The class lives in the ``_duckdb`` pybind11 module on current builds (and
    ``duckdb`` on older ones), so match either by substring."""
    cls = type(value)
    return cls.__name__ == "DuckDBPyRelation" and "duckdb" in (cls.__module__ or "")

relation_to_polars 

relation_to_polars(relation: Any) -> DataFrame

Materialize a DuckDB relation to a Polars DataFrame.

Source code in python/golit/data.py 
72
73
74
75
76
77
def relation_to_polars(relation: Any) -> pl.DataFrame:
    """Materialize a DuckDB relation to a Polars ``DataFrame``."""
    to_pl = getattr(relation, "pl", None)
    if callable(to_pl):
        return to_pl()
    return pl.from_arrow(relation.arrow())  # type: ignore[return-value]

sql 

sql(query: str, **frames: Any) -> DataFrame

Run a DuckDB SQL query over the given named frames, returning Polars.

Each keyword binds a name usable in the query (sql("… FROM data", data=df)); values may be Polars/pandas/Arrow frames. The result is fully materialized, so it memoizes and renders exactly like a Polars node output.

Source code in python/golit/data.py 
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
def sql(query: str, **frames: Any) -> pl.DataFrame:
    """Run a DuckDB SQL ``query`` over the given named frames, returning Polars.

    Each keyword binds a name usable in the query (``sql("… FROM data", data=df)``);
    values may be Polars/pandas/Arrow frames. The result is fully materialized, so
    it memoizes and renders exactly like a Polars node output."""
    con = _connection()  # reused per thread
    try:
        for name, frame in frames.items():
            con.register(name, frame)
        return con.sql(query).pl()  # fully materialized before the views are dropped
    finally:
        # Unbind this call's frames so they don't leak or collide with the next
        # query on the same (reused) connection.
        for name in frames:
            con.unregister(name)