An open framework for physical climate risk.

This is the technical reference: how to install, what the numbers mean, and how the workflows run. Climate Risk Commons turns hazard curves, asset locations and damage functions into comparable, auditable risk numbers, on a laptop, over open data. For why the project exists, start on the project site. If you would rather not write Python, use the without-code guide.

uv add crc-sdk                         # or: python -m pip install crc-sdk
git clone https://github.com/RiskThinking/crc-docs   # worked examples

Installing crc-sdk pulls in crc-framework, the Rust computation core.

What you can install today

The standard is no longer only a specification. Three pieces are published, versioned and runnable.

The shape of an analysis

Every analysis in the framework is the same three inputs meeting in one place. Learning where each one lives is most of the learning curve.

The output is one row per asset, hazard, horizon and pathway, with a value column for each return period you asked for, and metadata recording exactly how each number was produced.

Names you will meet

Climate Risk Commons

The whole of it: the specification, the reference implementation, the community that maintains them, and RiskThinkingFoundation, the neutral non-profit that stewards them. Shortened to CRC, which is where every package name comes from.

The CRC specification

The standard itself. Input contracts for climate hazard data, physical assets and impact functions, plus the canonical hazard schema, the output model and the conformance rules.

Reference pipeline

An end-to-end implementation of the spec. The runnable scripts live in crc-docs/pipelines/ and are built on crc-sdk.

crc_framework

The import name of the computation core. Distributions and metrics live here.

crc_sdk

The import name of the SDK. Data access, ingest and workflows live here, and it re-exports the stable framework API through crc_sdk.core, .fitting and .impacts.

Climate Digital Twin (VELO/CDT)

RiskThinking.AI's commercial high-fidelity data and enterprise platform. Separate from these packages, and reached through a commercial agreement.

Installation

Baseline install is one command. The extras exist because format adapters are heavy and most callers need none of them.

Pick an entry point

There are two reasonable ways in, and they answer different questions.

Run the worked examples

Best if you want to see a complete analysis before writing any code. Clone crc-docs and let uv resolve everything, including the notebook stack.

git clone https://github.com/RiskThinking/crc-docs
cd crc-docs
uv sync

# notebooks expect notebooks/ as the working directory,
# so their data/ cache lands next to them
uv run jupyter lab notebooks/

# pipelines run from the repo root
uv run python pipelines/asset_portfolio_pipeline.py

Add it to your own project

Best if you already know which surface you need. Install crc-sdk and add extras as the import errors tell you to.

uv add crc-sdk
# or, with the adapters for OS-Climate Zarr and H3 geometry
uv add "crc-sdk[zarr,geometry]"

# framework only: distributions, fitting and risk metrics, no data access layer
python -m pip install crc-framework

What comes in the box

DuckDB, Arrow, psutil and the remote-storage transports (fsspec, s3fs, gcsfs) are baseline dependencies, not extras. Every connector and workflow in the SDK is built on that stack, so gating it would just move the same install onto every real caller.

Everything else is a specific format adapter or a pure-geometry dependency. Importing crc_sdk or any subpackage never needs more than the baseline.

One dependency pip cannot give you

PMTiles generation shells out to tippecanoe. Both tippecanoe and tile-join must be on PATH; they are expected to be part of your runtime image rather than a Python install, so there is no extra for them.

from crc_sdk.geometry.pmtiles import require_tippecanoe, require_tile_join

require_tippecanoe()   # raises with install instructions if missing
require_tile_join()

Platforms and Python versions

We recommend Python 3.12 or later for new environments. Choose a version supported by your selected packages and extras. For current requirements and prebuilt wheels, check crc-sdk on PyPI and crc-framework on PyPI. Wheel availability depends on your Python version, operating system and architecture.

A compatible prebuilt wheel needs no Rust toolchain. For source builds and extra-specific dependencies, follow the SDK and framework instructions on GitHub.

Verify the install

python -c "import crc_framework; print(crc_framework.__doc__)"

Then check that the SDK's engine layer imports and that the framework API is reachable through it:

import crc_sdk
from crc_sdk.workflows import HazardDataset
from crc_sdk.core import TabulatedDistribution   # re-exported from crc_framework

curve = TabulatedDistribution.from_return_periods(
    [10, 50, 100], [0.4, 1.1, 1.6], tail="upper",
)
print(curve.ppf(0.99))   # the 100-year value

DuckDB thread, memory and spill-directory limits are detected automatically and can be overridden with environment variables; see Resources and tuning.

Quickstart

Flood depths for a small portfolio, offline, in about ten minutes. No cloud credentials, no account, no hazard data to download.

1. Get the repository and the environment

git clone https://github.com/RiskThinking/crc-docs
cd crc-docs
uv sync

uv sync reads the lockfile, so you get the exact dependency set the notebooks were executed against.

2. Run a whole analysis before reading any of it

uv run python pipelines/asset_portfolio_pipeline.py

Outputs land in pipeline_output/. Every notebook in the ordered track has a headless twin like this one, which makes the examples usable as regression checks and as a starting point for your own scripts.

3. Open the notebook that matches it

uv run jupyter lab notebooks/

Start with asset_portfolio_evaluation.ipynb. Launch from the repo root as shown so the notebook's data/ cache lands beside it rather than in your working directory.

4. Point the same workflow at your own assets

This is the whole portfolio surface. An Arrow table in, a Parquet file out, one value column per return period.

import pyarrow as pa
from crc_sdk.workflows import HazardDataset

assets = pa.table({
    "asset_id":  ["warehouse-a", "warehouse-b"],
    "longitude": [6.9603, 7.5010],
    "latitude":  [50.9375, 51.0030],
    "sector":    ["logistics", "manufacturing"],
})

result = (
    HazardDataset.local("flood.parquet")
    .for_assets(assets)
    .select(horizons=[2050], pathways=["ssp585"])
    .return_periods([25, 50, 100, 250, 500, 1000])
    .write_parquet("portfolio-flood.parquet")
)

You get value_rp25 through value_rp1000. The column names asset_id, longitude, latitude and cell_index are inferred, so a conventional schema needs no configuration at all.

5. Turn depth into damage

An impact function replaces the sampled hazard values with event-aligned impact values before writing.

from crc_sdk.impacts import PiecewiseLinearImpact

damage_curve = PiecewiseLinearImpact(
    exposure=[0.0, 0.2, 1.0, 2.0],
    impact=[0.0, 0.0, 0.25, 1.0],
)

impacted = (
    HazardDataset.local("flood.parquet")
    .for_assets(assets)
    .return_periods([25, 100, 250])
    .impact(
        damage_curve,
        name="flood_damage_ratio",
        value_unit="fraction",
        value_semantics="damage ratio",
    )
    .write_parquet("portfolio-impact.parquet")
)

Now value_rp100 is the damage ratio at the 100-year flood, not the depth. The return period still identifies the source hazard event.

Learning path

Notebooks in two ordered and unordered tracks, sharing the same packages. Most have a headless pipeline twin that does the same work without Jupyter. Every notebook also opens in Colab with no install.

Asset portfolios

Ordered, offline, and the right place to start. It reads the checked-in Cologne fixtures under fixtures/os_climate/, so it runs without AWS access or any other credentials.

Spatial and thematic

Unordered, and they fetch public inputs over the network. Best read once you know what a canonical hazard dataset is.

Running them

The two working directories differ, and it matters.

# notebooks: run with notebooks/ as the working directory
uv run jupyter lab notebooks/

# pipelines: run from the repo root
uv run python pipelines/flood_admin_pipeline.py

notebooks/data/

Cached notebook inputs and outputs. Gitignored.

pipeline_output/

Everything the headless pipelines write. Gitignored.

fixtures/os_climate/

Checked-in Cologne Parquet fixtures. This is what makes the portfolio track offline.

Probability and distributions

Everything the framework computes passes through a distribution interface. Two conventions do most of the work, and getting them straight prevents an entire class of quiet error.

Non-exceedance probability is the currency

Every metric API takes a non-exceedance probability q in [0, 1]. Return periods are accepted in exactly one place: when constructing a tabulated distribution.

For an upper-tail hazard, a return period T converts as q = 1 − 1/T. So the 100-year return level is queried at q=0.99, and the 500-year at q=0.998.

One interface, five shapes

All distributions expose pdf, cdf, ppf, quantiles and sample. Evaluation methods accept both scalars and NumPy arrays. Choose the shape by the form of the data you actually have, not by what you wish you had.

Tabulated curves refuse to guess

TabulatedDistribution does not extrapolate. Querying outside its declared probability range raises, unless you opt in explicitly.

from crc_framework import TabulatedDistribution

curve = TabulatedDistribution.from_return_periods(
    [5, 10, 25, 50, 100],
    [0.2, 0.6, 1.1, 1.5, 1.9],
    tail="upper",
)

curve.ppf(0.99)    # fine: the 100-year knot is the edge of support
curve.ppf(0.999)   # raises: beyond the declared range

loose = TabulatedDistribution.from_return_periods(
    [5, 10, 25, 50, 100],
    [0.2, 0.6, 1.1, 1.5, 1.9],
    tail="upper", extrapolate=True,
)

This is a deliberate design choice, and it is the behaviour you want. A silent extrapolation from a 100-year source map to a 1000-year answer is exactly the kind of number nobody can defend later.

Interpolation, extrapolation, and what your source supports

Portfolio evaluation warns when a requested return period falls outside the source support recorded in the dataset metadata. With EFAS source rasters covering RP10 to RP500, asking for RP250 is interpolation, and asking for RP1000 is extrapolation. Both return numbers. Only one of them is defensible without a footnote.

Fitting curves

Fitting is always explicit. Metrics sample the distribution you hand them and never quietly substitute a fitted curve, which is what makes the modelling choice auditable.

Three entry points

Supported families

Pass family="auto", the default, to select among them. Pass a name to fit one family. Use fit_all to inspect every candidate, and quality_metrics to compute diagnostics for a distribution you have already chosen.

import numpy as np
from crc_framework import EmpiricalDistribution, FitConstraints, fit_distribution

observations = np.array([0.1, 0.2, 0.4, 0.7, 1.1, 1.8])
empirical = EmpiricalDistribution(observations)

fit = fit_distribution(
    empirical,
    family="auto",
    constraints=FitConstraints(probability=0.99, maximum_value=3.0),
)

print(fit.distribution.family, fit.diagnostics.ks_pvalue)
print(fit.distribution.ppf(0.5))

FitConstraints is an acceptance constraint, not a hint. It lets you reject a fit that puts an implausible value at a probability you care about.

Fitting published return-period knots

This is the common case when you are canonicalising a public hazard map. fit_quantiles minimises weighted differences in value space between your knots and the family's PPF.

from crc_framework import TabulatedDistribution, fit_quantiles

curve = TabulatedDistribution.from_return_periods(
    [5, 10, 25, 50, 100],
    [0.2, 0.6, 1.1, 1.5, 1.9],
    tail="upper",
)
fit = fit_quantiles(curve, family="gumbel_r")

Zero-heavy hazards need a stated point mass

A flood-depth map over mostly dry land is mostly zeros. The framework will not infer an exact point mass from sparse zero-valued knots, because the inference is not sound. Supply the probability yourself.

from crc_framework import fit_hurdle_quantiles

fit = fit_hurdle_quantiles(
    curve,
    family="gumbel_r",
    atom_probability=0.5,   # half the support sits on the atom
    atom_location=0.0,      # and the atom is at zero depth
)

Notebook 1 in the ordered track, hurdle_fit_primer, reconstructs one of these from a canonical row and walks its diagnostics. It is worth an hour.

Lossless against lossy

TabulatedDistribution is the lossless interpolation of the knots you supplied. Parametric and hurdle fits are lossy. If your system has to reconstruct its source exactly, persist the original probability and value pairs separately.

Impact functions

An impact function maps exposure to damage. It has two deliberately distinct operations, and choosing the wrong one is the most common subtle error in the whole framework.

Two operations, two meanings

Built-in transforms

LinearImpact, SigmoidImpact and PiecewiseLinearImpact support both operations. CallableImpact adapts a vectorised NumPy callable for point evaluation; CallableTransform also supports distribution transformation.

import numpy as np
from crc_framework import CallableImpact, LinearImpact

impact_function = LinearImpact(slope=0.25, maximum=1.0)

# event-aligned: two events in, two impacts out, same order
event_impacts = impact_function.evaluate(np.array([0.5, 2.0]))

# distributional: an exposure distribution in, an impact distribution out
impact = impact_function(exposure)

custom = CallableImpact(lambda values: np.clip(values / 2.0, 0.0, 1.0))

The registry, keyed by risk factor and context

impacts.for_factor(...) selects a registry-backed climate transform. Supply a TransformContext with the geography, building type and any historical values it needs; pass overrides when your application must replace transform parameters.

from crc_framework import RiskFactor, TransformContext, impacts

impact = impacts.for_factor(
    RiskFactor.CFLOOD,
    context=TransformContext(
        continent="Europe",
        building_type="Commercial buildings",
    ),
)(exposure)

Wiring context from asset columns

In a portfolio run, the context usually varies per asset. ImpactContextColumns maps asset columns onto context fields.

from crc_sdk.impacts import impacts
from crc_sdk.workflows import ImpactContextColumns

registry_request = request.impact(
    impacts.for_factor("inundation"),
    context=ImpactContextColumns(
        country="country",
        continent="continent",
        building_type="building_type",
        historic_mean="historic_mean",
    ),
    name="inundation_impact",
    value_unit="fraction",
    value_semantics="damage ratio",
)

• The generated H3 cell_index is always supplied to the framework impact context.
• Configured context columns are read from every asset, including when they are not kept as output passthrough columns.
• Stored registry context provides fallback values for fields with no asset value.
• Impact metadata records the event-aligned interpretation, source units and semantics, output units and semantics, function name and type, and the context-column mapping.

Lambdas cannot be parallelised

Framework impact objects and top-level Python callables run in the existing process pool. Lambdas and closures are not picklable, so they run serially when the worker count is implicit. Asking for more than one worker with a lambda raises rather than silently degrading.

from crc_sdk.workflows import ExecutionOptions

# a lambda plus explicit parallelism is an error
.impact(lambda depth: np.clip(depth / 2.0, 0.0, 1.0), name="ratio")
.write_parquet("out.parquet", execution=ExecutionOptions(max_workers=1))

# promote it to a module-level function to use the pool
def depth_to_ratio(values):
    return np.clip(values / 2.0, 0.0, 1.0)

Microscores, VaR and CVaR

The path from a hazard distribution to a portfolio risk number runs through binary outcomes. It is a small API with one sharp edge: branch count grows as two to the power of the number of factors.

Microscores turn quantiles into outcomes

A microscore samples exposure and impact at the probabilities you name, and reduces them to a binary risk outcome.

from crc_framework import (
    RiskFactor, ScenarioMetadata, TabulatedDistribution,
    TransformContext, generate_microscores, impacts,
)

exposure = TabulatedDistribution.from_return_periods(
    periods=[10, 20, 50, 100, 200, 500],
    values=[0.1, 0.2, 0.5, 0.9, 1.4, 2.0],
    tail="upper",
)

impact = impacts.for_factor(
    RiskFactor.CFLOOD,
    context=TransformContext(
        continent="Europe",
        building_type="Commercial buildings",
    ),
)(exposure)

suite = generate_microscores(
    exposure,
    impact=impact,
    probabilities=[0.95, 0.99],
    metadata=ScenarioMetadata(factor=RiskFactor.CFLOOD.value),
)

suite.scores

The sampled exposure and impact values behind every score.

suite.at(q)

The binary outcome at a requested non-exceedance probability.

suite.exposure_statistics

Summary diagnostics for the exposure side.

suite.impact_statistics

Summary diagnostics for the impact side.

Aggregating factors into VaR and CVaR

Each microscore becomes a BinaryOutcome. Combining outcomes produces the full set of independent downside and upside branches, and risk is computed at one or more confidence levels.

from crc_framework import BinaryOutcome, compute_risk, compute_spanning_set

outcomes = [
    BinaryOutcome("flood", downside_probability=0.05, downside_impact=0.4),
    BinaryOutcome("fire",  downside_probability=0.10, downside_impact=0.2),
]

branches = compute_spanning_set(outcomes)
risk = compute_risk(outcomes, levels=[0.80, 0.95])
level_95 = risk.at(0.95)

print(len(branches))                  # 4: two outcomes per factor
print(level_95.var, level_95.cvar)

Cap the branches before they cap you

Branch generation grows as 2**n for n factors. Ten factors is a thousand branches; twenty is a million. Set max_branches on compute_spanning_set or compute_risk to impose a limit your application can actually defend.

RiskLevel.attribution reports each factor's contribution to VaR and CVaR, which is usually the number a committee actually wants to see.

Spatial lookups

Spatial helpers take an H3 cell as an integer or a string, normalise it to H3 resolution 4, and return None when no reference mapping exists. That None is meaningful: handle it rather than asserting.

from crc_framework import lookup_geography, lookup_ipcc_region

cell = 600550049193132031
geography = lookup_geography(cell)
region = lookup_ipcc_region(cell)

if geography is not None:
    print(geography.continent, geography.countries)

lookup_continent returns a continent name, lookup_ipcc_region returns the IPCC region, and lookup_geography returns a Geography with the continent and the intersecting ISO3 country codes.

Canonical hazard datasets

One versioned Arrow and Parquet contract holds every fitted hazard. Understanding this file is understanding the standard: it is the thing two teams can compare.

One self-describing file

Each dataset is a single Parquet file, expanded by H3 cell so it can be joined spatially. You choose the full destination path and filename. Writes go through DuckDB, so supplying a configured connection lets the same API use its filesystems, extensions, secrets and settings.

The row key

The logical row key is (hazard_name, horizon, pathway, cell_index, source_id). Canonical files are sorted by that key by default, which enables predicate pruning and merge joins.

Curve columns are a tagged union

Rows carry the parameters needed to reconstruct a framework distribution. curve_kind tells you which shape you are holding, and which columns are populated.

This layout avoids a sprawl of redundant status, fit-stage, interpolation and reason columns. Dataset metadata and run manifests carry ordered-family and aggregate treatment provenance once, rather than on every row.

Metadata carries the meaning

Dataset-wide facts are stored once as a complete JSON payload under the Parquet key crc.hazard.metadata. Without it the numbers are ambiguous, so read it before you trust a file someone hands you.

Evaluation writes its own record under crc.hazard.evaluation: value unit, value semantics, and the complete mapping from return period to probability to column name.

Cells are a conservative superset

Boundary candidate generation uses H3 overlap coverage, not centre polyfill. The integer join therefore returns a conservative superset, which is then refined exactly.

• Where source WKB is present, the join is refined with ST_Covers(source_geometry, asset_point).
• Rows without WKB keep cell-level precision.
• source_id and spatial_match, which is either exact_geometry or h3_cell, stay in the output so you can see which happened.

Resolution estimates report measured coverage error and expanded row count; ingest policy selects and records the dataset resolution.

Writing a canonical stream

write_hazard_stream has two modes, and the choice is a memory-against-storage trade.

Canonical schema, metadata, uniqueness and every downstream curve and percentile API are identical either way. Only the physical row order differs.

Measured on one machine

These figures measure the persistence pass only, on one machine. Treat them as evidence of the trade-off, not as throughput guarantees.

Storage reduction from ordering was consistent across both samples. The small timing difference changed direction between them, so do not plan around it. Whichever mode you choose, reserve process headroom beyond the engine limit for Arrow and Python buffers and the final Parquet write.

Ingesting public hazards

Acquisition is an immutable, lazy workflow. Builder calls do no network work at all, so you can print a plan and read it before anything is fetched or fitted.

Plan first, then materialise

Builder methods and explain() never resolve a release or fetch a raster. materialize() resolves latest once, records the pinned version in the cache manifest and in canonical provenance, caches the AOI crops, fits the curves, and hands back an ordinary HazardDataset.

from crc_sdk.workflows import HazardDataset, JRCFloodPolicy

plan = (
    HazardDataset.efas(version="latest")
    .for_area((7.75, 49.75, 8.45, 50.25))
    .cache("cache/efas", mode="reuse")
    .source_periods("all")
    .canonicalize(policy=JRCFloodPolicy.curated(h3_resolution=10))
)

print(plan.explain())          # no network access yet
hazard = plan.materialize("hazards/efas-rhine.parquet")

mode="reuse"

Use the cache if it is there, fetch what is missing.

mode="offline"

Fail rather than touch the network. Pair it with prefetch().

mode="refresh"

Re-resolve the release and replace cached crops.

mode="stream"

Read remotely with no persistent source cache.

Cache manifests pin the resolved version, years, bounds, source URLs, local objects and checksums. Run prefetch(), then switch the same plan to mode="offline", and the run is reproducible without network access.

Source periods are not evaluation periods

This distinction catches people. Source periods choose which rasters are used for fitting, and default to every period in the resolved release. Evaluation periods are what you ask the fitted curve for afterwards.

result = (
    plan.for_assets(assets)
    .select(hazard_names=["RiverineInundation"], pathways=["historical"])
    .return_periods([50, 100, 250, 500])
    .write_parquet("outputs/flood-depth.parquet")
)

The compact chain lazily materialises a deterministic canonical file inside the configured cache, then delegates to the same local portfolio evaluator that HazardDataset.local(...) uses. Requests beyond the source return-period support produce a warning; see interpolation and extrapolation.

JRC flood: GLOFAS and EFAS

The two releases are laid out differently, and the dataset descriptions encode that difference so an AOI workflow never exposes tiles or filenames to you.

JRC and EDO drought

EDO Soil Moisture Index data uses the same lazy workflow, with complete years reduced to compact annual-minimum AOI cache objects before fitting.

from crc_sdk.workflows import EDODroughtPolicy, HazardDataset

plan = (
    HazardDataset.smi(version="latest")
    .for_area((9.5, 50.5, 10.5, 51.5))
    .years("all_complete")
    .cache("cache/edo-smi", mode="reuse")
    .canonicalize(policy=EDODroughtPolicy.curated(h3_resolution=6))
)

hazard = plan.materialize("hazards/edo-smi.parquet")

The curated policy uses the lower return-period tail and requires at least twenty complete years. Metadata stores both that tail and the Gringorten support of the selected annual record, so evaluation picks the correct lower tail on its own and warns on extrapolation.

OS-Climate return-period rasters

OS-Climate rasters are canonicalised with OSClimateIngestPolicy and canonicalize_os_climate. You must choose the distribution family, and for a zero-heavy hazard you must provide an explicit HurdleFitPolicy: the SDK will not infer an exact point mass from sparse knots.

from crc_sdk.workflows import OSClimateIngestPolicy, canonicalize_os_climate

Plain curves are fitted with fit_quantiles; hurdle curves with fit_hurdle_quantiles. Persisted rows are queried back through LocalProvider and HazardQuery. The zarr extra is required for OSClimateProvider and ZarrRaster.

Private buckets

Authenticated remote sources are configured the idiomatic DuckDB way: raw secret DDL, passed as setup_sql so it runs on connect, right after extensions load. There is deliberately no secret-builder type in the SDK, because DuckDB's own secret DDL is already the documented interface.

import os
from crc_sdk.connectors.duckdb import DuckDBConnection, sql_quote

setup_sql = []
key_id, secret = os.getenv("GCS_ACCESS_KEY"), os.getenv("GCS_ACCESS_SECRET")
if key_id and secret:
    setup_sql.append(
        f"CREATE OR REPLACE SECRET gcs (TYPE GCS, KEY_ID {sql_quote(key_id)}, "
        f"SECRET {sql_quote(secret)})"
    )

con = DuckDBConnection.for_analytics(work_dir, setup_sql=setup_sql).connect()

Connectors and sources

Connectors are source-format readers and query engines. Ingest adapters do the conversion into the canonical contract; connectors themselves stay simple on purpose.

Where things live

crc_sdk.core

The stable public API of crc_framework, re-exported.

crc_sdk.fitting

Fitting entry points, re-exported from the framework.

crc_sdk.impacts

Impact transforms and the registry, re-exported from the framework.

crc_sdk.connectors

External formats and query engines. DuckDB connection helpers, Zarr and GeoTIFF ingest, the lazy pipeline seam.

crc_sdk.providers

Storage and dataset discovery.

crc_sdk.geometry

Geometry conversion, H3 indexing and polyfill, raster-to-H3 sampling, coverage writers, PMTiles.

crc_sdk.schema

Columnar data contracts.

crc_sdk.types

SDK-owned Pydantic configuration and metadata.

crc_sdk.workflows

The coordination layer, where data access meets computation. Most of what you will call day to day.

The lazy process seam

DuckDBRelationSource, ArrowBatchSource and DuckDBPipeline form one common seam. Native SQL and Parquet adapters return relations; chunk stores yield bounded Arrow batches. Both feed the same immutable filter, project, aggregate and write pipeline.

Agricultural layers

Agricultural requests are immutable and bounded before they can scan. Builder calls perform no network I/O; relation(), to_arrow_reader() and write_parquet() are the execution points.

from crc_sdk.workflows import AgriculturalLayer

crop_mix = (
    AgriculturalLayer.usda_cdl()
    .resolution("30m")
    .for_area((-93.46, 42.14, -93.45, 42.15))
    .years(2025)
    .classes([1, 5])   # corn and soybeans
    .scan()
    .pipeline()
    .aggregate("count(*) AS sampled_pixels", groups="year, crop_code, crop_name")
)

for batch in crop_mix.to_arrow_reader():
    process(batch)

For global predicted field units, swap the source and keep the same surface:

fields = (
    AgriculturalLayer.ftw_fields()
    .in_country("FR")
    .for_area((2.0, 47.5, 3.0, 48.5))
    .years(2024)
    .confidence_at_least(80)
    .scan()
    .pipeline()
)
fields.write_parquet("outputs/france-fields.parquet")

The two sources take different routes for a real reason. DuckDB's community duckdb_zarr extension can scan ordinary remote Zarr v2 and v3 stores, but it cannot open Icechunk's versioned repository and session model. USDA CDL therefore uses the official Icechunk client for version resolution and chunk reads, then exposes bounded Arrow batches to DuckDB; it never materialises the full raster or a full AOI in memory. FTW GeoParquet stays native in DuckDB, applying bbox pruning before exact spatial filtering.

PMTiles

PMTilesBuild streams a GeoParquet source, either a single file or a Hive-partitioned dataset glob, into one .pmtiles archive in a single tiling pass. The GeoParquet to GeoJSON bridge is built in DuckDB spatial SQL (ST_AsGeoJSON, ST_ReducePrecision, ST_Transform) rather than shelling out to an external converter. Thread defaults and the pre-flight disk check are covered in Resources and tuning.

Public inputs used by the examples

Asset portfolios

Evaluate canonical curve parameters for a whole portfolio without returning to the source format and without refitting. One row per asset, hazard, horizon and pathway; one value column per return period. The minimal example is in the quickstart.

Four ways to hand over assets

The column names asset_id, longitude, latitude and cell_index are inferred. Reach for AssetPortfolio, PointColumns or CellColumn only when your asset schema is unconventional.

(
    HazardDataset.local("flood.parquet")
    .for_assets(assets_with_cells)
    .return_periods([25, 50, 100, 250, 500, 1000])
    .write_parquet("portfolio-flood.parquet")
)

How an asset meets a curve

1. Point assets are converted to the H3 resolution recorded by the canonical dataset.
2. The integer H3 join produces a conservative superset of candidates.
3. Where source WKB is present, the join is refined with ST_Covers(source_geometry, asset_point).
4. Rows with no WKB keep cell-level precision, and say so through spatial_match.

Adding an impact function

Sampling happens first, then impact.evaluate(...) runs on the row's value vector, so value_rp100 = impact(hazard_rp100). See Impact functions for why.

Always pass value_unit and value_semantics. They are written into the evaluation metadata, and they are the difference between a column of numbers and a column somebody else can interpret.

Execution options

Worker, batch and connection controls are grouped under ExecutionOptions on write_parquet, deliberately kept out of the analytical chain. Output evaluation is streamed in bounded Arrow batches to compressed Parquet, so a large portfolio does not have to fit in memory. Lambdas run serially; see why.

For rows you have already selected, distribution_from_hazard_row remains available as the low-level curve reconstruction utility.

Spatial

The other track. Instead of asking what a portfolio is exposed to, it asks what a place is exposed to, and renders the answer as tiles you can put on a map.

The spatial flow

Two notebooks cover it. flood_risk_by_province runs OS-Climate to H3, joins administrative boundaries, enriches with Overture places and writes PMTiles. jrc_global_flood_hazard streams a GeoTIFF or COG and samples it to H3 using JRC LISFLOOD. Both fetch their public inputs over the network.

H3 indexing

Two paths exist because two situations exist. H3Indexer does DuckDB-native polyfill; polyfill_wkb does vectorised batch polyfill over Arrow data and needs the geometry extra for h3ronpy.

Because candidate generation uses overlap coverage rather than centre polyfill (see Canonical hazard datasets), a spatial result can look like it has more cells than you expected: the extras are dropped at the refinement step, not before it.

Constructors tune themselves

OSClimateProvider, ZarrRaster and H3Indexer have no natural caller-supplied directory of their own, so they build a resource-tuned connection by default through DuckDBConnection.for_analytics rather than a bare untuned one. This scales out of the box with no configuration.

Passing an explicit connection or con always wins and skips that entirely. Otherwise the spill directory defaults to a stable location under the system temp directory through default_work_dir(), not a fresh one per call, and can be set per call with each constructor's work_dir parameter, or globally with CRC_DUCKDB_WORK_DIR.

Resources and tuning

The SDK detects DuckDB limits when asked and relays them through the connection config. The defaults are deliberately conservative about threads, for a reason worth understanding.

How the thread count is derived

Resource limits are detected on request, through RuntimeResources.detect or DuckDBConnection.for_analytics.

threads = min(cpus, usable_RAM / GiB_per_thread)

usable_RAM     ~= 60% of detected memory
GiB_per_thread  = 2.5 by default

Spatial work through GEOS often gets slower when over-threaded, which is why the ceiling is memory-derived rather than simply the core count. memory_limit and max_temp_directory_size remain hard process caps.

Environment variables

Or skip the detection entirely and pass an explicit config dict.

PMTiles threading is different on purpose

Unlike DuckDB's GEOS-throttled default, tippecanoe_threads and duckdb_threads in PMTilesBuild default to every detected core, because tippecanoe's tile building has no documented per-thread memory ceiling to protect against.

A pre-flight budget check raises a clear error if a source is estimated to exceed available scratch disk, rather than degrading into a slower multi-batch fallback. If it raises, give it more scratch disk or narrow the scope.

Choosing a write mode

See Writing a canonical stream for the measured trade-off. In short: keep the default ordered=True unless you are memory-bound, and reserve headroom above the engine limit for Arrow and Python buffers and the final Parquet write.

Before a long run

• Call explain() on the plan and read it. Builder calls do no I/O, so this costs nothing.
• prefetch() the source cache, then switch to mode="offline" so a network wobble cannot kill hour three.
• Point work_dir at a disk with room to spill, not at a small container overlay.
• Check that requested return periods sit inside the source support, or accept the extrapolation warning knowingly.
• If you are using an impact function, make sure it is a module-level function rather than a lambda before asking for workers.

Troubleshooting

Common setup and analysis issues, each with a pointer to the page that explains it.

Setting up

ImportError naming an extra

Not a broken install. Extras are imported lazily and the error names exactly what to add. See Installation.

tippecanoe is not pip-installable

PMTiles needs tippecanoe and tile-join on PATH. Check with require_tippecanoe() before a long job, not after.

Notebook working directory

Launch as uv run jupyter lab notebooks/. Pipelines run from the repo root. Getting this backwards scatters cache directories.

Python version or missing wheel

Use Python 3.12 or later and check platform guidance.

Getting numbers

A return period is not a probability

Metrics take non-exceedance q. Upper tail: q = 1 − 1/T. See Probability and distributions.

Tabulated curves raise outside their range

By design. Pass extrapolate=True only when you mean it.

Requested period outside source support

You get a warning, not an error, and you get a number. Read the warning.

Knots are not observations

Passing a tabulated or fitted distribution to fit_distribution is an error. Use fit_quantiles.

No point mass is inferred for you

Zero-heavy hazards need an explicit HurdleFitPolicy or atom_probability.

Fitted values are not source pixels

If you need bit-for-bit reproduction, persist the knots yourself or use a tabulated curve.

Gumbel has no shape

curve_shape is legitimately null for Gumbel families. It is not a missing value to impute.

Impacts and portfolios

Two operations, one object

impact.evaluate(values) is event-aligned; impact(distribution) transforms a distribution. See Impact functions.

Lambdas are not picklable

They run serially, and asking for more than one worker raises. Promote them to module-level functions.

Ambiguous or missing matches raise

Multiple curves for one asset, hazard, horizon and pathway is an error, not something to average.

Units are yours to declare

value_unit and value_semantics land in metadata and are how the next person reads your column.

At scale

Branches are exponential

compute_spanning_set grows as 2**n. Set max_branches.

Ordered writes cost memory

A global sort roughly tripled peak RSS on a two-million-row sample. Use ordered=False when containers are tight.

Engine limits are not process limits

memory_limit caps DuckDB, not Arrow buffers, Python objects or the Parquet writer. Leave headroom.

Thread count below your core count

That is the memory-derived ceiling. Override with CRC_DUCKDB_THREADS if you know better.

Spill directory in a container

The system temp default is often wrong in a container. Set CRC_DUCKDB_WORK_DIR.

Claims about the outputs

Multi-scenario notebooks are stresses

Sensitivity stresses on local tails, not calibrated projections.

FTW fields and CDL pixels are not exposure

Remote-sensing units and land-cover observations, not parcels, ownership, acreage or yield.

Open data changes precision, not credibility

The engine and schema match the commercial stack; precision depends on input resolution, so report it.

Licence, versions and contributing

The packages are on PyPI and the code is AGPL. Hosted access and participation in the Commons are separate, covered on the project site.

The code licence

crc-sdk and crc-framework are licensed AGPL-3.0-or-later. Install them from PyPI today; no application, no account. Parts of the numerical distribution implementation derive from or are informed by SciPy and Cephes, whose notices are preserved in the package's third-party notices.

AGPL obligations are worth reading with your legal office before you embed either package in a service you distribute. Commercial use of RiskThinking.AI's hosted platform or data is a separate question again; see what is open and what is commercial.

Lock your versions

Resolve compatible releases together and commit uv.lock alongside pyproject.toml. Use the lockfile to reproduce the environment:

uv add crc-sdk crc-framework
uv lock
uv sync --locked

Contributing

Fixes, tooling, docs and open impact functions go through the published contribution process. The most useful early contribution is unglamorous: run the reference pipeline on the sample data and report what you found. Spec gaps you hit feed into the roadmap directly.

If your component works with the reference pipeline on reference data, it works with the ecosystem. Conformance is what makes it plug-compatible, and it is the claim you can put in front of your own users.

Where to raise things

GitHub issues

Bugs and spec gaps in crc-sdk, crc-framework or crc-docs.

opensource@riskthinking.ai

Contributions, conformance questions, the early access preview, and building on the standard.

academic@riskthinking.ai

Research access, institutional enrolment, and any conversation about high-fidelity data.