amotl

"""
### About

Task: Import a CSV file with special features into CrateDB.
Source: https://guided-path.s3.us-east-1.amazonaws.com/demo_climate_data_export.csv

The program applies two transformations before the data is ready for importing into
CrateDB. For the import procedure, it uses a performance path with pandas/SQLAlchemy.

- Convert coordinates in JSON list format to WKT POINT format.

amotl

import sqlalchemy as sa


def workload():
    """
    Install
    uv pip install sqlalchemy-cratedb psycopg2-binary

    Invoke
    python cratedb_percentile_with_args.py

amotl

import sqlalchemy as sa


def workload():
    """
    Install
    uv pip install sqlalchemy-cratedb psycopg2-binary

    Invoke
    python cratedb_groupby_function.py

amotl

import sqlalchemy as sa


def workload():
    """
    Install
    uv pip install sqlalchemy-cratedb psycopg2-binary

    Invoke
    python cratedb_functionname_nested.py

amotl

TESTDRIVE=foobar

amotl

"""
## About

`crate-python` uses `orjson` for JSON serialization.

## Errors

- TypeError: Type is not JSON serializable: numpy.ndarray
- TypeError: Type is not JSON serializable: recarray

amotl

To optimally synchronize data between MongoDB and CrateDB, you should use a Change Data Capture (CDC) integration, which is available as a managed feature in CrateDB Cloud. This allows you to keep your MongoDB data continuously and efficiently synchronized with a table in CrateDB. Here’s a concise guide on how to do this:

---

1. Use CrateDB Cloud’s MongoDB CDC Integration

CrateDB Cloud (preview feature, see docs) can continuously import and sync data from MongoDB (e.g., MongoDB Atlas) using Change Streams.

Key Features:

amotl

services:

  nginx:
    image: nginx:1.27
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
    ports:
      - "8080:80"
    restart: unless-stopped

amotl

#!/usr/bin/env python3
# /// script
# requires-python = ">=3.9"
# dependencies = [
#     "click",
#     "geopandas",
#     "geopy",
# ]
# ///
"""

amotl

#!/usr/bin/env python3
"""
Prerequisite:

    docker run --rm --name=cratedb \
        --publish=4200:4200 --publish=5432:5432 \
        --env=CRATE_HEAP_SIZE=2g crate/crate:nightly \
        -Cdiscovery.type=single-node

Variants: