Skip to content

Instantly share code, notes, and snippets.

@tddschn

tddschn/qdb-cli-short-tour

Created April 24, 2025 03:08
Show Gist options
  • Save tddschn/e2d0e0da3928abc7c071a429dcde44ec to your computer and use it in GitHub Desktop.
Save tddschn/e2d0e0da3928abc7c071a429dcde44ec to your computer and use it in GitHub Desktop.
Download ZIP
Raw
qdb-cli-short-tour
# << a short tour of questdb-cli >>
# querying the public demo instance, print the data in psql table format
$ qdb-cli --port 443 --host https://demo.questdb.io exec --psql -q 'trades limit 20'
+----------+--------+----------+------------+-----------------------------+
| symbol | side | price | amount | timestamp |
|----------+--------+----------+------------+-----------------------------|
| ETH-USD | sell | 2615.54 | 0.00044 | 2022-03-08T18:03:57.609765Z |
| BTC-USD | sell | 39270 | 0.001 | 2022-03-08T18:03:57.710419Z |
| ETH-USD | buy | 2615.4 | 0.002 | 2022-03-08T18:03:57.764098Z |
| ETH-USD | buy | 2615.4 | 0.001 | 2022-03-08T18:03:57.764098Z |
| ETH-USD | buy | 2615.4 | 0.00042698 | 2022-03-08T18:03:57.764098Z |
| ETH-USD | buy | 2615.36 | 0.025936 | 2022-03-08T18:03:58.194582Z |
| ETH-USD | buy | 2615.37 | 0.0350084 | 2022-03-08T18:03:58.194582Z |
| ETH-USD | buy | 2615.46 | 0.172602 | 2022-03-08T18:03:58.194582Z |
| ETH-USD | buy | 2615.47 | 0.14811 | 2022-03-08T18:03:58.194582Z |
| BTC-USD | sell | 39265.3 | 0.000127 | 2022-03-08T18:03:58.357448Z |
| BTC-USD | sell | 39265.3 | 0.000245 | 2022-03-08T18:03:58.357448Z |
| BTC-USD | sell | 39265.3 | 7.3e-05 | 2022-03-08T18:03:58.357448Z |
| BTC-USD | sell | 39263.3 | 0.00392897 | 2022-03-08T18:03:58.357448Z |
| ETH-USD | buy | 2615.35 | 0.0224587 | 2022-03-08T18:03:58.612275Z |
| ETH-USD | buy | 2615.36 | 0.0324461 | 2022-03-08T18:03:58.612275Z |
| BTC-USD | sell | 39265.3 | 6.847e-05 | 2022-03-08T18:03:58.660121Z |
| BTC-USD | sell | 39262.4 | 0.00046562 | 2022-03-08T18:03:58.660121Z |
| ETH-USD | buy | 2615.62 | 0.00044 | 2022-03-08T18:03:58.682070Z |
| ETH-USD | buy | 2615.62 | 0.00044 | 2022-03-08T18:03:58.682070Z |
| ETH-USD | buy | 2615.62 | 0.00044 | 2022-03-08T18:03:58.682070Z |
+----------+--------+----------+------------+-----------------------------+
# export the whole table (180 MB, be careful)
$ qdb-cli --port 443 --host https://demo.questdb.io exp 'trips' > trips.csv
# import the copy in your local instance
# let's configure the CLI to use your local instance first
$ qdb-cli gen-config
# edit the config file to set your local instance
# lightning fast local import!
# the imp command can infer table name using different rules, install it and run --help to see
$ qdb-cli imp --name trips trips.csv --partitionBy WEEK --timestamp pickup_datetime
+-----------------------------------------------------------------------------------------------------------------+
| Location: | trips | Pattern | Locale | Errors |
| Partition by | WEEK | | | |
| Timestamp | pickup_datetime | | | |
+-----------------------------------------------------------------------------------------------------------------+
| Rows handled | 1000000 | | | |
| Rows imported | 1000000 | | | |
+-----------------------------------------------------------------------------------------------------------------+
| 0 | cab_type | VARCHAR | 0 |
| 1 | vendor_id | VARCHAR | 0 |
| 2 | pickup_datetime | TIMESTAMP | 0 |
| 3 | dropoff_datetime | TIMESTAMP | 0 |
| 4 | rate_code_id | VARCHAR | 0 |
| 5 | pickup_latitude | DOUBLE | 0 |
| 6 | pickup_longitude | DOUBLE | 0 |
| 7 | dropoff_latitude | DOUBLE | 0 |
| 8 | dropoff_longitude | DOUBLE | 0 |
| 9 | passenger_count | INT | 0 |
| 10 | trip_distance | DOUBLE | 0 |
| 11 | fare_amount | DOUBLE | 0 |
| 12 | extra | DOUBLE | 0 |
| 13 | mta_tax | DOUBLE | 0 |
| 14 | tip_amount | DOUBLE | 0 |
| 15 | tolls_amount | DOUBLE | 0 |
| 16 | ehail_fee | DOUBLE | 0 |
| 17 | improvement_surcharge | DOUBLE | 0 |
| 18 | congestion_surcharge | DOUBLE | 0 |
| 19 | total_amount | DOUBLE | 0 |
| 20 | payment_type | VARCHAR | 0 |
| 21 | trip_type | VARCHAR | 0 |
| 22 | pickup_location_id | INT | 0 |
| 23 | dropoff_location_id | INT | 0 |
+-----------------------------------------------------------------------------------------------------------------+
# check schema to confirm the import
$ qdb-cli schema trips
CREATE TABLE 'trips' (
cab_type VARCHAR,
vendor_id VARCHAR,
pickup_datetime TIMESTAMP,
dropoff_datetime TIMESTAMP,
rate_code_id VARCHAR,
pickup_latitude DOUBLE,
pickup_longitude DOUBLE,
dropoff_latitude DOUBLE,
dropoff_longitude DOUBLE,
passenger_count INT,
trip_distance DOUBLE,
fare_amount DOUBLE,
extra DOUBLE,
mta_tax DOUBLE,
tip_amount DOUBLE,
tolls_amount DOUBLE,
ehail_fee DOUBLE,
improvement_surcharge DOUBLE,
congestion_surcharge DOUBLE,
total_amount DOUBLE,
payment_type VARCHAR,
trip_type VARCHAR,
pickup_location_id INT,
dropoff_location_id INT
) timestamp(pickup_datetime) PARTITION BY WEEK WAL
WITH maxUncommittedRows=500000, o3MaxLag=600000000us;
# rename commands for your convenience (run something like `RENAME TABLE 'test.csv' TO 'myTable'`; under the hood)
$ qdb-cli rename trips taxi_trips_feb_2018
{
"status": "OK",
"message": "Table 'trips' renamed to 'taxi_trips_feb_2018'"
}
@tddschn
Copy link
Author

tddschn commented Apr 24, 2025

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment