Skip to content

Instantly share code, notes, and snippets.

View Swoorup's full-sized avatar
🎲
Focusing

Swoorup Joshi Swoorup

🎲
Focusing
View GitHub Profile
@Swoorup
Swoorup / elasticsearch_playground.es
Created May 29, 2025 01:50
Playground to test elasticsearch
/// OPENSEARCH TUTORIAL: Comprehensive Query Examples for Bakery Management System
/// This tutorial covers: Basic CRUD, Search, Aggregations, Multi-search, and Percolator queries
/// Use Elasticsearch VSCode extension for easy execution
/// Get Elasticsearch version information
GET /
/// ===================================================
/// STEP 1: INDEX SETUP AND DATA PREPARATION
@Swoorup
Swoorup / dtype_macros.rs
Last active April 5, 2025 16:15
Slightly different take than dtype_dispatch
//! # DType Macros
//!
//! This module provides a set of macros and traits for defining and working with strongly-typed
//! enums and their associated tokens. These macros are designed to:
//!
//! - **Generate Tokens**: Define unique tokens for each variant of an enum using `build_dtype_tokens`.
//! - **Define Enums**: Create enums with compile-time checks to ensure all variants are valid tokens
//! using `build_dtype_enum`.
//! - **Dynamic Downcasting**: Enable safe downcasting of enum variants to their underlying types
//! or containers.
@Swoorup
Swoorup / morton_space_fill.rs
Created May 28, 2024 11:31
64-bit, 128-bit bit interleaved with (+/-) axis.
mod bit_util_u64 {
const SIGN_BIT_FLIP: u64 = 0x8000_0000_0000_0000;
// magic numbers for bit interleaving
const MAGIC: [u128; 7] = [
0x5555555555555555_5555555555555555,
0x3333333333333333_3333333333333333,
0x0F0F0F0F0F0F0F0F_0F0F0F0F0F0F0F0F,
0x00FF00FF00FF00FF_00FF00FF00FF00FF,
@Swoorup
Swoorup / dirty_index_set.rs
Last active May 30, 2025 12:47
Gpu Buffer Tracking
use std::ops::Range;
use derive_more::{Deref, DerefMut, Into};
#[derive(Debug, Deref, DerefMut, Hash, PartialEq, Eq, Clone, Into)]
pub struct DirtyIndexRange(pub(super) Range<usize>);
impl DirtyIndexRange {
pub fn one(index: usize) -> Self {
Self(index..index + 1)
@Swoorup
Swoorup / TaskValidation.fs
Created June 15, 2023 05:31
Task Validation CE with FsToolkit.ErrorHandling
open System.Threading.Tasks
open FsToolkit.ErrorHandling
open FsToolkit.ErrorHandling.Operator.Validation
type TaskValidation<'a, 'err> = Task<Validation<'a, 'err>>
[<RequireQualifiedAccess>]
module TaskValidation =
let inline map f (tv: TaskValidation<'a, 'err>) : TaskValidation<'b, 'err> = TaskResult.map f tv
let inline bind (f: 'a -> TaskValidation<'b, 'err>) (tv: TaskValidation<'a, 'err>) : TaskValidation<'b, 'err> = TaskResult.bind f tv
@Swoorup
Swoorup / ViperAppDemo.tsx
Last active March 1, 2023 16:27
Viper charts demo
import { useEffect, useRef } from 'react';
import './App.css';
import { DataUpdates, HistoricalDataRequest, Price, UTCTimestamp, Viper as ViperCharts } from "@viper-charts/viper-charts";
// import "@viper-charts/viper-charts/dist/style.css";
import { ViperDatabase } from "./ViperDatabase";
const db = createDB();
function onSaveViperSettings(settings: any) {
@Swoorup
Swoorup / Makefile.toml
Created December 19, 2022 09:49
duckdb shared library helper for cargo-make
[config]
# skip_core_tasks = true
# skip_git_env_info = true
# skip_crate_env_info = true
# skip_rust_env_info = true
[env]
CARGO_MAKE_EXTEND_WORKSPACE_MAKEFILE = true
SHARED_LIB_DIR = "${CARGO_MAKE_WORKSPACE_WORKING_DIRECTORY}/shared_lib"
LIBRARY_EXTENSION = { source = "${CARGO_MAKE_RUST_TARGET_OS}", default_value = "unknown", mapping = {"linux" = "so", "macos" = "dylib" } }
@Swoorup
Swoorup / restartable.rs
Created October 2, 2022 16:28
Restartable stream on complete or error using a connect function
use std::task::Poll;
use futures::{Future, FutureExt, Stream};
use pin_project::pin_project;
use tracing::debug;
#[pin_project(project = StreamStateProj)]
#[derive(Debug, Clone)]
enum StreamState<F, S>
where
@Swoorup
Swoorup / StreamCache.scala
Created July 17, 2022 16:12
Cache for redis streams with offsets
package lib.cache
import cats.effect.std.{UUIDGen, Supervisor}
import cats.effect.syntax.all.*
import cats.effect.{Async, Clock, Fiber, Outcome, Ref, Resource}
import cats.syntax.all.*
import fs2.Stream
import io.chrisdavenport.mapref.MapRef
import io.chrisdavenport.rediculous.RedisCommands
import io.chrisdavenport.rediculous.RedisConnection

https://web.archive.org/web/20110219163448/http://howtohft.wordpress.com/2011/02/15/how-to-build-a-fast-limit-order-book/

The response to my first few posts has been much larger than I’d imagined and I’d like to thank everyone for the encouragement.

If you’re interested in building a trading system I recommend first reading my previous post on general ideas to keep in mind.

My first really technical post will be on how to build a limit order book, probably the single most important component of a trading system. Because the data structure chosen to represent the limit order book will be the primary source of market information for trading models, it is important to make it both absolutely correct and extremely fast.

To give some idea of the data volumes, the Nasdaq TotalView ITCH feed, which is every event in every instrument traded on the Nasdaq, can have data rates of 20+ gigabytes/day with spikes of 3 megabytes/second or more. The individual messages average about 20 bytes each so this means handling