Nguyễn Đức Long oNddleo

Technical summary Ultra Scale Kafka system

Core

Apache Kafka
- Usage: Message broker infrastructure, topic management, partition replication
- Key Components: Kafka Streams API for processing, AdminClient for topic management
Apache Avro

Usage: Schema definition, binary serialization/deserialization

Thread and Worker Models for Kafka Event Processing

I'll implement worker/thread models for processing the Kafka order events in Java and compare with another language (Go), including pros and cons of each approach.

Java Implementation: Thread Pool Worker Model

import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.apache.kafka.clients.consumer.ConsumerRecords;
import org.apache.kafka.clients.consumer.KafkaConsumer;

Kafka Event Definition for Order Management in Trading

Here's a comprehensive definition for Kafka events in an order management trading system with hash verification for message producers:

{
  "eventType": "ORDER_CREATED",
  "version": "1.0",
  "timestamp": "2025-03-10T08:45:32.215Z",
 "orderId": "ORD-12345678",

I'll provide you with an example of using the sane_finances library to export S&P Dow Jones Indices (SPDJI) data in Python.

from decimal import Decimal
from datetime import date, datetime
import pandas as pd
from sane_finances.sources.spdji.v2021.exporters import SpdjIndicesExporter
from sane_finances.sources.spdji.v2021.parsers import SpdjIndicesParser

def export_spdji_index(index_code, start_date, end_date, output_file=None):

Detailed Trading System Design

Let's delve deeper into the architecture, data models, and flows for your trading system:

Data Layer: Detailed Design

Database Schema

Market Data (TimescaleDB/InfluxDB)

Here's a Dockerfile that uses Maven 4 and JDK 23:

FROM eclipse-temurin:23-jdk-jammy

# Set Maven version
ENV MAVEN_VERSION 4.0.0
ENV MAVEN_HOME /usr/share/maven
ENV MAVEN_CONFIG "/root/.m2"

Lựa chọn giữa Kafka và Redis Streams cho Event Driven Architecture

Khi xây dựng hệ thống Event Driven, việc lựa chọn công nghệ stream phù hợp là quyết định quan trọng. Dưới đây là phân tích để giúp bạn quyết định giữa Kafka và Redis Streams:

Tiêu chí chính để lựa chọn

Chọn Kafka khi:

Hệ thống có quy mô lớn và phức tạp

Nhiều microservices (10+ services)

Ví dụ Producer và Consumer Redis Stream và Đo Hiệu Năng 100K Request

Dưới đây là ví dụ về cách triển khai Producer và Consumer với Redis Stream, cùng với cách đo hiệu năng khi xử lý 100K request:

1. Cài đặt Redis

Trước khi bắt đầu, hãy đảm bảo bạn đã cài đặt Redis (phiên bản 5.0+)

# Với Docker

Here are examples of Kafka producer and consumer in Go (Golang) that connect to a Bitnami Kafka instance in Kubernetes via port forwarding:

Kafka Producer in Go

package main

import (
	"context"
	"fmt"

	# Kaarma Question

	- The maximum number of people (default) when unchecking unlimited guest? Which event for that? All of event type or which one?
	- Does visible only to group of friends in only private event?
	- Send invitations, polls is feature for private event?
	- In public template event and schedule event -> all user can see that?