Gilles Philippart gphilipp

Release	Blog Post	What you’re missing out if you're using this version	Notable Changes	Date	New Features	Improvements	Bug Fixes
3.8(29 Jul 2024)	What's new in Apache Kafka 3.8	0 new features, 0 improvements, 0 bug fixes	KIP-858: JBOD in KRaft is now GA.KIP-390: Adds support for specifying compression levels.KIP-848: The new Consumer Rebalance Protocol, available in preview.KIP-993: Restricts file access by File and Directory ConfigProviders.KIP-974: Introduces Docker image for GraalVM-based native Kafka broker (great for faster test runs 🚀).KIP-1018: Introduces `remote.fetch.max.wait.ms` for DelayedRemoteFetch requests.KIP-1028:

Install MSYS2 https://www.msys2.org/
```
winget install --id=MSYS2.MSYS2  -e
```
Open the MSYS2 terminal and update the package database and core system packages:

	Delta Lake (Read)	Delta Lake (Write)	Iceberg (Read)	Iceberg (Write)

I've exported my environment variables to make it easier to run the commands:

export SCHEMA_REGISTRY_URL=xxxxx
export SCHEMA_REGISTRY_API_KEY=xxxxx
export SCHEMA_REGISTRY_API_SECRET=xxxxx
export BASIC_AUTH_USER_INFO=$SCHEMA_REGISTRY_API_KEY:$SCHEMA_REGISTRY_API_SECRET
export BOOTSTRAP_SERVERS=SASL_SSL://xxxxx.xxxxx.aws.confluent.cloud:9092
export KAFKA_API_KEY=xxxxx
export KAFKA_API_SECRET=xxxxx

Run Confluent platform

git clone https://github.com/confluentinc/cp-docker-images
cd cp-docker-images/examples/cp-all-in-one
git checkout v5.2.1
docker-compose up -d

# wait a minute for services to be started
docker-compose ps

	{:user {:dependencies [[spyscope "0.1.6"]
	[org.clojure/tools.trace "0.7.9"]
	[vvvvalvalval/scope-capture "0.1.4"] ]
	:plugins [[lein-ancient "0.6.15"]]
	:injections [(require 'spyscope.core)
	(require 'sc.api)]}}

	#!/bin/bash

	#kafka helper functions riped off from cddr: https://gist.github.com/cddr/65b03448d34bb34636c3b2738ff14e3e

	export BROKER_LIST=kafka-4:9092
	export SCHEMA_REGISTRY_URL="https://schema-registry.fc-uat.us"
	export ZOOKEEPER_ADDRESS="localhost:2181"

	# just do that once
	#sudo ifconfig en0 alias 192.168.4.1 broadcast 192.168.4.255

	#!/bin/bash

	#kafka helper functions riped off from cddr: https://gist.github.com/cddr/65b03448d34bb34636c3b2738ff14e3e

	export BROKER_LIST=kafka-4:9092
	export KAFKA_DIR=$CONFLUENT_HOME
	export SCHEMA_REGISTRY_URL="https://schema-registry.fc-uat.us"
	export ZOOKEEPER_ADDRESS="localhost:2181"

	sudo ifconfig en0 alias 192.168.4.1 broadcast 192.168.4.255

	avro~=1.11.3
	fastavro~=1.11.3

Gilles Philippart gphilipp

Exactly once processing serie