Skip to content

Instantly share code, notes, and snippets.

Show Gist options
  • Save shivanshtalwar0/0706eec9a1fd88c1a4f133c21b26af38 to your computer and use it in GitHub Desktop.
Save shivanshtalwar0/0706eec9a1fd88c1a4f133c21b26af38 to your computer and use it in GitHub Desktop.
Learn how to deploy **Pulsar 4.x.x** on **Docker Compose** for easy, scalable, and efficient setup of your **Apache Pulsar** messaging platform. Follow our simple guide to quickly launch Pulsar components like brokers, zookeepers, and bookies in a containerized environment.
networks:
pulsar:
driver: bridge
services:
zookeeper:
image: apachepulsar/pulsar:4.0.4
container_name: zookeeper
restart: on-failure
networks:
- pulsar
volumes:
- ./data/zookeeper:/pulsar/data/zookeeper
environment:
- metadataStoreUrl=zk:zookeeper:2181
- PULSAR_MEM=-Xms256m -Xmx256m -XX:MaxDirectMemorySize=256m
command: |
bash -c "bin/apply-config-from-env.py conf/zookeeper.conf && \
bin/generate-zookeeper-config.sh conf/zookeeper.conf && \
exec bin/pulsar zookeeper"
healthcheck:
test: ["CMD", "bin/pulsar-zookeeper-ruok.sh"]
interval: 10s
timeout: 5s
retries: 30
# Init cluster metadata
pulsar-init:
container_name: pulsar-init
hostname: pulsar-init
image: apachepulsar/pulsar:4.0.4
networks:
- pulsar
environment:
- PULSAR_MODE=cluster
- PULSAR_CLUSTER_NAME=my-cluster
- PULSAR_ZOOKEEPER_SERVERS=zookeeper:2181
- PULSAR_WEB_SERVICE_URL=http://pulsar:8080
- PULSAR_CLUSTER=my-cluster
entrypoint: >
/bin/bash -c "pulsar initialize-cluster-metadata --cluster cluster-a --web-service-url http://broker:8080 --zookeeper zookeeper:2181 --broker-service-url pulsar://broker:6650 --configuration-store zookeeper:2181"
depends_on:
zookeeper:
condition: service_healthy
# Start bookie
bookie:
image: apachepulsar/pulsar:4.0.4
container_name: bookie
restart: on-failure
networks:
- pulsar
environment:
- clusterName=cluster-a
- zkServers=zookeeper:2181
- metadataServiceUri=metadata-store:zk:zookeeper:2181
# otherwise every time we run docker compose uo or down we fail to start due to Cookie
# See: https://github.com/apache/bookkeeper/blob/405e72acf42bb1104296447ea8840d805094c787/bookkeeper-server/src/main/java/org/apache/bookkeeper/bookie/Cookie.java#L57-68
- advertisedAddress=bookie
- BOOKIE_MEM=-Xms512m -Xmx512m -XX:MaxDirectMemorySize=256m
depends_on:
zookeeper:
condition: service_healthy
pulsar-init:
condition: service_completed_successfully
# Map the local directory to the container to avoid bookie startup failure due to insufficient container disks.
volumes:
- ./data/bookkeeper:/pulsar/data/bookkeeper
command: bash -c "bin/apply-config-from-env.py conf/bookkeeper.conf && exec bin/pulsar bookie"
# Start broker
broker:
image: apachepulsar/pulsar:4.0.4
container_name: broker
hostname: broker
restart: on-failure
networks:
- pulsar
environment:
- metadataStoreUrl=zk:zookeeper:2181
- zookeeperServers=zookeeper:2181
- clusterName=cluster-a
- managedLedgerDefaultEnsembleSize=1
- managedLedgerDefaultWriteQuorum=1
- managedLedgerDefaultAckQuorum=1
- advertisedAddress=broker
- advertisedListeners=external:pulsar://127.0.0.1:6650
- PULSAR_MEM=-Xms512m -Xmx512m -XX:MaxDirectMemorySize=256m
depends_on:
zookeeper:
condition: service_healthy
bookie:
condition: service_started
ports:
- "6650:6650"
- "8080:8080"
command: bash -c "bin/apply-config-from-env.py conf/broker.conf && exec bin/pulsar broker"
@shivanshtalwar0
Copy link
Author

Deploying Pulsar 4.x.x on Docker Compose

Deploying Pulsar 4.x.x on Docker Compose is a streamlined and efficient way to run Apache Pulsar, a distributed messaging and event streaming platform, in a containerized environment. This guide will walk you through the steps to set up Pulsar 4.x.x with Docker Compose, enabling seamless deployment, easy scalability, and quick management of your Pulsar clusters.

Whether you're looking to experiment with Pulsar or deploy it in production, Docker Compose simplifies the process by automating the setup of Pulsar components, including brokers, zookeepers, and bookies.

Follow our step-by-step instructions to deploy Pulsar 4.x.x with just a few simple commands, ensuring a robust and flexible messaging solution for your applications.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment