Created
January 16, 2014 15:02
-
-
Save hpirosha/8456360 to your computer and use it in GitHub Desktop.
Using high level consumer from Kafka 0.8.0 distribution. There are two kafka and single zookeeper node.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| 2014-01-16 20:29:08:228 [INFO ] [main] [TestKafkaConsumer] - Zookeeper url :192.168.64.210:2181 | |
| 2014-01-16 20:29:08:403 [INFO ] [main] [VerifiableProperties] - Verifying properties | |
| 2014-01-16 20:29:08:428 [INFO ] [main] [VerifiableProperties] - Property auto.commit.interval.ms is overridden to 6000 | |
| 2014-01-16 20:29:08:428 [INFO ] [main] [VerifiableProperties] - Property group.id is overridden to consumer.group.1 | |
| 2014-01-16 20:29:08:428 [INFO ] [main] [VerifiableProperties] - Property zookeeper.connect is overridden to 192.168.64.210:2181 | |
| 2014-01-16 20:29:08:428 [INFO ] [main] [VerifiableProperties] - Property zookeeper.session.timeout.ms is overridden to 10000 | |
| 2014-01-16 20:29:08:429 [INFO ] [main] [VerifiableProperties] - Property fetch.wait.max.ms is overridden to 100 | |
| 2014-01-16 20:29:08:429 [INFO ] [main] [VerifiableProperties] - Property consumer.timeout.ms is overridden to 100 | |
| 2014-01-16 20:29:08:465 [INFO ] [main] [ZookeeperConsumerConnector] - [consumer.group.1_Impetus-L292-1389884348463-3e0deb32], Connecting to zookeeper instance at 192.168.64.210:2181 | |
| 2014-01-16 20:29:13:050 [INFO ] [main] [ZookeeperConsumerConnector] - [consumer.group.1_Impetus-L292-1389884348463-3e0deb32], starting auto committer every 6000 ms | |
| 2014-01-16 20:29:13:076 [DEBUG] [main] [ZookeeperConsumerConnector] - [consumer.group.1_Impetus-L292-1389884348463-3e0deb32], entering consume | |
| 2014-01-16 20:29:13:106 [INFO ] [main] [ZookeeperConsumerConnector] - [consumer.group.1_Impetus-L292-1389884348463-3e0deb32], begin registering consumer consumer.group.1_Impetus-L292-1389884348463-3e0deb32 in ZK | |
| 2014-01-16 20:29:13:144 [INFO ] [main] [ZookeeperConsumerConnector] - [consumer.group.1_Impetus-L292-1389884348463-3e0deb32], end registering consumer consumer.group.1_Impetus-L292-1389884348463-3e0deb32 in ZK | |
| 2014-01-16 20:29:13:147 [INFO ] [consumer.group.1_Impetus-L292-1389884348463-3e0deb32_watcher_executor] [ZookeeperConsumerConnector] - [consumer.group.1_Impetus-L292-1389884348463-3e0deb32], starting watcher executor thread for consumer consumer.group.1_Impetus-L292-1389884348463-3e0deb32 | |
| 2014-01-16 20:29:13:153 [DEBUG] [main] [ZookeeperConsumerConnector] - [consumer.group.1_Impetus-L292-1389884348463-3e0deb32], Adding topicThreadId (in-events,consumer.group.1_Impetus-L292-1389884348463-3e0deb32-1) and queue [] to topicThreadIdAndQueues data structure | |
| 2014-01-16 20:29:13:153 [DEBUG] [main] [ZookeeperConsumerConnector] - [consumer.group.1_Impetus-L292-1389884348463-3e0deb32], Adding topicThreadId (in-events,consumer.group.1_Impetus-L292-1389884348463-3e0deb32-0) and queue [] to topicThreadIdAndQueues data structure | |
| 2014-01-16 20:29:13:156 [DEBUG] [main] [ZookeeperConsumerConnector] - [consumer.group.1_Impetus-L292-1389884348463-3e0deb32], adding topic in-events and 2 streams to map. | |
| 2014-01-16 20:29:13:168 [INFO ] [main] [ZookeeperConsumerConnector] - [consumer.group.1_Impetus-L292-1389884348463-3e0deb32], begin rebalancing consumer consumer.group.1_Impetus-L292-1389884348463-3e0deb32 try #0 | |
| 2014-01-16 20:29:13:313 [DEBUG] [main] [ZkUtils$] - Partition map for /brokers/topics/in-events is Map(0 -> List(1), 1 -> List(0)) | |
| 2014-01-16 20:29:13:317 [INFO ] [main] [ConsumerFetcherManager] - [ConsumerFetcherManager-1389884353027] Stopping leader finder thread | |
| 2014-01-16 20:29:13:317 [INFO ] [main] [ConsumerFetcherManager] - [ConsumerFetcherManager-1389884353027] Stopping all fetchers | |
| 2014-01-16 20:29:13:318 [INFO ] [main] [ConsumerFetcherManager] - [ConsumerFetcherManager-1389884353027] All connections stopped | |
| 2014-01-16 20:29:13:319 [INFO ] [main] [ZookeeperConsumerConnector] - [consumer.group.1_Impetus-L292-1389884348463-3e0deb32], Cleared all relevant queues for this fetcher | |
| 2014-01-16 20:29:13:320 [DEBUG] [main] [ConsumerIterator] - Clearing the current data chunk for this consumer iterator | |
| 2014-01-16 20:29:13:320 [DEBUG] [main] [ConsumerIterator] - Clearing the current data chunk for this consumer iterator | |
| 2014-01-16 20:29:13:320 [INFO ] [main] [ZookeeperConsumerConnector] - [consumer.group.1_Impetus-L292-1389884348463-3e0deb32], Cleared the data chunks in all the consumer message iterators | |
| 2014-01-16 20:29:13:321 [INFO ] [main] [ZookeeperConsumerConnector] - [consumer.group.1_Impetus-L292-1389884348463-3e0deb32], Committing all offsets after clearing the fetcher queues | |
| 2014-01-16 20:29:13:321 [INFO ] [main] [ZookeeperConsumerConnector] - [consumer.group.1_Impetus-L292-1389884348463-3e0deb32], Releasing partition ownership | |
| 2014-01-16 20:29:13:323 [INFO ] [main] [ZookeeperConsumerConnector] - [consumer.group.1_Impetus-L292-1389884348463-3e0deb32], Consumer consumer.group.1_Impetus-L292-1389884348463-3e0deb32 rebalancing the following partitions: List(0, 1) for topic in-events with consumers: List(consumer.group.1_Impetus-L292-1389884348463-3e0deb32-0, consumer.group.1_Impetus-L292-1389884348463-3e0deb32-1) | |
| 2014-01-16 20:29:13:325 [INFO ] [main] [ZookeeperConsumerConnector] - [consumer.group.1_Impetus-L292-1389884348463-3e0deb32], consumer.group.1_Impetus-L292-1389884348463-3e0deb32-1 attempting to claim partition 1 | |
| 2014-01-16 20:29:13:328 [DEBUG] [main] [PartitionTopicInfo] - initial consumer offset of in-events:1: fetched offset = -1: consumed offset = -1 is -1 | |
| 2014-01-16 20:29:13:328 [DEBUG] [main] [PartitionTopicInfo] - initial fetch offset of in-events:1: fetched offset = -1: consumed offset = -1 is -1 | |
| 2014-01-16 20:29:13:329 [DEBUG] [main] [ZookeeperConsumerConnector] - [consumer.group.1_Impetus-L292-1389884348463-3e0deb32], in-events:1: fetched offset = -1: consumed offset = -1 selected new offset -1 | |
| 2014-01-16 20:29:13:329 [INFO ] [main] [ZookeeperConsumerConnector] - [consumer.group.1_Impetus-L292-1389884348463-3e0deb32], consumer.group.1_Impetus-L292-1389884348463-3e0deb32-0 attempting to claim partition 0 | |
| 2014-01-16 20:29:13:331 [DEBUG] [main] [PartitionTopicInfo] - initial consumer offset of in-events:0: fetched offset = -1: consumed offset = -1 is -1 | |
| 2014-01-16 20:29:13:331 [DEBUG] [main] [PartitionTopicInfo] - initial fetch offset of in-events:0: fetched offset = -1: consumed offset = -1 is -1 | |
| 2014-01-16 20:29:13:331 [DEBUG] [main] [ZookeeperConsumerConnector] - [consumer.group.1_Impetus-L292-1389884348463-3e0deb32], in-events:0: fetched offset = -1: consumed offset = -1 selected new offset -1 | |
| 2014-01-16 20:29:13:345 [INFO ] [main] [ZookeeperConsumerConnector] - [consumer.group.1_Impetus-L292-1389884348463-3e0deb32], consumer.group.1_Impetus-L292-1389884348463-3e0deb32-0 successfully owned partition 0 for topic in-events | |
| 2014-01-16 20:29:13:348 [INFO ] [main] [ZookeeperConsumerConnector] - [consumer.group.1_Impetus-L292-1389884348463-3e0deb32], consumer.group.1_Impetus-L292-1389884348463-3e0deb32-1 successfully owned partition 1 for topic in-events | |
| 2014-01-16 20:29:13:349 [INFO ] [main] [ZookeeperConsumerConnector] - [consumer.group.1_Impetus-L292-1389884348463-3e0deb32], Updating the cache | |
| 2014-01-16 20:29:13:349 [DEBUG] [main] [ZookeeperConsumerConnector] - [consumer.group.1_Impetus-L292-1389884348463-3e0deb32], Partitions per topic cache Map(in-events -> List(0, 1)) | |
| 2014-01-16 20:29:13:350 [DEBUG] [main] [ZookeeperConsumerConnector] - [consumer.group.1_Impetus-L292-1389884348463-3e0deb32], Consumers per topic cache Map(in-events -> List(consumer.group.1_Impetus-L292-1389884348463-3e0deb32-0, consumer.group.1_Impetus-L292-1389884348463-3e0deb32-1)) | |
| 2014-01-16 20:29:13:352 [INFO ] [main] [ZookeeperConsumerConnector] - [consumer.group.1_Impetus-L292-1389884348463-3e0deb32], Consumer consumer.group.1_Impetus-L292-1389884348463-3e0deb32 selected partitions : in-events:0: fetched offset = -1: consumed offset = -1,in-events:1: fetched offset = -1: consumed offset = -1 | |
| 2014-01-16 20:29:13:353 [INFO ] [consumer.group.1_Impetus-L292-1389884348463-3e0deb32-leader-finder-thread] [ConsumerFetcherManager$LeaderFinderThread] - [consumer.group.1_Impetus-L292-1389884348463-3e0deb32-leader-finder-thread], Starting | |
| 2014-01-16 20:29:13:354 [INFO ] [main] [ZookeeperConsumerConnector] - [consumer.group.1_Impetus-L292-1389884348463-3e0deb32], end rebalancing consumer consumer.group.1_Impetus-L292-1389884348463-3e0deb32 try #0 | |
| 2014-01-16 20:29:13:356 [INFO ] [pool-2-thread-1] [TestKafkaConsumer] - inside run | |
| 2014-01-16 20:29:13:356 [INFO ] [pool-2-thread-2] [TestKafkaConsumer] - inside run | |
| 2014-01-16 20:29:13:372 [INFO ] [consumer.group.1_Impetus-L292-1389884348463-3e0deb32-leader-finder-thread] [VerifiableProperties] - Verifying properties | |
| 2014-01-16 20:29:13:373 [INFO ] [consumer.group.1_Impetus-L292-1389884348463-3e0deb32-leader-finder-thread] [VerifiableProperties] - Property metadata.broker.list is overridden to 192.168.64.210:9092,192.168.64.210:9093 | |
| 2014-01-16 20:29:13:373 [INFO ] [consumer.group.1_Impetus-L292-1389884348463-3e0deb32-leader-finder-thread] [VerifiableProperties] - Property request.timeout.ms is overridden to 30000 | |
| 2014-01-16 20:29:13:373 [INFO ] [consumer.group.1_Impetus-L292-1389884348463-3e0deb32-leader-finder-thread] [VerifiableProperties] - Property client.id is overridden to consumer.group.1 | |
| 2014-01-16 20:29:13:392 [INFO ] [consumer.group.1_Impetus-L292-1389884348463-3e0deb32-leader-finder-thread] [ClientUtils$] - Fetching metadata from broker id:0,host:192.168.64.210,port:9092 with correlation id 0 for 1 topic(s) Set(in-events) | |
| 2014-01-16 20:29:13:397 [DEBUG] [consumer.group.1_Impetus-L292-1389884348463-3e0deb32-leader-finder-thread] [BlockingChannel] - Created socket with SO_TIMEOUT = 30000 (requested 30000), SO_RCVBUF = 8192 (requested -1), SO_SNDBUF = 102400 (requested 102400). | |
| 2014-01-16 20:29:13:398 [INFO ] [consumer.group.1_Impetus-L292-1389884348463-3e0deb32-leader-finder-thread] [SyncProducer] - Connected to 192.168.64.210:9092 for producing | |
| 2014-01-16 20:29:13:417 [INFO ] [consumer.group.1_Impetus-L292-1389884348463-3e0deb32-leader-finder-thread] [SyncProducer] - Disconnecting from 192.168.64.210:9092 | |
| 2014-01-16 20:29:13:419 [DEBUG] [consumer.group.1_Impetus-L292-1389884348463-3e0deb32-leader-finder-thread] [ClientUtils$] - Successfully fetched metadata for 1 topic(s) Set(in-events) | |
| 2014-01-16 20:29:13:428 [DEBUG] [consumer.group.1_Impetus-L292-1389884348463-3e0deb32-leader-finder-thread] [ConsumerFetcherManager$LeaderFinderThread] - [consumer.group.1_Impetus-L292-1389884348463-3e0deb32-leader-finder-thread], {TopicMetadata for topic in-events -> | |
| Metadata for partition [in-events,0] is partition 0 leader: 1 (192.168.64.210:9093) replicas: 1 (192.168.64.210:9093) isr: 1 (192.168.64.210:9093) isUnderReplicated: false | |
| Metadata for partition [in-events,1] is partition 1 leader: 0 (192.168.64.210:9092) replicas: 0 (192.168.64.210:9092) isr: 0 (192.168.64.210:9092) isUnderReplicated: false} | |
| 2014-01-16 20:29:13:439 [INFO ] [ConsumerFetcherThread-consumer.group.1_Impetus-L292-1389884348463-3e0deb32-0-1] [ConsumerFetcherThread] - [ConsumerFetcherThread-consumer.group.1_Impetus-L292-1389884348463-3e0deb32-0-1], Starting | |
| 2014-01-16 20:29:13:443 [DEBUG] [consumer.group.1_Impetus-L292-1389884348463-3e0deb32-leader-finder-thread] [BlockingChannel] - Created socket with SO_TIMEOUT = 30000 (requested 30000), SO_RCVBUF = 65536 (requested 65536), SO_SNDBUF = 8192 (requested -1). | |
| 2014-01-16 20:29:13:451 [DEBUG] [consumer.group.1_Impetus-L292-1389884348463-3e0deb32-leader-finder-thread] [PartitionTopicInfo] - reset fetch offset of ( in-events:0: fetched offset = 10: consumed offset = -1 ) to 10 | |
| 2014-01-16 20:29:13:451 [DEBUG] [consumer.group.1_Impetus-L292-1389884348463-3e0deb32-leader-finder-thread] [PartitionTopicInfo] - reset consume offset of in-events:0: fetched offset = 10: consumed offset = 10 to 10 | |
| 2014-01-16 20:29:13:452 [INFO ] [consumer.group.1_Impetus-L292-1389884348463-3e0deb32-leader-finder-thread] [ConsumerFetcherManager] - [ConsumerFetcherManager-1389884353027] Adding fetcher for partition [in-events,0], initOffset -1 to broker 1 with fetcherId 0 | |
| 2014-01-16 20:29:13:454 [INFO ] [ConsumerFetcherThread-consumer.group.1_Impetus-L292-1389884348463-3e0deb32-0-0] [ConsumerFetcherThread] - [ConsumerFetcherThread-consumer.group.1_Impetus-L292-1389884348463-3e0deb32-0-0], Starting | |
| 2014-01-16 20:29:13:455 [DEBUG] [consumer.group.1_Impetus-L292-1389884348463-3e0deb32-leader-finder-thread] [BlockingChannel] - Created socket with SO_TIMEOUT = 30000 (requested 30000), SO_RCVBUF = 65536 (requested 65536), SO_SNDBUF = 8192 (requested -1). | |
| 2014-01-16 20:29:13:457 [DEBUG] [consumer.group.1_Impetus-L292-1389884348463-3e0deb32-leader-finder-thread] [PartitionTopicInfo] - reset fetch offset of ( in-events:1: fetched offset = 0: consumed offset = -1 ) to 0 | |
| 2014-01-16 20:29:13:457 [DEBUG] [consumer.group.1_Impetus-L292-1389884348463-3e0deb32-leader-finder-thread] [PartitionTopicInfo] - reset consume offset of in-events:1: fetched offset = 0: consumed offset = 0 to 0 | |
| 2014-01-16 20:29:13:458 [INFO ] [consumer.group.1_Impetus-L292-1389884348463-3e0deb32-leader-finder-thread] [ConsumerFetcherManager] - [ConsumerFetcherManager-1389884353027] Adding fetcher for partition [in-events,1], initOffset -1 to broker 0 with fetcherId 0 | |
| 2014-01-16 20:29:19:074 [DEBUG] [Kafka-consumer-autocommit-1] [ZookeeperConsumerConnector] - [consumer.group.1_Impetus-L292-1389884348463-3e0deb32], Committed offset 10 for topic in-events:0: fetched offset = 10: consumed offset = 10 | |
| 2014-01-16 20:29:19:085 [DEBUG] [Kafka-consumer-autocommit-1] [ZookeeperConsumerConnector] - [consumer.group.1_Impetus-L292-1389884348463-3e0deb32], Committed offset 0 for topic in-events:1: fetched offset = 0: consumed offset = 0 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment