错误信息如下:
3864318 96724 141 92 0 2014-09-13
7529584 79828 729 31 0 2014-09-13
9052481 95904 236 19 1 2014-09-13
4498600 37499 515 64 0 2014-09-13
1684086 71814 646 8 -1 2014-09-13
9246161 7491 1088 0 1 2014-09-13
4845436 96147 930 40 1 2014-09-13
8226377 78643 560 4 -1 2014-09-13
9758545 90410 173 96 1 2014-09-13
771673 80504 475 50 -1 2014-09-13
[2015-04-20 16:13:55,079] ERROR fetching topic metadata for topics [Set(order)] from broker [ArrayBuffer(id:0,host:storm1,port:9092)] failed (kafka.utils.Utils$:106)
kafka.common.KafkaException: fetching topic metadata for topics [Set(order)] from broker [ArrayBuffer(id:0,host:storm1,port:9092)] failed
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:72)
at kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82)
at kafka.producer.async.DefaultEventHandler$$anonfun$handle$1.apply$mcV$sp(DefaultEventHandler.scala:67)
at kafka.utils.Utils$.swallow(Utils.scala:172)
at kafka.utils.Logging$class.swallowError(Logging.scala:106)
at kafka.utils.Utils$.swallowError(Utils.scala:45)
at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:67)
at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105)
at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:94)
at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45)
Caused by: java.nio.channels.ClosedChannelException
at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
... 9 more
[2015-04-20 16:13:55,091] ERROR Failed to collate messages by topic, partition due to: fetching topic metadata for topics [Set(order)] from broker [ArrayBuffer(id:0,host:storm1,port:9092)] failed (kafka.producer.async.DefaultEventHandler:97)
[2015-04-20 16:13:55,202] ERROR fetching topic metadata for topics [Set(order)] from broker [ArrayBuffer(id:0,host:storm1,port:9092)] failed (kafka.utils.Utils$:106)
kafka.common.KafkaException: fetching topic metadata for topics [Set(order)] from broker [ArrayBuffer(id:0,host:storm1,port:9092)] failed
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:72)
at kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82)
at kafka.producer.async.DefaultEventHandler$$anonfun$handle$2.apply$mcV$sp(DefaultEventHandler.scala:78)
at kafka.utils.Utils$.swallow(Utils.scala:172)
at kafka.utils.Logging$class.swallowError(Logging.scala:106)
at kafka.utils.Utils$.swallowError(Utils.scala:45)
at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:78)
at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105)
at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:94)
at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45)
Caused by: java.nio.channels.ClosedChannelException
at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
... 9 more
[2015-04-20 16:13:55,205] ERROR Failed to collate messages by topic, partition due to: fetching topic metadata for topics [Set(order)] from broker [ArrayBuffer(id:0,host:storm1,port:9092)] failed (kafka.producer.async.DefaultEventHandler:97)
[2015-04-20 16:13:55,307] ERROR fetching topic metadata for topics [Set(order)] from broker [ArrayBuffer(id:0,host:storm1,port:9092)] failed (kafka.utils.Utils$:106)
kafka.common.KafkaException: fetching topic metadata for topics [Set(order)] from broker [ArrayBuffer(id:0,host:storm1,port:9092)] failed
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:72)
at kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82)
at kafka.producer.async.DefaultEventHandler$$anonfun$handle$2.apply$mcV$sp(DefaultEventHandler.scala:78)
at kafka.utils.Utils$.swallow(Utils.scala:172)
at kafka.utils.Logging$class.swallowError(Logging.scala:106)
at kafka.utils.Utils$.swallowError(Utils.scala:45)
at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:78)
at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105)
at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:94)
at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45)
Caused by: java.nio.channels.ClosedChannelException
at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
... 9 more
[2015-04-20 16:13:55,309] ERROR Failed to collate messages by topic, partition due to: fetching topic metadata for topics [Set(order)] from broker [ArrayBuffer(id:0,host:storm1,port:9092)] failed (kafka.producer.async.DefaultEventHandler:97)
[2015-04-20 16:13:55,414] ERROR fetching topic metadata for topics [Set(order)] from broker [ArrayBuffer(id:0,host:storm1,port:9092)] failed (kafka.utils.Utils$:106)
kafka.common.KafkaException: fetching topic metadata for topics [Set(order)] from broker [ArrayBuffer(id:0,host:storm1,port:9092)] failed
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:72)
at kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82)
at kafka.producer.async.DefaultEventHandler$$anonfun$handle$2.apply$mcV$sp(DefaultEventHandler.scala:78)
at kafka.utils.Utils$.swallow(Utils.scala:172)
at kafka.utils.Logging$class.swallowError(Logging.scala:106)
at kafka.utils.Utils$.swallowError(Utils.scala:45)
at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:78)
at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105)
at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:94)
at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45)
Caused by: java.nio.channels.ClosedChannelException
at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
... 9 more
[2015-04-20 16:13:55,416] ERROR Failed to collate messages by topic, partition due to: fetching topic metadata for topics [Set(order)] from broker [ArrayBuffer(id:0,host:storm1,port:9092)] failed (kafka.producer.async.DefaultEventHandler:97)
[2015-04-20 16:13:55,522] ERROR fetching topic metadata for topics [Set(order)] from broker [ArrayBuffer(id:0,host:storm1,port:9092)] failed (kafka.utils.Utils$:106)
kafka.common.KafkaException: fetching topic metadata for topics [Set(order)] from broker [ArrayBuffer(id:0,host:storm1,port:9092)] failed
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:72)
at kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82)
at kafka.producer.async.DefaultEventHandler$$anonfun$handle$2.apply$mcV$sp(DefaultEventHandler.scala:78)
at kafka.utils.Utils$.swallow(Utils.scala:172)
at kafka.utils.Logging$class.swallowError(Logging.scala:106)
at kafka.utils.Utils$.swallowError(Utils.scala:45)
at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:78)
at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105)
at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:94)
at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45)
Caused by: java.nio.channels.ClosedChannelException
at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)
at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)
at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)
at kafka.producer.SyncProducer.send(SyncProducer.scala:113)
at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:58)
... 9 more
[2015-04-20 16:13:55,526] ERROR Failed to send requests for topics order with correlation ids in [0,8] (kafka.producer.async.DefaultEventHandler:97)
[2015-04-20 16:13:55,527] ERROR Error in handling batch of 10 events (kafka.producer.async.ProducerSendThread:103)
kafka.common.FailedToSendMessageException: Failed to send messages after 3 tries.
at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90)
at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:105)
at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:94)
at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:45)
如果使用Mac或windows开发测试kafka发送数据时,会有这个错误,原因是在Send message之前需要填写ProducerConfig,但是config中的内容需要使用服务器的hostName,为了使开发环境识别hostName,需要修改本机的hosts文件。
对于Mac下的hosts文件修改方法如下:
http://blog.csdn.net/seven_zhao/article/details/45151597