Kafka-python問題解決

消費者代碼

from kafka import KafkaConsumer

consumer = KafkaConsumer('reimport-app-douyin2', auto_offset_reset='earliest',bootstrap_servers='bigdata-test01:6667')

for msg in consumer:

print(msg)

主要是加了一個auto_offset_reset='earliest'，查看所有的Kafka消息

mac

1、python創建的kafka消費者報錯

kafka.errors.UnsupportedCodecError: UnsupportedCodecError: Libraries for snappy compression codec not found

解決方式：

brew install snappy

brew install python-snappy

windows

2、python創建kafka消費者報錯

AttributeError: module 'snappy' has no attribute 'decompress'

解決：

在win系統的python安裝python-snappy庫

從以下去下載對應的python-snappy版本

https://www.lfd.uci.edu/~gohlke/pythonlibs/#python-snappy