一、Kafka监控的几个指标
1、lag:多少消息没有消费
lag=logsize-offset
2、logsize:Kafka存的消息总数
3、offset:已经消费的消息
二、查看zookeeper配置
cat /home/app/zookeeper/zookeeper/conf/zoo.cfg | egrep -v "^$|^#"
clientPort=2181
三、查看kafka配置
cat /home/app/kafka/kafka/config/server.properties | egrep -v "^$|^#"
port=9092
host.name=192.168.38.87
zookeeper.connect=192.168.38.87:2181,192.168.38.88:2181
四、查看kafka的group name
cd /home/app/zookeeper/zookeeper/bin
./zkCli.sh -server 192.168.38.87:2181
ls /consumers/
lijieGroup
quit
五、查看kafka的topic_name
/home/app/kafka/kafka/bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --group=lijieGroup --zookeeper=192.168.38.87:2181
六、修改zabbix配置文件
因为zabbix用户不能调用kafka的脚本,需要root用户启动zabbix_agent
vim /opt/zabbix/zabbix_agentd.conf
AllowRoot=1
User=root
Include=/opt/zabbix/zabbix_agentd.conf.d/
vim /opt/zabbix/zabbix_agentd.conf.d/kafka_status.conf
UserParameter=kafka.lag[*],/home/zabbix_scripts/kafka_mon.sh $1 $2 lag
UserParameter=kafka.offset[*],/home/zabbix_scripts/kafka_mon.sh $1 $2 offset
UserParameter=kafka.logsize[*],/home/zabbix_scripts/kafka_mon.sh $1 $2 logsize
chown -R zabbix:zabbix /opt/zabbix/zabbix_agentd.conf.d/kafka_status.conf
chmod -R 777 /opt/zabbix/zabbix_agentd.conf.d/kafka_status.conf
七、创建监控脚本
mkdir -pv /home/zabbix_scripts/
vim /home/zabbix_scripts/kafka_mon.sh
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
|
#!/bin/bash kafka_ip= "127.0.0.1" kafka_port=2181 topic_name=$1 group_id=$2 pn=$3 /usr/local/kafka_2 .11-0.11.0.0 /bin/kafka-run-class .sh kafka.tools.ConsumerOffsetChecker --topic=$topic_name --group=$group_id --zookeeper=$kafka_ip:$kafka_port | grep - v Offset > /tmp/kafka-tp- ${topic_name}-${group_id}.info Offset=0 logSize=0 Lag=0 while read line do Offset=$((${Offset}+` echo $line | awk '{print $4}' `)) logSize=$((${logSize}+` echo $line | awk '{print $5}' `)) Lag=$(($Lag+` echo $line | awk '{print $6}' `)) done < /tmp/kafka-tp- ${topic_name}-${group_id}.info case $pn in offset|Offset) echo $Offset ;; logsize|logSize) echo $logSize ;; lag|Lag) echo $Lag ;; *) echo Error ;; esac |
八、给脚本和对应文件权限
chown -R zabbix:zabbix /home/zabbix_scripts/kafka_mon.sh
chmod -R 777 /home/zabbix_scripts/kafka_mon.sh
touch /tmp/kafka-tp-RouterOnOfflineStateChange-lijieGroup.info
chmod 777 /tmp/kafka-tp-RouterOnOfflineStateChange-lijieGroup.info
chown zabbix:zabbix /tmp/kafka-tp-RouterOnOfflineStateChange-lijieGroup.info
chmod 777 /home/app/kafka/kafka/bin/kafka-run-class.sh
九、重启zabbix
/etc/init.d/zabbix-agent restart
十、监控上增加3个键值
kafka.offset[RouterOnOfflineStateChange,lijieGroup]
kafka.logsize[RouterOnOfflineStateChange,lijieGroup]
kafka.lag[RouterOnOfflineStateChange,lijieGroup]
本文地址: https://www.xiongge.club/1602.html
转载请注明:熊哥club → zabbix监控kafka消费
©熊哥club,本站推荐使用的主机:阿里云,CDN建议使用七牛云。

关注微信公众号『熊哥club』
免费提供IT技术指导交流
关注博主不迷路~