本站提供互联网编程技术交流分享,部分技术教程不断更新中,请随时关注或联系我寻求帮助 有事点我吧同时也欢迎有兴趣的朋友进行投稿。

django实现zookeeper的监控

前端 熊哥club 201℃ 0评论

版本信息

$ python

Python 2.7.5 (default, Jul 13 2018, 13:06:57)

[GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux2

Type “help”, “copyright”, “credits” or “license” for more information.

 

>>> import django
>>> print(django.get_version())

1.11.14

 

创建名为mysite的项目

在第三方软件Pyharm中:

  • file菜单–》New Project按钮
  • 左菜单选择Django,内容目录选择:/home/weis/PycharmProjects/mysite
  • 参数不用配
  • Create按钮

生成目录:

mysite/

__init__.py

manage.py

settings.py

urls.py

mysite

venv

 

终端创建名为zkTarget的应用

Terminal

$ python manage.py startapp zkTarget

 

生成目录:

mysite/

zkTarget/

__init__.py

models.py

tests.py

views.py

 

 

浏览器上输入地址127.0.0.1:8000/,django运行顺序:

1、配置文件:mysite/settings.py

添加应用,在INSTALLED_APPS参数插入值:

‘zkTarget’,

2、url文件:mysite/urls.py

插入:
from zkTarget import views
url(r’^$’, views.display_meta, name=’index’),

 

3.1、模型(Model),即数据存取层。文件:zkTarget/model.py

# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import models

# Create your models here.
class Targetmntr(models.Model):
time = models.DateTimeField(‘%Y-%m-%d %H:%M’)#:%S
server = models.CharField(max_length=60)
key = models.CharField(max_length=60)
value = models.CharField(max_length=60)
def __str__(self):
return u’%s %s %s %s’ % (self.time, self.server, self.key, self.value)

class Targetcons(models.Model):
time = models.DateTimeField(‘%Y-%m-%d %H:%M’)
server = models.CharField(max_length=60)
key = models.CharField(max_length=60)
value = models.CharField(max_length=60)
client = models.CharField(max_length=60)
def __str__(self):
return u’%s %s %s %s %s’ % (self.time, self.server, self.key, self.value, self.client)

class Controlkey(models.Model):
command = models.CharField(max_length=10)
key = models.CharField(max_length=60)
site = models.CharField(max_length=60)
check = models.CharField(max_length=60)
remark = models.CharField(max_length=100)
def __str__(self):
return u’%s %s %s %s %s’ % (self.command, self.key, self.site, self.check, self.remark)

class Controlserver(models.Model):
server = models.CharField(max_length=60)
check = models.CharField(max_length=60)
remark = models.CharField(max_length=100)
def __str__(self):
return u’%s %s %s’ % (self.server, self.check, self.remark)

 

终端中输入:

$ python manage.py check                 #验证模型的有效性
$ python manage.py makemigrations zkTarget   #为模型的改变生成迁移文件
$ python manage.py sqlmigrate zkTarget 0001  #返回对应的 SQL
$ python manage.py migrate                  #在数据库中创建对应的表

使用是django默认自带的sqlite数据库,执行后按配置文件(settings.py)中配置项生成db.sqlite3文件。

3.2、导入固定指标数据

导入要抓的指标和服务器ip等信息。

$ python data_initial_key_and_server.py:

#!/usr/bin/env python
#coding:utf-8
”’
删除表Controlkey、表Controlserver中的数据,且重新插入数据
”’
import os
os.environ.setdefault(“DJANGO_SETTINGS_MODULE”, “mysite.settings”)

import django
django.setup()

def main():
from zkTarget.models import Controlkey, Controlserver

#表Controlkey的“command”四字命令字段参数
commandlist = [‘mntr’, ‘cons’]

#表Controlkey的“key”字段参数
mntrkeylist = [
[‘zk_avg_latency’, 11], \
[‘zk_max_latency’, 11], \
[‘zk_min_latency’, 11], \
[‘zk_packets_received’, 11], \
[‘zk_num_alive_connections’, 21], \
[‘zk_outstanding_requests’, 21], \
[‘zk_approximate_data_size’, 21], \
[‘zk_open_file_descriptor_count’, 21], \
[‘zk_max_file_descriptor_count’, 21]
]
conskeylist = [
[‘queued’, 12], \
[‘sent’, 12], \
[‘to’, 12], \
[‘minlat’, 22], \
[‘avglat’, 22], \
[‘maxlat’, 22]
]
keylist = [mntrkeylist, conskeylist]

#表Controlserver的“server”服务器ip+端口字段参数
serverlist = [‘127.0.0.1 2181’]

Controlkey.objects.all().delete()
for c in range(len(commandlist)):
for key in keylist[c]:
data = Controlkey(command=commandlist[c], key=key[0], site=key[1], check=1, remark=”)
data.save()

Controlserver.objects.all().delete()
for server in serverlist:
data = Controlserver(server=server, check=1, remark=”)
data.save()

if __name__ == “__main__”:
main()
print(‘Done!’)

3.4、导入历史数据

导入历史四字命令数据。

$ python data_initial_mntr_and_cons.py:

#!/usr/bin/env python
#coding:utf-8
”’
删除表Targetmntr、表Targetcons中的数据,且重新插入数据
”’
import os
os.environ.setdefault(“DJANGO_SETTINGS_MODULE”, “mysite.settings”)

import django
django.setup()

import json
from sqlapi import savemntr, savecons

def main():
#json.dumps(data)用于将dict类型的数据转成str,因为如果直接将dict类型的数据写入json文件中会发生报错,因此在将数据写入时需要用到该函数。
#json.loads(data)用于将str类型的数据转成dict。
write(‘mntr’)
write(‘cons’)

def write( command ):
from zkTarget.models import Targetmntr, Targetcons

f = open(‘zookeeper_’+command+’_command.txt’)
for line in f:
linestrip = line.strip(‘\n’)
if linestrip !=”:
linedict = json.loads(linestrip)
for onedict in linedict:
if command == ‘mntr’:
for onedict in linedict:
savemntr(onedict[‘time’], onedict[‘server’], onedict[‘key’], onedict[‘value’])
else:
for onedict in linedict:
savecons(onedict[‘time’], onedict[‘server’], onedict[‘client’], onedict[‘key’], onedict[‘value’])
f.close()
print Targetmntr.objects.filter(key = ‘zk_avg_latency’)[0:2]
print Targetcons.objects.filter(key = ‘queued’)[0:2]

if __name__ == “__main__”:
main()
print(‘Done!’)

$ data_initial_mntr_and_cons.py文件中调用的文件sqlapi.py:

#!/usr/bin/env python
#coding:utf-8

import django
django.setup()
from zkTarget.models import Targetmntr, Targetcons

def savemntr( time, server, key, value ):
print ‘—-call savemntr(‘, time, ‘,’, server, ‘,’, key, ‘,’, value, ‘):’
obj, created = Targetmntr.objects.get_or_create(
time=time,
server=server,
key=key,
defaults={‘value’: value}
)

def savecons( time, server, client, key, value ):
print ‘—-call savemntr(‘, time, ‘,’, server, ‘,’, client, ‘,’, key, ‘,’, value, ‘):’
obj, created = Targetcons.objects.get_or_create(
time=time,
server=server,
client=client,
key=key,
defaults={‘value’: value}
)

 

3.5、每分钟执行抓四字命令指标文件。data_minute.py

#!/usr/bin/env python
#coding:utf-8
”’
每分钟执行一次zookeeper四字命令,存入数据库表Targetmntr、表Targetcons
”’
import os
os.environ.setdefault(“DJANGO_SETTINGS_MODULE”, “mysite.settings”)

import django
django.setup()

import datetime
from sqlapi import savemntr, savecons

def main():
from zkTarget.models import Controlkey, Controlserver

mntrKeys = Controlkey.objects.values(“key”).filter(command=’mntr’)
consKeys = Controlkey.objects.values(“key”).filter(command=’cons’)
Time = datetime.datetime.now() + datetime.timedelta(hours=8)

for ip in Controlserver.objects.values(“server”):
echo = “echo mntr | nc ” + ip[“server”];
keyValue = os.popen(echo).read().split(‘\n’);
for index in range( len(keyValue) – 1 ):
key_value = keyValue[index].split(‘\t’);
if key_value[0] in mntrKeys:
savemntr(Time, ip[“server”], key_value[0], key_value[1])

echo = “echo cons | nc ” + ip[“server”];
echoString = os.popen(echo).read();
echoString = echoString.replace(‘ /’,”);
echoString = echoString.replace(‘)’,”);
echoString = echoString.split(‘\n’);
for client in range( len(echoString) – 2 ):
echoSplit1 = echoString[client].split(‘[‘);
echoclient = echoSplit1[0];
echoSplit2 = echoSplit1[1].split(‘(‘);
keyValue = echoSplit2[1].split(‘,’);
for index in range(len(keyValue)):
key_value = keyValue[index].split(‘=’);
if key_value[0] in consKeys:
savecons(Time, ip[“server”], echoclient, key_value[0], key_value[1])
print Time, ‘Get target of zookeeper, done!’

if __name__ == “__main__”:
main()

4、视图(View),即表现层。视图文件:zkTarget/views.py

# -*- coding: utf-8 -*-
“””
“””
from __future__ import unicode_literals

import os
os.environ.setdefault(“DJANGO_SETTINGS_MODULE”, “mysite.settings”)

import django
django.setup()

import json
import datetime
from django.shortcuts import render_to_response
from zkTarget.models import Targetmntr, Targetcons, Controlkey, Controlserver
from data_minute import main

#显示到页面的方法
def display_meta(request):
print ‘–call display_meta():’
mainList = getmainList()
seriesList = get_series_or_line(“seriesList”)
timeList, lineList = get_line_and_time()

return render_to_response(“zk.html”, {
“mainid”: mainList,
“mainList”: json.dumps(mainList),
“seriesList”: json.dumps(seriesList),
“lineList”: json.dumps(lineList),
“timeList”: json.dumps(timeList)
})

#获取变量lineList和timeList的列表值
def get_line_and_time():
main()
print ‘–call get_line_and_time():’
mainList = getmainList()
lineList = get_series_or_line(“lineList”)
mntrdata = Targetmntr.objects.order_by(“time”)
consdata = Targetcons.objects.order_by(“time”)
timeList = []

# 得到timeList、lineList
for m in range(len(mainList)):
timeList.append([])
command, site = splitmain(mainList[m])
current = datetime.datetime.strptime(‘1970-01-01 00:00:01’, ‘%Y-%m-%d %H:%M:%S’)
if command == “mmtr”:
for d in mntrdata:
t = datetime.datetime.strptime(d.time.strftime(“%Y-%m-%d %H:%M:%S”), ‘%Y-%m-%d %H:%M:%S’)
if current < t:
current = t
timeList[m].append(d.time.strftime(“%Y-%m-%d %H:%M:%S”))
for l in lineList[m]:
l[“data”].append(0)
for l in lineList[m]:
if d.server == l[“server”] and d.key == l[“key”]:
l[“data”][-1] = int(d.value)
else:
for d in consdata:
t = datetime.datetime.strptime(d.time.strftime(“%Y-%m-%d %H:%M:%S”), ‘%Y-%m-%d %H:%M:%S’)
if current < t:
current = t
timeList[m].append(d.time.strftime(“%Y-%m-%d %H:%M:%S”))
for l in lineList[m]:
l[“data”].append(0)
for l in lineList[m]:
if d.server == l[“server”] and d.key == l[“key”]:
l[“data”][-1] = int(d.value)

return timeList, lineList

#获取变量seriesList或lineList的列表值
def get_series_or_line(x):
print ‘—-call get_series_or_line(‘, x, ‘):’
mainList = getmainList()
servertable = Controlserver.objects.values(“server”)
keytable = Controlkey.objects.order_by(“site”)

List = []
for m in range(len(mainList)):
List.append([])
command, site = splitmain(mainList[m])
for k in keytable:
if k.site == site:
for s in servertable:
if x == “seriesList”:
Json = {
“type”: “line”,
“name”: s[“server”] + “-” + k.key,
“data”: []
}
elif x == “lineList”:
Json = {
‘server’: s[‘server’],
‘key’: k.key,
‘lineName’: s[“server”] + “-” + k.key,
‘data’: []
}
List[m].append(Json)
return List

#获取变量mainList的列表值
def getmainList():
print ‘—-call getmainList():’
mainList = []
for k in Controlkey.objects.order_by(“site”):
main = k.command+k.site
if main not in mainList:
mainList.append(main)
return mainList

#将变量main拆分
def splitmain(main):
print ‘—-call splitmain(‘, main, ‘):’
command = main[0:4]
site = main[4:]
return command, site

 

5、static静态文件载入

在目录zkTarget下新建与配置文件默认的固定目录名为static的目录存放静态文件。

将js文件放到此目录下。目录结构如下:

mysite/

zkTarget/

static/

js/

echarts.min.js

jquery-3.2.1.min.js

6、模板(Template),即业务逻辑层。目录:zkTarget/templates

父模板base.html:

<!–基础模板–>
<!DOCTYPE html>
<html>
<head>
<meta charset=”utf-8″>

<!–载入js包–>
{% load static %}
<script src=”{% static “js/jquery-3.2.1.min.js” %}” type=”text/javascript”></script>
<script src=”{% static “js/echarts.min.js” %}” type=”text/javascript”></script>

<title>
<!–定义父模板标签:title–>
{% block title %}
{% endblock %}
</title>
</head>
<body>
<h1>zookeeper四字命令监控指标</h1>
<h3>–echarts2.0 django1.11.14 python2.7.5</h3>

<!–定义父模板标签:content–>
{% block content %}
{% endblock %}

</body>
</html>

 

子模板zk.html:

<!–页面内容模板–>

<!–extends字段:装载父模板base.html–>
{% extends “base.html” %}

<!–重载父模板的块:content,开始标签为:block 块名–>
{% block content %}

<!–使用py后端传输过来的json键:mainid,获取对应的值–>
{% for main in mainid %}
<div id=”{{ main }}” style=”width: 1300px;height:200px;float:left;”></div>
{% endfor %}
<script type=”text/javascript”>

//载入图像
load();

//每分钟更新一次页面。或者用HTML DOM setInterval() 方法
setTimeout(function () {
window.location.reload();
},60000);//1秒=1000毫秒

//载入图像方法
function load(){
//使用py后端传输过来的json键:mainList、seriesList、lineList、timeList,获取对应的值
//Django出于安全考虑对接收的数据中大于号、小于号、单引号和“&”5个符号自动转义,所以视图接收数据时,用safe过滤器关闭自动转义
var mainList = {{ mainList|safe }};
var seriesList = {{ seriesList|safe }};
var lineList = {{ lineList|safe }};
var timeList = {{ timeList|safe }};

for (var n=0;n<mainList.length;n++){
var myChart = echarts.init(document.getElementById(mainList[n]));

//图表的配置项和数据
myChart.setOption({

//标题
title: {
text: mainList[n]
},
tooltip: {},
legend: {},
dataset: {},

//x轴信息
xAxis: {
type: ‘category’,
splitNumber:10,
data: []
},

//y轴信息
yAxis: {
max: function(value) {
return value.max*1.01;
},
min: function(value) {
return value.min*0.99;
}
},

//图例信息
series: seriesList[n]
});

myChart.showLoading();

//获取valueList
var valueList = [];
for (var l=0;l<lineList[n].length;l++){
valueList.push({
type: ‘line’,
name: lineList[n][l][“lineName”],
data: lineList[n][l][“data”]
});
}

//使用刚指定的配置项和数据显示图表。
myChart.setOption({
xAxis: {
data: timeList[n]
},

//系列列表。通过健type决定图表类型,name决定系列名称,data决定系列中的数据内容数组
series: valueList
});

myChart.hideLoading();
}
}

</script>

<!–重载父模板的块:content,结束标签为:endblock–>
{% endblock %}

 

配置管理页面

创建一个管理员账号:

$ python manage.py createsuperuser     Username: adminEmail address: admin@localhostPassword: a12345678Password (again): a12345678

将Models数据库加入到Admin管理中。在admin.py文件中插入:

from zkTarget.models import Targetmntr, Targetcons, Controlkey, Controlserver
admin.site.register(Targetmntr)
admin.site.register(Targetcons)
admin.site.register(Controlkey)
admin.site.register(Controlserver)

启动项目

$ python manage.py runserver

http://127.0.0.1:8000/

django学习网址

  • Django官网(不容易理解,可作为api查询)。https://docs.djangoproject.com/zh-hans/2.0/contents/
  • The Django Book网(讲解非常详细)。http://djangobook.py3k.cn/2.0/
  • 自学学堂网(讲解不详细,实例较为丰富)。https://code.ziqiangxuetang.com/django/django-tutorial.html

本文地址: http://www.xiongge.club/1352.html

转载请注明:熊哥club » django实现zookeeper的监控

喜欢 (1)
[您的支持是我最大的动力]
分享 (0)
发表我的评论
取消评论
表情

Hi,您需要填写昵称和邮箱!

  • 昵称 (必填)
  • 邮箱 (必填)
  • 网址
×
订阅图标按钮