利用prometheus文件服务发现功能,可以方便自定义主机或虚拟机的监控。
   
  1. /etc/prometheus/prometheus.yml配置文件
  文件服务发现配置参考 job_name: 'vm' 段
  # my global config global:   scrape_interval:     15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.   evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.   # scrape_timeout is set to the global default (10s).    # Attach these labels to any time series or alerts when communicating with   # external systems (federation, remote storage, Alertmanager).   external_labels:       monitor: 'codelab-monitor'  # Load rules once and periodically evaluate them according to the global 'evaluation_interval'. rule_files:   # - "first.rules"   # - "second.rules"  # A scrape configuration containing exactly one endpoint to scrape: # Here it's Prometheus itself. scrape_configs:   # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.   - job_name: 'prometheus'      # metrics_path defaults to '/metrics'     # scheme defaults to 'http'.      static_configs:       - targets: ['localhost:9090']  scrape_configs:   - job_name: 'vm'     file_sd_configs:       - files:           - /etc/prometheus/files/vm.yml         refresh_interval: 5s 
  
 2. /etc/prometheus/files/vm.yml配置文件内容
  [   {     "targets": [ "192.168.1.6:9273"],     "labels": {       "hosttype": "vm"     }   } ] 
   
  3. 通过docker启动prometheus
  docker run -d  -p 9090:9090 \ -v /etc/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml \ -v /etc/prometheus/files:/etc/prometheus/files \ prom/prometheus:v1.8.2
   
  4. 检查prometheus的target,是否包含files.yml文件中配置的target
  http://192.168.1.6:9090/targets
  
  5. 动态生成/etc/prometheus/files.yml 
  如果想动态修改target,可以通过http请求,获取主机列表,然后动态生成/etc/prometheus/files/vm.yml文件。
  下面是动态生成vm.yml例子。通过http请求,获取虚拟机的ip地址列表,然后动态生成vm.yml文件。
  #!/usr/bin/python  import json import logging import os import requests import time  logging.basicConfig(level=logging.INFO,                     format='%(asctime)s %(filename)s[line:%(lineno)d] %(levelname)s %(message)s',                     datefmt='%Y-%m-%d %H:%M:%S',                     filename='/var/log/prometheus-vm-http.log',                     filemode='w')  console = logging.StreamHandler() console.setLevel(logging.INFO) formatter = logging.Formatter('%(name)-12s: %(levelname)-8s %(message)s') console.setFormatter(formatter) logging.getLogger('').addHandler(console)  def refresh():     url = "http://域名/api/vm/ip"     resp = requests.get(url, timeout=10)     if resp.status_code == 200:         logging.info("get " + url + " ok")          data = [             {                 "targets": map(lambda x: x + ":9273", resp.json()),                 "labels": {                     "hosttype": "vm"                 }             }         ]          with open('/etc/prometheus/files/vm.yml.new', 'w') as f:             json.dump(data, f, indent=2)             f.flush()             os.fsync(f.fileno())          os.rename('/etc/prometheus/files/vm.yml.new', '/etc/prometheus/files/vm.yml')      else:         logging.error("get " + url + " failed. status_code=%d, resp=%s" % (resp.status_code, resp.text))  if __name__ == '__main__':     while True:         try:             refresh()         except Exception, e:             logging.error(e)          time.sleep(10)