Merge pull request #27 from dj-wasabi/release_0.7.0

Updating to release 0.7.0

.travis.yml

@@ -9,9 +9,12 @@ before_install:
 - sudo apt-get install -o Dpkg::Options::="--force-confold" --force-yes -y docker-engine
 
 install:
-- pip install molecule
+- pip install molecule ansible docker-py
 
 script:
+- molecule --version
+- ansible --version
 - molecule test
+
 notifications:
   webhooks: https://galaxy.ansible.com/api/v1/notifications/

CHANGELOG.md

@@ -5,6 +5,32 @@ Below an overview of all changes in the releases.
 
 Version (Release date)
 
+0.7.0 (2017-02-23)
+
+  * Replace action by modules #26 (By pull request: tszym (Thanks!))
+  * Use yum repository to install telegraf on RedHat #25 (By pull request: tszym (Thanks!))
+  * Remove for-loop in extra-plugin template #24 (By pull request: emersondispatch (Thanks!))
+  * Update Debian.yml #23 (By pull request: zend0 (Thanks!))
+  * extra plugins tags #21 (By pull request: oboukili (Thanks!))
+  * Input tags support #20 (By pull request: szibis (Thanks!))
+  * Fix telegraf confguration permissions #19 (By pull request: szibis (Thanks!))
+
+0.6.0 (2017-01-02)
+
+  * Fix the Influxdb repo for "hybrid" debian distros (like "jessie/sid") #9 (By pull request: Ismael (Thanks!))
+  * Do "become" for the steps that require root access on Debian #10 (By pull request: Ismael (Thanks!))
+  * Fix the Influxdb repo for "hybrid" debian distros (like "jessie/sid") #11 (By pull request: Ismael (Thanks!))
+  * Removed imports #12
+  * Fixing molecule #15
+  * set telegraf hostname in defaults. #13 (By pull request: romainbureau (Thanks!))
+  * use version_compare filter … #14 (By pull request: lhoss (Thanks!))
+  * support missing agent settings upto telegraf v1.1 #16 (By pull request: lhoss (Thanks!))
+  * update the README with the latest v0.13 - v1.1 agent settings #17 (By pull request: lhoss (Thanks!))
+
+0.5.1 (2016-08-24)
+
+  * fixed issue with ansible not getting the package #6 (By pull request: thecodeassassin (Thanks!))
+
 0.5.0 (2016-07-17)
 
   * Removed Test Kitchen tests

README.md

@@ -1,7 +1,6 @@
-dj-wasabi.telegraf
-=========
+# dj-wasabi.telegraf
 
-Build status:
+## Build status:
 
 [![Build Status](https://travis-ci.org/dj-wasabi/ansible-telegraf.svg?branch=master)](https://travis-ci.org/dj-wasabi/ansible-telegraf)
 
@@ -13,24 +12,38 @@ Design goals are to have a minimal memory footprint with a plugin system so that
 
 (https://github.com/influxdb/telegraf)
 
-Requirements
-------------
+## Requirements
+
 
 No requirements. (Yes, an Influxdb server somewhere on the network will help though ;-) )
 
-Role Variables
---------------
+## Upgrade
+### 0.7.0
+
+There was an issue:
+
+    If I configure a telegraf_plugins_extra, run ansible, delete the plugin and run ansible again, the plugin stays on the machine.
+
+
+
+## Role Variables
 
 The following parameters can be set for the Telegraf agent:
 
 * `telegraf_agent_version`: The version of Telegraf to install. Default: `1.0.0`
-* `telegraf_agent_rpm_url`: The full path to the RPM file located on a webserver. 
-* `telegraf_agent_deb_url`:  The full path to the DEB file located on a webserver.
 * `telegraf_agent_interval`: The interval configured for sending data to the server. Default: `10`
-* `telegraf_agent_debug`: Setting the Telegraf in debug mode. Default: `False`
+* `telegraf_agent_debug`: Run Telegraf in debug mode. Default: `False`
 * `telegraf_agent_round_interval`: Rounds collection interval to 'interval' Default: True
 * `telegraf_agent_flush_interval`: Default data flushing interval for all outputs. Default: 10
 * `telegraf_agent_flush_jitter`: Jitter the flush interval by a random amount. Default: 0
+* `telegraf_agent_collection_jitter`: Jitter the collection by a random amount. Default: 0 (since v0.13)
+* `telegraf_agent_metric_batch_size`: The agent metric batch size. Default: 1000  (since v0.13)
+* `telegraf_agent_metric_buffer_limit`: The agent metric buffer limit. Default: 10000  (since v0.13)
+* `telegraf_agent_quiet`: Run Telegraf in quiet mode (error messages only). Default: `False` (since v0.13)
+* `telegraf_agent_logfile`: The agent logfile name. Default: '' (means to log to stdout) (since v1.1)
+* `telegraf_agent_omit_hostname`: Do no set the "host" tag in the agent. Default: `False` (since v1.1)
+
+Full agent settings reference: https://github.com/influxdata/telegraf/blob/master/docs/CONFIGURATION.md#agent-configuration
 
 You can set tags for the host running telegraf:
 
@@ -45,6 +58,8 @@ Specifying an output. The default is set to localhost, you'll have to specify th
 	    config:
 	      - urls = ["http://localhost:8086"]
 	      - database = "telegraf"
+            tagpass:
+              - diskmetrics = ["true"]
 
 The config will be printed line by line into the configuration, so you could also use:
 
@@ -71,7 +86,7 @@ With the property `telegraf_plugins_default` it is set to use the default set of
 	  - plugin: swap
 	  - plugin: netstat
 
-Every telegraf agent has these as an default configuration.
+Every telegraf agent has these as a default configuration.
 
 The 2nd parameter `telegraf_plugins_extra` can be used to add plugins specific to the servers goal. Following is an example for using this parameter for MySQL database servers:
 
@@ -84,6 +99,7 @@ The 2nd parameter `telegraf_plugins_extra` can be used to add plugins specific t
 
 Telegraf plugin options:
 
+* `tags` An k/v tags to apply to a specific input's measurements. Can be used on any stage for better filtering for example in outputs.
 * `pass`: An array of strings that is used to filter metrics generated by the current plugin. Each string in the array is tested as a prefix against metric names and if it matches, the metric is emitted.
 * `drop`: The inverse of pass, if a metric name matches, it is not emitted.
 * `tagpass`: (added in Telegraf 0.1.5) tag names and arrays of strings that are used to filter metrics by the current plugin. Each string in the array is tested as an exact match against the tag name, and if it matches the metric is emitted.
@@ -95,18 +111,19 @@ An example might look like this:
 	telegraf_plugins_default:
 	  - plugin: disk
 	    interval: 12
+            tags:
+                - diskmetrics = "true"
 	    tagpass:
 	      - fstype = [ "ext4", "xfs" ]
     	  - path = [ "/opt", "/home" ]
 
 
 
-Dependencies
-------------
+## Dependencies
+
 No dependencies
 
-Example Playbook
-----------------
+## Example Playbook
 
     - hosts: servers
       roles:
@@ -117,18 +134,20 @@ The following have contributed to this Ansible role:
 
   * aferrari-technisys
   * stvnwrgs
+  * lhoss
+  * thecodeassassin
+  * Ismael
+  * romainbureau
 
-# Molecule
+## Molecule
 
 This roles is configured to be tested with Molecule. You can find on this page some more information regarding Molecule: https://werner-dijkerman.nl/2016/07/10/testing-ansible-roles-with-molecule-testinfra-and-docker/
 
-License
--------
+## License
 
 BSD
 
-Author Information
-------------------
+## Author Information
 
 Please let me know if you have issues. Pull requests are also accepted! :-)
 

defaults/main.yml

@@ -3,16 +3,22 @@
 
 #telegraf_agent_version: 0.10.1
 telegraf_agent_version: 1.0.0
-telegraf_agent_version_sub_l: _beta2
-telegraf_agent_version_sub_u: -beta2
+telegraf_agent_hostname: "{{ ansible_fqdn }}"
 telegraf_agent_interval: 10
 telegraf_agent_debug: False
 telegraf_agent_round_interval: True
 telegraf_agent_flush_interval: 10
 telegraf_agent_flush_jitter: 0
 
-telegraf_agent_rpm_url: https://dl.influxdata.com/telegraf/releases/telegraf-{{ telegraf_agent_version }}{{ telegraf_agent_version_sub_l }}.x86_64.rpm
-telegraf_agent_deb_url: https://dl.influxdata.com/telegraf/releases/telegraf_{{ telegraf_agent_version }}{{ telegraf_agent_version_sub_u }}_amd64.deb
+#v0.13 settings (not sure if supported in older version):
+telegraf_agent_collection_jitter: 0
+telegraf_agent_metric_batch_size: 1000
+telegraf_agent_metric_buffer_limit: 10000
+telegraf_agent_quiet: False
+
+#v1.1 settings:
+telegraf_agent_logfile: ""
+telegraf_agent_omit_hostname: False
 
 telegraf_agent_tags:
 

handlers/main.yml

@@ -2,8 +2,8 @@
 # handlers file for ansible-telegraf
 
 - name: "Restart Telegraf"
-  action: service
-          name=telegraf
-          state=restarted
-          enabled=yes
-  sudo: yes
+  service:
+    name: telegraf
+    state: restarted
+    enabled: yes
+  become: yes

molecule.yml

@@ -2,6 +2,12 @@
 ansible:
   playbook: playbook.yml
 
+driver:
+  name: docker
+
+verifier:
+  name: testinfra
+
 docker:
   containers:
   - name: telegraf-centos

playbook.yml

@@ -2,14 +2,14 @@
 - hosts: all
   pre_tasks:
     - name: "Installing which on CentOS"
-      action: yum
-              name=which
-              state=installed
+      yum:
+        name: which
+        state: installed
       when: ansible_distribution == 'CentOS'
     - name: "Installing wget on Debian"
-      action: apt
-              name=wget
-              state=installed
+      apt:
+        name: wget
+        state: installed
       when: ansible_distribution == 'Debian'
   roles:
     - role: ansible-telegraf

tasks/Debian.yml

@@ -1,14 +1,60 @@
 ---
 
-- name: "Fetch telegraf installation package | Debian"
-  action: get_url
-          url={{ telegraf_agent_deb_url }}
-          dest=/tmp/telegraf_{{ telegraf_agent_version }}-1_amd64.deb
-          mode=0440
+- name: Ensure the system can use the HTTPS transport for APT.
+  stat:
+    path: /usr/lib/apt/methods/https
+  register: apt_https_transport
+  tags:
+    - telegraf
+    - packages
+
+- name: Install APT HTTPS transport.
+  apt:
+    name: "apt-transport-https"
+    state: present
+  when: not apt_https_transport.stat.exists
+  become: yes
+  tags:
+    - telegraf
+    - packages
+
+- name: Download Influxdb apt key.
+  apt_key:
+    url: "https://repos.influxdata.com/influxdb.key"
+    state: present
+  become: yes
+  tags:
+    - telegraf
+    - packages
+
+- name: Add Influxdb repository (using LSB).
+  apt_repository:
+    repo: "deb https://repos.influxdata.com/{{ ansible_distribution|lower }} {{ ansible_lsb.codename }} stable"
+    filename: "influxdb"
+    state: present
+  become: yes
+  tags:
+    - telegraf
+    - packages
+  when: ansible_lsb is defined
+
+- name: Add Influxdb repository.
+  apt_repository:
+    repo: "deb https://repos.influxdata.com/{{ ansible_distribution|lower }} {{ ansible_distribution_release }} stable"
+    filename: "influxdb"
+    state: present
+  become: yes
+  tags:
+    - telegraf
+    - packages
+  when: ansible_lsb is not defined
 
 - name: "Install telegraf package | Debian"
-  action: apt
-          deb=/tmp/telegraf_{{ telegraf_agent_version }}-1_amd64.deb
-          state=installed
+  apt:
+    name: telegraf
+    state: installed
   notify: "Restart Telegraf"
-  sudo: yes
+  become: yes
+  tags:
+  - telegraf
+  - packages

tasks/RedHat.yml

@@ -1,14 +1,14 @@
 ---
 
 - name: "Enable EPEL"
-  action: yum
-          name=epel-release
-          state=installed
+  yum:
+    name: epel-release
+    state: installed
 
 - name: "Installing some dependencies"
-  action: yum
-          name={{ item }}
-          state=installed
+  yum:
+    name: "{{ item }}"
+    state: installed
   with_items:
     - python-pip
     - python-devel
@@ -17,22 +17,24 @@
     - openssl-devel
 
 - name: "Installing PIP dependensies"
-  action: pip
-          name={{ item }}
-          state=present
+  pip:
+    name: "{{ item }}"
+    state: present
   with_items:
     - urllib3
     - pyopenssl
     - ndg-httpsclient
 
-- name: "Fetch telegraf installation package | RedHat"
-  action: get_url
-          url="{{ telegraf_agent_rpm_url }}"
-          dest=/tmp/telegraf_{{ telegraf_agent_version }}.x86_64.rpm
-          mode=0440
+- name: "Add yum repository | RedHat"
+  yum_repository:
+    name: influxdb
+    description: InfluxDB Repository - RHEL $releasever
+    baseurl: https://repos.influxdata.com/rhel/$releasever/$basearch/stable
+    gpgcheck: yes
+    gpgkey: https://repos.influxdata.com/influxdb.key
 
 - name: "Install telegraf package | RedHat"
-  action: yum
-          name=/tmp/telegraf_{{ telegraf_agent_version }}.x86_64.rpm
-          state=installed
+  yum:
+    name: "telegraf-{{ telegraf_agent_version }}"
+    state: installed
   notify: "Restart Telegraf"

tasks/main.yml

@@ -9,40 +9,35 @@
   include: "Debian.yml"
   when: ansible_os_family == "Debian"
 
-- name: "Set fact by remove the dots from version"
-  set_fact:
-    telegraf_ver: "{{ telegraf_agent_version.replace('.','') }}"
-
 - name: "Copy the template for versions < 0.10.0"
   template:
     src: etc-opt-telegraf-telegraf.conf.j2
     dest: /etc/opt/telegraf/telegraf.conf
-    owner: root
-    group: root
-    mode: 0644
-  when: telegraf_ver < 100
-  sudo: yes
+    owner: telegraf
+    group: telegraf
+    mode: 0640
+  become: yes
+  when: telegraf_agent_version|version_compare('0.10.0', '<')
   notify: "Restart Telegraf"
 
 - name: "Copy the template for versions >= 0.10.0"
   template:
     src: telegraf.conf.j2
     dest: /etc/telegraf/telegraf.conf
-    owner: root
-    group: root
-    mode: 0644
-  when: telegraf_ver >= 100
-  sudo: yes
+    owner: telegraf
+    group: telegraf
+    mode: 0640
+  become: yes
+  when: telegraf_agent_version|version_compare('0.10.0', '>=')
   notify: "Restart Telegraf"
 
 - name: "Copy telegraf extra plugins"
   template:
     src: "telegraf-extra-plugin.conf.j2"
-    dest: "/etc/telegraf/telegraf.d/{{ item.plugin }}.conf"
-    owner: root
-    group: root
-    mode: 644
-  with_items: "{{ telegraf_plugins_extra }}"
+    dest: "/etc/telegraf/telegraf.d/extra-plugins.conf"
+    owner: telegraf
+    group: telegraf
+    mode: 0640
   when: "telegraf_plugins_extra is defined and telegraf_plugins_extra is iterable"
-  sudo: yes
+  become: yes
   notify: "Restart Telegraf"

templates/telegraf-extra-plugin.conf.j2

@@ -1,4 +1,3 @@
-
 {% if telegraf_plugins_extra is defined and telegraf_plugins_extra is iterable %}
 {% for item in telegraf_plugins_extra %}
 [[inputs.{{ item.plugin }}]]
@@ -10,6 +9,12 @@
     {{ items }}
 {% endfor %}
 {% endif %}
+{% if item.tags is defined and item.tags is iterable %}
+[inputs.{{ item.plugin }}.tags]
+{% for items in item.tags %}
+    {{ items }}
+{% endfor %}
+{% endif %}
 {% if item.tagpass is defined and item.tagpass is iterable %}
 [{{ item.plugin }}.tagpass]
 {% for items in item.tagpass %}

templates/telegraf.conf.j2

@@ -11,10 +11,20 @@
 [agent]
     interval = "{{ telegraf_agent_interval }}s"
     debug = {{ telegraf_agent_debug | lower }}
-    hostname = "{{ ansible_fqdn }}"
+    hostname = "{{ telegraf_agent_hostname }}"
     round_interval = {{ telegraf_agent_round_interval | lower }}
     flush_interval = "{{ telegraf_agent_flush_interval }}s"
     flush_jitter = "{{ telegraf_agent_flush_jitter }}s"
+{% if telegraf_agent_version|version_compare('0.13', '>=') %}
+    collection_jitter = "{{ telegraf_agent_collection_jitter }}s"
+    metric_batch_size = {{ telegraf_agent_metric_batch_size }}
+    metric_buffer_limit = {{ telegraf_agent_metric_buffer_limit }}
+    quiet = {{ telegraf_agent_quiet | lower }}
+{% endif %}
+{% if telegraf_agent_version|version_compare('1.1', '>=') %}
+    logfile = "{{ telegraf_agent_logfile }}"
+    omit_hostname = {{ telegraf_agent_omit_hostname | lower }}
+{% endif %}
 
 ###############################################################################
 #                                  OUTPUTS                                    #
@@ -44,6 +54,12 @@
     {{ items }}
 {% endfor %}
 {% endif %}
+{% if item.tags is defined and item.tags is iterable %}
+[inputs.{{ item.plugin }}.tags]
+{% for items in item.tags %}
+    {{ items }}
+{% endfor %}
+{% endif %}
 {% if item.tagpass is defined and item.tagpass is iterable %}
 [{{ item.plugin }}.tagpass]
 {% for items in item.tagpass %}

tests/test_docker.py

@@ -1,3 +1,7 @@
+from testinfra.utils.ansible_runner import AnsibleRunner
+
+testinfra_hosts = AnsibleRunner('.molecule/ansible_inventory').get_hosts('all')
+
 
 def test_telegraf_running_and_enabled(Service, SystemInfo):
     telegraf = Service("telegraf")
@@ -8,9 +12,9 @@ def test_telegraf_running_and_enabled(Service, SystemInfo):
 
 def test_telegraf_dot_conf(File):
     telegraf = File("/etc/telegraf/telegraf.conf")
-    assert telegraf.user == "root"
-    assert telegraf.group == "root"
-    assert telegraf.mode == 0o644
+    assert telegraf.user == "telegraf"
+    assert telegraf.group == "telegraf"
+    assert telegraf.mode == 0o640
     assert telegraf.contains('[[inputs.cpu]]')