Updating to 0.8.0

Use telegra_global_tags for oldest telegraf versions

.travis.yml

@@ -4,17 +4,12 @@ language: python
 services:
   - docker
 
-before_install:
-- sudo apt-get -qq update
-- sudo apt-get install -o Dpkg::Options::="--force-confold" --force-yes -y docker-engine
-
 install:
-- pip install molecule ansible docker-py
+  - pip install molecule ansible docker
 
 script:
-- molecule --version
-- ansible --version
-- molecule test
-
+  - molecule --version
+  - ansible --version
+  - molecule test
 notifications:
   webhooks: https://galaxy.ansible.com/api/v1/notifications/

CHANGELOG.md

@@ -5,6 +5,24 @@ Below an overview of all changes in the releases.
 
 Version (Release date)
 
+0.8.0 (2017-10-30)
+
+  * Updating to Molecule V2
+  * Test if LSB codename exists before using it #35 (By pull request: tszym (Thanks!))
+  * Remove useless packages on RedHat. fix #28 #36 (By pull request: tszym (Thanks!))
+  * Fix extra plugins by file / Change apt source filename / Change tags by global_tags #37 (By pull request: aarnaud (Thanks!))
+  * Use telegra_global_tags for oldest telegraf versions #38 (By pull request: tszym (Thanks!))
+
+0.7.0 (2017-02-23)
+
+  * Replace action by modules #26 (By pull request: tszym (Thanks!))
+  * Use yum repository to install telegraf on RedHat #25 (By pull request: tszym (Thanks!))
+  * Remove for-loop in extra-plugin template #24 (By pull request: emersondispatch (Thanks!))
+  * Update Debian.yml #23 (By pull request: zend0 (Thanks!))
+  * extra plugins tags #21 (By pull request: oboukili (Thanks!))
+  * Input tags support #20 (By pull request: szibis (Thanks!))
+  * Fix telegraf confguration permissions #19 (By pull request: szibis (Thanks!))
+
 0.6.0 (2017-01-02)
 
   * Fix the Influxdb repo for "hybrid" debian distros (like "jessie/sid") #9 (By pull request: Ismael (Thanks!))

README.md

@@ -1,7 +1,6 @@
-dj-wasabi.telegraf
-=========
+# dj-wasabi.telegraf
 
-Build status:
+## Build status:
 
 [![Build Status](https://travis-ci.org/dj-wasabi/ansible-telegraf.svg?branch=master)](https://travis-ci.org/dj-wasabi/ansible-telegraf)
 
@@ -13,19 +12,25 @@ Design goals are to have a minimal memory footprint with a plugin system so that
 
 (https://github.com/influxdb/telegraf)
 
-Requirements
-------------
+## Requirements
+
 
 No requirements. (Yes, an Influxdb server somewhere on the network will help though ;-) )
 
-Role Variables
---------------
+## Upgrade
+### 0.7.0
+
+There was an issue:
+
+    If I configure a telegraf_plugins_extra, run ansible, delete the plugin and run ansible again, the plugin stays on the machine.
+
+
+
+## Role Variables
 
 The following parameters can be set for the Telegraf agent:
 
 * `telegraf_agent_version`: The version of Telegraf to install. Default: `1.0.0`
-* `telegraf_agent_rpm_url`: The full path to the RPM file located on a webserver. 
-* `telegraf_agent_deb_url`:  The full path to the DEB file located on a webserver.
 * `telegraf_agent_interval`: The interval configured for sending data to the server. Default: `10`
 * `telegraf_agent_debug`: Run Telegraf in debug mode. Default: `False`
 * `telegraf_agent_round_interval`: Rounds collection interval to 'interval' Default: True
@@ -42,7 +47,7 @@ Full agent settings reference: https://github.com/influxdata/telegraf/blob/maste
 
 You can set tags for the host running telegraf:
 
-	telegraf_agent_tags:
+	telegraf_global_tags:
 	  - tag_name: some_name
 	    tag_value: some_value
 
@@ -53,6 +58,8 @@ Specifying an output. The default is set to localhost, you'll have to specify th
 	    config:
 	      - urls = ["http://localhost:8086"]
 	      - database = "telegraf"
+            tagpass:
+              - diskmetrics = ["true"]
 
 The config will be printed line by line into the configuration, so you could also use:
 
@@ -79,8 +86,8 @@ With the property `telegraf_plugins_default` it is set to use the default set of
 	  - plugin: swap
 	  - plugin: netstat
 
-Every telegraf agent has these as an default configuration.
- 
+Every telegraf agent has these as a default configuration.
+
 The 2nd parameter `telegraf_plugins_extra` can be used to add plugins specific to the servers goal. Following is an example for using this parameter for MySQL database servers:
 
 	cat group_vars/mysql_database
@@ -92,6 +99,7 @@ The 2nd parameter `telegraf_plugins_extra` can be used to add plugins specific t
 
 Telegraf plugin options:
 
+* `tags` An k/v tags to apply to a specific input's measurements. Can be used on any stage for better filtering for example in outputs.
 * `pass`: An array of strings that is used to filter metrics generated by the current plugin. Each string in the array is tested as a prefix against metric names and if it matches, the metric is emitted.
 * `drop`: The inverse of pass, if a metric name matches, it is not emitted.
 * `tagpass`: (added in Telegraf 0.1.5) tag names and arrays of strings that are used to filter metrics by the current plugin. Each string in the array is tested as an exact match against the tag name, and if it matches the metric is emitted.
@@ -103,18 +111,19 @@ An example might look like this:
 	telegraf_plugins_default:
 	  - plugin: disk
 	    interval: 12
+            tags:
+                - diskmetrics = "true"
 	    tagpass:
 	      - fstype = [ "ext4", "xfs" ]
     	  - path = [ "/opt", "/home" ]
 
 
 
-Dependencies
-------------
+## Dependencies
+
 No dependencies
 
-Example Playbook
-----------------
+## Example Playbook
 
     - hosts: servers
       roles:
@@ -130,17 +139,15 @@ The following have contributed to this Ansible role:
   * Ismael
   * romainbureau
 
-# Molecule
+## Molecule
 
 This roles is configured to be tested with Molecule. You can find on this page some more information regarding Molecule: https://werner-dijkerman.nl/2016/07/10/testing-ansible-roles-with-molecule-testinfra-and-docker/
 
-License
--------
+## License
 
 BSD
 
-Author Information
-------------------
+## Author Information
 
 Please let me know if you have issues. Pull requests are also accepted! :-)
 

defaults/main.yml

@@ -1,31 +1,25 @@
 ---
 # defaults file for ansible-telegraf
 
-#telegraf_agent_version: 0.10.1
-telegraf_agent_version: 1.0.0
+telegraf_agent_version: 1.4.0
 telegraf_agent_hostname: "{{ ansible_fqdn }}"
-telegraf_agent_version_sub_l: _beta2
-telegraf_agent_version_sub_u: -beta2
 telegraf_agent_interval: 10
 telegraf_agent_debug: False
 telegraf_agent_round_interval: True
 telegraf_agent_flush_interval: 10
 telegraf_agent_flush_jitter: 0
 
-#v0.13 settings (not sure if supported in older version):
+# v0.13 settings (not sure if supported in older version):
 telegraf_agent_collection_jitter: 0
 telegraf_agent_metric_batch_size: 1000
 telegraf_agent_metric_buffer_limit: 10000
 telegraf_agent_quiet: False
 
-#v1.1 settings:
+# v1.1 settings:
 telegraf_agent_logfile: ""
 telegraf_agent_omit_hostname: False
 
-telegraf_agent_rpm_url: https://dl.influxdata.com/telegraf/releases/telegraf-{{ telegraf_agent_version }}{{ telegraf_agent_version_sub_l }}.x86_64.rpm
-telegraf_agent_deb_url: https://dl.influxdata.com/telegraf/releases/telegraf_{{ telegraf_agent_version }}{{ telegraf_agent_version_sub_u }}_amd64.deb
-
-telegraf_agent_tags:
+telegraf_global_tags: []
 
 telegraf_agent_output:
   - type: influxdb

handlers/main.yml

@@ -2,8 +2,8 @@
 # handlers file for ansible-telegraf
 
 - name: "Restart Telegraf"
-  action: service
-          name=telegraf
-          state=restarted
-          enabled=yes
-  become: yes
+  service:
+    name: telegraf
+    state: restarted
+    enabled: yes
+  become: yes

meta/main.yml

@@ -6,15 +6,15 @@ galaxy_info:
   license: license BSD
   min_ansible_version: 1.2
   platforms:
-  - name: EL
-    versions:
-     - all
-  - name: Ubuntu
-    versions:
-    - all
-  - name: Debian
-    versions:
-    - all
+    - name: EL
+      versions:
+        - all
+    - name: Ubuntu
+      versions:
+        - all
+    - name: Debian
+      versions:
+        - all
   categories:
-   - monitoring
-dependencies: []
+    - monitoring
+dependencies: []

molecule.yml

@@ -1,24 +0,0 @@
----
-ansible:
-  playbook: playbook.yml
-
-driver:
-  name: docker
-
-verifier:
-  name: testinfra
-
-docker:
-  containers:
-  - name: telegraf-centos
-    ansible_groups:
-      - group1
-    image: milcom/centos7-systemd
-    image_version: latest
-    privileged: True
-  - name: telegraf-debian
-    ansible_groups:
-      - group2
-    image: maint/debian-systemd
-    image_version: latest
-    privileged: True

molecule/default/Dockerfile.j2

@@ -0,0 +1,7 @@
+FROM {{ item.image }}
+
+RUN if [ $(command -v apt-get) ]; then apt-get update && apt-get upgrade -y && apt-get install -y python sudo bash ca-certificates && apt-get clean; \
+    elif [ $(command -v dnf) ]; then dnf makecache && dnf --assumeyes install python python-devel python2-dnf bash && dnf clean all; \
+    elif [ $(command -v yum) ]; then yum makecache fast && yum update -y && yum install -y python sudo yum-plugin-ovl bash && sed -i 's/plugins=0/plugins=1/g' /etc/yum.conf && yum clean all; \
+    elif [ $(command -v zypper) ]; then zypper refresh && zypper update -y && zypper install -y python sudo bash python-xml && zypper clean -a; \
+    elif [ $(command -v apk) ]; then apk update && apk add --no-cache python sudo bash ca-certificates; fi

molecule/default/INSTALL.rst

@@ -0,0 +1,46 @@
+*******
+Install
+*******
+
+This set of playbooks have specific dependencies on Ansible due to the modules
+being used.
+
+Requirements
+============
+
+* Ansible 2.2
+* Docker Engine
+* docker-py
+
+Install OS dependencies on CentOS 7
+
+.. code-block:: bash
+
+  $ sudo yum install -y epel-release
+  $ sudo yum install -y gcc python-pip python-devel openssl-devel
+  # If installing Molecule from source.
+  $ sudo yum install libffi-devel git
+
+Install OS dependencies on Ubuntu 16.x
+
+.. code-block:: bash
+
+  $ sudo apt-get update
+  $ sudo apt-get install -y python-pip libssl-dev docker-engine
+  # If installing Molecule from source.
+  $ sudo apt-get install -y libffi-dev git
+
+Install OS dependencies on Mac OS
+
+.. code-block:: bash
+
+  $ brew install python
+  $ brew install git
+
+Install using pip:
+
+.. code-block:: bash
+
+  $ sudo pip install ansible
+  $ sudo pip install docker-py
+  $ sudo pip install molecule --pre

molecule/default/create.yml

@@ -0,0 +1,47 @@
+---
+- name: Create
+  hosts: localhost
+  connection: local
+  gather_facts: False
+  no_log: "{{ not lookup('env', 'MOLECULE_DEBUG') | bool }}"
+  vars:
+    molecule_file: "{{ lookup('env', 'MOLECULE_FILE') }}"
+    molecule_ephemeral_directory: "{{ lookup('env', 'MOLECULE_EPHEMERAL_DIRECTORY') }}"
+    molecule_scenario_directory: "{{ lookup('env', 'MOLECULE_SCENARIO_DIRECTORY') }}"
+    molecule_yml: "{{ lookup('file', molecule_file) | from_yaml }}"
+  tasks:
+    - name: Create Dockerfiles from image names
+      template:
+        src: "{{ molecule_scenario_directory }}/Dockerfile.j2"
+        dest: "{{ molecule_ephemeral_directory }}/Dockerfile_{{ item.image | regex_replace('[^a-zA-Z0-9_]', '_') }}"
+      with_items: "{{ molecule_yml.platforms }}"
+      register: platforms
+
+    - name: Discover local Docker images
+      docker_image_facts:
+        name: "molecule_local/{{ item.item.name }}"
+      with_items: "{{ platforms.results }}"
+      register: docker_images
+
+    - name: Build an Ansible compatible image
+      docker_image:
+        path: "{{ molecule_ephemeral_directory }}"
+        name: "molecule_local/{{ item.item.image }}"
+        dockerfile: "{{ item.item.dockerfile | default(item.invocation.module_args.dest) }}"
+        force: "{{ item.item.force | default(True) }}"
+      with_items: "{{ platforms.results }}"
+      when: platforms.changed or docker_images.results | map(attribute='images') | select('equalto', []) | list | count >= 0
+
+    - name: Create molecule instance(s)
+      docker_container:
+        name: "{{ item.name }}"
+        hostname: "{{ item.name }}"
+        image: "molecule_local/{{ item.image }}"
+        state: started
+        recreate: False
+        log_driver: syslog
+        command: "{{ item.command | default('sleep infinity') }}"
+        privileged: "{{ item.privileged | default(omit) }}"
+        volumes: "{{ item.volumes | default(omit) }}"
+        capabilities: "{{ item.capabilities | default(omit) }}"
+      with_items: "{{ molecule_yml.platforms }}"

molecule/default/destroy.yml

@@ -0,0 +1,16 @@
+---
+- name: Destroy
+  hosts: localhost
+  connection: local
+  gather_facts: False
+  no_log: "{{ not lookup('env', 'MOLECULE_DEBUG') | bool }}"
+  vars:
+    molecule_file: "{{ lookup('env', 'MOLECULE_FILE') }}"
+    molecule_yml: "{{ lookup('file', molecule_file) | from_yaml }}"
+  tasks:
+    - name: Destroy molecule instance(s)
+      docker_container:
+        name: "{{ item.name }}"
+        state: absent
+        force_kill: "{{ item.force_kill | default(True) }}"
+      with_items: "{{ molecule_yml.platforms }}"

molecule/default/molecule.yml

@@ -0,0 +1,28 @@
+---
+dependency:
+  name: galaxy
+driver:
+  name: docker
+lint:
+  name: yamllint
+  options:
+    config-file: molecule/default/yaml-lint.yml
+
+platforms:
+  - name: telegraf-centos
+    image: milcom/centos7-systemd
+    privileged: True
+  - name: telegraf-debian
+    image: maint/debian-systemd
+    privileged: True
+
+provisioner:
+  name: ansible
+  lint:
+    name: ansible-lint
+scenario:
+  name: default
+verifier:
+  name: testinfra
+  lint:
+    name: flake8

molecule/default/playbook.yml

@@ -2,14 +2,14 @@
 - hosts: all
   pre_tasks:
     - name: "Installing which on CentOS"
-      action: yum
-              name=which
-              state=installed
+      yum:
+        name: which
+        state: installed
       when: ansible_distribution == 'CentOS'
     - name: "Installing wget on Debian"
-      action: apt
-              name=wget
-              state=installed
+      apt:
+        name: wget
+        state: installed
       when: ansible_distribution == 'Debian'
   roles:
     - role: ansible-telegraf

molecule/default/tests/test_default.py

@@ -1,6 +1,9 @@
-from testinfra.utils.ansible_runner import AnsibleRunner
+import os
 
-testinfra_hosts = AnsibleRunner('.molecule/ansible_inventory').get_hosts('all')
+import testinfra.utils.ansible_runner
+
+testinfra_hosts = testinfra.utils.ansible_runner.AnsibleRunner(
+    os.environ['MOLECULE_INVENTORY_FILE']).get_hosts('all')
 
 
 def test_telegraf_running_and_enabled(Service, SystemInfo):
@@ -12,9 +15,9 @@ def test_telegraf_running_and_enabled(Service, SystemInfo):
 
 def test_telegraf_dot_conf(File):
     telegraf = File("/etc/telegraf/telegraf.conf")
-    assert telegraf.user == "root"
-    assert telegraf.group == "root"
-    assert telegraf.mode == 0o644
+    assert telegraf.user == "telegraf"
+    assert telegraf.group == "telegraf"
+    assert telegraf.mode == 0o640
     assert telegraf.contains('[[inputs.cpu]]')
 
 

molecule/default/yaml-lint.yml

@@ -0,0 +1,9 @@
+---
+
+extends: default
+
+rules:
+  line-length:
+    max: 120
+    level: warning
+  truthy: disable

tasks/Debian.yml

@@ -18,7 +18,7 @@
     - telegraf
     - packages
 
-- name: Download Influxdb apt key.
+- name: Download Telegraf apt key.
   apt_key:
     url: "https://repos.influxdata.com/influxdb.key"
     state: present
@@ -27,33 +27,34 @@
     - telegraf
     - packages
 
-- name: Add Influxdb repository (using LSB).
+- name: Add Telegraf repository (using LSB).
   apt_repository:
     repo: "deb https://repos.influxdata.com/{{ ansible_distribution|lower }} {{ ansible_lsb.codename }} stable"
-    filename: "influxdb"
-    state: present
-  tags:
-    - telegraf
-    - packages
-  when: ansible_lsb is defined
-
-- name: Add Influxdb repository.
-  apt_repository:
-    repo: "deb https://repos.influxdata.com/{{ ansible_distribution|lower }} {{ ansible_distribution_release }} stable"
-    filename: "influxdb"
+    filename: "telegraf"
     state: present
   become: yes
   tags:
     - telegraf
     - packages
-  when: ansible_lsb is not defined
+  when: ansible_lsb is defined and ansible_lsb.codename is defined
+
+- name: Add Telegraf repository.
+  apt_repository:
+    repo: "deb https://repos.influxdata.com/{{ ansible_distribution|lower }} {{ ansible_distribution_release }} stable"
+    filename: "telegraf"
+    state: present
+  become: yes
+  tags:
+    - telegraf
+    - packages
+  when: ansible_lsb is not defined or ansible_lsb.codename is not defined
 
 - name: "Install telegraf package | Debian"
-  action: apt
-          name=telegraf
-          state=installed
+  apt:
+    name: telegraf
+    state: installed
   notify: "Restart Telegraf"
   become: yes
   tags:
-  - telegraf
-  - packages
+    - telegraf
+    - packages

tasks/RedHat.yml

@@ -1,38 +1,15 @@
 ---
 
-- name: "Enable EPEL"
-  action: yum
-          name=epel-release
-          state=installed
-
-- name: "Installing some dependencies"
-  action: yum
-          name={{ item }}
-          state=installed
-  with_items:
-    - python-pip
-    - python-devel
-    - gcc
-    - libffi-devel
-    - openssl-devel
-
-- name: "Installing PIP dependensies"
-  action: pip
-          name={{ item }}
-          state=present
-  with_items:
-    - urllib3
-    - pyopenssl
-    - ndg-httpsclient
-
-- name: "Fetch telegraf installation package | RedHat"
-  action: get_url
-          url="{{ telegraf_agent_rpm_url }}"
-          dest=/tmp/telegraf_{{ telegraf_agent_version }}.x86_64.rpm
-          mode=0440
+- name: "Add yum repository | RedHat"
+  yum_repository:
+    name: influxdb
+    description: InfluxDB Repository - RHEL $releasever
+    baseurl: https://repos.influxdata.com/rhel/$releasever/$basearch/stable
+    gpgcheck: yes
+    gpgkey: https://repos.influxdata.com/influxdb.key
 
 - name: "Install telegraf package | RedHat"
-  action: yum
-          name=/tmp/telegraf_{{ telegraf_agent_version }}.x86_64.rpm
-          state=installed
+  yum:
+    name: "telegraf-{{ telegraf_agent_version }}"
+    state: installed
   notify: "Restart Telegraf"

tasks/main.yml

@@ -13,9 +13,9 @@
   template:
     src: etc-opt-telegraf-telegraf.conf.j2
     dest: /etc/opt/telegraf/telegraf.conf
-    owner: root
-    group: root
-    mode: 0644
+    owner: telegraf
+    group: telegraf
+    mode: 0640
   become: yes
   when: telegraf_agent_version|version_compare('0.10.0', '<')
   notify: "Restart Telegraf"
@@ -24,9 +24,9 @@
   template:
     src: telegraf.conf.j2
     dest: /etc/telegraf/telegraf.conf
-    owner: root
-    group: root
-    mode: 0644
+    owner: telegraf
+    group: telegraf
+    mode: 0640
   become: yes
   when: telegraf_agent_version|version_compare('0.10.0', '>=')
   notify: "Restart Telegraf"
@@ -35,10 +35,10 @@
   template:
     src: "telegraf-extra-plugin.conf.j2"
     dest: "/etc/telegraf/telegraf.d/{{ item.plugin }}.conf"
-    owner: root
-    group: root
-    mode: 0644
+    owner: telegraf
+    group: telegraf
+    mode: 0640
   with_items: "{{ telegraf_plugins_extra }}"
   when: "telegraf_plugins_extra is defined and telegraf_plugins_extra is iterable"
   become: yes
-  notify: "Restart Telegraf"
+  notify: "Restart Telegraf"

templates/etc-opt-telegraf-telegraf.conf.j2

@@ -1,8 +1,8 @@
 # Telegraf configuration
 
 [tags]
-{% if telegraf_agent_tags is defined and telegraf_agent_tags is iterable %}
-{% for item in telegraf_agent_tags %}
+{% if telegraf_global_tags is defined and telegraf_global_tags is iterable %}
+{% for item in telegraf_global_tags %}
     {{ item.tag_name }} = "{{ item.tag_value }}"
 {% endfor %}
 {% endif %}

templates/telegraf-extra-plugin.conf.j2

@@ -1,6 +1,3 @@
-
-{% if telegraf_plugins_extra is defined and telegraf_plugins_extra is iterable %}
-{% for item in telegraf_plugins_extra %}
 [[inputs.{{ item.plugin }}]]
 {% if item.interval is defined %}
     interval = "{{ item.interval }}s"
@@ -10,6 +7,12 @@
     {{ items }}
 {% endfor %}
 {% endif %}
+{% if item.tags is defined and item.tags is iterable %}
+[inputs.{{ item.plugin }}.tags]
+{% for items in item.tags %}
+    {{ items }}
+{% endfor %}
+{% endif %}
 {% if item.tagpass is defined and item.tagpass is iterable %}
 [{{ item.plugin }}.tagpass]
 {% for items in item.tagpass %}
@@ -40,5 +43,3 @@
     {{ items }}
 {% endfor %}
 {% endif %}
-{% endfor %}
-{% endif %}

templates/telegraf.conf.j2

@@ -1,8 +1,8 @@
 # Telegraf configuration
 
-[tags]
-{% if telegraf_agent_tags is defined and telegraf_agent_tags is iterable %}
-{% for item in telegraf_agent_tags %}
+[global_tags]
+{% if telegraf_global_tags is defined and telegraf_global_tags is iterable %}
+{% for item in telegraf_global_tags %}
     {{ item.tag_name }} = "{{ item.tag_value }}"
 {% endfor %}
 {% endif %}
@@ -54,6 +54,12 @@
     {{ items }}
 {% endfor %}
 {% endif %}
+{% if item.tags is defined and item.tags is iterable %}
+[inputs.{{ item.plugin }}.tags]
+{% for items in item.tags %}
+    {{ items }}
+{% endfor %}
+{% endif %}
 {% if item.tagpass is defined and item.tagpass is iterable %}
 [{{ item.plugin }}.tagpass]
 {% for items in item.tagpass %}

tests/inventory

@@ -1,6 +0,0 @@
-localhost
-[group1]
-telegraf-centos
-
-[group2]
-telegraf-debian

tests/test.yml

@@ -1,5 +0,0 @@
----
-- hosts: localhost
-  remote_user: root
-  roles:
-    - ansible-zabbix-agent