Support for simple custom columns via --cust-cols.

Author: fboender

README.md

@@ -47,6 +47,7 @@ Features
 * Gathered host facts and manual custom facts.
 * Adding and extending facts of existing hosts and manually adding entirely
   new hosts.
+* Custom columns
 
 
 Getting started

docs/usage.md

@@ -36,6 +36,8 @@ You can now open `overview.html` in your browser to view the results.
       -q, --quiet           Don't report warnings
       -c COLUMNS, --columns=COLUMNS
                             Show only given columns
+      -C CUST_COLS, --cust-cols=CUST_COLS
+                            Path to a custom columns definition file
       -l LIMIT, --limit=LIMIT
                             Limit hosts to pattern
       --exclude-cols=EXCLUDE_COLUMNS
@@ -92,7 +94,7 @@ For example, let's say we have the following `hosts` file:
     [cust.megacorp]
     db1.dev.megacorp.com   dtap=dev  comment="Old database server"
     db2.dev.megacorp.com   dtap=dev  comment="New database server"
-    test.megacorp.com      dtap=test 
+    test.megacorp.com      dtap=test
     acc.megacorp.com       dtap=acc  comment="24/7 support"
     megacorp.com           dtap=prod comment="Hosting by Foo" ext_id="SRV_10029"
 
@@ -273,7 +275,8 @@ same as `--columns`. For example:
                  -i hosts \
                  facts/
 
-If you want to add custom columns, please refer to 'Custom templates' section.
+If you want to add custom columns, please refer to [Custom
+columns](#custom-columns) section.
 
 
 ## Extending facts
@@ -285,7 +288,7 @@ hosts.
 
 Extended facts are basically the same as normal Ansible fact files. When you
 specify multiple fact directories, Ansible-cmdb scans all of the in order and
-overlays the facts. 
+overlays the facts.
 
 Note that the host *must still* be present in your hosts file, or it will not
 generate anything.
@@ -424,11 +427,73 @@ Generate the overview:
 
 The software items will be listed under the "*Custom facts*" heading.
 
+
+## Custom columns
+
+You can add custom columns to the host overview with the `-C` (`--cust-cols`)
+option. This allows you to specify
+[jsonxs](https://github.com/fboender/jsonxs) expressions to extract and
+display custom host facts. Such columns are fairly limited in what they can
+display. If you need a more powerful method of adding custom data to your
+CMDB, please refer to the [Custom templates](#custom-templates) section.
+
+Custom columns are currently only supported by the `html_fancy` and
+`html_fancy_split` templates!
+
+The `-C` option takes a parameter which is the path to a JSON file containing
+your custom column definitions. An example can be found in the
+`examples/cust_cols.json` file in the repo:
+
+    [
+        {
+            "title": "AppArmor",
+            "id": "apparmor",
+            "sType": "string",
+            "visible": true,
+            "jsonxs": "ansible_facts.ansible_apparmor.status"
+        },
+        {
+            "title": "Proc type",
+            "id": "proctype",
+            "sType": "string",
+            "visible": true,
+            "jsonxs": "ansible_facts.ansible_processor[2]"
+        }
+    ]
+
+This defines two new columns: 'AppArmor' and 'Proc type'. All keys are
+required.
+
+* `title` is what is displayed as the columns user-friendly title.
+* The `id` key must have a unique value, to differentiate between
+  columns.
+* The `sType` value determines how the values will be sorted in the host
+  overview. Possible values include `string` and `num`. `visible` determines
+  whether the column will be active (shown) by default.
+* The `jsonxs` expression points to an entry in the facts files for each host,
+  and determines what will be shown for the column's value for each host.
+  The easiest way to figure out a jsonxs expression is by opening one of the
+  gathered facts files in a json editor. Please see
+  [jsonxs](https://github.com/fboender/jsonxs) for info on how to write jsonxs
+  expressions.
+
+To use it:
+
+    ../ansible-cmdb/src/ansible-cmdb -C example/cust_cols.json -i example/hosts example/out/ > cmdb.html
+
+When opening the `cmdb.html` file in your browser, you may have to hit the
+'Clear settings' button in the top-right before the new columns show up or
+when you get strange behaviour.
+
+
 ## Custom templates
 
-If you want to add custom columns or other data to the output, you can create
-a custom template. Ansible-cmdb uses the [Mako templating
-engine](http://www.makotemplates.org/) to render output.
+Custom columns can be added with the `-C` param. See the [Custom
+columns](#custom-columns) section for more info. Custom columns are somewhat
+limited in the type of information they can display (basically only strings
+and numbers).  If you want to add more elaborate custom columns or other data
+to the output, you can create a custom template. Ansible-cmdb uses the [Mako
+templating engine](http://www.makotemplates.org/) to render output.
 
 For example, if you want to add a custom column to the `html_fancy` template:
 

example/cust_cols.json

@@ -0,0 +1,16 @@
+[
+    {
+        "title": "AppArmor",
+        "id": "apparmor",
+        "sType": "string",
+        "visible": false,
+        "jsonxs": "ansible_facts.ansible_apparmor.status"
+    },
+    {
+        "title": "First processor",
+        "id": "firstproc",
+        "sType": "string",
+        "visible": true,
+        "jsonxs": "ansible_facts.ansible_processor[2]"
+    }
+]

example/generate.sh

@@ -20,6 +20,7 @@ python2 ../src/ansible-cmdb.py -q -t markdown -i hosts out > gen_markdown_2.md
 python2 ../src/ansible-cmdb.py -q -t sql -i hosts out > gen_sql_2.md
 python2 ../src/ansible-cmdb.py -q -t html_fancy_split -i hosts out
 python2 ../src/ansible-cmdb.py -q -i hosts -f out_factcache > gen_fact_cache_2.html
+python2 ../src/ansible-cmdb.py -q -i hosts -C cust_cols.json out > gen_fact_cust_cols_2.html
 
 
 # Python v3
@@ -31,3 +32,4 @@ python3 ../src/ansible-cmdb.py -q -t markdown -i hosts out > gen_markdown_3.md
 python3 ../src/ansible-cmdb.py -q -t sql -i hosts out > gen_sql_3.md
 python3 ../src/ansible-cmdb.py -q -t html_fancy_split -i hosts out
 python3 ../src/ansible-cmdb.py -q -i hosts -f out_factcache > gen_fact_cache_3.html
+python3 ../src/ansible-cmdb.py -q -i hosts -C cust_cols.json out > gen_fact_cust_cols_2.html

src/ansible-cmdb.py

@@ -15,6 +15,7 @@
 import sys
 import os
 import logging
+import json
 from mako import exceptions
 import ansiblecmdb
 import ansiblecmdb.util as util
@@ -92,6 +93,30 @@ def get_hosts_files(option):
                     return [line.split('=', 1)[1].strip()]
 
 
+def get_cust_cols(path):
+    """
+    Load custom column definitions.
+    """
+    required_keys = ["title", "id", "sType", "visible", "jsonxs"]
+
+    with open(path, 'r') as f:
+        try:
+            cust_cols = json.load(f)
+        except json.decoder.JSONDecodeError as err:
+            sys.stderr.write("Invalid custom columns json file: {}\n".format(path))
+            sys.stderr.write("{}\n".format(err.args[0]))
+            sys.exit(1)
+
+    # Validate
+    for col in cust_cols:
+        for required_key in required_keys:
+            if required_key not in col:
+                sys.stderr.write("Missing required key '{}' in custom "
+                                 "column {}\n".format(required_key, col))
+                sys.exit(1)
+
+    return cust_cols
+
 def parse_user_params(user_params):
     """
     Parse the user params (-p/--params) and them as a dict.
@@ -126,6 +151,7 @@ def parse_user_params(user_params):
     parser.add_option("-d", "--debug", dest="debug", action="store_true", default=False, help="Show debug output")
     parser.add_option("-q", "--quiet", dest="quiet", action="store_true", default=False, help="Don't report warnings")
     parser.add_option("-c", "--columns", dest="columns", action="store", default=None, help="Show only given columns")
+    parser.add_option("-C", "--cust-cols", dest="cust_cols", action="store", default=None, help="Path to a custom columns definition file")
     parser.add_option("-l", "--limit", dest="limit", action="store", default=None, help="Limit hosts to pattern")
     parser.add_option("--exclude-cols", dest="exclude_columns", action="store", default=None, help="Exclude cols from output")
     (options, args) = parser.parse_args()
@@ -142,6 +168,10 @@ def parse_user_params(user_params):
 
     hosts_files = get_hosts_files(options.inventory)
 
+    cust_cols = []
+    if options.cust_cols is not None:
+        cust_cols = get_cust_cols(options.cust_cols)
+
     # Handle template params
     params = {
         'lib_dir': data_dir,  # Backwards compatibility for custom templates < ansible-cmdb v1.7
@@ -150,6 +180,7 @@ def parse_user_params(user_params):
         'log': log,
         'columns': None,
         'exclude_columns': None,
+        'cust_cols': cust_cols
     }
     params.update(parse_user_params(options.params))
     if options.columns is not None:

src/ansiblecmdb/data/tpl/html_fancy.tpl

@@ -13,6 +13,9 @@ skip_empty = to_bool(context.get('skip_empty', '0'))
 # Get column definitions from html_fancy_defs.html
 cols = var_cols(columns, exclude_columns)
 
+# Extend default columns with custom columns
+cols.extend(cust_cols)
+
 # Set the Javascript resource URL (local disk or CDN)
 if local_js is False:
   res_url = "https://cdn.datatables.net/1.10.2/"

src/ansiblecmdb/data/tpl/html_fancy_defs.html

@@ -279,7 +279,11 @@ <h2>Host overview</h2>
           % if skip_empty is False or 'ansible_facts' in host:
             <tr>\
               % for col in cols:
-                <td>${col["func"](host, **kwargs)}</td>\
+                % if "func" in col:
+                  <td>${col["func"](host, **kwargs)}</td>\
+                % elif "jsonxs" in col:
+                  <td>${col__cust(host, col=col, **kwargs)}</td>\
+                % endif
               % endfor
             </tr>
           % endif
@@ -469,6 +473,11 @@ <h3 class="toggle-collapse ${collapsed_class}" id="${host['name']}" data-host-na
 ## in the function scope in Mako for some reason, so if we need to use them, we
 ## need to do a `kwargs.get()`. I don't like Mako much.
 
+<%def name="col__cust(host, col, **kwargs)">
+  ## Special column function for custom columns that does a jsonxs lookup
+  ${jsonxs(host, col["jsonxs"], default='')}
+</%def>
+
 <%def name="col_name(host, **kwargs)">
   <%
   link_type = kwargs.get("link_type")

src/ansiblecmdb/data/tpl/html_fancy_split_overview.tpl

@@ -13,6 +13,9 @@ skip_empty = to_bool(context.get('skip_empty', '0'))
 # Get column definitions from html_fancy_defs.html
 cols = var_cols(columns, exclude_columns)
 
+# Extend default columns with custom columns
+cols.extend(cust_cols)
+
 # Set the Javascript resource URL (local disk or CDN)
 if local_js is False:
   res_url = "https://cdn.datatables.net/1.10.2/"