Example: CustomStats

HOMER 5

Custom JSON-HEP Stats

HOMER 5 implements a basic HTTP Push API powered by Kamailio and the xhttp and jansson modules. The new API supports parsing and indexing external custom data sources (in form of JSON objects) received via HTTP socket and using user customizable structures as illustrated in this example featuring RTP statistics produced by rtpsniff-json and statstrmr

Example JSON Report

The first thing is to analyze and define a delivery format identified by a "type" field - the order of fields doesn't matter as the document will get parsed, but remember only string and int types are supported.

Let's look at our JSON object:

{ "timestamp": 1455196990, "interval": 10, "streams": 2, "packets": 1000, "lost": 0, "late": 0, "lost_perc": 0.000000, "late_perc": 0.000000,"out_of_seq": 0, "delay_min": 41580, "delay_max": 19970578, "type": "rtp_stat" }

Example Configuration

Next we will extend the HOMER/Kamailio configuration to spawn an xHTTP API configured to handle our type-specific JSON object and use it to generate new statistics:

#!substdef "!HOMER_STATS_SERVER!tcp:MY_IP_ADDR:8888!g"

listen=HOMER_STATS_SERVER
....
loadmodule "xhttp.so"
loadmodule "jansson.so"
loadmodule "avpops.so"
...
modparam("htable", "htable", "d=>size=8;autoexpire=400")
modparam("xhttp", "url_match", "^/api/v1/stat")

...

route[CHECK_STATS] {
....

 #Generic stats
    sht_iterator_start("i1", "d");
    while(sht_iterator_next("i1")) {
                $var(key) = $(shtitkey(i1){s.select,2,:});
                sql_query("cb", "INSERT INTO stats_generic (from_date, to_date, type, total) VALUES($var(f_date), $var(t_date), '$var(key)', $shtitval(i1)) ON DUPLICATE KEY UPDATE total=(total+$shtitval(i1))/2");
    }
    sht_iterator_end("i1");
    sht_rm_name_re("d=>.*");
...
}


event_route[xhttp:request] {
        set_reply_close();
        set_reply_no_connect();
        xlog("L_WARN", "HTTP request received on $Rp, $hu\n");
        if($hu =~ "/api/v1/stats/push") {
                #Json is our body
                $var(json) = $rb;
                jansson_get("type", $var(json), "$var(n)");
                xlog("L_WARN","Type is $var(n)");
                if($var(n) == "rtp_stat") {
                         $var(i) = 0;
                         $(avp(x)[0]) = 'interval';
                         $(avp(x)[1]) = 'streams';
                         $(avp(x)[3]) = 'packets';
                         $(avp(x)[4]) = 'lost';
                         $(avp(x)[5]) = 'late';
                         $(avp(x)[6]) = 'lost_perc';
                         $(avp(x)[7]) = 'late_perc';
                         $(avp(x)[8]) = 'out_of_seq';
                         $(avp(x)[9]) = 'delay_min';
                         $(avp(x)[10]) = 'delay_max';
                         while(is_avp_set("$(avp(x)[$var(i)])")) {
                                  xlog("L_INFO", "Array value [$var(i)]: $(avp(x)[$var(i)])\n");
                                  jansson_get("$(avp(x)[$var(i)])", $var(json), "$var(d)");
                                  $var(n) = $(var(d){s.int});
                                  if($sht(d=>generic::$(avp(x)[$var(i)])) == $null) $sht(d=>generic::$(avp(x)[$var(i)])) = $var(n);
                                  else $sht(d=>generic::$(avp(x)[$var(i)])) = ($sht(d=>generic::$(avp(x)[$var(i)])) + $var(n))/2;                                  
                                  $var(i) = $var(i) + 1;
                         }
          }
          xhttp_reply("200", "Ok", "done", "");
          exit;
        }

        xhttp_reply("403", "Forbidden", "", "");
        exit;
}

Ship your data!

We're ready to start sending some data! In this example we are using RTP statistics produced by rtpsniff-json and sent to HOMER API using statstrmr to produce general RTP usage reports on large ranges.

Example Datasource (sipcapture charts)

In order for the data to be consumed, we need to make the widgets aware of it - this is achieved by extending the datasource.js file located inside the ui/widgets folder. An example follows:

{
            "name": "Generic",
            "type": "JSON",
            "settings": {
                "path": "statistic\/generic",
                        "query": "{\n   \"timestamp\": {\n          \"from\": \"@from_ts\",\n          \"to\":  \"@to_ts\"\n   },\n  \"param\": {\n        \"filter\": [ \n             \"@filters\"\n       ],\n       \"limit\": \"@limit\",\n       \"total\": \"@total\"\n   }\n}",
                "method": "GET",
                "limit": 200,
                "total": false,
                "eval": {
                    incoming: {
                        name: "test incoming",
                        value: "var object = @incoming; return object"
                    }
                },
                "timefields" : [
                    { "field": "from_ts", "desc": "From Timestamp" },
                    { "field": "to_ts", "desc": "To Timestamp" }
                ],
                "fieldvalues": [
                    { "field": "total", "desc": "All Packets" }
                ],
                "filters": [
		    { "type": "type", "desc": "Data Statistics", options: [
	                    { "value": "delay_max", "desc": "delay_max" },
	                    { "value": "interval", "desc": "interval" },
	                    { "value": "lost_perc", "desc": "lost_perc" },
	                    { "value": "late", "desc": "late" },
	                    { "value": "streams", "desc": "streams" },
	                    { "value": "out_of_seq", "desc": "out-of-seq" },
	                    { "value": "late_perc", "desc": "late_perc" },
	                    { "value": "lost", "desc": "lost" },
	                    { "value": "packets", "desc": "packets" },
	                    { "value": "delay_min", "desc": "delay_min" }
			] 
		    }
                ]
            }
        }

Once the datasource is configured, the new set will become available in connected widgets:

And voila'! Your personalized chart using custom data is now ready!