Pushing logs to loggly with fluentd

Setting up Fluentd log publisher to Loggly is straightforward thanks to the detailed tutorials can be found online. Some useful readings:

One gotcha: numeric fields in loggly

By default everything you send in your JSON will be a string because you parsed a log line with regular expression. Taking nginx access log for example:

Nginx config:

Fluentd config:

This results in logs appearing in Loggly nicely but when wanted to create timeline graphs the following error message appears:

Chart cannot be created because no numeric fields have been found. Please try again when there is more data.

Fix it

To fix it we need to cast our JSON attributes sent by fluentd to loggly. We can do this with fluentd typecast plugin and to restore the original tags we will use record reformer plugin.

Fluentd conf with the new changes:

What happens here is we changed the tag to the original tail plugin to have a raw prefix which we will match later. In the match raw.nginx.** section we will cast the code and size to integer and the request time, upstream time and gzip ratio to float.

The second part is optional but we probably don’t want to have all our events tagged with typed.raw. prefix therefore we can use record-reformer to remove the first to tag part.

There you go… Now you have nicely formatted log events where every attributes has the proper type which enables more sophisticated analysis on them.