Created
August 8, 2017 19:12
-
-
Save dannyk81/4bb95914509c1e8db6438536e352985f to your computer and use it in GitHub Desktop.
Fluentd v0.12 converting long epoch (milliseconds) to Date Time string with milleseconds precision
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
# Consider the record contains the time stamp of the event in a record key called 'timestamp' | |
# e.g. "timestamp": "1502217900063" | |
# The below will add a new record called `formatted_date` that will include an iso8601(3) formatted date string with milliseconds, | |
# the trick was to extract from the long epoch value the seconds & remaining milliseconds and convert it to microseconds since Time.at() accepts: | |
# `Time.at(seconds, microseconds_with_frac) → time` | |
<filter tag.*> | |
@type record_modifier | |
<record> | |
formatted_date ${Time.at(record['timestamp'].to_i/1000, record['timestamp'].to_i%1000*1000).utc.strftime('%Y-%m-%dT%H:%M:%S.%LZ')} | |
</record> | |
</filter> | |
# If you use the fluent-plugin-elasticsearch, you can tell the plugin to use `formatted_date` to generate the @timestamp, like so: | |
<match **> | |
type elasticsearch | |
time_key formatted_date | |
... | |
</match> |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Comment above loses milliseconds precision. Use
time ${record['timestamp'].to_f / 1000}
instead.