Send OpsWorks log files to Papertrail using the remote_syslog gem

1-OpsWorksPapertrail.md

1. Add remote_syslog to your Gemfile
2. Add papertrail.rake to lib/tasks
3. Add remote_syslog.yml to config/
4. Add before_restart.rb to deploy/
5. Set the PAPERTRAIL_PORT ENV var to the one provided by Papertrail for your system

Inspiration from Scott W. Bradley

before_restart.rb

run "bundle exec rake papertrail:start" # Start remote_syslog daemon to push logs to papertrail

Gemfile

# Probably only needed in production/staging environments
gem 'remote_syslog'

papertrail.rake

PAPERTRAIL_CONFIG = File.expand_path("../../../config/remote_syslog.yml", __FILE__)
PAPERTRAIL_PID    = File.expand_path("../../../tmp/pids/remote_syslog.pid", __FILE__)

namespace :papertrail do
  def papertrail_is_running?
    File.exists?(PAPERTRAIL_PID) && system("ps x | grep `cat #{PAPERTRAIL_PID}` 2>&1 > /dev/null")
  end

  desc "Start papertrail remote_syslog daemon."
  task :start => :stop do
    if papertrail_is_running?
      puts "Papertrail is already running."
    else
      sh "remote_syslog -c #{PAPERTRAIL_CONFIG} --pid-file #{PAPERTRAIL_PID} --dest-port #{ENV['PAPERTRAIL_PORT']}"
    end
  end

  desc "Stop papertrail remote_syslog daemon."
  task :stop do
    if File.exists? PAPERTRAIL_PID
      sh "kill `cat #{PAPERTRAIL_PID}`"
      rm_f PAPERTRAIL_PID
    end
  end

  desc "Show status of papertrail remote_syslog daemon."
  task :status do
    if papertrail_is_running?
      puts "Papertrail remote_syslog is running"
    else
      puts "Papertrail remote_syslog is stopped"
    end
  end
end

remote_syslog.yml

files:
  - /var/log/nginx/*.log
  - /srv/www/*/shared/log/*
exclude_files:
  - newrelic_agent
destination:
  host: logs.papertrailapp.com

Two fixes I'd suggest, line #6:

File.exists?(PAPERTRAIL_PID) && system("kill -0 `cat #{PAPERTRAIL_PID}` 2>&1 > /dev/null")

turns out to be a better way to check whether the process is actually running. And line #20:

if papertrail_is_running?

should be used to "properly" verify papertrail's status

Maybe a silly question but how do you actually set the ENV var? Do you add it to your stack's custom json somewhere?

@edejong yes, http://zaman.io/how-to-import-aws-opsworks-json-into-rails-app/ I've written a how to on that.