Google Analytics Real Time Visitors Widget for Dashing with OAuth2 Authentication

README.md

Description

A Dashing widget for displaying the number of current visitors (in real time) to your website, as reported by Google Analytics.

This widget is a fork of https://gist.github.com/mtowers/5986576

Dependencies

google-api-ruby-client

Add it to dashing's gemfile:

gem 'google-api-client'

and run bundle install.

Usage

To use this widget, you'll first need to set up a Google API project and attach it to the Google Analytics profile you wish to monitor, as well as request access to the Google Analytics Real Time API.

1. Request acces to the Google Analytics Real Time API

1. Go to http://goo.gl/7ZPlMI and fill the form

2. Create and download a new private key for Google API access.

1. Go to https://code.google.com/apis/console
2. Click 'Create Project'
3. Enable 'Analytics API' service and accept both TOS's
4. Click 'API Access' in the left-hand nav menu
5. Click 'Create an OAuth 2.0 Client ID'
6. Enter a product name (e.g. Dashing Widget) - logo and url are optional
7. Click 'Next'
8. Under Application Type, select 'Service Account'
9. Click 'Create Client ID'
10. Click 'Download private key' NOTE: This will be your only opportunity to download this key.
11. Note the password for your new private key ('notasecret')
12. Close the download key dialog
13. Find the details for the service account you just created and copy it's email address which will look something like this: [email protected] - you'll need it in your ruby code later

3. Attach your Google API service account to your Google Analytics profile

Note: you will need to be an administrator of the Google Analytics profile

1. Log in to your Google Analytics account: http://www.google.com/analytics/
2. Click 'Admin' in the upper-right corner
3. Select the account containing the profile you wish to use
4. Select the property containing the profile you wish to use
5. Select the profile you wish to use
6. Click the 'Users' tab
7. Click '+ New User'
8. Enter the email address you copied from step 13 above
9. Click 'Add User'

4. Locate the ID for your Google Analytics profile

1. On your Google Analytics profile page, click the 'Profile Settings' tab
2. Under 'General Information' copy your Profile ID (e.g. 654321) - you'll need it in your ruby code later

5. Start coding (finally)

1. Copy the visitor_count.rb file in to your dashing jobs\ folder.
2. Update the service_account_email, key_file, key_secret and profileID variables

    service_account_email = '[YOUR SERVICE ACCOUTN EMAIL]' # Email of service account
    key_file = 'path/to/your/keyfile.p12' # File containing your private key
    key_secret = 'notasecret' # Password to unlock private key
    profileID = '[YOUR PROFILE ID]' # Analytics profile ID.

3. Add the widget HTML to your dashboard

    <li data-row="1" data-col="1" data-sizex="1" data-sizey="1"> 
        <div data-id="visitor_count_real_time" data-view="Graph" data-title="Current Visitors"></div> 
    </li>

Notes

If you want to modify this plugin to pull other data from Google Analytics, be sure to check out the Google Analytics Query Explorer.

visitor_count_real_time.rb

require 'google/api_client'
require 'date'

# Update these to match your own apps credentials
service_account_email = '[YOUR SERVICE ACCOUNT EMAIL]' # Email of service account
key_file = 'path/to/your/keyfile.p12' # File containing your private key
key_secret = 'notasecret' # Password to unlock private key
profileID = '[YOUR PROFILE ID]' # Analytics profile ID.

# Get the Google API client
client = Google::APIClient.new(
  :application_name => '[YOUR APPLICATION NAME]', 
  :application_version => '0.01'
)

visitors = []

# Load your credentials for the service account
key = Google::APIClient::KeyUtils.load_from_pkcs12(key_file, key_secret)
client.authorization = Signet::OAuth2::Client.new(
  :token_credential_uri => 'https://accounts.google.com/o/oauth2/token',
  :audience => 'https://accounts.google.com/o/oauth2/token',
  :scope => 'https://www.googleapis.com/auth/analytics.readonly',
  :issuer => service_account_email,
  :signing_key => key)

# Start the scheduler
SCHEDULER.every '5s', :first_in => 0 do

  # Request a token for our service account
  client.authorization.fetch_access_token!

  # Get the analytics API
  analytics = client.discovered_api('analytics','v3')

  # Execute the query
  visitCount = client.execute(:api_method => analytics.data.realtime.get, :parameters => { 
    'ids' => "ga:" + profile_id,
    'metrics' => "ga:activeVisitors",
  })

  visitors << { x: Time.now.to_i, y: visits.to_i }

  # Update the dashboard
  send_event('visitor_count_real_time', points: visitors)
end

What is wrong? How to solve?

MINGW64 /c/Teste
$ ruby -v
ruby 2.3.3p222 (2016-11-21 revision 56859) [i386-mingw32]

MINGW64 /c/Teste
$ gem environment
RubyGems Environment:
  - RUBYGEMS VERSION: 2.6.11
  - RUBY VERSION: 2.3.3 (2016-11-21 patchlevel 222) [i386-mingw32]
  - INSTALLATION DIRECTORY: C:/Ruby23/lib/ruby/gems/2.3.0
  - USER INSTALLATION DIRECTORY: C:/Users/marcelovilela.mg/.gem/ruby/2.3.0
  - RUBY EXECUTABLE: C:/Ruby23/bin/ruby.exe
  - EXECUTABLE DIRECTORY: C:/Ruby23/bin
  - SPEC CACHE DIRECTORY: C:/Users/marcelovilela.mg/.gem/specs
  - SYSTEM CONFIGURATION DIRECTORY: C:/ProgramData
  - RUBYGEMS PLATFORMS:
    - ruby
    - x86-mingw32
  - GEM PATHS:
     - C:/Ruby23/lib/ruby/gems/2.3.0
     - C:/Users/marcelovilela.mg/.gem/ruby/2.3.0
  - GEM CONFIGURATION:
     - :update_sources => true
     - :verbose => true
     - :backtrace => false
     - :bulk_threshold => 1000
  - REMOTE SOURCES:
     - https://rubygems.org/
  - SHELL PATH:
     - C:\Users\marcelovilela.mg\bin
     - C:\Program Files\Git\mingw64\bin
     - C:\Program Files\Git\usr\local\bin
     - C:\Program Files\Git\usr\bin
     - C:\Program Files\Git\usr\bin
     - C:\Program Files\Git\mingw64\bin
     - C:\Program Files\Git\usr\bin
     - C:\Users\marcelovilela.mg\bin
     - C:\Ruby23\bin
     - C:\ProgramData\Oracle\Java\javapath
     - C:\Program Files\Common Files\Microsoft Shared\Windows Live
     - C:\Windows\system32
     - C:\Windows
     - C:\Windows\System32\Wbem
     - C:\Windows\System32\WindowsPowerShell\v1.0
     - C:\Program Files\Intel\DMIX
     - C:\Program Files\TortoiseSVN\bin
     - C:\python26
     - C:\python26\scripts
     - C:\cheetah-2.4.4\bin
     - C:\Program Files\nodejs
     - C:\Program Files (x86)\Skype\Phone
     - C:\Program Files\Git\cmd
     - C:\Users\marcelovilela.mg\AppData\Roaming\npm
     - C:\Python27\Scripts
     - C:\Python27
     - C:\Program Files\Git\usr\bin\vendor_perl
     - C:\Program Files\Git\usr\bin\core_perl

MINGW64 /c/Teste
$ gem list google-api-client -d
google-api-client (0.11.1)
    Authors: Steven Bazyl, Tim Emiola, Sergio Gomes, Bob Aman
    Homepage: https://github.com/google/google-api-ruby-client
    License: Apache 2.0
    Installed at: C:/Ruby23/lib/ruby/gems/2.3.0

    Client for accessing Google APIs

MINGW64 /c/Teste
$ ruby visitor_count.rb
C:/Ruby23/lib/ruby/site_ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require': cannot load such file -- google/api_client (LoadError)
        from C:/Ruby23/lib/ruby/site_ruby/2.3.0/rubygems/core_ext/kernel_require.rb:55:in `require'
        from visitor_count.rb:1:in `<main>'

We recently implemented this widget on our dashboard and applied some modifications.

We added the ability to set the timezone (activesupport) as our server is set to UTC.

We are storing 12 hours of results at 1 per minute instead of 1 per second (max 720) in memcached (dalli) so if the service is restarted it retains the graph data.

Add to smashings gem file:

gem 'activesupport'
gem 'dalli'

and run bundle install

require 'google/api_client'
require 'date'
require 'active_support/all'
require 'dalli'

Time.zone = '[YOUR TIMEZONE]'

# Update these to match your own apps credentials
service_account_email = '[YOUR SERVICE ACCOUNT EMAIL]' # Email of service account
key_file = 'path/to/your/keyfile.p12' # File containing your private key
key_secret = 'notasecret' # Password to unlock private key
profile_id = '[YOUR PROFILE ID]' # Analytics profile ID.

# Get the Google API client
client = Google::APIClient.new(:application_name => '[YOUR APPLICATION NAME]', 
  :application_version => '0.01')

# Load your credentials for the service account
key = Google::APIClient::KeyUtils.load_from_pkcs12(key_file, key_secret)
client.authorization = Signet::OAuth2::Client.new(
  :token_credential_uri => 'https://accounts.google.com/o/oauth2/token',
  :audience => 'https://accounts.google.com/o/oauth2/token',
  :scope => 'https://www.googleapis.com/auth/analytics.readonly',
  :issuer => service_account_email,
  :signing_key => key)

visitors = []

# Memcached initialization
options = { namespace: "dashboard", compress: true }
dc = Dalli::Client.new('localhost:11211', options)

# Query visitors data from memcached on start
visitors = dc.get('visitor_count') if visitors.empty?

# Initialize the graph data
send_event('visitor_count_real_time', points: visitors)

# Total number of values to store
max_amount = 720
  
# Start the scheduler
SCHEDULER.every '1m', :first_in => 0 do
  # Request a token for our service account
  client.authorization.fetch_access_token!

  # Get the analytics API
  analytics = client.discovered_api('analytics','v3')

  # Execute the query
  response = client.execute(:api_method => analytics.data.realtime.get, :parameters => {
    'ids' => "ga:" + profile_id,
    'metrics' => "rt:activeUsers",
  })

  # Get the current visit count from the query
  visit_count = response.data.rows[0][0].to_i
  
  visitors << { x: Time.now.to_i, y: visit_count }
  
  # Remove the oldest value if total number is more than allowed
  if visitors.size > max_amount
    visitors.shift
  end
  
  # Store visitors data in memcached
  dc.set('visitor_count', visitors)

  # Update the dashboard
  send_event('visitor_count_real_time', points: visitors)
end

I've done an update which does a few things:

• Instead of oauth I created a service account in google auth (Note the private key is base64 encoded which I did to avoid some encoding issues when passing it through environment variables we use Docker).
• I've set the timer to run every five seconds and store 24 hours of data
• I've set the query to group by dimensions as we're planning to add a view that shows the most popular pages

require 'base64'
require 'date'
require 'google/apis/analytics_v3'

# Decode the private key
ENV['GOOGLE_PRIVATE_KEY'] = Base64.strict_decode64(ENV['GOOGLE_PRIVATE_KEY']).gsub('\n', "\n")

scopes = %w(https://www.googleapis.com/auth/analytics.readonly)
authorization = Google::Auth::ServiceAccountCredentials.make_creds(scope: scopes)
Google::Apis::RequestOptions.default.authorization = authorization

# Schedule the token refresh
SCHEDULER.every '30m', first_in: 1800 do
  # Refresh the token for our service account
  authorization.fetch_access_token!
end

# Setup services and variables
Analytics = Google::Apis::AnalyticsV3
analytics = Analytics::AnalyticsService.new

dimensions = %w(rt:pagePath)
metrics = %w(rt:activeUsers)
sort = %w(rt:pagePath)

visitors = []
max_amount = 8640

# Schedule the data updating
SCHEDULER.every '10s', first_in: 0 do
  # Get the analytics API
  response = analytics.get_realtime_data("ga:#{ENV['GOOGLE_ANALYTICS_PROFILE_ID']}", 
    metrics.join(','), 
    dimensions: dimensions.join(','),
    sort: sort.join(','))

  visitors << { 
    x: Time.now.to_i,
    y: response.totals_for_all_results['rt:activeUsers'].to_i
  }
  visitors.shift if visitors.size > max_amount

  # Update the dashboard
  send_event('visitor_count_real_time', points: visitors)
end