Skip to content

Instantly share code, notes, and snippets.

@billwanjohi
Created October 29, 2013 06:50
Show Gist options
  • Save billwanjohi/7210089 to your computer and use it in GitHub Desktop.
Save billwanjohi/7210089 to your computer and use it in GitHub Desktop.
scrape sportstracker data
require 'watir-webdriver'
require 'csv'
require 'pry'
# TODO:
# scrape specific activities (e.g. running, cycling)
# plot scatter
# fit curve
begin
b = Watir::Browser.new :phantomjs
b.goto "http://www.sportstracklive.com/user/#{ENV['STL_USER']}"
CSV.open("bests.csv", "wb") do |csv|
b.wait_until { b.div(class: "perf1").exists? }
b.divs(class: "perf1").each do |pb|
category, speed, quantity =
[pb.div(style: "float:left").text] + pb.span.text.split(' / ')
speed.chomp!("mph")
ts = category.match(/mi$/) ? quantity : category
weeks = ts.match(/\d+(?=w)/).to_s.to_i
days = ts.match(/\d+(?=d)/).to_s.to_i + weeks * 7
hours = ts.match(/\d+(?=(h|hr))/).to_s.to_i + days * 24
minutes = ts.match(/\d+(?=(m|min))/).to_s.to_i + hours * 60
seconds = ts.match(/\d*\.?\d*+(?=(s|sec))/).to_s.to_f + minutes * 60
csv << [seconds, speed]
end
end
ensure
b.close
end
# csvsort -c 1 <bests.csv
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment