Skip to content

Instantly share code, notes, and snippets.

@anisotropi4

anisotropi4/.blocks

Last active June 29, 2020 07:31
Show Gist options
  • Save anisotropi4/757430340067ebfdd144bdd3ce92823c to your computer and use it in GitHub Desktop.
Save anisotropi4/757430340067ebfdd144bdd3ce92823c to your computer and use it in GitHub Desktop.
Download ZIP
Fizzy Knitting: Location of electrified rail in the UK
Raw
.blocks
Released license:MIT
height:780
border:no
Fizzy Knitting: Location of electrified rail in the UK
OpenStreetMap® open data is © OpenStreetMap contributors and used under the Open Street Map license https://www.openstreetmap.org/copyright
Open Data Institute Leeds scripts and configuration are used © Creative Commons BY 4.0 ODI Leeds 2018 https://creativecommons.org/licenses/by/4.0/
Raw
README.md

Fizzy Knitting GB Rail Electrification data

Open Street Map contains lots of data about the railway in mainland Britain. These scripts extract the full Open Street Map dataset and then filter for point associated with rail.

Creating the datafiles and associate geojson format report

Once the dependencies to create the report are met run the script:

$ ./prepublish

This will download, update to a point in time and create a set of geometric point, lines, multilinestrings, multipolygons and other_relations geojson objects filtered to contain rail related data.

Dependencies

These are environment and project dependencies.

Environment dependencies

Install the required OSM update and GDAL dependencies for the ogr2ogr tool

$ sudo apt install osmctools gdal-bin

python dependencies

For ease of use manage package python packages dependencies with a local virtual environment venv:

$ virtualenv venv
$ source venv/bin/activate
$ pip install geopandas
Raw
glue.py
#!/usr/bin/env python3
import json
from shapely.ops import transform
import geopandas as gp
def _set_precision(g, precision=5):
def _precision(precision):
return lambda x, y, z=None: tuple([round(x, precision),
round(y, precision),
round(z, precision)] if z
else
[round(x, precision),
round(y, precision)])
return transform(_precision(precision), g)
GDATA = gp.GeoDataFrame()
TAGS = []
for gtype in ['points', 'lines', 'multilinestrings', 'multipolygons', 'other_relations']:
GF1 = gp.read_file('great-britain-voltage-{}.json'.format(gtype))
if not GF1.empty:
GDATA = GDATA.append(GF1)
GDATA = GDATA.fillna('None').reset_index(drop=True)
GDATA['geometry'] = GDATA['geometry'].apply(_set_precision)
KEYS = ['osm_id', 'electrified', 'frequency', 'voltage', 'railway', 'geometry']
DATA = json.loads(GDATA[KEYS].to_json())
DATA = DATA['features']
for i in DATA:
i.pop('id', None)
i['properties'] = {k: v for k, v in i['properties'].items() if v != 'None'}
with open('output-all.json', 'w') as fout:
json.dump(DATA, fout)
KEYS = ['osm_id', 'electrified', 'frequency', 'voltage', 'railway']
KEYS += [i for i in set(GDATA.columns) - set(KEYS)]
for KEY in KEYS:
GDATA[KEY] = GDATA[KEY].replace({'None': '-'})
with open('tag-report.csv', 'w') as fout:
GDATA[KEYS].drop('geometry', axis=1).to_csv(fout, index=False)
Raw
great-britain.poly
none
1
1.983960E+00 5.133770E+01
5.807600E-02 5.010375E+01
-1.246112E+00 4.989032E+01
-2.027284E+00 4.977612E+01
-2.020900E+00 4.976352E+01
-2.031990E+00 4.976171E+01
-1.808598E+00 4.910263E+01
-1.835368E+00 4.900443E+01
-3.940324E+00 4.915592E+01
-7.024780E+00 4.970097E+01
-5.441616E+00 5.278017E+01
-5.206178E+00 5.377268E+01
-5.488813E+00 5.486510E+01
-6.208707E+00 5.533747E+01
-6.604158E+00 5.543057E+01
-7.148041E+00 5.566455E+01
-1.486751E+01 5.746894E+01
-1.499070E+01 5.768017E+01
-1.208287E+01 5.853333E+01
-1.637515E+00 6.113564E+01
-2.670263E-01 6.110300E+01
1.278458E-01 5.976591E+01
9.781600E-02 5.942198E+01
7.998970E-01 5.579959E+01
1.702395E+00 5.443427E+01
2.250000E+00 5.258000E+01
1.983960E+00 5.133770E+01
END
END
Raw
index.html
<!DOCTYPE html>
<html>
<head>
<title>Fizzy Knitting: OSM locations of rail electrification in the UK</title>
<meta charset="utf-8"/>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta http-equiv="Cache-control" content="No-Cache">
<link rel="shortcut icon" type="image/x-icon" href="docs/images/favicon.ico"/>
<link rel="stylesheet" href="https://unpkg.com/[email protected]/dist/leaflet.css"
integrity="sha512-xwE/Az9zrjBIphAcBb3F6JVqxf46+CDLwfLMHloNu6KEQCAWi6HcDUbeOfBIptF7tcCzusKFjFw2yuvEpDL9wQ==" crossorigin=""/>
<script src="https://unpkg.com/[email protected]/dist/leaflet.js"
integrity="sha512-gZwIG9x3wUXg2hdXF6+rVkLF/0Vi9U8D2Ntg4Ga5I5BZpVkVxlJWbSQtXPSiUTtC0TjtGOmxa1AJPuV0CPthew=="
crossorigin=""></script>
<script src="http://d3js.org/d3.v5.min.js"></script>
<div id="map" style="width: 1340px; height: 780px"></div>
</head>
<body>
<div id='map'></div>
<script type="text/javascript">
var radius = 3;
var weight = 1;
var linewidth = 2;
var log2 = Math.log(2.0);
var minZoom = 3;
var maxZoom = 18;
var map = L.map('map').setView([54.533, -2.53], 6);
//var voltages = ["25000", "1500", "750", "630", "600", "550"]
var voltages = ["660", "750", "1000", "1500", "3000", "6000", "11000", "15000", "20000", "25000", "50000"];
var colours = ["#4300BD","#5A00B2","#6F00A7","#86009C","#FFCC00","#FFAF00","#FF9400","#FF7D00","#FF6600","#FF4500","#F00000","#D00000"];
L.tileLayer("http://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png", {
maxZoom: maxZoom,
minZoom: minZoom,
attribution: 'Map data &copy; <a href="https://www.openstreetmap.org/">OpenStreetMap</a> contributors, ' + '<a href="https://creativecommons.org/licenses/by-sa/2.0/">CC-BY-SA</a>, ' + 'Imagery © <a href="https://www.mapbox.com/">Mapbox</a>',
id: 'osm.standard'
}).addTo(map);
d3.json("output-all.json").then(function(d) {
function onEachFeature(feature, layer) {
var this_feature = feature.properties;
var popupContent;
if (this_feature.name)
popupContent = this_feature.name;
else
popupContent = this_feature.railway;
var lookup = {
"osm_id": "OSM ID",
"place": "Place",
"natural": "Natural",
"place": "Place",
"highway": "Road",
"ref": "CRS",
"admin_level": "admin",
"boundary": "boundary",
"voltage": "voltage",
"frequency": "frequency",
"electrified": "electrified"
}
var k = Object.keys(this_feature).filter(i=>(i != "type" && i != "geometry" && i != "name" && i != "is_in" && i != "z_order" && i != "railway"))
for (var i = 0; i < k.length; i++) {
if (k[i] in this_feature) {
popupContent += '<br>' + lookup[k[i]] + ': ' + this_feature[k[i]];
}
}
layer.bindPopup(popupContent);
}
function get_colour(p) {
var colours = {
contact_line: {
550: '#e69c65',
600: '#bd5d16',
750: '#ffce36',
1500: '#ff893f',
25000: '#ff3818',
0: '#9ca3a9',
},
rail: {
550: '#b3e297',
630: '#7bc643',
750: '#09843b',
},
'4th_rail': {
630: '#0a3797',
750: '#11a7e0',
},
'contact_line;rail': {
'25000;750': '#fe85a8',
},
'contact_line;4th_rail': {
'25000;750': '#9d176s',
},
no: '#2c2e35',
fail: '#ff00ff',
}
var e = p.electrified;
var f = p.frequency;
var v = p.voltage;
var r = p.railway
if (v && Number(v) < 550) v = '550';
if (v && Number(v) > 25000) return colours['fail'];
if (e == 'yes') e = 'contact_line';
if (e == '3rd_rail') e = 'rail';
if (! e) e = 'no';
if (e in colours) return colours[e][v] || colours[e];
return colours['fail'];
}
L.geoJSON(d, {
style: function(feature) {
var c = get_colour(feature.properties);
switch (feature.geometry.type) {
case 'Point':
return {
color: c,
radius: radius,
weight: weight
};
case 'LineString':
return {
color: c,
weight: linewidth
};
case 'MultiLineString':
return {
color: c,
weight: linewidth
};
case 'MultiPolygon':
return {
weight: linewidth,
opacity: 1.0,
color: "Green",
fillColor: "Green",
fillOpacity: 0.2
};
case 'GeometryCollection':
return {
radius: radius,
weight: linewidth,
opacity: 1.0,
color: "Green",
fillColor: "GreenYellow",
fillOpacity: 0.2
};
default:
return {
weight: weight
};
}
},
onEachFeature: onEachFeature,
pointToLayer: function(feature, latlng) {
return L.circleMarker(latlng, {
opacity: 1,
fillOpacity: 0.8
});
}
}).addTo(map);
});
</script>
</body>
</html>
Raw
LICENSE
MIT License
Copyright (c) 2020 Will Deakin
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
Raw
osmconf-voltage.ini
#
# Configuration file for OSM import
#
# put here the name of keys for ways that are assumed to be polygons if they are closed
# see http://wiki.openstreetmap.org/wiki/Map_Features
closed_ways_are_polygons=aeroway,amenity,boundary,building,craft,geological,historic,landuse,leisure,military,natural,office,place,shop,sport,tourism
# comment to avoid laundering of keys ( ':' turned into '_' )
attribute_name_laundering=yes
# uncomment to report all nodes, including the ones without any (significant) tag
#report_all_nodes=yes
# uncomment to report all ways, including the ones without any (significant) tag
#report_all_ways=yes
[points]
# common attributes
osm_id=yes
osm_version=no
osm_timestamp=no
osm_uid=no
osm_user=no
osm_changeset=no
# keys to report as OGR fields
attributes=name,barrier,highway,ref,address,is_in,place,man_made,railway,natural,train,rail,ref:tiploc,naptan:AtcoCode,ele,layer,location,bridge,tunnel,covered,voltage,electrified,frequency,boundary,building,chainage_start,chainage_end,distance_marker,elr,ELR,fuel:electricity
# keys that, alone, are not significant enough to report a node as a OGR point
unsignificant=created_by,converted_by,source,time,attribution
# keys that should NOT be reported in the "other_tags" field
ignore=created_by,converted_by,source,time,note,openGeoDB:,fixme,FIXME
# uncomment to avoid creation of "other_tags" field
other_tags=no
# uncomment to create "all_tags" field. "all_tags" and "other_tags" are exclusive
#all_tags=yes
[lines]
# common attributes
osm_id=yes
osm_version=no
osm_timestamp=no
osm_uid=no
osm_user=no
osm_changeset=no
# keys to report as OGR fields
attributes=name,barrier,highway,ref,address,is_in,place,man_made,railway,natural,train,rail,ref:tiploc,naptan:AtcoCode,ele,layer,location,bridge,tunnel,covered,voltage,electrified,frequency,boundary,building,chainage_start,chainage_end,distance_marker,elr,ELR,fuel:electricity
# keys that should NOT be reported in the "other_tags" field
ignore=created_by,converted_by,source,time,note,openGeoDB:,fixme,FIXME
# uncomment to avoid creation of "other_tags" field
other_tags=no
# uncomment to create "all_tags" field. "all_tags" and "other_tags" are exclusive
#all_tags=yes
#computed_attributes must appear before the keywords _type and _sql
computed_attributes=z_order
z_order_type=Integer
# Formula based on https://github.com/openstreetmap/osm2pgsql/blob/master/style.lua#L13
# [foo] is substituted by value of tag foo. When substitution is not wished, the [ character can be escaped with \[ in literals
z_order_sql="SELECT (CASE [highway] WHEN 'minor' THEN 3 WHEN 'road' THEN 3 WHEN 'unclassified' THEN 3 WHEN 'residential' THEN 3 WHEN 'tertiary_link' THEN 4 WHEN 'tertiary' THEN 4 WHEN 'secondary_link' THEN 6 WHEN 'secondary' THEN 6 WHEN 'primary_link' THEN 7 WHEN 'primary' THEN 7 WHEN 'trunk_link' THEN 8 WHEN 'trunk' THEN 8 WHEN 'motorway_link' THEN 9 WHEN 'motorway' THEN 9 ELSE 0 END) + (CASE WHEN [bridge] IN ('yes', 'true', '1') THEN 10 ELSE 0 END) + (CASE WHEN [tunnel] IN ('yes', 'true', '1') THEN -10 ELSE 0 END) + (CASE WHEN [railway] IS NOT NULL THEN 5 ELSE 0 END) + (CASE WHEN [layer] IS NOT NULL THEN 10 * CAST([layer] AS INTEGER) ELSE 0 END)"
[multipolygons]
# common attributes
# note: for multipolygons, osm_id=yes instanciates a osm_id field for the id of relations
# and a osm_way_id field for the id of closed ways. Both fields are exclusively set.
osm_id=yes
osm_version=no
osm_timestamp=no
osm_uid=no
osm_user=no
osm_changeset=no
# keys to report as OGR fields
attributes=name,barrier,highway,ref,address,is_in,place,man_made,railway,natural,train,rail,ref:tiploc,naptan:AtcoCode,ele,layer,location,bridge,tunnel,covered,voltage,electrified,frequency,boundary,building,chainage_start,chainage_end,distance_marker,elr,ELR,fuel:electricity
# keys that should NOT be reported in the "other_tags" field
ignore=area,created_by,converted_by,source,time,note,openGeoDB:,fixme,FIXME
# uncomment to avoid creation of "other_tags" field
other_tags=no
# uncomment to create "all_tags" field. "all_tags" and "other_tags" are exclusive
#all_tags=yes
[multilinestrings]
# common attributes
osm_id=yes
osm_version=no
osm_timestamp=no
osm_uid=no
osm_user=no
osm_changeset=no
# keys to report as OGR fields
attributes=name,barrier,highway,ref,address,is_in,place,man_made,railway,natural,train,rail,ref:tiploc,naptan:AtcoCode,ele,layer,location,bridge,tunnel,covered,voltage,electrified,frequency,boundary,building,chainage_start,chainage_end,distance_marker,elr,ELR,fuel:electricity
# keys that should NOT be reported in the "other_tags" field
ignore=area,created_by,converted_by,source,time,note,openGeoDB:,fixme,FIXME
# uncomment to avoid creation of "other_tags" field
other_tags=no
# uncomment to create "all_tags" field. "all_tags" and "other_tags" are exclusive
#all_tags=yes
[other_relations]
# common attributes
osm_id=yes
osm_version=no
osm_timestamp=no
osm_uid=no
osm_user=no
osm_changeset=no
# keys to report as OGR fields
attributes=name,barrier,highway,ref,address,is_in,place,man_made,railway,natural,train,rail,ref:tiploc,naptan:AtcoCode,ele,layer,location,bridge,tunnel,covered,voltage,electrified,frequency,boundary,building,chainage_start,chainage_end,distance_marker,elr,ELR,fuel:electricity
# keys that should NOT be reported in the "other_tags" field
ignore=area,created_by,converted_by,source,time,note,openGeoDB:,fixme,FIXME
# uncomment to avoid creation of "other_tags" field
other_tags=no
# uncomment to create "all_tags" field. "all_tags" and "other_tags" are exclusive
#all_tags=yes
Raw
output-all.json
This file has been truncated, but you can view the full file.
Raw
prepublish
View raw

(Sorry about that, but we can’t show files that are this big right now.)

Raw
tag-report.csv
View raw

(Sorry about that, but we can’t show files that are this big right now.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment