---
title: RailMiles
description: "I built an app to track how far I've travelled on trains because of course I did"
publishedDate: 2023-06-22
tags: ["projects"]
---
If you know me, you know that I'm kind of a nerd about trains. It's not like I have [an entire page dedicated to what I think of certain types](/trains), or anything.

In a similar vein to that, I decided that I wanted to keep track of how far I'd travelled by train and that I was going to build a few pages on my website to do that for me.

Before I get into how all that was built and how that works, this is what the end product looks like:

<img class="border" alt="Screenshot of the UI" src="the-site.jpg">

The UI is fairly simple - there's a table of all the journeys logged in the last month (not pictured), a map of those journeys and some overall stats at the bottom of the page.

To add a journey, at the very least all you have to do is tell it the date you travelled on and the stations you started, changed and ended at:

<img class="border" alt="Screenshot of the new journey UI" src="add-new-journey.jpg">

# How it's built

When a new journey is entered into the system, there are two key things that need to happen to be able to show the user a map and update the mileage totals.

* First, the length of the journey needs to be worked out by looking up schedules
* Second, the physical places the train runs through need to be logged to generate the map

## Getting data

Currently, I use [RealTimeTrains](https://realtimetrains.co.uk) to source distance data. Their API is first used to look up the schedule for the trains that the journey was comprised of. A journey can have multiple legs - for example, if I'm travelling from Corby to Brighton, I would take two trains, one from Corby to London St Pancras and one from St Pancras to Brighton. For each leg in the journey, we look up the schedule for a train that took that route on the same day that the journey took place on, then pull the stations it passed through and the distance travelled from that train.

However, the RealTimeTrains API only allows searching services by the stations they call at (as opposed to just by the origin and terminus stations) on the day they ran. This means that, sometimes, we have to manually enter the service IDs of the trains that formed the journey to get accurate results, for example if we're logging them for a journey we took yesterday. In this case, the program skips straight to requesting the routing and distance information for that train and don't bother doing any schedule lookups.

While scheduling information can (and is) pulled from the RTT API, distance information for a service is not. RealTimeTrains only servces this data in the frontend, so if we want it, we have to do some simple web scraping.

![The distance numbers](thats-distance-baybe.jpg)

This is the actual code that pulls that information out of the HTML, after which the numbers are parsed and converted into miles.

```go
var waypoints [][3]string
doc.Find(".location.call,.location.pass").Each(func(i int, selection *goquery.Selection) {
    shortcode := shortcodeRegexp.FindString(
        selection.Find(".location a").Text(),
    )

    if shortcode == "" {
        // If this is a junction without a CRS code
        return
    }

    waypoints = append(waypoints, [3]string{
        shortcode,
        strings.TrimSpace(selection.Find("span.miles").Text()),
        strings.TrimSpace(selection.Find("span.chains").Text()),
    })
})
```
<span class="text--secondary">Note that here, a short/CRS code one of the three letter codes used to refer to a train station - eg BHM for Birmingham New Street, KGX for Kings Cross, or VIC for London Victoria.</span>

From all of this, we get a sequence of locations that the train passes through and the distance between them all. Now what?

## Generating maps

It's all well and good talking about abstract location data, but it'd be nice to put that on a map and visualise it. Enter: [OpenStreetMap](https://www.openstreetmap.org/about)!

Using OSM data, it becomes trivially easy to translate a CRS code into a latitude and longitude using the [Overpass API](https://wiki.openstreetmap.org/wiki/Overpass_API), which puts a layer on top of the raw OSM data to let you query it with a dedicated query language. In fact, to get a list of station locations (and while we're at it, names, since we only store short station codes instead of their full names in the database), all you need to run is the following query:

```
[out:json];
node["ref:crs"];
out geom;
```

... which gives you a load of JSON that's comprised of entries looking a little like this (you can play around with the query [here](https://overpass-turbo.eu/s/1wdQ)).


```json
{
  "type": "node",
  "id": 104734,
  "lat": 51.5656526,
  "lon": -1.7858762,
  "tags": {
    "name": "Swindon",
    "naptan:AtcoCode": "9100SDON",
    "network": "National Rail",
    "operator": "First Great Western",
    "platforms": "4",
    "public_transport": "station",
    "railway": "station",
    "ref:crs": "SWI",
    "toilets:wheelchair": "yes",
    "wheelchair": "yes",
    "wikidata": "Q3244572",
    "wikipedia": "en:Swindon railway station"
  }
}
```

I filtered this raw data so it only contained the stuff I was interested in, then embedded the file in the Go code that powers RailMiles.

Now we have train routes and the locations those routes correspond to, the last thing we have to do is to render those to a map.

I chose to do all the visual mapping stuff with [Leaflet](https://leafletjs.com/) using the default OpenStreetMap tiles and some fancy OpenRailwayMap tiles as an overlay to emphasize the main railway routes atop that. On the main page, the routes and stations visited in the last month are compiled into a blob of GeoJSON and fed to this chunk of code:

```html
<div id="journey-map"></div>
<script>
    let map = L.map("journey-map");

    L.tileLayer('https://tile.openstreetmap.org/{z}/{x}/{y}.png', {
        maxZoom: 19,
        attribution: '...'
    }).addTo(map);

    let ormOverlay = L.tileLayer('http://{s}.tiles.openrailwaymap.org/standard/{z}/{x}/{y}.png', {
        attribution: '...',
        minZoom: 2,
        maxZoom: 19,
        tileSize: 256,
        className: "tile-orm", // this is used to make the ORM tiles greyscale
    });

    let layerControl = L.control.layers({}, {"OpenRailwayMap": ormOverlay}).addTo(map);

    let gj = L.geoJSON("<< geoJSON goes here >>", {onEachFeature: (feature, layer) => {
        if (feature.properties && feature.properties.name) {
            layer.bindPopup(feature.properties.name);
        }
    }});
    gj.addTo(map);
    map.fitBounds(gj.getBounds())
</script>
```

# Problems

As much as I'd like to say that this is perfect and works 100% of the time - it doesn't. Sometimes it falls flat on its face and ceases to work. The main issues I know about are:

* I'm using web scraping - RealTimeTrains doesn't have distance data in their API, probably for a reason, so if they cotton on to what I'm doing, I wouldn't be surprised if they stop me from doing it
* Sometimes, distance data just isn't available and isn't shown on the page, which stops the whole thing from working.
* It's a bit slow

I have a couple of ideas of how to fix these. More posts to follow!
