A problem isn't always what it seems

I had a call today from somebody who was trying to use an Avaya IP Phone from their office at home. The phone has built-in IPSec VPN capability, and their phone switch is some distance away on – surprisingly – a DSL line. Call quality is not an issue, which is testament to Zen Internet‘s network quality.
In the office, the phone works fine. At home, it fails to establish a VPN, displaying an “Invalid PSK” error. Looking on the VPN router at the site with the softswitch, I see nothing untoward, so I set about asking the caller to tell me his default gateway address – which is the same address as the network range that the softswitch is no. No prizes for guessing why it doesn’t work.
Here I was, expecting a full-on IPSec debugging session, and it turns out to be IP addresses.

TransportHacker

I’ve been hacking away at a project called TransportHacker for the last month or two.  I’ve only shown it to a few people I know, and the reactions have been largely positive – thanks to Laura, Su, John and Ellis for your feedback.
I am not usually one for giving ‘sneak previews’, but in the spirit of getting feedback and ideas, I present – for your viewing pleasure – a sneak preview of TransportHacker.
TransportHacker sneak preview
TransportHacker takes its data from a variety of sources – currently some of those on the TfL Developers’ Area, although adding new data is straightforward. I’m putting TfL’s live traffic cameras up in the next few days, and experimenting with the Highways Agency’s DATEX II feed to see if I can include other parts of the country.
TubeHorus will be integrated, but also be available standalone – both this and TransportHacker will use geolocation to display only the parts of the world you’re near.
I have plans to make the data richer and more exciting – at present it’s just a mash-up of some data with no value-added logic, and that kinda thing can only go so far before it’s dull. When TfL get around to releasing real-time iBus data, things are really going to start hotting up – imagine being able to compare taking the tube from Highgate to Warren Street versus taking the bus. Colour in the 134’s route in red where it’s congested, and lighter colours where it’s free-flowing. If I included the Journey Planner API – when that’s ready for public consumption – I could augment the information it returns with real-time performance data for different modes of transport.
Now that all my Christmas shopping is sorted, it’s going to be a fun few weeks while I get TransportHacker in to a robust and workable product. Christmas is perfect for that sort of thing, although I better go easy on the mulled wine…

TfL re-release Trackernet API

Back in June, Transport for London released their Trackernet API to the public. This is about as close an insight in to how the tube network is performing as you can get without being there in person. Its enormous popularity caused their internal system to collapse, and the service was pulled. What a way to demonstrate the appetite for this data!
Ever since, there’s been frequent talk of whether the API is going to return or not. I, for one, have been particularly looking forward to the day it returns so I can get to work adding more feeds to TransportHacker. In the meantime, I’ve resurrected TubeHorus.
That day was yesterday.
I was invited to a press conference at 55 Broadway, London Underground’s headquarters, where the TfL Developer area was relaunched with additional feeds, and importantly, the Trackernet API. This time, to cope with demand, it’s been placed on the Microsoft Azure platform, although Microsoft’s representative was keen to point out that it’s not just for .NET applications.
Despite having to rush out of the press conference to catch a train out of town, I’ve had a few hours playing with the API, and it differs little from the original service. The biggest changes I can see are:

  • URL change – it’s on a different server and the URLs are RESTified
  • There’s little server-side filtering of the data, so you may end up pulling more than you actually need
  • Data freshness – the data is only pushed out to the cloud every 30 seconds. I know at least one person who was deeply unhappy about this
  • You need to register – free – to get the URL for the service, but it’s not locked down with an API key

TfL really haven’t had an easy task to get here, and I salute their efforts. TrackerNet was a system designed to take multiple sources of data from the trackside and other operational systems, and present them internally in a coherent manner, including to drive other internal systems. It was never envisaged that the general public would have access to it, and so architectural decisions were probably made regarding its sizing that precluded making its data available en-masse.
So, TfL have set the standard. If they can build a platform to disseminate their real-time information sensibly, why can’t the likes of National Rail Enquiries? Hopefully NRE will see the benefits of making their real-time data (and whilst we’re at it, static data too) available without onerous contracts and agreements. Heck, NRE already have a scaleable platform for their Live Departure Boards service that can handle train information for the whole country – why are they concerned about scaling?
On a less political note, we’ve also been promised access to the Journey Planner API within the next few months, and there were some murmurs about real-time bus information, but nothing concrete.

Open Transport Data

The Guardian published an article on National Rail Enquiries’ refusal to be sensible about licensing its data. Malcolm Barclay has mused on NRE’s inflexibility, claiming “They are stuck in the command & control mentality of the industrial age and have zero understanding of what open data is or it’s benefits”.
This whole debacle is reminiscent of Eric S Raymond’s “The Cathedral and The Bazaar” for me – where
TfL have had no problems making their data available free of change, and they’re working really hard to bring the Trackernet service back to life. Emer Coleman, Director of Digital Projects at the GLA, posted “But you can be assured that it definitely will be back and hopefully the solution will be so robust there will be no chance of it falling over like it did the last time. That is TfL’s main concern that once it foes back its there for good and in a robust way”. Hats off to them – TfL were caught by surprise with the popularity of the Trackernet API, and they’re tackling it head-on.
If TfL can be this innovative and forward-thinking, why can’t NRE?
NRE’s jealous data-guarding is not just limited to real-time train running information. If you want daily-updates of fares, timetable and routeing guide (the official definition of the routes you can use with ‘Any Permitted’ route tickets – a complicated beast that few people properly understand), you will need to part with £27,430 according to ATOC’s RSP Data Feeds document. That’s a ludicrous price that serves only to lock data on our country’s rail system away from prying eyes. Heck, it’s £600 if you want a CD with test/trial information.
This data is probably of most value to companies who are deeply involved in selling tickets, such as TheTrainLine and those who