Peter Hicks' Blog

Blog

  • Open Transport Data

    The Guardian published an article on National Rail Enquiries’ refusal to be sensible about licensing its data. Malcolm Barclay has mused on NRE’s inflexibility, claiming “They are stuck in the command & control mentality of the industrial age and have zero understanding of what open data is or it’s benefits”.
    This whole debacle is reminiscent of Eric S Raymond’s “The Cathedral and The Bazaar” for me – where
    TfL have had no problems making their data available free of change, and they’re working really hard to bring the Trackernet service back to life. Emer Coleman, Director of Digital Projects at the GLA, posted “But you can be assured that it definitely will be back and hopefully the solution will be so robust there will be no chance of it falling over like it did the last time. That is TfL’s main concern that once it foes back its there for good and in a robust way”. Hats off to them – TfL were caught by surprise with the popularity of the Trackernet API, and they’re tackling it head-on.
    If TfL can be this innovative and forward-thinking, why can’t NRE?
    NRE’s jealous data-guarding is not just limited to real-time train running information. If you want daily-updates of fares, timetable and routeing guide (the official definition of the routes you can use with ‘Any Permitted’ route tickets – a complicated beast that few people properly understand), you will need to part with £27,430 according to ATOC’s RSP Data Feeds document. That’s a ludicrous price that serves only to lock data on our country’s rail system away from prying eyes. Heck, it’s £600 if you want a CD with test/trial information.
    This data is probably of most value to companies who are deeply involved in selling tickets, such as TheTrainLine and those who

  • Invoking Shibboleet

    I’ve just come off the phone to my bank – First Direct – having asked them about a problem viewing previous months bank statements. It just doesn’t work and tells me “No statements have been produced for this account”. This is highly likely to be an account-specific of server-side problem.
    Upon being asked what operating system I’m using (Ubuntu 10.10), which browser I have (Firefox 3.6.11) and who my ISP is (Andrews & Arnold), I am talked through deleting all my cookies and the browser cache, shutting down the browser, starting it again and logging in.
    Hey presto, the problem has been summarily worked around since I’ve been load-balanced to a completely different server on First Direct’s network as I can clearly see from the URL bar.
    I was precariously close to uttering “Shibboleet” when asked who my ISP was.

  • The Cyclist

    A couple of months ago, I bought myself a bike, and catapulted head-first in to cycling to and from work.
    It hasn’t been easy – dealing with less courteous road users who try to kill me, pedestrians who walk out from between buses, cars that overtake leaving precious little room for me – it’s a very big learning experience. The benefits are worth it though – the money I’m saving on my Travelcard is going on cycling gear for the first few months, and my body is changing shape in to something rather more pleasant than “chubby”.
    Within London, TfL offer free or subsidised cycle training. I took up their offer and had a couple of two-hour sessions with a qualified and knowledgeable instructor. She showed me so many techniques to help me get around safely – a couple I was already doing, others I wasn’t and should, and some – such as roundabouts – that I didn’t dare tackle.
    The result is a big increase in my confidence, and the feeling that I am assertive and aware enough on the roads to make the majority of cycling fun. If you have the opportunity to do so, take the cycle training – it may not be there forever.

  • Converting OSGB36 (Eastings/Northings) to WGS84 (Longitude/Latitude) in Ruby

    The excellent people at the Greater London Assembly have released a list of bus stops and bus routes in London. The coordinates of each bus stop are in eastings and northings, and I wanted to convert these to longitude and latitude for my Ruby on Rails application.
    Using the proj4rb gem and some projection definitions from spatialreference.org – with some help from Harry Wood’s blog, I came up with the following code:

    #!/usr/bin/ruby
require 'rubygems'
gem 'proj4rb'
require 'proj4'
easting = 529978
northing = 186491
srcPoint = Proj4::Point.new(easting, northing)
srcProj = Proj4::Projection.new('+proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=400000 +y_0=-100000 +ellps=airy +datum=OSGB36 +units=m +no_defs')
dstProj = Proj4::Projection.new('+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs')
dstPoint = srcProj.transform(dstProj, srcPoint)
print "lat=#{dstPoint.lat * (180 / Math::PI)}n"
print "lon=#{dstPoint.lon * (180 / Math::PI)}n"

    To convert WGS84 to OSGB36:

    #!/usr/bin/ruby
require 'rubygems'
gem 'proj4rb'
require 'proj4'
latitude = 51.5623279577278
longitude = -0.126277004538848
srcPoint = Proj4::Point.new(longitude * (Math::PI / 180), latitude * (Math::PI / 180))
srcProj = Proj4::Projection.new('+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs')
dstProj = Proj4::Projection.new('+proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=400000 +y_0=-100000 +ellps=airy +datum=OSGB36 +units=m +no_defs')
dstPoint = srcProj.transform(dstProj, srcPoint)
print "easting=#{dstPoint.x}n"
print "northing=#{dstPoint.y}n"

  • Facebook Data 'Leak' – it's not news, people

    Security consultant Ron Bowles trawled Facebook’s public directory and published ‘data’ on 100,000,000 users. Some people are up-in-arms about the leak, crying that ‘something must be done’, but not what. Few people seem to have looked at the data and given a coherent response. In fact, there’s nothing exciting.
    The data is 2.8GB and I’ve spent the last twelve hours downloading it. Good news, everybody – it’s rubbish! Your bank details, email addresses and the name of your first born child are not there. Neither is your date of birth, your location, or a photo. A telephone directory contains more information on you.
    The data is simply a list of names and a large file containing URLs to entries in the directory with seemingly no relation to the names.
    There’s some other data which are derived works of the original file – these are described as follows:

    Filename                        Description
-------------------------------------------------------------------------
facebook.rb                     The script used to generate these files (v1)
facebook.nse                    The script that will be used for the second pass (v2)
facebook-urls                   The full URLs to every profile
facebook-names-original         All names, including duplicates
facebook-names-unique           All names, no duplicates
facebook-names-withcount        All names, no duplicates but with a count
facebook-firstnames-withcount   All first names (with count)
facebook-lastnames-withcount    All last names (with count)
facebook-f.last-withcount       All first initial last name (with count)
facebook-first.l-withcount      All first name last initial (with count)

    So in summary, it’s a load of hot air.
    What data appears for me? Looking through all the files – my name appears in facebook-names-unique once. And that’s it. According to Facebook’s directory, there are at least 192 people called Peter Hicks.

  • Getting fitter

    A month and a half ago, a friend of mine went to an absolute beginners fencing class. Having done next to no regular exercise other than walking to and from the tube, four hours of activity did me in and my thighs and legs ached for the next two days. Despite the physical symptoms, I was ready for more and signed myself up for a six week course which ended yesterday. I’ve had several lessons of private tuition too – it’s great fun, hard work, but I don’t notice the physical activity so much as the technique and tactics.
    In conjunction, and to get rid of the “IT Thighs” that I’ve developed after over a decade of running technology, I’ve taken up cycling. This is a great activity which is a lot faster than the bus, and exercises muscles I never before believed I had – which include my awareness of the back-streets of London.
    My goal for both of these is to shape up, tone up and attain thighs of steel. If I can lose two stone, I’ll be over the moon 🙂

  • Importing SSL certificates on Cisco IOS

    A requirement came up to use Cisco’s AnyConnect VPN on a router. For this, an SSL certificate and corresponding private key is required – I used CAcert.org.
    I will deliberately skip the detail of how to generate an RSA private key, create a CSR and get this signed by a CA. Straight to the chase – here is how to import the key and certificate on to an IOS router.
    Use openssl rsa -in foo.key -pubout to display the corresponding public key for your private key foo.key. This will begin with BEGIN PUBLIC KEY.
    Next, ensure your private key has a password – use openssl rsa -in foo.key. If it doesn’t, encrypt it with 3DES using openssl rsa -in foo.key -3des and specify a password.
    On the IOS device in question, use crypto key import rsa foo pem terminal to import the PEM encoded public key:

    cr(config)#crypto key import rsa foo pem terminal strongpassword
    % Enter PEM-formatted public General Purpose key or certificate.
    % End with a blank line or "quit" on a line by itself.
    -----BEGIN PUBLIC KEY-----
    ...

    Ensure there is a complete blank line after pasting the public key, and the router will then prompt:
    % Enter PEM-formatted encrypted private General Purpose key.
    % End with "quit" on a line by itself.
    Paste in the 3DES-encrypted key – it will begin BEGIN RSA PRIVATE KEY, and type ‘quit’ on a blank line at the end.
    That’s it. It’s not straightforward, and I know I’ll forget if I don’t write it down!

  • Real-time Tube Tracking

    I have a lot of respect for the folks at the Greater London Assembly, especially those who worked to get the Train Prediction API exposed and available.
    Many people have seen Matthew Somerville‘s Live Map of Underground Trains which was whipped up in a frighteningly short time.
    I’m working on a Rails interface to the Train Prediction API, with an ‘advanced’ mode for those who grok the tube. It’s a little rusty, and not even beta-quality, but it’s available for you to play with if you so wish.
    Here’s hoping that particular box stands up to the load 🙂

  • Reducing Nagios' CPU load when using SNMP plugins

    I have a virtual machine which has 100 hosts and 600 services being monitored through Nagios. 400 of these services are monitored via SNMP plugins.
    One of our system administrators mentioned that this VM was quite CPU hungry, so I set about trying to lighten things up. I found that the simple act of adding a -M MIB-NAME switch in to a service check has dropped the load average from around 1.7 to 0.8 over 15 minutes.
    Here’s how to do it:
    First, look for any check_snmp plugin which uses an textual OID. Convert this to a numeric OID by using snmptranslate IF-MIB::ifOperStatus -Of. Use the numeric OID shown in place of the textual OID – this will save a few CPU cycles.
    Unless you need to translate the returned values back to text – for example, if an enum is returned that needs to be translated in to text – don’t specify ‘-m’ on the command line at all. However, if you do need to translate the returned values, specify -m IF-MIB, or whichever MIB name appears before the :: when translating the textual OID to a numeric OID.
    I’m happy, our sysadmin team are happy 🙂

  • One wisdom tooth out

    I’ve had one of them out (warning: contains graphic images of an extracted tooth).
    It really wasn’t painful – the most uncomfortable thing was the injection in the palatal side of the gum, and I heard a noise not dissimilar to the squeezing of a cork in a wine bottle as it came out. One suture – I think – and a bite down on some gauze for a bit, and I’m fine.
    I am still a little high on the lidocaine, but that will pass.
    If you’re worried about having a wisdom tooth extracted – don’t be. The worst bit is the anticipation!