Author: Peter Hicks

Open Rail Data – Two Years On

After a brief, but really interesting visit to the former Bletchley PSB (or signalbox, if you’re less of a railway geek), I popped in to OpenTech 2013 to present an update to the presentation I gave two years ago.

In some ways, we’ve come a long way – in others, maybe not. Regardless, there’s scope for opening up more data to make us all more aware of what’s going on – suggestions immediately afterwards included getting data on cable theft incidents, counts of people going through ticket barriers at stations in real-time, plus passenger counts from trains.

My presentation is available if you missed it, or if you want to cut-out and keep. Exciting times 🙂

Really Useful Things

Here are five things that help my quest to reduce the amount of time I spend supporting and increase the amount of time I spend doing:

  • Google Apps for Business – £3.30 per user per month, and it means I can use Google Drive (although an Ubuntu client that isn’t InSync would rock) to hold PDFs and reference data I use all the time, and not have to worry about email hosting.
  • LastPass – password management. Much easier than having a GPG-encrypted text file that you sync to DropBox. Premium is only $12 (£8) per year.
  • GitHub – online project hosting using Git. I use GitHub by default for almost everything I do, private and public. A snip at $7 (£4.60) per month.
  • Atlassian Hosted JIRA – which, love it or hate it, works well for OpenTrainTimes bug tracking. Reasonably priced at $10 (£6) per month for up to 10 users.
  • Evernote Premium – write notes, sync them automatically and go search. I still love my Moleskine notebook for taking most meeting notes, and it looks great too – but sometimes you just gotta type electronically. £4 per month, but I really wish they’d come up with a sensible client for Ubuntu.

Installing an Olive on VirtualBox

Although it exists in many other places, I’ve not found a comprehensive set of instructions for installing JunOS 11.4 under VirtualBox that actually works. As I found, It isn’t too difficult, and only took me a day or so.
You’ll need to create a FreeBSD machine in VirtualBox with 1Gb of RAM and 5Gb of disk space. Select one or more network interfaces as the Intel PRO/1000 MT Desktop adapter. If you’re running on a UNIX system, additionally redirect the COM1 serial port to a host pipe called /tmp/com1. Use the command socat /tmp/com - to show the output from the serial console, which is useful after booting the Olive for the first time.

Installing FreeBSD

  • Download the FreeBSD 4.4 mini ISO from ftp://ftp-archive.freebsd.org/pub/FreeBSD-Archive/old-releases/i386/ISO-IMAGES/4.4/4.4-mini.iso
  • Create a FreeBSD machine with 1Gb of RAM. Create a VDI startup disk (either dynamically allocated or fixed size) of 4Gb
  • Edit the machine settings and enable network adapters as Intel PRO/1000 MT Desktop (82540EM) adapters. Although FreeBSD 4.4 won’t support these, JunOS 11.4 will.
  • Attach the ISO image to the CD/DVD drive in the machine, and boot it.
  • When FreeBSD boots, select ‘Skip kernel configuration and continue with installation’.
  • At the /stand/sysinstall menu, select a Standard installation.
  • At the ‘FDISK Partition Editor’ screen, delete any existing slices and create a single FreeBSD slice covering the entire disk by pressing ‘A’. Press ‘Q’ to finish – changes are automatically saved.
  • At the ‘Install Boot Manager for drive ad0?’ page, select ‘Standard’ so as not to install a boot manager.
  • At the ‘FreeBSD Disklabel Editor’ screen, create partitions as follows:
    • 1G filesystem mounted on /
    • 512M swap partition
    • 512M filesystem mounted on /config
    • Remaining space in a filesystem mounted on /var
  • Press ‘Q’ to finish – changes are automatically saved.
  • At the ‘Choose Distributions’ page, select ‘X’ to exit’.
  • At the ‘Choose Installation Media’ page, select ‘Install from a FreeBSD CD/DVD’.
  • The disk will now be partitioned, filesystems created and FreeBSD installed.
  • After installation, the following questions will appear. Answer ‘No’ to each:
    • Would you like to configure any Ethernet or SLIP/PPP network devices?
    • Do you want this machine to function as a network gateway?
    • Do you want to configure inetd and simple internet services?
    • Do you want to have anonymous FTP access to this machine?
    • Do you want to configure this machine as an NFS server?
    • Do you want to configure this machine as an NFS client?
    • Do you want to select a default security profile for this host?
    • Would you like to customise your system console settings?
  • Answer ‘Yes’ to “Would you like to set this machine’s time zone now?”. Select ‘No’ to “Is this machine’s CMOS clock set to UTC?”, then select ‘8 – Europe’, ’42 – United Kingdom’ then ‘1 – Great Britain’. Answer ‘Yes’ to “Does the abbreviation ‘BST’ look reasonable?”
  • Answer ‘No’ to “Would you like to enable Linux binary compatibility?”
  • Answer ‘No’ to “Does this system have a USB mouse attached to it?”, then select ‘Exit’ at the “Please configure your mouse” menu
  • Answer ‘No’ to the question regarding browsing the FreeBSD package collection.
  • Answer ‘No’ to the question regarding adding initial user accounts.
  • Set a password for the ‘root’ user.
  • Answer ‘No’ to the question regarding the last chance to set options.
  • Select ‘X’ to exit installation, detach the ISO image and select ‘Yes’ to the “Are you sure you wish to exit?” question.
  • The virtual machine will now restart.

Creating a JunOS installation image

Download junos-olive-patch.sh and run it against a standard JunOS installation image, for example:
user@host:~$ ./junos-olive-patch.sh jinstall-11.4R2.14-domestic.tgz
This will unpack and patch the installation file, replacing ‘checkpic’ in the pkgtools archive with a symbolic link to /bin/true so the package will install on an Olive.
To get this installation package on to VirtualBox, make it in to an ISO file using mkisofs:
user@host:~$ mkisofs jinstall-11.4R2.14-domestic.tgz > olive.iso
Attach the ISO image to the Olive in VirtualBox, then mount the ISO file on FreeBSD by typing mount /cdrom. Install the package by running pkg_add -f jinstall-11.4R2.14-domestic.tgz.
Reboot, and wait for the BTX loader screen to disappear – this may take several minutes. If you’re using socat to monitor the output of the console, you’ll see JunOS being installed.

Installing WebSphere MQ 7.5 on Ubuntu 12.04LTS

An item on my mental To Do list for some time has been getting WebSphere MQ installed on Ubuntu. I don’t use it in a production environment, therefore an IBM PartnerWorld Software Access agreement gets me the software for a year, and a virtual machine with Ubuntu circumvents the need to buy Red Hat Enterprise Linux.
With many things I do that could potentially turn out complicated or non-trivial, I document the entire process. I’m assuming you’re running a 64-bit version of Ubuntu 12.04LTS at this point, and that you’ve also read Quick Beginnings for Linux on IBM’s site.
First, install the ‘rpm’, ‘pax’ and ‘default-jre’ packages. IBM distribute MQ as a set of RPMs, and I decided to install these rather than rebuild them as .deb files.
Next, create /etc/sysctl.d/50-webspheremq.conf with the following:
kernel.msgmni=1024
kernel.shmmni=4096
kernel.shmmax=73834496
kernel.shmall=2097152
kernel.sem=500 256000 250 1024
fs.file-max=32768
net.ipv4.tcp_keepalive_time=500
Make these changes live by running sudo sysctl -p.
Run sudo ./mqlicence.sh, read and agree to the licence.
‘crtmqpmg’ expects to find ‘pax’ in /usr/bin, so create symbolic link to ‘pax’ by running sudo ln -s /bin/pax /usr/bin/pax, then create /usr/lib64 using sudo mkdir /usr/lib64 as the installation process will want to write files in there.
Create a set of MQ packages by running ./crtmqpkg $ID, where $ID is an installation identifier, such as ‘dev’ or ‘prod’. This may take some time. When it’s finished, change to /var/tmp/mq_rpms/$ARCH and install the following RPMs using the command sudo rpm -ivh --nodeps --force-debian:

  • MQSeriesRunTime
  • MQSeriesServer
  • MQSeriesSDK
  • MQSeriesClient
  • MQSeriesJava
  • MQSeriesSamples
  • MQSeriesGSKit
  • MQSeriesMan

Set this MQ installation as default using sudo /opt/mqm/bin/setmqinst -i -p /opt/mqm, then run dspmqver to verify the installation is correct.
Finally, add your user in to the ‘mqm’ group by running sudo addgroup $USER mqm. Log out and log in for these changes to take effect.

Opening Great Britain’s Rail Data

It’s my first time in Helsinki, and the weather is much the same as a September day in London – wet.
I finished preparing for my talk at OKFestival a little under 24 hours ago, and it went without a hitch. These things are normally OK once you’ve finished worrying about them.
Anyway, the slides and video (which no longer works as of February 2020) of my presentation on Open Train Times and Opening Great Britain’s Rail Data are now online. Enjoy!

The Chancellor's Autumn Statement

Many people reading my blog are interested in Open Data – here are the three important paragraphs from the Chancellor’s Autumn Statement earlier, as they relate to Open Data:
“1.125 Making more public sector information available will help catalyse new markets and innovative products and services as well as improving standards and transparency in public services. The Government will open up access to core public datasets on transport, weather and health, including giving individuals access to their online GP records by the end of this Parliament. The Government will provide up to £10 million over five years to establish an Open Data Institute to help industry exploit the opportunities created through release of this data.”
“A.146 Open Data Institute – The Government will provide up to £10 million over five years, with match-funding from industry and academia, to establish the world’s first Open Data Institute to help business exploit the opportunities created by release of public data”
“A.140 Rail fares data – The Government will consult in early 2012, through the Fares and Ticketing Review, on providing open access to rail fares data, giving passengers and business better information and enabling them to make the most cost-effective travel choices.”
The Cabinet Office website has further details in a PDF here.
I’ll leave it at that for the moment – other people will doubtless be writing their take on it, but I’ll leave you with one word from me: positive.

VirtualBox is too clever

I have a desktop machine connected to a Gigabit Ethernet switch, plugged in to my Cisco 1801 router. The switch is dumb and doesn’t support VLAN tagging, which is fine as I have everything in a single VLAN.
My desktop machine works fine with IPv6, however when I brought up a virtual machine on VirtualBox, IPv6 breaks for that particular VM. “ip addr show” tells me that it’s autoconfigured three IPv6 addresses, each on separate subnets – one of them correct, two wrong (but valid on other subnets here).
What’s going on? It turns out I’d set the port to my GigE switch up as a trunk port, with the native VLAN being the user access VLAN, and two other VLANs allowed. What this does is to send frames in VLAN 4 on to the wire untagged, and any untagged frames received automatically drop in to VLAN 4. However, any broadcast frames from VLANs 5 and 6 are sent on to the wire with an 802.1q header – and VirtualBox, being very clever, seems to pass these up to the virtual machine without their 802.1q headers.
I may file a bug – that’s too clever 🙂

SSL Certificate error whilst using rvm

Given that I have a lovely clean install of Ubuntu 11.10, I decided to use rvm to manage by Ruby installation. The only problem with that is the SSL certificate beginrescueend.com uses isn’t known to Ubuntu.
I’ve seen a post advocating using -k to ignore CA certificate validation, but that’s not the right thing to do. It works, but only by defeating a security mechanism.
Cut to the chase – how to get around it:

  • Read the error message curl produces, which suggests visiting http://curl.haxx.se/docs/sslcerts.html
  • From there, download cacert.pem and copy this to your home directory
  • Create ~/.curlrc, with the single line cacert = ~/cacert.pem
  • Test using curl https://rvm.beginrescueend.com/ > /dev/null – the CA error should not appear

Ubuntu 11.10 for Productive People

Following on from my blog post, “Dumbed Down Ubuntu”, I’ve spent the past 24 hours trying other distributions with varying levels of success.
From reading Mark Shuttleworth’s blog, I’ve found strong feelings out there on Ubuntu’s default user interface. I agree with some of them – that the new Ubuntu UI is not aimed at a certain demographic of people who are power users. Having installed Ubuntu 11.04 for a friend of mine, she loved the Unity interface, and I left her to it.
I’ve played with Fedora 15 and Linux Mint 11, but neither quite worked the way I wanted them to. A specific show-stopper in Fedora’s case was Spotify’s lack of RPM. For Linux Mint, it felt like a reskinned Ubuntu. One really interesting thing I foudn is thatZalman have really cool SATA enclosure which can act as a CD-ROM drive for ISO files, and that’s on my shopping list.
After trying Ubuntu in VirtualBox, and deciding to persevere with taming the UI, I came across a handful of steps to get the interface back to something that people like me will use. Here’s how:

  1. Install Ubuntu 11.10
  2. Install Gnome using apt-get install gnome
  3. Remove the scrollbar eye-candy using apt-get remove overlay-scrollbar
  4. Log in, but select Gnome (Classic) from the gear icon next to your username
  5. Install the Gnome Tweak Tool using apt-get install gnome-tweak-tool, select Fonts and set the text scaling factor to 0.8, and Hinting to ‘slight’, then under Theme, set the GTK+ them to Ambiance, the Icon theme to Gnome and Cursor theme to Adwaita (if they aren’t already)
  6. Alt-Right Click on the bar at the top of the screen, select Properties, Background, and set a solid colour of opaque #3F3E39
  7. Alt-Right Click on the bar again, and select New Panel. Alt-Right Click on the bar at the bottom, then add a Workspace Switcher and a Window List in the bottom right and bottom left corners

Ta-da. That’s a user interface that I’m more than happy to use, and importantly, one I think lots of other non-novices will enjoy using.
Something I forgot in my original post is to say how much I value choice. I tried using Windows over a decade ago as a desktop machine, but quickly became irritated with the way it worked. Slackware served me fine, Debian was great, and I started to use Ubuntu several years after that. I’ve tried a beta of Ubuntu 11.10, and I didn’t like it, so went and tried – for free – two other distributions. Neither quite did it for me, so I came back to square one and found out how to mould Ubuntu’s interface to the way I like it.
I hope some of you find my experiences useful, particularly how to tame the UI.

Dumbed Down Ubuntu

Whilst I am in full support of making computers accessible to everyone, I don’t believe that everyone should have to use them in the same way.
I’ve been an Ubuntu user for, I reckon, three years, and Debian for years before that. In that time, it’s been great to use – install, get on with doing what I do, minimal fuss. But like Windows, the people behind Ubuntu just had to make the user interface ‘easier’ and ‘better’, but they’ve done it at the expense of power-user functionality.
I am a power-user, I know what I’m doing. I don’t think it’s elitist to want to have a UI that lets me get on with my work, one that doesn’t have graphical elements fading on and off the page gently whilst you’re trying to work. One where you can grab the bottom left or right hand corners of a window to resize it, or even one where the close, minimise and maximise windows are in the same place all the time. Ubuntu is no longer that.
I can’t quite put my finger on what’s changed, but it no longer feels efficient. One of the bigger gripes I have is the ‘thingy’ on the left with the large icons. I don’t know the name of it, but it has a set of hideously large icons which come on-screen when I hover near them, and disappear after. I don’t want that – I want to be able to throw a handful of icons in the bar at the top of the screen, and access anything else via a drop-down menu.
I want decent scrollbars back, like in Chrome – not a control that appears outside the window when I hover my mouse in a specific place. My Terminal window is easy to re-size using the top left and top right corners, but next to impossible to resize at the bottom left and right – that’s the way I want to re-size my windows.
Apparently, I can Install the ‘Classic’ desktop in Ubuntu 11.10, but with plenty of caveats. Having done that, my desktop looks even less usable.
Sorry, Ubuntu – I think I’ll revert back to Debian, at least until you’ve sorted yourself out.