I have numerous HTTP services that I consume, each of them from different providers with different URLs. Some are updated every 60 seconds, some every hour, some less frequently. Regardless, I started to lose track of what was what, and looked for a solution.
Having used ActiveMQ for a couple of years for message queueing, I decided to look at Apache Camel and whether this could help out. The short answer is that is can, and here’s how.
The services I use fall in to three broad categories:
- Unauthenticated XML over HTTP
- Authenticated XML over HTTP
- Authenticated stream over HTTP
ActiveMQ already comes with Camel integration, so it’s a straightforward task of adding the following lines to activemq.xml:
Splitting the Camel configuration in to a separate file will help when upgrading.
*** CAMEL CONFIG ***
At the time of writing, ActiveMQ 5.9.1 doesn’t have all the necessary libraries bundled with it. Download the latest version of Apache Camel and copy the following libraries from the lib/ subdirectory in to lib/camel/ on the ActiveMQ server:
*** DO WE NEED http://archive.apache.org/dist/httpcomponents/commons-httpclient/3.0/binary/ JAR file in lib/optional? ***
The following Camel configuration will poll the TfL Tube Station Status feed every 30 seconds, outputting the response on to a topic TFL.TUBE.STATION.STATUS:
<description>TfL Tube Station Status</description>
Let’s take a look at each line in turn.
The first line sets up a timer called ‘tflTubeStatusTimer’, firing every 30 seconds.
The next line requests a URL:
The final line sends the data on to an ActiveMQ topic.