Author: Conal

Zotero, Web APIs, and data formats

I’ve been doing some work recently (for a couple of different clients) with Zotero, the popular reference management software. I’ve always been a big fan of the product. It has a number of great features, including the fact that it integrates with users’ browsers, and can read metadata out of web pages, PDF files, linked data, and a whole bunch of APIs.

zotero

One especially nice feature of Zotero is that you can use it to collaborate with a group of people on a shared library of data which is stored in the cloud and synchronized to the devices of the group members.
Continue reading Zotero, Web APIs, and data formats

Proxying: a trick to easily add features to existing websites and applications

At the start of last month I attended the LODLAM (Linked Open Data in Libraries, Archives and Museums) Summit in Sydney, in the lovely Mitchell Library of the State Library of New South Wales.

The Summit is organised as an “un-conference”. There is no pre-defined agenda; it’s organised by the participants themselves at the start of the day. It makes it a very participatory event; your brain is in top gear the whole time and everything is so interesting you end up feeling a bit stunned at the end of the day.

One of the features of the Summit was a series of very brief talks (“speedos”) on a variety of topics. At the last minute I decided I’d contribute a quick rant on a particular hobby-horse of mine: the value of using proxies to build web applications, Linked Open Data, and so on. Continue reading Proxying: a trick to easily add features to existing websites and applications

Old News for Twitter

Yesterday I finished a little development project to build a TwitterBot for New Zealand’s online newspaper archive Papers Past.

What’s a “TwitterBot”? It’s a software application that autonomously (robotically, hence “-bot”) sends tweets. There are a lot of TwitterBots tweeting about all kinds of things. Tim Sherratt has produced a few, including one called @TroveNewsBot which tweets links to articles from the Australian online newspaper archive of Trove, and this was a direct inspiration for my TwitterBot. Recently Hugh Rundle produced a TwitterBot called Aus GLAM Blog Bot that tweets links to blog posts by people blogging in the Australian GLAM (Galleries, Libraries, Archives and Museums) sector. People like me. I’m looking forward to seeing Hugh’s bot tweeting about my bot. Continue reading Old News for Twitter

Public OAI-PMH repository for Papers Past

I have deployed a publicly available service to provide access in bulk to newspaper articles from Papers Past — the National Library of New Zealand’s online collection of historical newspapers — via the DigitalNZ API.

The service allows access to newspaper articles in bulk (up to a maximum of 5000 articles), using OAI-PMH harvesting software. To gain access to the collection, point your OAI-PMH harvester to the repository with this URI:

https://papers-past-oai-pmh.herokuapp.com/ Continue reading Public OAI-PMH repository for Papers Past

Beta release of XProc-Z web server framework

I have at last released a “final” version of my web server framework, XProc-Z, for testing. The last features I had wanted to include were:

  • The ability for the XProc code in the web server to read information from its environment, so that a generic XProc pipeline can be customized by setting configuration properties.
  • Full support for sending and receiving binary files (i.e. non text files). XProc is really a language for processing XML, but I think it will be handy to be able to deal with binary files as well from time to time.
  • A few sample XProc pipelines, to demonstrate the capability of the platform.

XProc-Z-samples Continue reading Beta release of XProc-Z web server framework

LODLAM Challenge Entry

I have put an entry into the 2015 LODLAM Challenge, a competition for projects doing something with Linked Open Data in Libraries, Archives, or Museums.

Now my entry just needs people to view it and vote for it!

corbicula-video-screenshotMy entry is for a product called Corbicula, which I mostly wrote a couple of years ago, which I am now updating for release as a web application. It’s probably a month away from its first release, at least, but in the meantime, it’s still possible to VOTE FOR ME by clicking the “like” button.

Continue reading LODLAM Challenge Entry

Long running and asynchronous processes in XProc-Z

I added a useful feature to my web server framework XProc-Z, to allow it to run processes that take a long time – longer than the lifetime of an HTTP request.

Specifically, I had in mind running OAI-PMH harvests, in which a harvester may need to run for hours and make hundreds or thousands of HTTP requests of its own, in order to download a large set of metadata records. But any long running process faces the same issue: if a person makes a request to initiate a long running process, using a web browser, the server can’t afford to wait until the process is complete before it responds, because the user’s HTTP request will time out. Instead, the server needs to return a response to say “process has started…” and then to continue the processing in the background.
Continue reading Long running and asynchronous processes in XProc-Z

PROVisualizer – a tool for exploring archives

Last year I got a call from my mate Asa Letourneau, who works for the Public Record Office Victoria, the state archives of the Australian state of Victoria. Asa was plotting to develop a tool for visualizing the relationships between agencies and functions, as a way to help researchers find the records they need. I agreed to get on board, as a developer, and last month I got started on coding the visualization.

Continue reading PROVisualizer – a tool for exploring archives

THATCamp Canberra

Last week I travelled down from Brisbane to Canberra, to attend the THATCamp Canberra event at the National Library of Australia.

It’s been a very pleasant trip. I’ve been staying with friends I haven’t seen for a while, the spring weather has been lovely, and the environment here really is beautiful, with lots of leafy trees and birds everywhere.

It’s been a busy few days though; today at last I’ve had a chance to relax and reflect on the THATCamp, and I thought I’d jot down a few things.

Continue reading THATCamp Canberra