From Puppy to Maturity: Experiences in Developing Terrier

We will be taking part in the SIGIR 2012 Workshop on Open Source Information Retrieval. In particular, we have published a paper on the Terrier open source information retrieval platform, detailing the vision behind the platform, some recent developments in Terrier, as well as a roadmap for future releases.

As always, our vision for the Terrier platform is to continue empowering researchers and practitioners in information retrieval (IR) with up-to-date, easily adaptable, effective and scalable indexing and search approaches, allowing them to build and evaluate the next generation IR applications. 

In particular, Terrier will be moving towards feature-based retrieval, in line with the increasing importance of the learning-to-rank paradigm in modern information retrieval where machine-learned ranking functions combining multiple features are deployed. To do so, Terrier will be supporting the efficient and effective extraction of query-independent and query-dependent features.

To support scalability and efficiency, Terrier's data structures have undergone a major enhancement to support advanced dynamic pruning techniques, as well as the development of applications requiring distributed and real-time indexing and retrieval such as Twitter search.

Finally, the growth of the Terrier platform over the past decade into exciting new areas such as MapReduce indexing and crowdsourcing entails increased functionality, but also platform complexity. To avoid software bloat, we are moving from a monolithic release structure, to a system of periodic core releases and timely plugin expansions. The first such release will be the CrowdTerrier plugin, providing  researchers with an out-of-the-box tool to achieve fast and cheap relevance assessments.

A more comprehensive account of the forthcoming Terrier releases is detailed in our paper below:

From Puppy to Maturity: Experiences in Developing Terrier. Craig Macdonald, Richard McCreadie, Rodrygo Santos and Iadh Ounis. In Proceedings of the SIGIR 2012 Workshop on Open Source Information Retrieval. Portland, Oregon, USA. August 2012

We hope to see many colleagues joining us to work towards the objectives of the platform and enriching its functionalities. As always, we welcome suggestions and any feedback on the roadmap in the run up to the forthcoming Terrier 4.0.