HFM Technology - An Open Future

Gary Collier, co-CTO Man AHL, on why Man AHL is open-sourcing their data developments.

To succeed in the modern world of quantitative fund management, you’re going to need an open mind and even more open technology. Financial services organisations have been implementing open source technology for years. And why not? It’s often better quality software that delivers vastly better results. Even the Bloomberg terminal is going open source. But so far that trend has been all in one direction: take, take, take.

Taking advantage of an open source community isn’t enough anymore. Times have changed. To win the next decade of technology innovation, you need to give back and genuinely engage with open source projects.

Developers are hugely important people in technology driven organisations. And to work with the best developers you need to be unconstrained by conventional thinking about what’s proprietary and what’s not.

That’s why at Man AHL we recently announced we’re opensourcing a tick store that improved our data processing performance by 25x. It was built on the open source MongoDB database and replaced a proprietary solution. It’s now freely available to download on Github (a site developers use to share code). In this article I’ll explain why that’s only the beginning.

At Man AHL, we have a few core beliefs, and chief among them is that clients come first. Many firms would make the same claim. But where we differ is the approach we take to delivering a client-first business: We apply rigorous science to the financial markets. To be scientific we believe you need a culture of genuine freedom and collaboration.

One of the areas my team is responsible for is data infrastructure. How can we make the underlying technology faster, more efficient and easier to use? One important piece of data infrastructure that has changed most in recent years is the database. That’s the layer of software which stores and manages the data for an application. A slow or difficult to use database could make a system unusable, a good database can make it unbeatable.

MongoDB is part of a new generation of databases that don’t handle data in the traditional row and column fashion used by relational databases. It looks at data as whole JSON (JavaScript Object Notation) documents, which makes things more flexible for a developer and means that the application performs well at a massive scale. You won’t be surprised to hear, it’s open source too.

Two years ago we got to a point where we anticipated scalability issues with our tick store (a specialised type of data tool for storing high frequency financial services market data). It’s a crucial part of how our quantitative researchers make decisions.

The old version had the potential to begin to hinder their ability to do their job, so we knew we had to change. After a few trials, we created our own tick store using MongoDB. Called Arctic, the tick store is now the primary market data store for all our market data. By building Arctic with MongoDB, and the open source programming language Python, we realised a 40x cost saving when compared to the legacy time series data store. Processing performance improved by 25x. These were incredible results, and we could have stopped there.

We knew this new powerful tick store could have potential applications far beyond financial services. For example, it could also be deployed for a range of similarly demanding workloads such as internet of things applications or sentiment analysis. So earlier this year we decided to make it open source. But why give it away? Despite what you read, humans still matter. To have truly state-of-the-art technology, you need the best developers. The best can be hard to find.

The latest generation of coders were brought up in an open and collaborative world. They want to use open source software and they want to be able to contribute back to the communities that gave them so much. To be an attractive home for good developers, I believe you need to have a genuinely free and collaborative philosophy towards people and software. Making Arctic open source is a natural extension of that philosophy.

Arctic is just one example of the work we do to excite and challenge our developer community. We’re also active sponsors and participants at PyData (a gathering for users and developers of the Python programming language) and we recently launched a competition to find talented student developers in the UK and Ireland. The competition has students control a player in the classic game Hexplode.

It all comes back to those core principles. We want the best people, using the best technology, to apply a data-driven, scientific approach to finance. Right now, that’s open source software and the developers who are experts in it.

The focus is not just about making money today. It’s about providing value tomorrow, next week and next year. When you’re focus is truly on client value it doesn’t make sense to hide away your team’s triumphs and failures. That’s why the future of finance is wide open.