How much does it cost to rent a private property in your area?


An ONS experiment with using Zoopla data for official stats

When the Urban Big Data Centre met with the Office for National Statistics to work together on experimenting with the use of big data in compiling official statistics, we were all excited by the possibilities. The Financial Times were too! We had already embarked on our own research using data gathered from the Zoopla housing website over time, to investigate the private rented sector. ONS has been exploring a number of areas using big data, including using machine learning to identify caravans in Zoopla data. In this blog, Nigel Henretty of the ONS Public Policy Division explores the reasoning and methods behind his investigation of Zoopla data, obtained from the UBDC’s data collections, for possible future use in compiling official national statistics. For more details, view his slides (PDF 866KB).

Clare Tilden (Data Sharing and Supplier Manager at ONS), said: “We have enjoyed working closely with the Urban Big Data Centre, who so helpfully supplied us with the Zoopla data they have been collecting for research use. This collaboration is an excellent example of publicly funded organisations sharing resources and expertise for the wider benefit of society. We hope these initial experiments using big data from UBDC will expand into other areas soon”.

How much does it cost to rent a private property in your area?

I’d really like to be able to answer that question, and so would housing policy makers in local authorities up and down the country. Given that the size of the private rented sector has near enough doubled over the past decade, it might come as a surprise to you that we can’t actually measure average rent prices for small areas using official data. At the risk of embarking on months of migraine-inducing data wrangling, I asked myself a dangerous question. How hard can it be?

Long story short: Quite hard.

There are already some official stats that can help point us in the right direction. First, there’s the snappily titled Index of Private Housing Rental Prices, which tells us about inflation of rent prices at the national and regional level. Then there’s the Valuation Office Agency’s private rental market statistics, which provide an indication of average rent prices at the local authority level based on a sample of rental data. Trouble is, neither of these was originally set up to provide small area statistics (because of the model-based method and the sampling respectively) and that’s what policy makers are now really after in trying to understand their local areas.

There is some hope though. A few years ago we started using admin data on property transactions from the Land Registry to produce House Price Statistics for Small Areas (the first rule of housing stats is that they must take longer to pronounce than produce). The address-level data we use for this allows us to produce small area statistics on the price paid for residential properties. Since then, a whole range of address-level admin data has become available, including some property website data which might just help us crack the small area rent price problem.

Zoopla. It’s pretty much the Amazon website of the UK property market these days, and as such holds data which offers a rich source of information about properties for sale and for rent. We’ve used these data to work out the average advertised monthly rent price at the small area level for the whole of Great Britain between 2010 and 2016. Let’s call this ‘Advertised Private Rent Price Statistics for Small Areas’ and maintain the theme of ridiculously long titles shall we? We’ve also looked at the number of rental property listings over this period, to get an indicator of rental market activity.
Fig 1: Understanding Zoopla Data


Our initial analysis of the first outputs suggests that the data do provide us with a reasonably comparable set of rent prices to the current official statistics for the larger geographies. The fact that we have some first outputs to analyse also suggests that we’ve found a broadly sensible way to handle and geo-reference the rather large amount of admin data.

Like all good statistics though there are of course many, many caveats and limitations which we need to understand and hopefully overcome. Here are just a few of those, starting with the most challenging:

  • The data relate to properties advertised for rent and so doesn’t cover all rented properties. The difficulty is in describing the extent to which advertised rent prices are representative of all rent prices.
  • Not all properties advertised for rent are included in the data. Some letting agents only use their own website, or other property websites, to advertise properties for rent.
  • Not all rented properties are advertised at all, anywhere. A substantial part of the private rented sector is casual, and these properties may never appear in property website data or any admin data.
  • Some areas appear to have no properties advertised for rent in the entire time-series of data and we don’t really know why.

We now need to work with the users of rent price statistics to find out if this new output meets their requirements, and whether we can use other data sources to overcome some of the limitations. We’d also like to produce information about the rent prices of different property types and sizes. Ultimately we aim to have a suite of rent price statistics that are comparable across geographies, which will help policy makers genuinely meet the demand for housing in their areas.

In the statistical utopia of the future, I suspect all this will be possible automatically and programmatically from a constant stream of linked and open big data. Until that happens, I still have a job.

(This blog was first published on National Statistical on 13 October 2017)

Author: Nigel Henretty

Nigel Henretty is part of the Public Policy Division at the Office for National Statistics.

Leave a comment

Your comment