Reviewing Airbnb – using a different metric to measure short-term let activity during lockdown

In the third of our blogs on the impact of the current crises on the short-term lets market, we use a different metric to assess the change in activity in Edinburgh and Glasgow and update our analysis to the end of April 2020.

In previous blogs in this series, we have looked at the impact of the current COVID-19 crisis up to the 21st March 2020. Those analyses used a complex methodology to identify those who had cancelled their bookings. However, a simpler approach is to look at the number of reviews left by visitors in order to get an estimate of Airbnb activity. One weakness in this approach is that not all visitors leave reviews. It has been estimated that somewhere between 50% and 90% of visitors leave reviews(1). There may be differences between the type of guest who leaves a review and those that don’t. Nevertheless, using reviews gives us a very clear indication of the activity of Airbnb listings in the months before and following lockdown.

Up until now, we have used data from the American website Inside Airbnb. While the authors are clear about the methods they use and publish their code, they also have a clear campaigning stance. From early this year the Urban Big Data Centre began to scrape Airbnb.com to widen the number of cities within the UK for which data was available and to ensure clear and objective methods and analysis. For this blog, we extracted daily reviews left by the customers who visited Edinburgh and Glasgow from January to April 2020. We excluded visits without reviews, or reviews that are generated automatically from a cancelled booking.

Table 1: Listings and Reviews for Glasgow and Edinburgh January-April 2019-2020

While the numbers of Airbnb listings have continued to grow in Edinburgh and Glasgow, reviews for the two cities have reduced. This does not give us the whole picture and, while there appears to be fewer reviews, examining this over time provides a much clearer picture of the effect of the current crisis on Airbnb activity.

We have graphed the number of reviews over each four-month period, aligning the dates in the two periods (2019 and 2020) and removing the 29th February in 2020. We also highlight three different dates on the graphs: the Airbnb cancellation policy which allowed users to cancel with a full refund; the date of the UK lockdown; and the date that Airbnb blocked new reservations.

Figure 1: Number of reviews over time January to April 2019 and 2020

We see similar patterns in both cities with a reduction in reviews to zero by the end of April. In Edinburgh, the number of reviews starts to reduce much earlier and by the time lockdown is announced reviews are already down to almost zero. While the number of Airbnb listings in Glasgow is much smaller, there are increased reviews in 2020, compared to 2019 and these do not appear to tail off until Airbnb introduce their cancellation policy (14th March). However, the overriding picture in both cities is that activity had decreased considerably by the introduction of lockdown measures and was minimal by the end of April. We will continue to monitor this to examine how quickly the market rebounds when lockdown restrictions in Scotland are relaxed.

Notes

1. Some articles stated that Airbnb estimated 90% of customers leave a review. But review rate also depends on where the cities are. Inside Airbnb estimates 50% of customers leave a review in Edinburgh.

Project Team

Research team (and the authors of this blog): Dr Mark Livingston; Dr Yang Wang; Dr David McArthur; Professor Nick Bailey

Data science team: Nikos Ves; Dr Andrew McHugh