Planning London DataHub - London
City / Region
London (United Kingdom)
In a nutshell
The Problem
To efficiently plan for and deliver services for Londoners, we need to understand how the city is changing, both on a micro and macro level. No dataset existed that told us how much development was due to take place in any part of the capital, how much floor space it was creating, meaning we were unable to quantify its impact, and what was happening in the planning system resulting in delays to housing delivery.
The Solution
The Planning London Datahub ("PLDH")is a collaboratively built dataset that collects data on all development proposals across the 35 planning authorities that can consent schemes, and creates a single uniform open dataset that enables the data to be used for any purpose.
All of the data that is submitted as part of a planning application form, is now collected in this central database, and matched with data provided daily by each of the planning authorities about the development consenting process (such as valid dates, decisions, restrictions/conditions on development.
Additional datasets are regularly added, such as whether developments are commenced and completed, appeal data and compliance data, enabling it to be described as a digital twin of the planning system in London.
The system is fully automated, meaning that there is no human intervention in creating the dataset, which now collects data on over 120,000 developments across London in any year.
Because of the digital set up, and local business process, the project involved over 450 people working across 43 organisations.
The Background
Prior to the PLDH, each of the 35 planning authorities provided skeleton data on permitted large developments meaning that all planning policy was developed, infrastructure planned and decisions made on data relating to c.4000 development proposals in any year. To maintain this service, it cost c.£750K per annum to taxpayers.
This project depoliticised the data, as well as reduced the operating costs of providing data.
This service is realised by the Greater London Authority, and the 32 London Boroughs, the City of London Corporation and the 2 Development Corporations).
What makes the service user-centric?
This project was developed using agile principles, meaning we started with a problem, and developed an understanding of the users experiencing the problem, as well as all of the other users of the data.
In this case we completed a discovery project exploring how the current datasets were developed and how the limited data set was then used. This blog from 2018 sets out some of the learnings https://chiefdigitalofficer4london.medium.com/improving-london-wide-planning-data-what-we-found-665de6b27d1a
Before landing on any technical solution the GLA instructed consultants, Atkins, to explore the personas of the service users, and develop epics of their experiences. This involved officers from each of the 35 planning authorities as well as the development community. To test the learnings we partnered with Plymouth City Council, who scrutinised the conclusions and tested them against their experience.
At every step in the process of the development of the solutions all partners to the project to able to explore options and solutions, resulting in a product that requires little or no human intervention to create the dataset. All parties signed up at the outset of the project to the core principle that the solution had to result in less work, rather than any work arounds, meaning that the final product is designed with variation in the process meaning has things change, it has adaptable solutions built into it, so no redesign is required. This includes both datasets, and mechanisms to create new ones.
The current phase of the project is predominantly focussed on improved data quality, because that is what the current end users of the data have requested. The most recent blog provides an update for all users on this https://sway.office.com/z20nJ7ozcDthz5MA?ref=Link
The next phase of the project is focussed on making the data accessible to Londoners, so they can see for themselves change as it is happening. We are starting the research on this at the moment.
To see an example of the data in action visit https://data.london.gov.uk/dataset/pld---monthly-planning-statistics and scroll through the different pages of analytics.
In this case we completed a discovery project exploring how the current datasets were developed and how the limited data set was then used. This blog from 2018 sets out some of the learnings https://chiefdigitalofficer4london.medium.com/improving-london-wide-planning-data-what-we-found-665de6b27d1a
Before landing on any technical solution the GLA instructed consultants, Atkins, to explore the personas of the service users, and develop epics of their experiences. This involved officers from each of the 35 planning authorities as well as the development community. To test the learnings we partnered with Plymouth City Council, who scrutinised the conclusions and tested them against their experience.
At every step in the process of the development of the solutions all partners to the project to able to explore options and solutions, resulting in a product that requires little or no human intervention to create the dataset. All parties signed up at the outset of the project to the core principle that the solution had to result in less work, rather than any work arounds, meaning that the final product is designed with variation in the process meaning has things change, it has adaptable solutions built into it, so no redesign is required. This includes both datasets, and mechanisms to create new ones.
The current phase of the project is predominantly focussed on improved data quality, because that is what the current end users of the data have requested. The most recent blog provides an update for all users on this https://sway.office.com/z20nJ7ozcDthz5MA?ref=Link
The next phase of the project is focussed on making the data accessible to Londoners, so they can see for themselves change as it is happening. We are starting the research on this at the moment.
To see an example of the data in action visit https://data.london.gov.uk/dataset/pld---monthly-planning-statistics and scroll through the different pages of analytics.
What impact has the service had?
For the first time, we have a single dataset that enables a clear picture of what has been proposed to be built, its position in the consenting process, what has been permitted, together with core data about the developments (such as size, thermal performance, developer and data points to enable it to be matched to other datasets.
The data was not previously held by any of the 35 local planning authorities in any format that enabled it to be accessed or analysed, meaning that decisions about planning the future of the city were being made on a limited evidence base, and arguably investment decisions were being made by both public and private bodies without knowledge of where population growth was taking place, and potentially could have had great impact.
Now decisions are being taken based on data about 120,000+ development proposals in any year, rather than 4000, and full rather than partial datasets are provided for each of the development proposals.
The database is still in its infancy, having only collected data since November 2020, and it is recognised that it still has many opportunities to enable its impact to be felt. However has provided a blue print which Government are currently putting in place legislation to enable a national similar solution to be delivered.
The fully automated, live dataset requires no human invention. Historically each of the 35 planning authorities employed an officer to provide the data manually, these resources have now been able to be redeployed in many cases as the workload has been reduced. In practice this has achieved a net saving of approximately £750,000 per year to the tax payers, or released that time to enable boroughs planning teams to be more impactful.
The data can be accessed by anyone, as an open dataset we do not monitor how it is used, however we do ask users where appropriate to show us what they have discovered and suggest how it can be improved.
Case Study - Infrastructure Mapping Application
www.london.gov.uk/programmes-strategies/better-infrastructure/data-and-innovation-tools/infrastructure-mapping-application
The IMA is a collaborative tool designed to enable infrastructure providers to share data about where they are intending to carry out street works, and collaborate to enable other providers to join in with the works, limiting the disruption to citizens.
The PLDH now feeds their platform, enabling the utility providers to see where developments are being proposed, which developments are consent, being built, as well was what is in the pipeline and plan more effectively to limit disruption in providing utilities for each of the sites. This in turn means that they are able to limit the disruption for citizens as well as plan to ensure there is sufficient capacity in the network for all future developments.
In practice this has meant a better level of service for citizens and a more holistic approach to ensuring developments are only consented in locations were they are capable of delivering capacity on each of the networks. This has not previously been possible.
Planning for future Capacity - Thames Water
(see slides attached)
As a Water and Drainage provider Thames Water have an obligation to provide services to all new developments in London, with an obligation to enable connection on demand.
Thames Water now take a live data feed through an API of all of the data, and map the data, combining it with their own datasets to (1) ensure that they understand all developments are capable of being built (2) model what that might mean in terms of future demand, and (3) ensure that all developers are proactively contacted, ensuring that only lawful connections take place.
The impact of this has meant that areas operating beyond capacity are better managed, including pressure on demand, and less late and unlawful connections are made, resulting in both cost savings and service improvements for citizens.
Population Calculations - Health Service Provision
Core to public services is ensuring that population changes are properly accounted for and public services capacity adjusted to meet their need. Upto 30% of all development proposals in London relate to extensions to existing housing stock, meaning extensions to enable higher occupation (e.g. additional bedrooms etc) meaning increased population and pressure on existing public services.
The Health Authorities in London now access this data to enable them to model projected changes in population, and consequently the likely need for changes in service provision based not only on new dwellings, but changes to household size and typology. Meaning better funded, more effective public services for our citizens.
We are aware of numerous uses being made by private bodies ensuring that the data drives change to risk modelling, supply chains of materials as well as investment decisions.
The data was not previously held by any of the 35 local planning authorities in any format that enabled it to be accessed or analysed, meaning that decisions about planning the future of the city were being made on a limited evidence base, and arguably investment decisions were being made by both public and private bodies without knowledge of where population growth was taking place, and potentially could have had great impact.
Now decisions are being taken based on data about 120,000+ development proposals in any year, rather than 4000, and full rather than partial datasets are provided for each of the development proposals.
The database is still in its infancy, having only collected data since November 2020, and it is recognised that it still has many opportunities to enable its impact to be felt. However has provided a blue print which Government are currently putting in place legislation to enable a national similar solution to be delivered.
The fully automated, live dataset requires no human invention. Historically each of the 35 planning authorities employed an officer to provide the data manually, these resources have now been able to be redeployed in many cases as the workload has been reduced. In practice this has achieved a net saving of approximately £750,000 per year to the tax payers, or released that time to enable boroughs planning teams to be more impactful.
The data can be accessed by anyone, as an open dataset we do not monitor how it is used, however we do ask users where appropriate to show us what they have discovered and suggest how it can be improved.
Case Study - Infrastructure Mapping Application
www.london.gov.uk/programmes-strategies/better-infrastructure/data-and-innovation-tools/infrastructure-mapping-application
The IMA is a collaborative tool designed to enable infrastructure providers to share data about where they are intending to carry out street works, and collaborate to enable other providers to join in with the works, limiting the disruption to citizens.
The PLDH now feeds their platform, enabling the utility providers to see where developments are being proposed, which developments are consent, being built, as well was what is in the pipeline and plan more effectively to limit disruption in providing utilities for each of the sites. This in turn means that they are able to limit the disruption for citizens as well as plan to ensure there is sufficient capacity in the network for all future developments.
In practice this has meant a better level of service for citizens and a more holistic approach to ensuring developments are only consented in locations were they are capable of delivering capacity on each of the networks. This has not previously been possible.
Planning for future Capacity - Thames Water
(see slides attached)
As a Water and Drainage provider Thames Water have an obligation to provide services to all new developments in London, with an obligation to enable connection on demand.
Thames Water now take a live data feed through an API of all of the data, and map the data, combining it with their own datasets to (1) ensure that they understand all developments are capable of being built (2) model what that might mean in terms of future demand, and (3) ensure that all developers are proactively contacted, ensuring that only lawful connections take place.
The impact of this has meant that areas operating beyond capacity are better managed, including pressure on demand, and less late and unlawful connections are made, resulting in both cost savings and service improvements for citizens.
Population Calculations - Health Service Provision
Core to public services is ensuring that population changes are properly accounted for and public services capacity adjusted to meet their need. Upto 30% of all development proposals in London relate to extensions to existing housing stock, meaning extensions to enable higher occupation (e.g. additional bedrooms etc) meaning increased population and pressure on existing public services.
The Health Authorities in London now access this data to enable them to model projected changes in population, and consequently the likely need for changes in service provision based not only on new dwellings, but changes to household size and typology. Meaning better funded, more effective public services for our citizens.
We are aware of numerous uses being made by private bodies ensuring that the data drives change to risk modelling, supply chains of materials as well as investment decisions.
How was the service co-created?
The Mayor of London has no statutory powers to deliver projects requiring data from each of the 35 Planning Authorities. Using his convening powers, the boroughs together chose to develop this solution together, in partnership with Government, their own back office system providers and the Planning Portal (the submission system in the UK).
At the start of the project, each of the Authorities had their own business processes, separate contractual relationships with their own back office system providers, and little leverage to achieve change. Working together across the 43 organisations, the team were able to navigate each of the systems to provide novel solutions to receive and extract data, and standardising it to enable it to become uniform in format. This required not just technical specialisms, but business process knowledge, meaning buy in from across each of the organisations was key to making this dataset happen.
Over 450 people have been involved in developing this technical solution so far, across 22 different technical solutions.
At a political level the project needed to develop trust through depoliticising data, meaning that only datapoints relevant to end users are taken, and needed to be agreed by parties before the project progressed. This was delivered using an existing collaborative framework in place through London Councils, creating space for politicians to discuss the project and understand the risks. The impact of this has been that the principle of the data sharing has not become a political issue, instead enabling it to be used by professionals across the built environment.
The GLA now operates a DataTaskforce working with planning authorities across London to find ways to improve the quality of the data, make the data more accessible to them, and skill up members to understand how to use data to develop insights into how London (and their areas) are changing.
Moving Forwards
The PLDH is designed to be extended, built in Elastic Search, it is planned to add additional related built environment datasets to increase its use, such as data around infrastructure funding, and building standards data, which enable it to be used for modelling changes to the carbon efficiency of building stock and the city's response to the climate crisis in the built environment.
At the start of the project, each of the Authorities had their own business processes, separate contractual relationships with their own back office system providers, and little leverage to achieve change. Working together across the 43 organisations, the team were able to navigate each of the systems to provide novel solutions to receive and extract data, and standardising it to enable it to become uniform in format. This required not just technical specialisms, but business process knowledge, meaning buy in from across each of the organisations was key to making this dataset happen.
Over 450 people have been involved in developing this technical solution so far, across 22 different technical solutions.
At a political level the project needed to develop trust through depoliticising data, meaning that only datapoints relevant to end users are taken, and needed to be agreed by parties before the project progressed. This was delivered using an existing collaborative framework in place through London Councils, creating space for politicians to discuss the project and understand the risks. The impact of this has been that the principle of the data sharing has not become a political issue, instead enabling it to be used by professionals across the built environment.
The GLA now operates a DataTaskforce working with planning authorities across London to find ways to improve the quality of the data, make the data more accessible to them, and skill up members to understand how to use data to develop insights into how London (and their areas) are changing.
Moving Forwards
The PLDH is designed to be extended, built in Elastic Search, it is planned to add additional related built environment datasets to increase its use, such as data around infrastructure funding, and building standards data, which enable it to be used for modelling changes to the carbon efficiency of building stock and the city's response to the climate crisis in the built environment.
Additional Documents
