Yesterday I was at RE:PUBLICA XI to give a talk on Open Government Data in the opening session of the “open” stream. The crammed to over-capacity room was a nice indicator of the growing attention and interest being generated by open data, and especially open governernment data. Slides online here and below.
As announced on Friday on the UK Government’s data.gov.uk, I am one of the members of the UK Government’s newly formed Public Sector Transparency Board.
From the announcement:
The Public Sector Transparency Board, which was established by the Prime Minister, met yesterday for the first time.
The Board will drive forward the Governmentâ€™s transparency agenda, making it a core part of all government business and ensuring that all Whitehall departments meet the new tight deadlines set for releasing key public datasets. In addition, it is responsible for setting open data standards across the whole public sector, listening to what the public wants and then driving through the opening up of the most needed data sets.
Chaired by Francis Maude, the Minister for the Cabinet Office, the other members of the Transparency Board are Sir Tim Berners-Lee, inventor of the World Wide Web, Professor Nigel Shadbolt from Southampton University, an expert on open data, Tom Steinberg, founder of mySociety, and Dr Rufus Pollock from Cambridge University, an economist who helped found the Open Knowledge Foundation.
In the words of Francis Maude:
â€œIn just a few weeks this Government has published a whole range of data sets that have never been available to the public before. But we donâ€™t want this to be about a few releases, we want transparency to become an absolutely core part of every bit of government business. That is why we have asked some of the countryâ€™s and the worldâ€™s greatest experts in this field to help us take this work forward quickly here in central government and across the whole of the public sector.â€
Yesterday, in a speech on “Building Britainâ€™s Digital Future”, UK Prime Minister Gordon Brown announced wide-ranging plans to open up UK government data. In addition to a general promise to extend the existing commitments to “make public data public” the PM announced:
- The opening up of a large and important set of transport data (the NaPTAN dataset)
- A commitment to open up a significant amount of Ordnance Survey data from the 1st April (though details of which datasets not yet specified)
- By the Autumn an online e-“domesday” book giving “an inventory of all non-personal datasets held by departments and arms-length bodies
- A new “institute” for web science headed by Tim Berners-Lee and Nigel Shadbolt and with an initial Â£30m in funding
This speech is a significant indication of a further commitment to the “making public data public” policy announced in the Autumn.
It’s great to see this as, a year ago it seemed as if government policy was set to largely ignore the research in the Models of Public Sector Information Provision by Trading Funds report (authored by myself, David Newbery and Professor Bently back in 2008) whose basic conclusions was that that government data which was digital, bulk and ‘upstream’ should be made available at marginal cost.
More detailed excerpts (with emphasis added)
Opening up data
In January we launched data.gov.uk, a single, easy-to-use website to access public data. And even in the short space of time since then, the interest this initiative has attracted – globally – has been very striking. The site already has more than three thousand data sets available – and more are being added all the time. And in the past month the Office for National Statistics has opened up access for web developers to over two billion data items right down to local neighbourhood level.
The Department for Transport and the transport industry are today making available the core reference datasets that contain the precise names and co-ordinates of all 350 thousand bus stops, railway stations and airports in Britain.
Public transport timetables and real-time running information is currently owned by the operating companies. But we will work to free it up – and from today we will make it a condition of future franchises that this data will be made freely available.
And following the strong support in our recent consultation, I can confirm that from 1st April, we will be making a substantial package of information held by ordnance survey freely available to the public, without restrictions on re-use. Further details on the package and governmentâ€™s response to the consultation will be published by the end of March.
And I can also tell you today that in the autumn the Government will publish online an inventory of all non-personal datasets held by departments and arms-length bodies – a â€œdomesday bookâ€ for the 21st century.
The programme will be managed by the National Archives and it will be overseen by a new open data board which will report on the first edition of the new domesday book by April next year. The Government will then produce its detailed proposals including how this work can be extended to the wider public sector.
To inform the continuing development of making public data public, the National Archives will produce a consultation paper on a definition of the â€œpublic taskâ€ for public data, to be published later this year.
The new domesday book will for the first time allow the public to access in one place information on each set of data including its size, source, format, content, timeliness, cost and quality. And there will be an expectation that departments will release each of these datasets, or account publicly for why they are not doing so.
Any business or individual will be free to embed this public data in their own websites, and to use it in creative ways within their own applications.
So our goal is to replace this first generation of e-government with a much more interactive second generation form of digital engagement which we are calling Mygov.
Companies that use technology to interact with their users are positioning themselves for the future, and government must do likewise. Mygov marks the end of the one-size-fits-all, man-from-the-ministry-knows-best approach to public services.
Mygov will constitute a radical new model for how public services will be delivered and for how citizens engage with government – making interaction with government as easy as internet banking or online shopping. This open, personalised platform will allow us to deliver universal services that are also tailored to the needs of each individual; to move from top-down, monolithic websites broadcasting public service information in the hope that the people who need help will find it – to government on demand.
And rather than civil servants being the sole authors and editors, we will unleash data and content to the community to turn into applications that meet genuine needs. This does not require large-scale government IT Infrastructure; the â€˜open sourceâ€™ technology that will make it happen is freely available. All that is required is the will and willingness of the centre to give up control.
The lead article of Prospect Magazine’s February issue is a piece by by James Crabtree and Tom Chatfield entitled “Mashing the State”. It’s an in-depth look at the recent launch of data.gov.uk and its place in the wider context of government policy in relation to information — as well as information’s relation to governance (that “mashing” of the state …).
Where Does My Money Go gets a mention as does the “Cambridge” paper on pricing models at trading funds.
There’s an interesting 6 month fellowship at OPSI for work on economics of public sector information being funded by ESRC and National Archives. Deadline for applications is 6th August:
Valuing information: an economic analysis of public sector information and its re-use
Length of Fellowship: Six months
Proposed start date: Autumn 2009
Applications to be submitted as soon as possible (and by 6 August)
Location of Fellowship: The National Archives’ sites (Central London and Kew)
As part of its Placement Fellowship Scheme, the Economic and Social Research Council (ESRC) and The National Archives welcome applications from academic economists interested in working in a research capacity in the Office of Public Sector Information (OPSI). OPSI is part of The National Archives, a member of the Ministry of Justice family, working to set standards, deliver access and encourage the re-use of PSI.
The Placement Fellowship Scheme encourages social science researchers to spend time within a partner organisation to undertake policy relevant research and to develop the research skills of partner employees. The Fellowship will be jointly funded by the ESRC and OPSI while the Fellow remains employed by his or her institution.
See the document below for further details on the Placement Fellowship: http://www.nationalarchives.gov.uk/documents/esrc-placement-fellowship-june-09.pdf
The Government announced last summer a further review of how trading funds supply PSI. The results of this review had been expected with the budget.
However, instead of the results of a review, trading funds were included in the report of the Operational Efficiency Programme in the section on “Asset management and sales” in the “final report”. Box 3A p.41 summarized the trading fund assessment exercise:
The first phase of the Trading Fund Assessment considered how a number of Government businesses could open up the information they create or hold as a result of carrying out their core public duties. The businesses were Met Office, Land Registry, Ordnance Survey, Companies House, Driver and Vehicle Licensing Agency and UK Hydrographic Office.
The Assessment identified key principles of good practice relating to information produced by all Trading Funds. These principles are:
- information easily available â€“ where possible at low or marginal cost;
- clear and transparent pricing structures for the information, with different parts of the business accounted for separately;
- simple and transparent licences to facilitate the re-use of information for purposes other than that for which it was originally created; and
- clearly and independently defined â€“ with input from customers and stakeholders â€“ core purposes (â€œpublic tasksâ€) of the organisations.
The Office of Public Sector Information will provide enhanced oversight and governance to ensure application of these principles across the Trading Funds that create significant amounts of information.
A new business strategy for Ordnance Survey has been developed (see Box 3.H) which also will ensure easier and simpler access to high-quality information. Further work on the future business plans and models for specific Trading Funds – as well as consideration of the effectiveness of the Trading Fund model â€“ will now be incorporated into the Operational Efficiency Programme.
So what we have is:
- A vague (“where possible”) commitment to “low or marginal cost” pricing but with “low” undefined — thereby leaving plenty of ‘wiggle room’. In any case the main PSI trading funds are explicitly excepted from this it seems — see below.
- Some centralization of oversight in OPSI (though not clear what power OPSI will have)
- Public tasks that are clearly and independently defined (though not clear who ensures independence)
- More pricing transparency within trading funds (though again little detail as to how this will be managed or enforced)
There were separate, specific, assessments for 3 of the trading funds mentioned in Box 3A: the Land Registry (box 3.E p.45), the Met Office (box 3.F p.46) and Ordnance Survey (box 3.H p.47). Each of these assessments consisted of just a few paragraphs (the assessments are excerpted in full below).
The Land Registry and Met Office assessments were, in essence, “pats on the back” with clear endorsements of their current operational model — albeit with an encouragement to expand commercial operations and be more efficient. Pricing policies weren’t mentioned.
For Ordnance Survey the tone was slightly different with a stated need for the OS to be “more customer-focused and commercially driven”. However, again there was no mention at all of pricing policies.
Where was the assessment of marginal cost pricing (or other pricing model) for “raw” bulk data — the recommended option from the Cambridge study (of which I was a co-author)? Where the detailed discussion of the regulatory model that needs to put in place to ensure that the system works well? Entirely absent! This is truly disappointing and one can only feel that the a serious opportunity has been missed here.
Trading Fund Assessments from the OEP Report
Box 3.E Land Registry
Land Registry maintains and develops a stable and effective land registration system throughout England and Wales, providing the cornerstone for the creation and free movement of interests in land. Giving a state-backed security for title to registered estates and interests in land for the whole of England and Wales, and ready access to up-to-date and guaranteed land information, enables confident dealings in property and security of title.
In addition, Land Registry produces property price reports and delivers a range of non- statutory added-value products and services. Land Registry is committed to providing high quality, cost-effective services which are delivered promptly to all customers. A review of the business model was undertaken as part of the OEP. This concluded that in light of current market conditions and recognising the need to retain responsibility for the creation, recording and guaranteeing of title to land within Government, the following improvements to the operating framework of the business have been identified and will be delivered;
- realising significant efficiency savings through a programme which includes estate and operational rationalisation and market testing of support functions that will result in a more streamlined, resourceful organisation;
- developing opportunities for the provision of wider commercial services and products;
- identifying synergies with the functions and data requirements of other public sector bodies with a view to achieving efficiency improvements through greater collaboration; and
- exploring opportunities to accelerate these initiatives through joint ventures and/or outsourcing of activities to third party providers.
Box 3.F Met Office
The Met Office is a world-leading provider of weather forecasts and climate change modelling and advice to the general public, specialist customers throughout the public sector and an increasing number of private sector customers.
It is essential that the Met Office’s unified approach to short, medium and long term forecasting and climate modelling, which is the most efficient and sophisticated in the world, is preserved. The Met Office also performs a number of key government roles, especially in international data collaboration and UK representation. In order to maintain the quality of its services it will require long-term investment and the freedom to develop its operations. There remains potential to expand commercial operations at the Met Office beyond those already provided, possibly through the introduction of private capital in some areas.
Over the coming months the project team will:
- work closely with the MOD as the owner department and HM Treasury to identify improvements to its business model, ownership structure and financial framework in order to reduce the administrative burden, maximise its development and to fully exploit the market opportunities open to it;
- work with other public sector bodies to achieve efficiency improvements through greater collaboration or transfer of functions;
- explore increased commercial activities, for example weather warnings to industry and helping business understand the impact of climate change;
- seek opportunities for private sector partners to develop specific services to complement the Met Office’s business; and
- maximise operational freedoms and reduce bureaucracy in the interface between the Met Office and the MOD.
Box 3.H: Ordnance Survey
Ordnance Survey collects, maintains and publishes high quality and up-to-date geographical information for the whole of Great Britain. Ordnance Survey provides data and services to customers both directly and indirectly through its network of commercial partners. The Government is committed to stimulating innovation in the geographical information market, increasing competition where it would be beneficial to consumers and to making geographical data and services more easily available.
The OEP has concluded so far that Ordnance Survey needs to be more customer-focused and commercially driven. The Government is therefore publishing a new commercial strategy for the Ordnance Survey on their website. The new strategy balances the requirement to maintain the highest quality standards with the need to significantly enhance ease of access to geographic data and services for both commercial and non-commercial use.
The new strategy seeks to equip Ordnance Survey to thrive in and better support competition and innovation in a wider geographical information market that is being transformed by advances in technology. It is a significant and ambitious programme of change. The Government has set key milestones for delivery in 6 and 12 monthsâ€™ time and beyond, as well as a process for independent review and challenge of progress. If sufficient progress is not made to promote competition and innovation in these timescales, the Government will consider further reforms. Opportunities to accelerate the delivery of initiatives through introducing further commercial experience and capabilities will be fully explored over the coming year.
When our report on PSI provision at trading funds was published alongside the budget in March, the Government promised further investigation. They have now commissioned another review this time done by the Shareholder Executive/HMT (full ToR on Shareholder Executive website:
The review, announced as part of Budget 2008, will be conducted by the Shareholder Executive and will consider the benefits to the wider UK economy from any potential changes to the trading funds’ business model. The review will examine the impact on the trading funds’ business models of any changes to the current pricing, accessing and licensing regimes with the aim of: * distinguishing more clearly what information is required by Government for public policy * ensuring that this information is available as widely as possible in order to maximise the benefits to the wider UK economy, at a price that balances the provision of such access with the need for users to make a fair contribution to the cost of collecting the information in the long term. The policy objectives of each of the trading funds will not form part of the assessment, but the review will consider the future of the trading fund model and how it impacts on the delivery of these objectives.
The priority group of Trading Funds to be included in the assessment includes Ordnance Survey, Meteorological Office, UK Hydrographic Office, HM Land Registry, Driver and Vehicle Licensing Agency and Companies House. Others are listed in Annex A of the Terms of Reference. The Review will be carried out within the framework of current data protection legislation.
Update: my slides from a presentation at the European Commission in May 2008 are now online
The research report entitled Models of Public Sector Information Provision via Trading Funds was published today to coincide with its mention in the budget (para 3.49). This report was commissioned by HMT and BERR and jointly written by myself, Professor Newbery and Professor Bently (all at Cambridge University).
The formal announcement and details of the context in which the report was commissioned can be found here on BERRâ€™s website. To quote at some length from the section dealing with this report:
As part of the response [to the OFTâ€™s report on the Commercial Use of Public Information], the Government commissioned Cambridge University to do some analysis specifically around the pricing of public sector information held by trading funds[i].
This analysis has been released today as a Study Report (11 March 2008 – see Related Documents), and it sets out estimates of the costs and benefits of marginal-cost pricing, based on the assumptions used by the Cambridge team and the data they were able to collect.
Going forward, the Government will look closely at the public sector information held by trading funds to distinguish more clearly what is required by Government for public tasks, ensuring this information is made available as widely as possible for use in actual and potential downstream markets.
In the lead up to the next spending review, it will also ensure that it is priced appropriately. The underlying principle will be that information collected for public purposes will be made available at a price that balances the need for access while ensuring customers pay a fair contribution to the cost of collecting this information in the long-term.
[i] The study was commissioned jointly by the Department for Business, Enterprise and Regulatory Reform and HM Treasury in July 2007. The terms of reference for the study are available upon request by contacting the BERR Enquiry Unit
Meanwhile the budget had this to say (chapter 3 and chapter 5)
3.49 [Commercial Use of Public Sector Information] The Office of Fair Tradingâ€™s (OFT) market study into the commercial use of public information15 highlighted important issues around access to public sector information for commercial or other re-use. The Government commissioned Cambridge University to analyse the pricing of this information. This analysis is published alongside Budget 2008. The Government will look closely at public sector information held by trading funds to distinguish more clearly what is required by Government for public tasks and ensure that this information is made available as widely as possible for use in downstream markets. In the lead up to the next Spending Review the Government will ensure that information collected for public purposes is priced so that the need for access is balanced with ensuring that customers pay a fair contribution to the cost of collecting this information in the long term. These issues will be considered in conjunction with the assessment of trading funds (see Chapter 5).
5.14 [Asset management and sales] The Government is committed to managing the stock of public assets to deliver optimal results for society and the economy. Following the 2007 CSR, departments are publishing asset management strategies setting out how they will deliver best value from their assets. To support the introduction of asset management strategies the Treasury and Shareholder Executive will undertake an assessment of the governance, business plans and future development strategies of each of the trading funds and a selection of public corporations (excluding those already subject to central government review).