miscellaneous factZ – The online home of Rufus Pollock

We’ve looked at the size of the public domain extensively in earlier posts.

The basic take away from the analysis was the finding that, based on library catalogue data, for books in the UK, approximately 15-20% of work was in the public domain — with public domain work being pretty old (70 years plus, due to the life+70 nature of copyright).

An interesting question to ask then is: how large would the public domain be if copyright had not been extended from its original length of 14 years with (possible) 14 year renewal (14+14) set out in Statute of Anne back in 1710? And how does this compare with how the situation, back when 14+14 was in “full swing”, say, 1795?

Furthermore, what about if copyright today was a simple 15 years — the point estimate for the optimal term of copyright found in paper on this subject? Well here’s the answer:

Number and percentage of public domain works based on various scenarios based on Cambridge University Library catalogue data.

That’s right folks: based on the data available, if copyright had stayed at its Statute of Anne level, 52% of the books available today would in the public domain compared to an actual level of 19%. That’s around 600,000 additional items that would be in the public domain including works like Virginia Woolf’s (d. 1941) the Waves, Salinger’s Catcher in the Rye (pub. 1951) and Marquez’s Chronicle of a Death Foretold (pub. 1981).

For comparison, in 1795 78% of all extant works were in the public domain. A figure which we’d be close to having if copyright was a simple 15 years (in that case the public domain would be a substantial 75%).

To put this in visual terms, what the public domain is missing out as a result of copyright extension is the yellow region in the following figure: those are the set of works that would be public domain under 14+14 but aren’t under current copyright!

The Public Domain of books today (red), under 14+14 (yellow), and published output (black)

Update: I’ve posted the main summary statistics file including per-year counts. I’ve also started a CKAN data package: eupd-data for this EUPD-related data.

This Thursday (11th March) I’m speaking at the Forum Virium’s Open Up the City event in Helsinki.

This year their focus is on “open data, design, interfaces and innovation” and I’m speaking under the title “Open Data: What, Why, How?”.

I was recently asked to put together a short document outlining my main policy recommendations in the area of “innovation, creativity and IP”. Below is what I prepared.

General IP Policy

Recommendation: IP policy, and more generally innovation policy, should aim at the improvement of the overall welfare of UK society and citizens and not just at promoting innovation and creativity

Innovation is, of course, a major factor in the improvement of societal welfare — but not the only factor, access to the fruits of that innovation is also important.

IP rights are monopolies and such monopolies when over-extended do harm rather than good. The provision of IP rights must balance the promotion of innovation and creativity with the need for adequate access to the results of those efforts both by consumers and those who would seek to innovate and create by building upon them. A policy which aims purely at maximizing innovation, via the use of IP rights, will almost certainly be detrimental to societal welfare, since it will ignore the negative consequences of extending IP on access to innovation and knowledge. As such, IP policy is about having “enough, but not too much”.

This basic point is often overlooked. To help minimize the risk of this occurring in future it is suggested that this basic purpose — of promoting the welfare of UK citizens — be explicitly embedded within the goals of organisations and departments tasked with handling policies related to innovation and IP.

Recommendation: Move away from a focus on intellectual property to look at innovation and information policy more widely

IP rights are but one tool for promoting innovation and often a rather limited one. The focus should be on the general problem — promoting societal welfare through innovation and access to innovation — not on one particular solution to that problem.

Provision and Pricing of Public Section Information

Background

Public sector information (PSI) is information held by a public sector organisation, for example a government department or, more generally, any entity which is majority owned and/or controlled by government. Classic examples, of public sector information in most countries would include, among many others: geospatial data, meteorological information and official statistics.

While much of the data or information used in our society is supplied from outside the public sector, compared to other parts of the economy, the public sector plays an unusually prominent role. In many key areas, a public sector organization may be the only, or one among very few, sources of the particular information it provides (e.g. for geospatial and meteorological information). As such, the policies adopted regarding maintenance, access and re-use of PSI can have a very significant impact on the economy and society more widely.

Funding for public sector information can come from three basic sources: government, ‘updaters’ (those who update or register information) and ‘users’ (those who want to access and use it). Policy-makers control the funding model by setting charges to external groups (’updaters’ or ‘users’) and committing to make up any shortfall (or receive any surplus) that results. Much of the debate focuses on whether ‘users’ should pay charges sufficient to cover most costs (average cost pricing) or whether they should be given marginal cost access — which equates to free when the information is digital. However, this should not lead us to neglect the third source of funding via charges for ‘updates’.

Policy-makers must also to concern themselves with the regulatory structure in which public sector information holders operate. The need to provide government funding can raise major commitment questions while the fact that many public sector information holders are the sole source of the information they supply raise serious competition and efficiency issues.

Recommendation: Make digital, non-personal, upstream PSI available at marginal cost (zero)

The case for pricing public sector information to users at marginal cost (equal to zero for digital data) is very strong for a number of complementary reasons. First, the distortionary costs of average rather than marginal cost pricing are likely to be high. Second, the case for hard budget constraints to ensure efficient provision and induce innovative product development is weak. As such, digital upstream public sector information is best funded out of a combination of ‘updater’ fees and direct government contributions with users permitted free and open access. Appropriately managed and regulated, this model offers major societal benefits from increased provision and access to information-based services while imposing a very limited funding burden upon government.

Recommendation: Regulation should be transparent, independent and empowered. For every public sector information holder there should be a single, clear, source of regulatory authority and responsibility, and this ‘regulator’ should be largely independent of government.

This is essential if any pricing-policy is to work well and is especially important for marginal-cost pricing where the Government may be providing direct funding to the information holder. Policy-makers around the world have had substantial experience in recent years with designing these kinds of regulatory systems and this is, therefore, not an issue that should be especially difficult to address.

Copyright Term

Background

The optimal term of copyright has been a very live policy issue over the last decade. Recently, in the European Union, and especially in the UK, there has been much debate over whether to extend the term of copyright in sound recordings from its current 50 years.

The basic trade-off inherent in copyright is a simple one. On the one hand, increasing copyright yields benefits by stimulating the creation of new works but, on the other hand, it reduces access to existing works (the welfare ‘deadweight’ loss). Choosing the optimal term, that is the length of protection, presents these two countervailing forces particularly starkly. By extending the term of protection, the owners of copyrights receive revenue for a little longer. Anticipating this, creators of work which were nearly, but not quite, profitable under the existing term will now produce work, and this work will generate welfare for society both now and in the future. At the same time, the increase in term applies to all works including existing ones — those created under the term of copyright before extension. Extending term on these works prolongs the copyright monopoly and therefore reduces welfare by hindering access to, and reuse of, these works.

Recommendation: Reduce Copyright Term – And Certainly Do Not Extend It

Current copyright term is significantly over-extended. Calculations performed in the course of my own work indicate that optimal copyright term is likely around 15 years and almost certainly below 40 (the breadth of the estimates here are a direct reflection of the existing data limitations but this upper bound is still (far) below existing terms).

Even a simple present-value calculation would indicate that the incentives for creativity today offered by extra term 50 years or more in the future are negligible — while the effect on access to knowledge can be very substantial, especially when term extensions are applied retrospectively (as they almost always are).

It is also noteworthy that recent extensions, such as that for authorial copyright in the US (the CTEA) and the proposed extension of recording copyright in the EU, have been opposed well-nigh unanimously by academic economists and other IP scholars. Policy-making in this area should be evidence-based and designed to promote the broader welfare of society as a whole. Policies that appear to reflect nothing more than special-interest lobbying will only perpetuate the “marked lack of public legitimacy” which the Gowers report lamented, discouraging those who wish to contribute constructively to future Government policy-making in these areas, and making enforcement ever harder — effective enforcement, after all, depends on consent borne of respect as well as obedience coerced through punishment.

This Wednesday (27th of January) at 1pm I’m giving one of Cambridge University Library’s regular lunch-time talks on Openness and Libraries. Attendance is free and anyone can come along!

Update (28th Jan): talk is done and slides are now up.

Blurb

Over the past few years, open licensing (http://www.opendefinition.org/) has facilitated the explosive growth of a ‘knowledge commons’. To give a few prominent examples: Open Access journals, Open Educational Resources and Open Data in scientific research have all been enabled by licenses which permit material to be freely re-used and re-distributed. This outpouring of support for openness has led to an incredible rise in community-led development and innovative uses.

Bibliographic records are a key part of our shared cultural heritage and essential to anyone working with cultural materials (books, music, films etc). Opening up those records for access and re-use offer a variety of benefits.

First, it would allow libraries to share records more efficiently and improve quality more rapidly through better, easier feedback. Second, easier access to catalogue data would spur development of the multifarious services, technologies and research that use that data, including, for example, search engines, book or music websites, researchers working on information production, journalists writing on orphan works, as well as many other areas we cannot even imagine in advance.

With a growing number of Government agencies and public institutions making data open, is is now time for the library community to do likewise?

Here we are going to apply the results on Public Domain “proportions” derived in our previous post and thereby obtain best estimates of the UK public domain.

The logic is simple, and similar to that in our first post in the series: we will take the Public Domain proportions from Table 3 of our last post and combine with our (conservative) estimates for output based on library catalogues. Here are the results:

UK Public Domain Totals Based on Cambridge University Library Data. Note, as discussed in previous posts, figures from British Library are approximately 3x larger (both for Public Domain and total items).

Total (Black) and Public Domain (Red) Items per year based on the CUL Catalogue.

Zooming in to the pre-1960 period to get more detail:

Total (Black) and Public Domain (Red) Items per Year based on the CUL Catalogue for pre-1960 period.

I’m one of the co-organizers of a workshop on Public Domain Calculators workshop taking place next week, on the 10th and 11th of November, at Emmanuel College, University of Cambridge.

Hosted by the Open Knowledge Foundation in association with the Centre for Intellectual Property and Information Law at the University of Cambridge, it’s a meeting of European experts on copyright and the digital public domain taking place as part of the Communia project.

The purpose of the workshop is to produce materials such as legal flow charts and public domain “algorithms” which will help with the representation of different national copyright laws and the determination of public domain status.

Details of the meeting are as follows:

• When: 10-11th November 2009
• Where: Emmanuel College, Cambridge
• Wiki: http://wiki.okfn.org/PublicDomainCalculators/Meeting
• Participate: Free but space is limited. If you are interested in coming, email the organizers at: info@okfn.org

Background

There is often a tendency to talk of ‘the public domain’ and of works falling out of copyright and ‘into the public domain’ – as though there is a single set of works which are out of copyright all over the world. In fact, of course, there are different national laws about the nature and duration of copyright in different types of works – and hence what is in the public domain is different in different countries.

Efforts are currently underway to build a series of public domain calculators – which will help to determine whether or not a given work is in copyright in a given jurisdiction. At the time of writing groups and individuals in more than 17 jurisdictions are assisting in this effort.

Continues the series of post related to analyzing catalogue data, here are some stats on author “significance” as measured by the number of book entries (’items’) for that author in the Cambridge University Library catalogue from 1400-1960 (there being 1m+ such entries).

I’ve termed this measure “significance” (with intentional quotes) as it co-mingles a variety of factors:

• Prolificness — how many distinct works an author produced (since usually each work will get an item)
• Popularity — this influences how many times the same work gets reissued as a new ‘item’ and the library decision to keep the item
• Merit — as for popularity

The following table shows the top 50 authors by “significance”. Some of the authors aren’t real people but entities such as “Great Britain. Parliament” and for our purposes can be ignored. What’s most striking to me is how closely the listing correlates with the standard literary canon. Other features of note:

• Shakespeare is number 1 (2)
• Classics (latin/greek) authors are well-represented with Cicero at number 2 (4), Horace at 5 (9) followed Homer, Euripides, Ovid, Plato, Aeschylus, Xenophon, Sophocles, Aristophanes and Euclid.
• Surprise entries (from a contemporary perspective): Hannah More, Oliver Goldsmith, Gilbert Burnet (perhaps accounted by his prolificity).
• Also surprising is limited entries from 19th century UK with only Scott (26), Dickens (28) and Byron (41)

Top 50 authors based on CUL Catalogue 1400-1960

The other thing we could look at is the overall distribution of titles per author (and how it varies with rank — a classic “is it a power law” question). Below are the histogram (NB log scale for counts) together with a plot of rank against count (which equates, v. crudely, to a transposed plot of the tail of the histogram …). In both cases it looks (!) like a power-law is a reasonable fit given the (approximate) linearity but this should be backed up with a proper K-S test.

Histogram of items-per-author distribution (log-log)

Rank versus no. of items (log-log)

TODO

• K-S tests
• Extend data to present day
• Check against other catalogue data
• Look at occurrence of people in title names
• Look at when items appear over time

Colophon

Code to generate table and graphs in the open Public Domain Works repository, specifically method ‘person_work_and_item_counts’ in this file: http://knowledgeforge.net/pdw/hg/file/tip/contrib/stats.py

Last week I was in Oxford to give a talk at the IP Seminar on “How Long Should Copyright Last?”. I have now posted the slides from the talk online.

In addition to covering the basic outline of the optimal term calculation, I was also able to give some results from the recent research on the public domain (see slide 58 onwards).

I’m posting up some work-in-progress entitled Exploring Patterns of Knowledge Production (link to full pdf). Below I’ve excerpted the introduction plus list of motivational questions. Comments (and critique) very welcome!

Introduction

In what follows the term ‘knowledge’ is here used broadly to signify all forms of information production including those involved in technological innovation, cultural creativity and academic advance.

Today, thanks to rapid advances in IT, we have available substantial datasets pertaining both to the extent and the structure of knowledge production across disciplines, space and time.

Especially recent is the availability of good ’structural’ data — that is data on the linkages and relationships of different pieces of knowledge, for example as provided by citation information. This new material allows us to explore the “patterns of knowledge production” in deeper and richer ways than ever previously possible and often using entirely new methods.

For example, it has long been accepted that innovation and creativity are cumulative processes, in which new ideas build upon old. However, other than anecdotal and case-study material provided by historians of ideas and sociologists of science there has been little data with which to study this issue — and almost none of a comprehensive kind that would make possible a systematic examination.

However, the recent availability of comprehensive databases containing ‘citation’ information have allowed us to begin really examining the extent to which new work builds upon old — be it a new technology as represented by a patent or a new idea in academia as represented by a paper, builds upon old.

Similar opportunities present themselves in relation to identifying the creation of new fields of research or technology, and tracing their evolution over time. Here the existence of extensive “structural information” as presented, for example, by citation databases, enables new systematic approaches — for example, can new fields be identified (or perhaps defined) as points in ‘knowledge space’ far away from the existing loci of effort? or, alternatively, by the nature of its connections to the existing body of work?

Structural information of this kind can also be used in charting other changes in the life-cycle of knowledge creation. For example, to offer a specific conjecture, a field entering decline, though still exhibiting a similar level of output (papers etc) and even citations to a field in rude health, may display a citation structure which is markedly different — for example, more clustered within the field itself. Thus, by using this additional structural information we may be able to gain insights not available with simpler approaches.

At the same time, structure must also play a central role in any attempt to estimate knowledge related ‘output’ measures. This is of course not true for other forms of ‘output’, for example that of corn of steel, where we have relatively well-defined objective measures available: tonnes of such-and-such a quality.

But knowledge is different: the most obvious metrics, such as number of patents or papers produced, seem entirely inadequate: one particular innovation or paper may be ‘worth’ as much as a hundred or a thousand others.

The issue here is that, compared to corn or steel, knowledge is extremely inhomogeneous, or put slightly differently, quality (or significance) differs very substantially across the individual pieces of knowledge (papers, patents etc).

Thus, any serious attempt to measure the progress of knowledge must must find some way to do this quality-adjustment and structural information seems essential to this.

What specific questions might we explore with such datasets?

The following is a (non-exhaustive) list of the kinds of questions one might explore using these new datasets:

• Can we use structure to infer information about quality of individual items? Clearly the answer is yes, for example by using a citation-based metric where a work’s value is estimated based on its citation by others.
• Can we then use this information together with more global structure of the production network to gain a better idea of total (quality-adjusted) output. This would allow one to chart progress, or the lack of it, over time?
• Can we use structural information to investigate the life-cycle of fields? For example, can we see fields ‘dying out’ or the onset of diminishing returns? Can we see new fields coming into existence and their initial growth patterns?
• What about productivity per capita and its variation across the population? It is likely that one would need to focus here within a discipline as it would be difficult to directly compare across disciplines, at least when using quality adjusted productivity.
• Do the structures of knowledge production vary over time and across disciplines and does this have implications for their productivity? Can we compare the structure of evolution in technology or economics with that in ‘natural’ evolution and, if not, what are the primary differences?
• How do other (observable) attributes related to the producers of knowledge (their collaboration with others, their geographical location) affect the structures we observe and the associated outcomes (output, productivity) already discussed above?
• Do different policies (for example openness vs. closedness — weak vs. strong IP) have implications for the structure of production and hence for output and productivity?
• Is knowledge production (in a particular area) ergodic or path-dependent? Crudely: do we always end up in the same place or do small shocks have large long-term effects?

Last week I was at the ATRIP Conference to give an invited talk on “How Long Should Copyright Last?”, based on my paper: Forever Minus a Day? Calculating the Optimal Term of Copyright.

Slide are here, and you can find the text of the accompanying introduction below (I plan to write up the full exposition as a short essay — but that is to come).

Most ATRIP participants were lawyers not economists, so this was an opportunity to do a more non-technical presentation (so no equations!). As with most economics, the fundamentals of calculating copyright term are simple: it is just a demand curve plus “welfare analysis” (a fancy name for adding up social benefits and costs), and shorn of “obfuscating” algebra these matters should be understandable by anyone.

How Long Should Copyright Last: Introduction

Before I begin it is important to note that in considering copyright and its term, we must leave to one side the questions of attribution and integrity — their existence and term can and should be considered quite separately from the ‘economic’ rights that form the core of copyright as it operates today.

This small caveat done, I beg your indulgence for a brief historical excursion. In particular, I ask you to cast your mind back a century and a half and more to the Houses of Parliament in the February of 1841.

[[Picture of Serjeant Talfourd]]

As many of you will be aware Serjeant Talfourd had, by this point, been doggedly pursuing a new copyright act for four years — since 1837. Originally wide in scope the Bill had been narrowed and the attention of both supporters and critics alike had come to focus on a single feature of that Act: the proposed extension in the term of protection. Specifically Talfourd’s Act proposed changing the then rule of 28 years or life (whichever being the longer) to life plus sixty — remarkably close to the life plus 70 of today.

[[Picture of Macaulay]]

By February 1841 Talfourd’s Bill had failed no less than 4 times. On its fifth attempt it had reached a second reading and on the fifth of February it came before the House. After a brief introduction by Talfourd — mindful that this was not the first time the matter had been discussed — Thomas Babbington Macaulay rose to speak. In a masterly disquisition, both in content and rhetoric, Macauley set out his opposition to the Bill, and did so so tellingly that the motion was defeated. Talfourd, who lost his seat at the next election, and therefore only saw his Bill pass in the hands of another — and in much reduced form — remained forever embittered by Macaulay’s intervention — coming so late and so decisively in the process.

To read Macaulay’s speech, and, for that matter, the views expressed on all sides in that debate, is to be struck by how little has changed.

[[Valenti Picture]]

When Jack Valenti and Mary Bono are found in recent times calling for a term of ‘Forever Minus a Day’ one hears the echoes of Serjeant Talfourd all those years ago, just as one can hear echoes of those who oppose extensions today of the likes of Henry Warburton, a radical politician and vehement opponent of Talfourd, who claimed the extension was “a robbery upon the public” and that copyright ought to be fixed, “only on such a term of years as would prove a sufficient inducement for authors to write good books”.

And the analogy is telling in other ways. Though Talfourd’s Bill was beaten back by a swell of opposition year after year eventually it was passed — albeit in reduced form and by Lord Mahon — with this success attributable to a persistence made possible not, primarily, by the size, but by the concentration of the interests who sought its passage. Like Fabius Cunctator the proponents of extension, sustained by deep reservoirs of emotional and financial commitment, can afford to wait, able to return, as necessary, again and again, until an opportune moment presents itself for the attainment of their purposes — for the opposition to extension, though broad is ’shallow’ and therefore more easily dissipated by distraction and division.

[[Philosophical Differences]]

Even more striking are the similarity in the issues that occupy centre stage in this debate. First, the fundamental ‘philosophical’ question — which colours all of discussion — of whether we confer copyright because it is a natural right — which should therefore last forever — or for ‘utilitarian’ purposes, that is the public good — in which case it almost certainly should not. Second, descending from these lofty heights of principle, what is the actual effect is copyright? In particular, does it operate to raise price and restrict access — that is: is it a monopoly?; and what specifically are the benefits that accrue to the producers of copyrightable works, and what costs to the public and others who wish to use and reuse them.

I think it is clear that economists — or any group for that matter — have no great claim to authority on answering this first question of principle, for it seems, ultimately, one of opinion. That said, I would note two points which must raise grave doubts as to the existence of any fundamental natural right from which copyright might spring.

First, term limited in all jurisdictions. Second, the breadth of copyright’s application both in subject matter, quality and ownership. For can we truly convince ourselves that “eternal expressions of the human spirit”, worthy of exclusivity for all time, subsist in an advert for toothpaste; or convince ourselves of the special status of the creator when so much copyright today, perhaps even the majority, is immediately, and indeed often automatically, assigned from the ‘creator’ to a corporation.

However, it is not my intention to enter into this debate any further here. Rather, in the interests of ‘full disclosure’ I wish only to make clear my views — and those of economists generally — on the matter, namely that copyright is not a natural right but is created and maintained for the purpose of promoting and securing the public good, no more, no less. (These are views which can come as no surprise given the nature of this talk — an analysis of term only makes sense if its basis is a utilitarian one!)

[[My Views]]

Furthermore, let me also make clear, right at the outset, my view, and one again I think shared by almost all economists, that copyright is a monopoly. This is not to say that copyright is bad — far from it. But to deny that copyright is a monopoly is to obscure its basic nature and operation — an obscuration that has, furthermore and unfortunately, been most common and attractive to those pursuing copyright’s enlargement.

[[The Big M Word]]

And what is the general tendency of monopoly — to echo Macaulay once again? It is indeed to raise prices and limit access. Now, of course, we may debate the precise extent of these effects, but there can be no denying that the very purpose of copyright’s existence is to confer on a single entity — the copyright holder — the power to control the dissemination, and hence the price, of all instances of a particular good — i.e. all copies of a given work.

This is the very definition of a monopoly and the fact that there may exist other goods, other works, which compete with that one makes no difference — a monopoly of apples is no less a monopoly because one does not control oranges. Of course, the existence and proximity of substitutes will alter the affect of the monopoly, but one must be cautious here: close substitutes may limit the negative effects of the copyright monopoly but they will, for the very same reasons, also limit the gains (those increased revenues for copyright-holder).

Returning then to Macaulay whose expression of the matter I cannot better:

[[Macaulay Again]]

“It is good that authors be remunerated; and the least exceptionable way of remunerating them is by a monopoly. Yet monopoly is evil. For the sake of the good we must submit to the evil, but the evil ought not to last a day longer than is necessary for securing the good.”

Our task then is to answer the implicit question: how long should copyright last (so as to not be a day longer than is needed)? More specifically what are the degrees of benefit and harm created by copyright’s monopoly and at what level should term be set to achieve the most advantageous balance of the two?