How to Make Good Innovation Policy
Whether you agree with it or not, governments around the world these days spend a lot of time, money and effort on attempts to 'foster innovation', which can mean anything from more basic scientific research, to growing patent applications, to increasing R&D expenditure (public and incentives for private), encouraging new firm creation or creating the 'right climate' for entrepreneurship.
I've recently been looking at various benchmarking exercises that attempt to compare innovation activites, policies and frameworks (i.e. institutional settings) amongst regions (whether they be cities, states or countries).
The benchmark for innovation benchmarking is the OECD's Science, Technology and Innovation Scoreboard, a data series published bienially which covers its members and 9 non-member, but emerging, countries.
Last year the OECD trialled a new technique using 'composite indicators', which try to parcel up the more than 75 individual measures produced for the STI Scoreboard, along with survey data from the World Economic Forum and the IMD, into more useful meta-indicators called, for example, 'technology diffusion', 'innovation financing' and 'market conditions'. The ultimate goal is to rank places by virtue of both their innovation outcomes, and their innovation inputs/drivers. The 2004 report on the project is here.
Anyway, one of the problems with the approach they've adopted is that the indicators included in each 'composite' measure have to be chosen by hand, relying on the 'informed judgement' of the analyst. This leads to things like 'the number of patents in triadic patent families' being used to measure regional performance in the 'innovation activity' composite, while leaving out, for example, new firm creation rates.
I think there is a way to do this meta-benchmarking better: it's called fuzzy clustering. Fuzzy clustering is a statistical technique which allows elements in a population to belong to more than one group/segment simultaneously, and which then produces a measure of the strength of membership for each element in each group. It allows you to build up a 'map' of which things or places are most like (or not like) other things or places. In essence, the data speak for themselves and are allowed to fall 'naturally' into groups, rather than being a priori segmented on the basis of 'informed judgement'. Economic geography - or place-based success - starts to look a lot different from this perspective.
There are many well-recognised methodological problems with standard benchmarking exercises, especially issues related to data collection, coverage and comparability.
But I think there is a deeper problem with these attempts to benchmark: they presume that linear ranking is meaningful. By concentrating on the production of league tables - that rank 'better' over 'worse' - they miss all the nuances of place-based success in a globalised - i.e. increasingly specialised and therefore differentiated - world.
The worst-better-best approach also makes it difficult to distinguish between the drivers of success as opposed to the outcomes of success. On the most recent figures (2004) Australia rates in the top 3 within OECD countries for public investment in research (along with Finland and Sweden). Yet the best (or at least most widely-used) measure of successful outcomes that we have at the moment (growth in GDP per hour worked = "productivity") shows that in the same year Australia did 'better' than Sweden, Finland, the UK and Switzerland, but 'worse' than the US, Hungary, and the Czech and Slovak Republics.
How to make sense of this, especially from a policy-making perspective? What is cause, and what is effect?
Ultimately I think this is a problem of inherently linear and static growth theories (including the new/endogenous and so-called Schumpeterian variants) that fail to acknowledge the dynamic development capacity of an economic system. Economic growth and its statistical off-sider, productivity growth, are the wrong long-term targets. Potts' recent post on maximising novelty-throughput speaks to this, but the credibility battle (not least how does one model the creation and consequences of novelty) is far from won.
In the meantime, I think there are policy-relevant, practically-useful alternatives to linear innovation-ranking mechanisms. Using fuzzy clustering, one would produce an 'innovation map', that shows (only) what places are innovatively *like* other places. A series of such maps would show how these dis/similarities are changing over time. The problem of cause and effect is circumvented by the fact that all indicators can be lumped in together. This may sound messy, but given theoretical uncertainties about these relationships it sure beats an a priori attribution of dependence vs. independence for particular variables, as must happen in regression-based analysis.
If I was making innovation policy for Australia, I'd rather know that we are like Israel but not like France in terms of patent creation per capita, and that we're like the US but not like Singapore in terms of public subsidy for R&D. I don't need to know that we came 16th in the OECD rankings for productivity (where France 'beat' us) and 12th in Richard Florida's Global Creativity index (where we 'win' hands down over Singapore). Encouraging high-tech start-ups? Give me Bangalore over California. Creating excellence in human capital development? I want us to be like Benelux, Israel and, actually, Singapore, but not like the US or China.
These are, at worst, subjective and at best, normative goals. But all innovation-related policy is like this, because we understand so little about how growth and development work, and even less about how they can be encouraged.
Using league tables to gauge innovation performance, and future capacity, simply provides a boost to short-run thinkers' egos. It's a bit like soccer in Australia - now that we've arrived, there's no question for most people that we've always had the capacity to 'do soccer well'. But it took someone (Lowy) with resources, and the foresight that comes from an understanding of context, to get us there in the first place.
Innovation-performance league tables are mechanisms to boost the same kind of success-in-sport pride. But they promote laziness in terms of long-term strategic development. They lead to the cut-and-paste approach to innovation policy ('if it's worked there it will work here') which ultimately underpins the zero-sum game of pursuing 'best practice'. All the literature on national innovation systems and innovation-led growth cautions against blind duplication of policy strategy; it instead tells us that becoming different is what matters, and that policy should always, therefore, be made with regard to context. League tables don't provide context, but global innovation maps might.
Down with linear rankings, I say. Maps of like and not-like are less exciting perhaps (because you can't tell who's winning); but they might be much more useful for good strategic policy making.
I've recently been looking at various benchmarking exercises that attempt to compare innovation activites, policies and frameworks (i.e. institutional settings) amongst regions (whether they be cities, states or countries).
The benchmark for innovation benchmarking is the OECD's Science, Technology and Innovation Scoreboard, a data series published bienially which covers its members and 9 non-member, but emerging, countries.
Last year the OECD trialled a new technique using 'composite indicators', which try to parcel up the more than 75 individual measures produced for the STI Scoreboard, along with survey data from the World Economic Forum and the IMD, into more useful meta-indicators called, for example, 'technology diffusion', 'innovation financing' and 'market conditions'. The ultimate goal is to rank places by virtue of both their innovation outcomes, and their innovation inputs/drivers. The 2004 report on the project is here.
Anyway, one of the problems with the approach they've adopted is that the indicators included in each 'composite' measure have to be chosen by hand, relying on the 'informed judgement' of the analyst. This leads to things like 'the number of patents in triadic patent families' being used to measure regional performance in the 'innovation activity' composite, while leaving out, for example, new firm creation rates.
I think there is a way to do this meta-benchmarking better: it's called fuzzy clustering. Fuzzy clustering is a statistical technique which allows elements in a population to belong to more than one group/segment simultaneously, and which then produces a measure of the strength of membership for each element in each group. It allows you to build up a 'map' of which things or places are most like (or not like) other things or places. In essence, the data speak for themselves and are allowed to fall 'naturally' into groups, rather than being a priori segmented on the basis of 'informed judgement'. Economic geography - or place-based success - starts to look a lot different from this perspective.
There are many well-recognised methodological problems with standard benchmarking exercises, especially issues related to data collection, coverage and comparability.
But I think there is a deeper problem with these attempts to benchmark: they presume that linear ranking is meaningful. By concentrating on the production of league tables - that rank 'better' over 'worse' - they miss all the nuances of place-based success in a globalised - i.e. increasingly specialised and therefore differentiated - world.
The worst-better-best approach also makes it difficult to distinguish between the drivers of success as opposed to the outcomes of success. On the most recent figures (2004) Australia rates in the top 3 within OECD countries for public investment in research (along with Finland and Sweden). Yet the best (or at least most widely-used) measure of successful outcomes that we have at the moment (growth in GDP per hour worked = "productivity") shows that in the same year Australia did 'better' than Sweden, Finland, the UK and Switzerland, but 'worse' than the US, Hungary, and the Czech and Slovak Republics.
How to make sense of this, especially from a policy-making perspective? What is cause, and what is effect?
Ultimately I think this is a problem of inherently linear and static growth theories (including the new/endogenous and so-called Schumpeterian variants) that fail to acknowledge the dynamic development capacity of an economic system. Economic growth and its statistical off-sider, productivity growth, are the wrong long-term targets. Potts' recent post on maximising novelty-throughput speaks to this, but the credibility battle (not least how does one model the creation and consequences of novelty) is far from won.
In the meantime, I think there are policy-relevant, practically-useful alternatives to linear innovation-ranking mechanisms. Using fuzzy clustering, one would produce an 'innovation map', that shows (only) what places are innovatively *like* other places. A series of such maps would show how these dis/similarities are changing over time. The problem of cause and effect is circumvented by the fact that all indicators can be lumped in together. This may sound messy, but given theoretical uncertainties about these relationships it sure beats an a priori attribution of dependence vs. independence for particular variables, as must happen in regression-based analysis.
If I was making innovation policy for Australia, I'd rather know that we are like Israel but not like France in terms of patent creation per capita, and that we're like the US but not like Singapore in terms of public subsidy for R&D. I don't need to know that we came 16th in the OECD rankings for productivity (where France 'beat' us) and 12th in Richard Florida's Global Creativity index (where we 'win' hands down over Singapore). Encouraging high-tech start-ups? Give me Bangalore over California. Creating excellence in human capital development? I want us to be like Benelux, Israel and, actually, Singapore, but not like the US or China.
These are, at worst, subjective and at best, normative goals. But all innovation-related policy is like this, because we understand so little about how growth and development work, and even less about how they can be encouraged.
Using league tables to gauge innovation performance, and future capacity, simply provides a boost to short-run thinkers' egos. It's a bit like soccer in Australia - now that we've arrived, there's no question for most people that we've always had the capacity to 'do soccer well'. But it took someone (Lowy) with resources, and the foresight that comes from an understanding of context, to get us there in the first place.
Innovation-performance league tables are mechanisms to boost the same kind of success-in-sport pride. But they promote laziness in terms of long-term strategic development. They lead to the cut-and-paste approach to innovation policy ('if it's worked there it will work here') which ultimately underpins the zero-sum game of pursuing 'best practice'. All the literature on national innovation systems and innovation-led growth cautions against blind duplication of policy strategy; it instead tells us that becoming different is what matters, and that policy should always, therefore, be made with regard to context. League tables don't provide context, but global innovation maps might.
Down with linear rankings, I say. Maps of like and not-like are less exciting perhaps (because you can't tell who's winning); but they might be much more useful for good strategic policy making.
posted by Kate Morrison at
8:54 pm
5 Comments:
'Statistical' means too many things to be useful in a practical sense. It's a static view (and notions of state, static and statistic are related at least etymologically).
Normative questions are simply unavoidable when dealing with notions of success in human-based systems. Whatever the methodology, in economic analysis there must be always a subjectively-chosen explanandum. Recent popular debates on happiness, old debates on the value of growth, and new attempts to taxonomise innovation capacity all suffer from the same problem of disntiguishing between cause and effect.
Drivers vs. outcomes are what we need to focus on, with a realisation that this dichotomy will always be drawn along normative lines, by necessity.
Statistical techniques, or computational methodologies, are only tools. But ideas in what counts as good policy are just as important as ideas that count for good economic development.
We need better selection mechanisms all round.
By
Kate Morrison, at 19 November, 2005 01:13
Predictably, I don't think we need any specific government policy to promote innovation. Free trade and cognate policies that focus people on production rather than rent seeking will do whatever can be done in that respect.
By
Rafe, at 20 November, 2005 06:53
Some speculation on the ecology of innovation can be found in a review of essays on Australian science.
http://www.the-rathouse.com/revessayscience.html
"In view of the current agitation over education and science policy two areas of investigation call for urgent attention. One might be called "the ecology of intellectual achievement". This concerns the personal, institutional and cultural factors which influence creativity and the growth of knowledge. The other is a similarly ecological investigation of the influences which promote commercial application of research findings (the 'D' part of R&D).
It appears that pure and applied work can flourish in partnership if a number of conditions are met. First, talented people are required who are interested in both practical and theoretical problems. Second, they should have high standards and high expectations of achievement. These attitudes tend to be assimilated by contact with gifted and inspiring teachers or colleagues early in life. They are killed by the inductivist, 'just collect the facts' method and the conformist, follow the Professor" ethos of Kuhn's 'normal science'. In each case the antidote is the Popperian spirit of conjecture and refutation. Thirdly, institutional and personal linkages are required to carry ideas backwards and forwards between the study/laboratory and the factory/farm.
Finally, to promote commercial application of ideas, industry needs to operate in a competitive environment, with the world as a potential market, instead of sheltering behind protective walls. Unfortunately most of the people who write about science policy start with the premise that more government involvement is required, more committees, more central direction to 'pick winners' for favoured (protected) treatment. It seem that the sad lesson of the New Protection (tariffs plus central wage fixing) has not been learned."
By
Rafe, at 20 November, 2005 21:12
If only we could put "Popperian spirit" in the water supply, like flouride.
Rafe, what are "cognate policies that focus people on production rather than rent seeking"?
Do you mean less easily-accessed welfare? I completely support that.
But how 'bout focusing people not just on production, but on creation of things that did not before exist, i.e. entrepreneurship. I am skeptical that government can achieve this (can we put entrepreneurial spirit in the water supply as well?)
But if efforts are going to be made with tax-payer $$ anyway I'd prefer the focus to be on optimising throughput of novelty. A policy target based on such a view would seek to maximise the number of start-ups seed funded, rather than on proportions of new firm failure or success. Ideally seed funds would come from private sources; but unfortunately we are really quite bad at doing venture capital well at this stage (are we a developed economy in this respect? no, not at all).
I'd like to change this. In the meantime, while seed-funds come from the public purse, I'd like to see much better selection mechanisms at work. Sometimes I have the impression that, in my neighbourhood, public service bureaucrats are making strategic decisions about innovation financing. Hayek must be turning in his grave.
By
Kate Morrison, at 20 November, 2005 21:48
"cognate policies that focus people on production" in addition to free trade in goods would include pay in proportion to productivity rather than hours spent on the job or having a militant union. Yes, reduced access to welfare and at the bottom end of the labour market you would have no minimum wage so the slow and unskilled could find some kind of work.
By
Rafe, at 21 November, 2005 19:56
Post a Comment
<< Home