Innovation policies and their evaluation
Some lessons from the Tuscan experience
Marco Mariani
(IRPET – Tuscany’s Regional Institute for Economic Planning)
SEE PROJECT - THEMATIC WORKSHOP 3:
Evaluating design and innovation policies
Florence, 10-11 May 2010
Viewpoints on innovation policies and their evaluation
Two different
approaches to
innovation
Two different
conceptions of
innovation policies
Different evaluation
objectives and tools
The rationale of support policies
 Approach A: innovation process at firm level  market
failure because of insufficient individual incentives to
R&D  public support to single enterprises (e.g. firm
subsidies, tax benefits)
 Approach B: innovation resulting from webs of
relationships  systemic failures  public support to
networking and university-industry linkages (e.g.
support to joint network projects)
Consequent evaluation goals and methodology
 Approach A: has the firm performed additional R&D, has it
become more innovative or competitive solely because of
public incentives?
Methodology: the performance of the aided enterprise is compared
to that of a non-aided “twin”, according to a counterfactual, quasiexperimental paradigm (e.g. differences in differences; matching)
 Approach B: has a network of new innovation-generative
relationships come into being, as a result of public support?
And is this network able to make firms more competitive?
Methodology: networks elicited by policies are analysed in their
structure and performance by means of social network analysis
tools; then they are compared to pre-existing networks (if any)
Tuscan examples of Approach A
Theory: mainstream micro-economic theory
Main goals: impact of public aid on innovation (R&D expenses, patents,
etc.), sales, productivity, # employees, labour cost, mark-up/pricing power…
Methods: propensity-score matching
Mattei and Mauro (2007)
Specific regional
programme for SMEs
2000-2006: 266 treated,
721 non-treated
Merito et al (2010)
National programmeSpecial fund for applied
research 1998-2004: 185
treated, 925 non-treated
Irpet Eval. Group (2010)
2 regional programmes
2000-2006:
ongoing project
Approach A: Evaluation questions (Q) and results (R)
Q: Has the programme had any impact on the expansion of sales and number of
employees of beneficiary enterprises?
R: YES. The increase in the number of employees occurs in the long run (M et al)
The impact on sales and employees is stronger for enterprises that benefited
of repayable loans; weaker for enterprises that benefited of non-repayable
subsidies (M&M).
Q: Has the programme had any impact on the amount of skilled labour?
R: YES in the short run, and in particular for SMEs (M et al)
Q: Has the programme had any impact on the labour productivity of beneficiary
enterprises?
R: NO (M et al)
Q: Has the programme had any impact on the patenting activity of beneficiary
enterprises?
R: Scarcely, and only in the short run. The policy has probably displaced existing
R&D investments, instead of stimulating new ones (M et al)
Tuscan examples of Approach B
Theory: micro-foundations of innovative systems; agent-based
perspective to innovation processes
Main goals: identification of emergent structures; contribution to the
formation of innovation clusters
Methods: Social network analysis tools
Russo and Rossi (2009)
RPIA 2002-2004: 36/14
projects, 409/203 agents
Bellandi and Caloffi (2010)
All network programmes
2000-2006:
122 projects, 908 agents
Caloffi and Mariani (2010)
All network programmes
2000-2006 on advanced
mechanics: 30 projects,
289 agents
The programme’s basic idea: networks are conductive of innovation, especially those involving
heterogeneous agents (SMEs, universities, service providers)
Broad sectoral-technological focus (no explicit focus on design activities)
Approach B: Evaluation questions (Q) and results (R)
Q: To what extent has the programme supported pre-existing networks of
relationships or mobilized new ones?
R: YES in advanced mechanics, optronics, medical devices (R&R, B&C, C&M);
NO in biotech and traditional sectors (B&C)
Q: Has the programme succeeded in promoting the creation of well-performing
(turnover, patents…) networks capable of integrating heterogeneous
competencies and to foster systemic effects in the regional economy?
R: YES, but in limited technological areas (R&R; B&C)
Q: What are the features of the emergent network structure resulting from the
policy interventions?
R: Some desirable agents and links are often missing (R&R, B&C; C&M)
Q: Has the policy been able to mobilize a wide range of co-localised
competencies and to favour the formation of an innovation cluster?
R: To a very limited extent (C&M)
Lines for future research
How can existing methodologies for the
evaluation of micro/meso-impact be rethought o re-adapted in the case of public
policies promoting design?
Some references on the Tuscan/Italian case
Bellandi, M., Caloffi, A. (2010), “An analysis of regional policies promoting networks for
innovation”, European Planning Studies, 18(1), pp. 67-82.
Irpet Evaluation Group- Bonaccorsi A., Brancati R., Mariani M., Mealli F., Trivellato U.
(2010), “L’impatto degli incentivi regionali alla R&S. Il caso della Toscana”, in progressforthcoming.
Bondonio D., Martini A. (2001), “Using Event History Analysis to Evaluate the Impact
of Investment Subsidies Targeted to Youth-Owned Firms”, Atti del convegno SIS 2001
“Processi e metodi statistici di valutazione”, Roma Tor Vergata, giugno, pp. 277-282.
Caloffi A., Mariani M. (2010), “Designing policy support to innovation clusters. An
overlapping network approach”, paper submitted for presentation to the EUNIP
Conference 2010, Faculty of Economics, Universitat Rovira i Virgili, Reus, 9-11 June.
Mattei A., Mauro V. (2007), “Analisi e valutazione delle politiche di sostegno alle
imprese artigiane in Toscana”, IRPET, febbraio.
Russo, M., Rossi, F. (2009), “Cooperation partnerships and innovation. A complex
systems perspective to the design, management and evaluation of a EU regional
innovation policy programme”, Evaluation, 15(1), pp. 75-100.
Merito M., Giannangeli S., Bonaccorsi A., (2010), “Do Incentives to Industrial R&D
Enhance Research Productivity and Firm Growth? Evidence from the Italian Case”,
International Journal of Technology Management , 49(1-3), pp. 25-48.