United We Stand. Evaluating,
Cooperating, and other Unlikely
Stories of Evaluation Capacity
Building in Italy
Martina Bolli, Silvia Ciampi, Francesco Giordano
& Laura Tagle
Italy’s National Evaluation System
Content
1. Cohesion policy in Italy: long-term
background & contrasting directions
2. The Italian evaluation policy
3. Efforts & (unlikely) achievements…
plus some drawbacks
COHESION POLICY IN
ITALY:
LONG TERM BACKGROUND
& CONTRASTING
DIRECTIONS
2007-2013 - Evaluation as one of the
pillars of a (much needed) renewal of
Italy’s regional policy
• Citizen-centered policies
• Strong focus on essential services—rather
than on procedures & expenditure
• Emphasis on local knowledge
• Responsibility to regional governments
• Links with national policies
• Greater importance to administrative
capacity
The regional (= cohesion) & rural
development policy in Italy
•
Intentionality of territorial goals
•
Additionality vis-à-vis ordinary policy
Resources come from EU & National funds
Levels of programming: a complex
framework
• National “umbrella” strategies for regional & rural
development policies
• 7 Ministries develop their own programs
• Each of the 21 Regions develops
– its multi-sector strategy
– 5 Programs (each program is funded by one source)
“Naturally”, each of the 5 programs is usually
managed by a separate Directorate
“Programs” end up being bureaucratic constructs
rather than guides for action
Over time, silos have been created
Sources of money
•
•
•
•
Coordination organisms
• Ministry for economic
EU Regional Fund
development
National funds
• Ministry for agriculture
EU Social Fund
EU Rural Development • Ministry for labour
Other departments
….Beyond silos
2007-2013: intentional efforts to bridge gaps &
introduce policy dialogue among and within
authorities in a new perspective
The National Strategy for 2007-2013
tried to make the various sources of
funding work together through common:
•
•
•
•
•
Programming
Implementation
Monitoring
Surveillance
Evaluation
At the same time, EU
regulations ask for separate
programmes by EU source
of money (& within Italy the
momentum for integration
dwindles)
21 Regions (70%)
6 Ministries (30%)
Centre-North: 15%
South: 85%
3bln€ to be given to Southern
Regions on the basis of
performance in waste, water,
education and social services
Regional development strategy
Between 2007 & 2010: lower national resources (other than national
cofinancing)
•different distribution North-South
•the instruments needed to implement nationally-relevant innovations (among
which greater integration) dwindle
THE ITALIAN
EVALUATION POLICY
Evaluation capacity building/1
From EU evaluation requirements towards a
national policy
• Institutional building:
– Creation of Evaluation Units within executive branch of
Regional authorities
– Network of Evaluation Units
• Specialized:
– Regional and Rural Development policies only
• Both demand & supply side
– Improve regional authorities’ ability to Demand for & Use
evaluations
– Improve Evaluation Units ability to Manage and Conduct
evaluations
Evaluation capacity building/2
• Organized guidance & support (National
Evaluation System)
• Enabling approach:
– No rigid prescriptions
– Reputational mechanisms
• Legitimize & support innovators at regional
level
Evaluation policy /1
• Nationally-funded interventions must be evaluated--not only
those funded by EU
• Objects:
– effects of a well-defined (however complex) intervention or group
of interventions on a problem, a social group, or an area—rather
than a bureaucratic construct (e.g., a multisectoral “program”—
tipically operating on a large area, affecting millionsof people)
– across bureaucratic borders: the evaluation object may have been
financed by more than one source
– selectivity: choose to answer defined questions about “crucial &
controversial issues”
Evaluation policy/2
• Each Region drafts an Evaluation Plan:
– What is evaluated (and what is not)
– Who (internally or externally) performs each evaluation
– When
– Organization of evaluation function
– Resources (human and financial) devoted to evaluations
The National Evaluation System
• UVAL – Ministry of Economic Development
(Coordinates)
• INEA – Ministry for Agricultural, Food and Forestry
Policies
• ISFOL – Ministry of Labour
• Department for Equal Opportunities
• Evaluation Units of Regional Authorities
Its composition shows a clear intention to work
together across silos
& involving Regions in a federal function
The National Evaluation System
Each component differs from the others
• Central units:
– Have differing rules, interpretations, goals, tasks &
status
• Regional evaluation units:
– Conduct or manage evaluations…
– … but also provide project selection, planning,
knowledge management to the Region
– Have background and skills closer to regional fund
policies (or national funds) rather than rural
development or active labour policies
EFFORTS & (UNLIKELY)
ACHIEVEMENTS… PLUS
SOME DRAWBACKS
A “federal” function performed across
silos & by central & regional units
Common guidance - include & adjust for differing rules,
traditions, goals
Personalized support to Regions (all managing authorities
together)
Ensure quality & evaluation of evaluations (....)
Observe processes & disseminate information (jointly)
Facilitate dissemination (conference-like meetings, workshops,
create & maintain eval database www.dps.tesoro/valutazione.it)
Some results
• All Regions developed an Evaluation Plan
http://www.dps.tesoro.it/uval_linee_valutazione.asp#10
• Evaluations have been launched:
– and even completed: 50!
(in addition to the mandatory ex
post evaluations of rural development)
– and a lofty 60% available on the internet
• Evaluation units (established between 2000 &
2002) have started conducting internal evaluations
Still far from perfection but …
Why we are happy anyway…
• Activities started (and 50 concluded) even though
– (or just because?) there are no enforceable
requirements (no sanctions)
– there is still strong inertia in the system
• old evaluations fulfilled needs of managing authorities,
national coordinators, and Commission rapporteurs
• restricted market with “marriages” between managing
authorities & evaluators
– still little pressure from social partners & general
public
• Innovations go together: evaluations seem to be
used as an instrument by Regions willing to improve
• Innovations implemented where there has been
more support from central state (not only on
evaluation)
Thank you
[email protected]
[email protected]
[email protected]
Please send us feedback at these e-mail
addresses