UQ
Uncertainty Quantification
in Monte Carlo simulation
Maria Grazia Pia
INFN Sezione di Genova
Matej Batic, Gabriela Hoff, Paolo Saracco
Collaborators:
Politecnico Milano, Fondazione Bruno Kessler, MPI HLL, Univ. Darmstadt, XFEL,
UC Berkeley, State Univ. Rio de Janeiro, Hanyang Univ. (Korea)
INFN CCR
Frascati, 5-7 October 2011
Maria Grazia Pia, INFN Genova
Simulation validation
Geant4 was first released on 15 December 1998
Geant4 reference paper (2003) is the most cited
publication in Thomson-Reuters’ Nuclear Science
and Technology category
Large user community worldwide, multi-disciplinary applications
What fraction of Geant4 has been validated to date?
By whom?
Geant4 kernel
Use cases
Uncertainty quantification in simulation (not only
Monte Carlo) is a major research topic these days
Essential to critical applications
Maria Grazia Pia, INFN Genova
Shall I trust my simulation?
How much can I trust my simulation?
Does it have predictive value?
Validation
Geant4
Intrinsic limits to simulation validation
Maria Grazia Pia, INFN Genova
Epistemic uncertainties
Epistemic uncertainties originate from lack of knowledge
Relatively scarce attention so far in Monte Carlo simulation
Studies in deterministic simulation (especially for critical applications)
Possible sources in Monte Carlo simulation
incomplete understanding of fundamental physics processes, or
practical inability to treat them thoroughly
non-existent or conflicting experimental data for a physical
parameter or model (for validation)
applying a physics model beyond the experimental conditions in
which its validity has been demonstrated
Epistemic uncertainties affect the reliability of simulation results
Can we quantify them?
Maria Grazia Pia, INFN Genova
Uncertainty quantification
Epistemic uncertainties are difficult to quantify
due to their intrinsic nature
No generally accepted method of measuring epistemic
uncertainties
and their contributions to reliability estimation
Various formalisms developed in the field of deterministic
simulation
Interval analysis
Dempster-Shafer theory of evidence
Not always directly applicable in Monte Carlo simulation
Adapt, reinterpret, reformulate existing formalisms
Develop new ones specific to Monte Carlo simulation
Maria Grazia Pia, INFN Genova
Benefits of quantifying uncertainties
Epistemic uncertainties are reducible
Can be reduced or suppressed by extending knowledge
New experimental measurements
Uncertainty quantification gives us guidance about
What to measure
What experimental precision is needed/adequate
Priorities: which uncertainties generate the worst systematic
effects
Measurements are not always practically possible
Uncertainty quantification to control systematics
Especially important in critical applications
Maria Grazia Pia, INFN Genova
Warm-up exercise
p stopping powers
Water ionisation potential
d-ray production
Multiple scattering
Nuclear elastic
Nuclear inelastic
Cross sections
Preequilibrium
Nuclear deexcitation
Intranuclear cascade
EGS5, EGSnrc
Penelope
MCNP(X)
PHITS
SHIELD-HIT
FLUKA
SPAR, CALOR, CEM, LAHET, INUCL,
GHEISHA, Liège INCL, Bertini
Maria Grazia Pia, INFN Genova
M. G. Pia et al., Epistemic uncertainties
in proton depth dose simulation, IEEE
Trans. Nucl. Sci., Oct. 2010
How well do we know basic physics parameters
in the simulation?
Atomic electron binding energies
EGS4
EGSnrc
EGS5
FLUKA
GEANT 3
BEB
total
ToI 1978 (Shirley-Uppsala)
e- on N
EADL
ToI 1996 (Larkins(Sevier 1972))
exp
.
NIST/Lotz
Unknown
Modified Bearden&Burr
Geant4
EADL, Carlson+Williams, ToI 1978, (Bearden&Burr)
MCNP
Carlson
Penelope
Carlson, ToI 1978
GUPIX
Sevier 1979
ISICS
Bearden & Burr, Williams
Maria Grazia Pia, INFN Genova
X-ray Data
Booklet
Williams
Basic test: radioactive decay
Experimental data: Z.W. Bell, ORNL (TNS Senior Editor)
Collaboration: S. Hauf, Tech. Univ. Darmstadt, M. Kuster, XFEL
Maria Grazia Pia, INFN Genova
Resources for UQ
Complex mathematics involved
Dempster-Shafer theory of evidence
Heavyweight computation
Sensitivity analysis, interval analysis
Large number of modeling alternatives
Toolkit
Complex experimental scenarios
Typical of HEP experiments
Pilot project to evaluate the feasibility of UQ
Maria Grazia Pia, INFN Genova
Conclusions
Further detail in publications
Evaluation of systematic effects associated with
The impact of epistemic
epistemic uncertainties
Sensitivity analysis (~interval analysis)
More refined methods: Dempster-Shafer
Methods specific to Monte Carlo simulation?
uncertainties depends on
the experimental
application environment
Complementary statistical methods contribute to
identify and quantify effects
Qualitative appraisal is not adequate
Epistemic uncertainties are reducible
Can be reduced or suppressed by extending knowledge
New experimental measurements
Uncertainty quantification gives us guidance about
What to measure
What experimental precision is needed/adequate
Priorities: which uncertainties generate the worst systematic effects
Maria Grazia Pia, INFN Genova
INFN UQ
Project on Geant4 uncertainty quantification
Complementary expertise
Geant4 developers
Experimental groups
Theory
Mathematics/statistics
Collaboration with other institutes
Maria Grazia Pia, INFN Genova