Electromagnetic physics
validation
Katsuya Amako,Susanna Guatelli, Vladimir
Ivanchenko, Michel Maire, Barbara
Mascialino, Koichi Murakami, Sandra Parlati,
Andreas Pfeiffer, Maria Grazia Pia, Takashi
Sasaki, Lazslo Urban
Geant4 Workshop
Catania, October 4th-9th 2004
Barbara Mascialino
Geant4 Workshop
Catania, October 4-9 2004
The project
• The project is based on a geographically spread
collaboration:
INFN Genova
INFN Gran Sasso
Standard Group
THANKS TO KOICHI
KEK
MURAKAMI, TAKASHI
SASAKI, KATSUYA AMAKO
FOR THE VERY FRUITFUL
COLLABORATION!
Preliminary results were presented at last Geant4 Workshop and at IEEE-NSS in
Portland. Now the project has reached a mature state.
Barbara Mascialino
Geant4 Workshop
Catania, October 4-9 2004
Aim of the project
• Project for the validation of all Geant4 electromagnetic models
against established references
• The project s made-up by two parts:
PHYSICAL TEST
GOODNESS-OF-FIT
TESTING
Goodness-of-Fit
statistical toolkit
Chi-squared stability study
test50
Quantitative statistical comparisons allow:
- an evaluation of Geant4 physics goodness
- how the specific models behave in the same experimental condition
POSSIBILITY OF CHOOSING THE MOST APPROPRIATE MODEL
Barbara Mascialino
Geant4 Workshop
Catania, October 4-9 2004
First phase:
validation against the NIST database
Photon Attenuation Coefficient
Photon Cross Sections
(attenuation coefficients with only one process activated)
Electrons CSDA range and Stopping Power
(no multiple scattering, no energy fluctuations)
Protons CSDA range and Stopping Power
(no multiple scattering, no energy fluctuations)
Alpha particles CSDA range and Stopping Power
(no multiple scattering, no energy fluctuations)
Elements: Be, Al, Si, Fe, Ge, Ag, Cs, Au, Pb, U
Energy range: 1 keV – 100 GeV
Testing activity has been automatised (thanks to Sandra Parlati and Koichi Murakami)
Barbara Mascialino
Geant4 Workshop
Catania, October 4-9 2004
Photons: attenuation coefficient
χ2/ν stability study
Be
Barbara Mascialino
Geant4 Workshop
Z dependency?
Catania, October 4-9 2004
Photon attenuation coefficient: statistical results
NIST – XCOM
LowE Livermore
NIST – XCOM
LowE Penelope
NIST – XCOM
Standard
χ2 /ν
p-value
χ2 /ν
p-value
χ2 /ν
p-value
Be
Al
Si
1.26
0.31
0.32
0.16
1
1
0.08
0.07
0.10
1
1
1
1.01
0.34
0.56
0.45
1
0.97
Fe
Ge
Ag
Cs
0.11
0.15
0.22
0.19
1
1
1
1
0.08
0.05
0.15
0.37
1
1
1
1
0.20
0.28
0.45
0.81
1
1
1
0.75
Au
0.08
1
0.05
1
0.71
0.86
Pb
U
0.13
0.04
1
1
0.08
0.06
1
1
0.52
0.75
0.99
0.93
Barbara Mascialino
Geant4 Workshop
Catania, October 4-9 2004
Photons: photoelectric cross section
χ2/ν stability study
Be
Cs
Z dependency?
Barbara Mascialino
Geant4 Workshop
Catania, October 4-9 2004
Photon photoelectric cross section: statistical
results NIST – XCOM
NIST – XCOM
NIST – XCOM
LowE Livermore
LowE Penelope
χ2 /ν
p-value
χ2 /ν
p-value
χ2 /ν
p-value
Be
Al
Si
0.05
0.05
0.07
1
1
1
0.05
0.03
0.02
1
1
1
0.82
0.27
0.17
0.63
1
1
Fe
Ge
Ag
Cs
0.09
0.21
0.07
0.52
1
1
1
0.94
0.05
0.08
0.05
0.21
1
1
1
1
0.11
0.07
0.07
1.18
1
1
1
0.27
Au
0.14
1
0.15
1
0.57
0.93
Pb
U
0.25
0.35
1
1
0.35
0.37
1
0.99
0.51
0.42
0.96
0.99
Barbara Mascialino
Geant4 Workshop
Standard
Catania, October 4-9 2004
Photons: Compton cross section
The 1keV deviation effect is
evident in both LowE Penelope
and Standard packages
As an example, let us consider Ag:
χ2/ν =
Barbara Mascialino
with the
without the
1 keV point 1 keV point
LowE
Penelope
17.30
0.46
Standard
9.0
1.65
Geant4 Workshop
Catania, October 4-9 2004
Photon Compton cross section: statistical results
NIST – XCOM
LowE Livermore
NIST – XCOM
LowE Penelope
NIST – XCOM
Standard
χ2 /ν
p-value
χ2 /ν
p-value
χ2 /ν
p-value
Be
Al
Si
0.12
0.07
1.61
1
1
0.09
0.07
0.18
0.15
1
1
1
0.53
0.49
0.60
0.85
0.86
0.77
Fe
Ge
Ag
Cs
0.30
0.13
0.09
0.14
1
1
1
1
0.69
1.68
0.65
0.36
0.75
0.07
0.79
0.97
0.27
0.38
0.46
0.62
0.99
0.96
0.92
0.72
Au
0.16
1
0.41
0.94
1.04
0.41
Pb
U
0.23
0.11
1
1
1.31
0.13
0.21
1
1.18
0.08
0.29
1
Barbara Mascialino
Geant4 Workshop
Catania, October 4-9 2004
Compton cross sections
χ2/ν stability study
(without the E=1 keV point)
Ge
χ2/ν stability study
Si
Pb
Au
Barbara Mascialino
Geant4 Workshop
Catania, October 4-9 2004
Photons: pair production cross section
χ2/ν stability study
Be
(not compatible with the NIST)
Beryllium
deviations
χ2/ν stability study
Barbara Mascialino
Geant4 Workshop
Catania, October 4-9 2004
Removing the 1 keV point
Photon pair production cross section: statistical
results NIST – XCOM
NIST – XCOM
NIST – XCOM
LowE Livermore
LowE Penelope
χ2 /ν
p-value
χ2 /ν
p-value
χ2 /ν
p-value
Be
Al
Si
0.05
0.07
0.09
1
1
1
8.07
0.83
0.76
<0.001
0.61
0.69
7.01
1.07
0.93
<0.001
0.39
0.52
Fe
Ge
Ag
Cs
0.07
0.04
0.06
0.24
1
1
1
1
0.32
0.25
0.15
0.54
0.99
1
1
0.89
0.30
0.18
0.23
0.30
0.99
1
1
0.99
Au
0.08
1
0.09
1
0.23
1
Pb
U
0.11
0.09
1
1
0.08
1.18
1
1
0.22
0.26
1
0.99
Barbara Mascialino
Geant4 Workshop
Standard
Catania, October 4-9 2004
Photons: Rayleigh cross section
Si
Al
χ2/ν stability study
Ge
Fe
Au
U
Pb
χ2/ν stability study
deviations
Barbara Mascialino
Geant4 Workshop
Catania, October 4-9 2004
NIST – XCOM
LowE Livermore
NIST – XCOM
LowE Penelope
χ2 /ν
p-value
χ2 /ν
p-value
Be
Al
Si
0.27
1.15
0.68
0.99
0.97
0.77
0.17
24.86
25.08
1
<0.001
<0.001
Fe
Ge
Ag
Au
0.27
17.97
1.10
1.90
1
<0.001
0.36
0.05
11.08
1.06
1.74
18.99
<0.001
0.39
0.08
<0.001
Pb
8.40
<0.001
22.10
<0.001
U
11.31
<0.001
28.19
<0.001
Barbara Mascialino
Geant4 Workshop
Test results are not consistent
Photon Rayleigh cross section: statistical results
Catania, October 4-9 2004
Critical discussion of this result
•
•
The disagreement between NIST
reference data and data coming
from the recent library EPDL97
(provided by Lawrence Livermore
National Laboratory) within the
range of energies between 1 keV
and 1 MeV has been already
underlined and discussed in a
recent paper by Zaidi*.
In his paper Zaidi concluded that
EPDL97 is the most up-dated,
complete and consistent data
library available at the moment.
For these features, it should be
considered as a standard.
* Zaidi
H., 2000, Comparative evaluation of photon cross section libraries for
materials of interest in PET Monte Carlo simulation IEEE Transaction on Nuclear
Science 47 2722-35
Barbara Mascialino
Geant4 Workshop
Catania, October 4-9 2004
Electrons: stopping power
χ2/ν stability study
The three models
are equivalent
Strange effect
(as a function of Z)
NIST – ESTAR
LowE Livermore
χ2/ν = -0.032 + 0.0074 Z
R2=0.995
p<0.0001
NIST – ESTAR
LowE Penelope
BEST
FIT
BEST
FIT
χ2/ν = -0.032 + 0.0074 Z
R2=0.995
p<0.0001
NIST – ESTAR
Standard
BEST
FIT
χ2/ν = -0.046 + 0.0073 Z
R2=0.989
p<0.0001
Barbara Mascialino
Geant4 Workshop
Catania, October 4-9 2004
Electrons stopping power: statistical results
NIST – ESTAR
LowE Livermore
NIST – ESTAR
LowE Penelope
NIST – ESTAR
Standard
χ2 /ν
p-value
χ2 /ν
p-value
χ2 /ν
p-value
Be
Al
Si
0.03
0.06
0.07
1
1
1
0.02
0.05
0.06
1
1
1
0.02
0.06
0.06
1
1
1
Fe
Ge
Cs
Au
0.15
0.18
0.36
0.57
1
1
1
0.96
0.14
0.17
0.35
0.56
1
1
1
0.96
0.13
0.15
0.31
0.54
1
1
1
0.97
Pb
0.58
0.95
0.57
0.96
0.56
0.97
U
0.65
0.91
0.64
0.92
0.64
0.92
Barbara Mascialino
Geant4 Workshop
Catania, October 4-9 2004
Electrons: CSDA range
χ2/ν stability study
Ag
(to be explained)
The three models
are equivalent
Barbara Mascialino
Geant4 Workshop
Catania, October 4-9 2004
Electrons CSDA range: statistical results
NIST – ESTAR
LowE Livermore
NIST – ESTAR
LowE Penelope
NIST – ESTAR
Standard
χ2 /ν
p-value
χ2 /ν
p-value
χ2 /ν
p-value
Be
Al
Si
0.21
0.05
0.04
1
1
1
0.24
0.05
0.04
1
1
1
0.17
0.04
0.04
1
1
1
Fe
Ge
Ag
Cs
0.02
0.02
0.10
0.009
1
1
1
1
0.02
0.01
1.06
0.006
1
1
0.39
1
0.02
0.02
0.02
0.008
1
1
1
1
Au
0.005
1
0.004
1
0.005
1
Pb
U
0.005
0.004
1
1
0.004
0.003
1
1
0.004
0.003
1
1
Barbara Mascialino
Geant4 Workshop
Catania, October 4-9 2004
Protons and alpha particles
• LowE Ziegler 85
• LowE Ziegler 2000
• ICRU
At low energies: free electron gas model
At middle energies (~ MeV): parametrisations
At high energies: Bethe Bloch
• Standard
NIST database
Statistical comparison cannot lead to a real physics validation, but
we can only compare two different models (NIST – Ziegler)
Barbara Mascialino
Geant4 Workshop
Catania, October 4-9 2004
Protons: stopping power
LowE ICRU
Standard
LowE Ziegler 85
lOWe Ziegler 2000
NIST - PSTAR
Barbara Mascialino
χ2/ν stability study
Geant4 Workshop
Catania, October 4-9 2004
Protons stopping power: statistical results
NIST – PSTAR
LowE ICRU49
NIST – PSTAR
NIST – PSTAR
NIST – PSTAR
Standard
LowE Ziegler85 LowE Ziegler2000
χ2 /ν
p
χ2 /ν
p
χ2 /ν
Be
Al
Si
0.008
0.02
0.008
1
1
1
0.41
0.05
0.41
1
1
1
Fe
Ge
Ag
Au
0.02
0.12
0.06
0.08
1
1
1
1
0.03
0.53
0.15
0.27
Pb
0.06
1
U
0.10
1
Barbara Mascialino
χ2 /ν
p
0.66
0.06
0.81
0.94 0.01
1
0.02
0.79 0.01
1
1
1
1
1
1
1
0.03
0.72
0.10
0.17
1
0.89
1
1
0.02
0.11
0.04
0.09
1
1
1
1
0.12
1
0.17
0.21 0.05
1
1.28
0.12
1.28
0.12 0.09
1
Geant4 Workshop
p
Catania, October 4-9 2004
Protons: CSDA range
χ2/ν stability study
LowE ICRU
Standard
LowE Ziegler 85
LowE Ziegler 2000
NIST - PSTAR
Barbara Mascialino
Geant4 Workshop
Catania, October 4-9 2004
Protons CSDA range: statistical results
NIST – PSTAR
LowE ICRU49
NIST – PSTAR
NIST – PSTAR
NIST – PSTAR
Standard
LowE Ziegler85 LowE Ziegler2000
χ2 /ν
p
χ2 /ν
p
χ2 /ν
Be
Al
Si
0.005
0.03
0.02
1
1
1
0.11
0.01
0.29
1
1
1
1.03
0.01
0.32
Fe
Ge
Ag
Au
0.06
0.13
0.08
0.09
1
1
1
1
0.20
0.26
0.34
0.91
1
1
1
0.62
0.18
0.27
0.15
0.47
Pb
0.09
1
0.21
1
U
0.11
1
1.23
0.16
Barbara Mascialino
Geant4 Workshop
χ2 /ν
p
0.41 0.47
1
0.27
0.99 0.23
1
1
1
p
1
1
1
1
0.32
0.24
0.15
0.14
1
1
1
1
0.17
0.99 0.12
1
1.18
0.21 0.13
1
Catania, October 4-9 2004
Alpha particles: stopping power
WORK IN PROGRESS
LowE ICRU
Standard
LowE Ziegler 77
NIST - ASTAR
Barbara Mascialino
Geant4 Workshop
Catania, October 4-9 2004
Alpha particles: CSDA range
WORK IN PROGRESS
LowE ICRU
Standard
LowE Ziegler 77
NIST - ASTAR
Barbara Mascialino
Geant4 Workshop
Catania, October 4-9 2004
Statistical comparisons
(II)
Concerning alpha particles, this is the second iteration
of production and analysis since last July.
This because thanks to the quantitative analysis we
could detect a conceptual flaw in physics tables
treatment for both protons and alpha particles.
Systematic data analysis allowed to improve the
physical models.
Barbara Mascialino
Geant4 Workshop
Catania, October 4-9 2004
SUMMARY:
photons and electrons
• Low Energy Livermore is the most compatible
with the NIST reference (Rayleigh scattering is a
special case)
• Low Energy Penelope is quite compatible with
NIST reference except for some problems
exhibited in Compton scattering and pair
production cross sections
• Standard electrons are compatible with NIST,
photons are quite compatible, but exhibit some
problems
Barbara Mascialino
Geant4 Workshop
Catania, October 4-9 2004
SUMMARY:
protons and alpha particles
• While NIST represents an established reference
for photon and electron processes, the reference
for protons and alpha processes in controversial
at least in the lower energy ranges.
• Two reference data compilations ICRU/NIST and
Ziegler.
• Quantitative comparisons available for all NIST
quantities for protons and alpha particles.
Barbara Mascialino
Geant4 Workshop
Catania, October 4-9 2004
Conclusions
• Validation of all Geant4 Electromagnetic models
against the NIST database
• Quantitative statistical analysis on all the
comparisons
• Fully automated testing system (thanks to
Sandra Parlati and Koichi Murakami)
• Objective comparison among Geant4 models
(with respect to the NIST reference)
• Mature project and results will be presented at
IEEE-NSS – paper submitted for publication next
month
Barbara Mascialino
Geant4 Workshop
Catania, October 4-9 2004
Future perspectives
• Final states
• angular distributions and spectra
• The first results will be shown and
discussed in the parallel section Physics
Book introductory talk by Susanna Guatelli
Barbara Mascialino
Geant4 Workshop
Catania, October 4-9 2004