Maurizio Pighin, University of Udine, Italy
Venezia, 14-16 Marzo 2001
Using IR Techniques Catalogue,
Reuse and Maintain
Existing Software Objects
Maurizio Pighin
Dipartimento di Matematica e Informatica
Università di Udine, Italy
e-mail: [email protected]
Venezia, 14-16 marzo 2001
Workshop SALADIN
Slide 1
Maurizio Pighin, University of Udine, Italy
Introduction
• Using IR tools to catalogue
software life-cycle
• Work done up-to-date
– First Model Definition
– Experimental Analysis
– Result Assessment
• Future perspectives
– Model extension to “specification
– Definition of mobility “feature”
– New Experimental Analysis
Venezia, 14-16 marzo 2001
Workshop SALADIN
Slide 2
Maurizio Pighin, University of Udine, Italy
Introduction
• Existing SW Objects
–
–
–
–
–
Specifications
Data-Model
Code
Test Chain
…
• Software Object Catalogue
– Repository of heterogeneous information
Venezia, 14-16 marzo 2001
Workshop SALADIN
Slide 3
Maurizio Pighin, University of Udine, Italy
Introduction
• Difficulties
–
–
–
–
Catalogue organization definition
Adopted methodology to fill it
Supported techniques to analyze it
…
• with Existing Software Objects
– The economic break-even point is very high
• Novel approach
– Understand how heterogeneous information
must be organized
– Application of Information Retrieval techniques
– Experimental evaluation on these techniques
Venezia, 14-16 marzo 2001
Workshop SALADIN
Slide 4
Maurizio Pighin, University of Udine, Italy
The Proposed Methodology
• First Step
– Verification on code-objects the soundness of
proposal
• ALICE (AppLication of Information Retrieval
to Catalogues of Existing software) is the
Experimental System produced
Venezia, 14-16 marzo 2001
Workshop SALADIN
Slide 5
Maurizio Pighin, University of Udine, Italy
The Proposed Methodology
• Search Functionalities which must be at disposal
of user
•
Structure of objects which must be catalogued
• task-analysis of Sw-engineer work permits us to
built a basic functionalities model
Venezia, 14-16 marzo 2001
Workshop SALADIN
Slide 6
Maurizio Pighin, University of Udine, Italy
The Proposed Methodology
• Four basic search methodologies on a Software
Catalogue
– semantics-driven search
– data-definition driven search
– structure-driven search
– syntax-driven search
• These search mechanisms can be combined in more
complex strategies
• From mechanism we found the feature which
defines the keys of catalogue search
Venezia, 14-16 marzo 2001
Workshop SALADIN
Slide 7
Maurizio Pighin, University of Udine, Italy
The Proposed Methodology
• The structure can automatically be built by
SW-Objects and can be indexed in the
catalogue
• Depending from object format and from
requested feature specialized parsers must
be built which analyze the objects end extract
interesting feature references
Venezia, 14-16 marzo 2001
Workshop SALADIN
Slide 8
Maurizio Pighin, University of Udine, Italy
The IR tool
• ALICE is built on Z-PRISE (statistical engine)
• We built the specialized parser which analyzes and
indexes an industrial SW package (about 150.000
“C” code lines)
• We defined a specialized client which easily interacts
with query, features and the ranked list extracted by
Z-Prise engine
Venezia, 14-16 marzo 2001
Workshop SALADIN
Slide 9
Maurizio Pighin, University of Udine, Italy
Venezia, 14-16 marzo 2001
Workshop SALADIN
Slide 10
Maurizio Pighin, University of Udine, Italy
The experiment
•
•
•
•
Ten students, one expert
Three management problems
20 minutes for every problem
Operation objective analysis
– SW-Monitor with operation-logAnalisi
soggettiva
• Operation Subjective analysis
– Questionnaries
• Semantic differential
• Likert Scale
(used the know-how of project IRNLI-Information
Retrieval, Natural Language Interface)
Venezia, 14-16 marzo 2001
Workshop SALADIN
Slide 11
Maurizio Pighin, University of Udine, Italy
Results
Table 1: Results of the experiment
Score
Measured Effectiveness
Precision
Recall
Perceived Effectiveness
Accurate
Satisfactory
Appropriate
Complete
Process Satisfaction
Satisf. Search Strategies
Search Complete
Target Reached
Noise avoided
Keyword Adequate
Easy Kind Query
Easy Choice Search field
System Satysfaction
Efficienct
Satisfactory
Reliable
Flexible
Fast
Usable
Relaxing
Restful
Simple
St.Dev.
0.46
79.98
1.95
70.71
7.05
74.99
9.86
75.23
4.64
69.17
68.53
82.25
80.09
80.09
77.49
71.65
64.99
61.65
73.35
65.05
78.35
79.95
83.55
82.25
86.15
78.79
72.29
83.12
62.77
64.07
61.90
TOTAL RESULTS
Venezia, 14-16 marzo 2001
Mean
68.85
Workshop SALADIN
Slide 12
Maurizio Pighin, University of Udine, Italy
Le Valutazioni
• Score mechanism
• Interesting results about
– Effectiveness
– Easy of use
– Efficiency
• The system seem powerful and sound
• This part of work is presented at
– International Conference on Software
Maintenance and Reuse, CSMR 2001, Lisboa
– Information Software Technology, Vol 51/2-3,
Elsevier Science Publications, Clare (IR), 2000,
pp. 131-138
Venezia, 14-16 marzo 2001
Workshop SALADIN
Slide 13
Maurizio Pighin, University of Udine, Italy
The Actual/Future Steps
• Attention on other SW objects
– Specification mechanism
• UML
• E-R
• Informal specification
• Actual activities
– “Feature” definition
• Particular attention at “mobility” aspects (PhD-Thesis
Gian Pietro Picco – Milan Politecnico team)
–
–
–
–
–
Venezia, 14-16 marzo 2001
Mobility code and data
Security
Traslation
Communication
…..
Workshop SALADIN
Slide 14
Maurizio Pighin, University of Udine, Italy
Reference schemas
Venezia, 14-16 marzo 2001
Workshop SALADIN
Slide 15
Maurizio Pighin, University of Udine, Italy
I Passi Attuali/Futuri
• … Continue Activity
– Definition of Data Base on which work
(Ericsson - UTMS specification Team)
– Application of more specification schemas
– Parser (feature) definition
– Search interface definition
– Search schemas (feature) definition
– Test on a significant Data Base
– Assessment of results
Venezia, 14-16 marzo 2001
Workshop SALADIN
Slide 16
Maurizio Pighin, University of Udine, Italy
The potential results
• (Reasonable) Automatic Cataloguing of
existing specifications
• Homogenization of specifications defined in
heterogeneously (like UTMS specification
team)
• Use of powerful cataloguing and search IR
tools
Venezia, 14-16 marzo 2001
Workshop SALADIN
Slide 17