HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Model Transformations in
Non-Functional Analysis
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Assistant Prof. Dr.-Ing. Steffen Becker
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
1
Personal Background
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
§  Studied Wirtschaftsinformatik in Darmstadt
§  Meta-Modelling Focus
§  Component Meta-Models and Marketplaces
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
§  Research Assistant in Oldenburg
§  Palladio Component (Meta-)Model
§  Model-Driven Architecture-Based Quality Prediction
§  Research Assistant in Karlsruhe
§  Department Manager at FZI
§  Disputation in Oldenburg to become
Dr.-Ing. in a performance engineering context
§  Since April 2010 Juniorprofessor at U Paderborn
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
2
HEINZ NIXDORF INSTITUT
Outline
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Motivation
Conclusions
& Outlook
Process
Trade-off
Analysis
Input Models
& Annotations
Model
Completions
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
3
HEINZ NIXDORF INSTITUT
Outline
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Motivation
Conclusions
& Outlook
Process
Trade-off
Analysis
Input Models
& Annotations
Model
Completions
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
4
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Example: SAP Business By Desgin
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
•  Development costs >300 Mio. €,
est. TTM + 1 year (was 3 finally)
•  Handles 10 users instead of the
planned 1000
•  Cite: „Main issue is the system‘s design, which
disallows server to access free capacities on other
machines…“
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
5
Context?
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
System Type
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
•  Web-based system
•  Three-tier business information system
Characteristics
•  Request-Response communication, often blocking
•  Open world assumptions, users come whenever they like
Metrics of Interest
•  Response time for requests, Utilization, Throughput
•  PROFOD
•  Costs
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
6
HEINZ NIXDORF INSTITUT
What might have happened?
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Requirements
Engineering
Acceptance
Test
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
System
Design
System
Test
Implementation
Model Transformations in Non-Functional Analysis, S. Becker
Unit
Testing
22.06.12
77 7
HEINZ NIXDORF INSTITUT
What might have happened?
§  Capture non-functional
properties
§  Typical issues
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Requirements
Engineering
Acceptance
Test
System Design
§  Imprecise
“The system has to be scalable,
reliable, performant, … and should not cost anything”
§  Ignore context
“The system should be unavailable only 3 min / year”
§  Overly strict
“The system should always react in 1sec”
Implementation
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
System
Test
Model Transformations in Non-Functional Analysis, S. Becker
Unit
Testing
22.06.12
8
Examples: HowTo do it better
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Response time of service bookFlight is
1ms
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Service fails 1h per year in the mean
Development of the system cost 10.000€
Average utilisation of server CPU is 80%
Mean repair time of the server is 1h
Throughput must at least be 5M
transactions per hour
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
9
HEINZ NIXDORF INSTITUT
What might have happened?
§  Design architecture,
classes, components, …
§  Typical issues
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Requirements
Engineering
Acceptance
Test
System Design
§  Ignore non-functional properties
“We can fix anything later…”
§  Reason on them
Informal (SAAM, ATAM), based on experience
§  Document non-functional rationales for decisions
Implicit in patterns or class designs
§  No context again
“A cache in front of a data access
is always good…”
Implementation
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
System
Test
Model Transformations in Non-Functional Analysis, S. Becker
Unit
Testing
22.06.12
10
HEINZ NIXDORF INSTITUT
What might have happened?
§  Test single entities
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Requirements
Engineering
Acceptance
Test
System Design
System
Test
§  Typical issues
§  Ignore non-functional properties
“We can fix anything later…”
§  No context
“I do not know the context of my module, and here it
performs fine!”
§  No explicit non-functional tests
Mainly test of module functionality
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Implementation
Model Transformations in Non-Functional Analysis, S. Becker
Unit
Testing
22.06.12
11
HEINZ NIXDORF INSTITUT
What might have happened?
§  Test drive composed
system
§  Typical issues
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Requirements
Engineering
Acceptance
Test
System Design
§  For the first time we consider
non-functional properties explicitly
§  Create context
“Target is one million users, can we test that in our lab?
What will it cost?”
Implementation
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
System
Test
Unit
Testing
§  If issues arise
§  Re-Implement, Re-Design, Re-Negotiate
§  Project failures are likely
§  Most kinds of architecture rework is very costly
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
12
Compared to: What is Engineering?
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
“[..] creative application of scientific principles to design or develop
structures, machines, apparatus, [..] to forecast their behavior under
specific operating conditions; all as respects an intended function,
economics of operation and safety to life and property.”
Systematic
design of
system
System model
Forecast
of quality
aspects
Requirement fulfilment
Environment model
Requirements model
Economic operation
Safety properties
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
13
HEINZ NIXDORF INSTITUT
Engineering history
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Engineering
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Industrial
production
Craftsmanship
• Customer and
developer
often identical
• Work division
• Specialised
education
• Tools
developed by
third-parties
Optimized Quality
of products and
processes
requires
• Scientific
foundations /
theory
• Guidelines to
apply methods
to real world
problems
• Talent and
experience
guide
development
based on [Shaw&Garlan95]
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
14
HEINZ NIXDORF INSTITUT
Development Cycles
Testing
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Requirements
Model-based
Software Engineering
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Analysis
Model
Implementation
Architecture
Prediction
Design
Feedback
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
15
HEINZ NIXDORF INSTITUT
Civil Engineering Analogy
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Model
Analysis
Results
Theory
$#B




















65
M
/=
µ
M
)
η
M
/
RG M


C4


;
)(;L)L=
M
/)(;
M
/) ;
M
-*NL*L/=
M
*0B ;
39 >
>
$(?A2 D
&# > %/B2




[http://de.wikipedia.org/wiki/Statik_%28Physik%29]
Model Transformations in Non-Functional Analysis, S. Becker
!
3F HF
[http://de.wikipedia.org/wiki/Baustatik]
22.06.12
16
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
An Instance is Model-Driven
Performance Engineering
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Design
5 ms
10 ms
25 ms
Performance
Model
Model
Transformation
Implementation
Analysis /
Simulation
Response Time,
Throughput,
Utilisation
[Koziolek et al., 2006]
Tool Supported
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
17
HEINZ NIXDORF INSTITUT
Different alternative designs...
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Requirements
Software Architect / Designer
Alternative 1
Assume: Same
functionality,
different QoS
Alternative 2
Alternative 3
Which alternative do we
implement?
[SWA09]
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
18
HEINZ NIXDORF INSTITUT
Example…
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
20.000 €
25.000 €
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
1
Service 2
0,01%
0,10%
10 ms
5 ms
SOA?
QoS?
Java
C / C++
Legacy Application
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
19
What is evaluated?
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Examples
§  Encryption component: Length of the key
§  Security vs. performance
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
§  Message formats: priority of message processing
§  Expressiveness / transmission security vs. performance (of the
whole system)
§  After such design alternatives are identified, they can be
evaluated:
§  Detailed design
§  Application of analysis methods
§  Prototyping
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
20
Further examples?
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
§  On site assignment:
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
§  Collect some alternative design or architectural patterns
§  List their variants?
§  What is their expected impact wrt. quality properties?
§  Examples
§ 
§ 
§ 
§ 
§ 
MVC vs. its variants?
MVC vs PAC?
Cache
Pooling
...
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
21
HEINZ NIXDORF INSTITUT
Outline
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Motivation
Conclusions
& Outlook
Process
Trade-off
Analysis
Input Models
& Annotations
Model
Completions
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
22
HEINZ NIXDORF INSTITUT
Process
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Model, e.g.,
using UML2
Interpretation
& Feedback
Solve
Model Transformations in Non-Functional Analysis, S. Becker
Add quality
annotations
Transform
into analysis
model
22.06.12
23
HEINZ NIXDORF INSTITUT
Model
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Software Architects create a model of the SuS
•  SW Models primarily focus on software & hardware –
not QoS issues
•  Software architects create and maintain models
•  Can they do this?
•  A question for empirical Software Engineering
Many languages exist, e.g.,
• 
• 
• 
• 
UML2 + MARTE
PalladioCM
PUMA
ProgressCM
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
24
Annotations
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Models need QoS annotations, e.g.,
•  Resource demands
•  Failure rates
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Requirements?
•  Easy to specify
•  Tool support to prevent mistakes
•  Easy access from model transformations
State of the Art?
•  MARTE
•  Extensive & complex standard
•  Parsers not well standardized
•  Specialised languages
•  Commonly have precise syntax & semantics
•  Some accessible by model-transformations
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
25
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Screenshot
PCM StoEx language
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
26
Transform
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Generates analysis model
•  (L)QN
•  Markov Chain
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Requirements?
•  Should be fully automated and easily accessible by software
architects
•  Again, a question for empirical software engineering
•  In practice experiments not done often enough!
•  Full automation implies all information is in the model
Development?
•  Needs expert both for software models and analysis theory
•  Additionally, expertise in MDD technologies required
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
27
HEINZ NIXDORF INSTITUT
Solve
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Automated step to generate analysis results
•  Throughput
•  PROFOD
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Usually done by running external solvers like
•  PRISM
•  Java Modelling Tools (QN tool)
•  SimQPN (Petri Net based)
•  CSP Checker (Process algebra)
•  …
Issues?
•  Typical issue: Models and tools do not scale for arbitrary inputs
•  à Transformations often need to abstract!
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
28
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
PCM:
Example transformations & solvers…
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
29
HEINZ NIXDORF INSTITUT
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Why so many solvers…?
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
30
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Disclaimer: This is joint work
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
§  Presentation of the results of a series of experiments
§  Performed over 1 year duration
§  Submitted to Transactions of Software Engineering (TSE),
currently under review
§  Results of joint work
§ 
§ 
§ 
§ 
Karlsruhe Institute of Technology
University of Zurich
University of Paderborn
Gameforge
Experiment Design: Research Questions
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
R1:What is the quantified error introduced
R2: Whatby
is the
runtime solver?
of each solver?
a particular
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Simulation
Layered QN
Queued PetriNets
Components/Services
Hardware Ressources
System Usage
Prototype
0011
0101
HEINZ NIXDORF INSTITUT
Experiment: Test Models
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Media Store
1 user in closed workload
WebGUI
MediaStore
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Digital
Watermarking
1 request/sec
Database
Cache
Media
Database
Process Control System
Server 2
Work
Order
Session
JDBC Driver
Message
Sender
Session
Mfg
Session
JDBC
Connection Pool
48 request/
sec
Business Reporting System
Server 1
Server 3
Tomcat
Webserver
Database
Server 4
Graphical
Reporting
Core
Graphics
Engine
Online
Reporting
Core
Online
Engine
C17
C5
C6
C7
C8
C9
C10
C28
C12
C13
C15
C14
C1
C16
C3
Oracle Database Server
Delegate
Work Order
Session
Server 3
C4
C11
1 request/
sec
150 requests/
sec
15 requests/
sec
Server 1
Oracle WebLogic Server
MySQL
Database
Server
Application Server
SPECjEnterprise2010
45 requests/
sec
C2
C18
C19
C20
C21
C22
C23
C24
C25
C26
C27
Server 2
Scheduler
Cache
User
Management
3 CPU cores
48 request/
sec
Experiment: Analysis Models
HEINZ NIXDORF INSTITUT
=1
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
UsageScenario_BRS_usage_scenario_1_Entry
UsageScenario_BRS_usage_scenario_1_Task { }
Start2
[0]
EntryLevelSystemCall3
[0]
UsageScenario_Loop_13
[0]
EntryLevelSystemCall57
[0]
EntryLevelSystemCall64
[0]
EntryLevelSystemCall65
[0]
Stop66
[0]
(1)
(5)
(1)
InnerCoreReportingEngine_IAdmin_maintain_23242526_Entry
UsageScenario_Loop_13_Entry
WebServer_IHTTP_processGraphicalReport_4558_Entry
UsageScenario_BRS_usage_scenario_1_Processor
InnerCoreReportingEngine_IAdmin_maintain_23242526_Task { }
StartAction_aName__PmVo4lekEd6zR8NNmb-sJg_232425_26
[0]
InternalAction_doMaintenance__tUnPoFekEd6zR8NNmb-sJg_232425_260
[0]
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
StopAction_aName__PmVo41ekEd6zR8NNmb-sJg_232425_26
[0]
UsageScenario_Loop_13_Task
WebServer_IHTTP_processGraphicalReport_4558_Task
{ }
{ }
Start14
[0]
StartAction_aName__Kb5aYF6hEd6D97fyAyMq6A_45_58
[0]
UsageScenario_Loop_15
InternalAction_acceptRequest__gvxEEF6sEd6D97fyAyMq6A_45_580
[0]
[0]
EntryLevelSystemCall35
LAN_ExternalCallAction_aName__l5Vm8F6hEd6D97fyAyMq6A_45_58_SYNCH
[0]
[0]
EntryLevelSystemCall37
ExternalCallAction_aName__l5Vm8F6hEd6D97fyAyMq6A_45_58
[0]
[0]
EntryLevelSystemCall38
LAN_ExternalCallAction_aName__l5Vm8F6hEd6D97fyAyMq6A_45_58_return
[0]
[0]
StopAction_aName__Kb5aYV6hEd6D97fyAyMq6A_45_58
EntryLevelSystemCall48
[0]
[0]
Stop56
[0]
(1)
(1)
(1)
(1)
(5)
(1)
(1)
(1)
WebServer_IHTTP_processLoginLogout_4536_Entry
WebServer_IHTTP_processLoginLogout_456_Entry
UsageScenario_Loop_15_Entry
WebServer_IHTTP_processGraphicalView_4539_Entry
WebServer_IHTTP_processOnlineReport_4549_Entry
Scheduler_IBRSSystem_report_7859_Entry
InnerCoreReportingEngine_IAdmin_maintain_23242526_Processor
UsageScenario_Loop_13_Processor
WebServer_IHTTP_processGraphicalReport_4558_Processor
WebServer_IHTTP_processLoginLogout_4536_Task
WebServer_IHTTP_processLoginLogout_456_Task
{ }
UsageScenario_Loop_15_Task
{ }
WebServer_IHTTP_processGraphicalView_4539_Task
{ }
WebServer_IHTTP_processOnlineReport_4549_Task
{ }
{ }
StartAction_aName__7gTfQdo5EdyCSNQKlT1LxA_45_36
StartAction_aName__7gTfQdo5EdyCSNQKlT1LxA_45_6
[0]
[0]
Start16
StartAction_aName__JdWO4GyAEd68zLmG_5OTdQ_45_39
StartAction_aName__LCybQF6hEd6D97fyAyMq6A_45_49
[0]
[0]
[0]
Scheduler_IBRSSystem_report_7859_Task { }
StartAction_aName__WnOesNo5EdyCSNQKlT1LxA_78_59
[0]
BranchAction_branchForRequestType__P30yANo6EdyCSNQKlT1LxA_45_36
BranchAction_branchForRequestType__P30yANo6EdyCSNQKlT1LxA_45_6
EntryLevelSystemCall17
InternalAction_acceptView__L-bEgGyAEd68zLmG_5OTdQ_45_390
InternalAction_acceptRequest__msfhMF6sEd6D97fyAyMq6A_45_490InternalAction_schedule__0QdhwLpDEd2fi_j9dEpB2g_78_590
[0]
[0]
[0]
[0]
[0]
[0]
+
+
1
1
Stop34
ExternalCallAction_aName__ebMmAFjxEd6UHd_VhJH9SQ_78_59
LAN_ExternalCallAction_aName__NtSaEGyAEd68zLmG_5OTdQ_45_39_SYNCH
LAN_ExternalCallAction_aName__L8754F6iEd6D97fyAyMq6A_45_49_SYNCH
[0]
[0]
[0]
[0]
StartAction_aName__QQIk0VhJEd6UHd_VhJH9SQ_45_36
StartAction_aName__N32e0FhJEd6UHd_VhJH9SQ_45_6
[0]
[0]
BranchAction_GraphicalOrOnline__a8tj4FenEd6zR8NNmb-sJg_78_59
ExternalCallAction_aName__NtSaEGyAEd68zLmG_5OTdQ_45_39
ExternalCallAction_aName__L8754F6iEd6D97fyAyMq6A_45_49
[0]
[0]
[0]
LAN_ExternalCallAction_aName__T3QcgFhJEd6UHd_VhJH9SQ_45_36_SYNCH
LAN_ExternalCallAction_aName__SMQ6UFhJEd6UHd_VhJH9SQ_45_6_SYNCH
[0]
[0]
+
1
LAN_ExternalCallAction_aName__NtSaEGyAEd68zLmG_5OTdQ_45_39_return
LAN_ExternalCallAction_aName__L8754F6iEd6D97fyAyMq6A_45_49_return
[0]
[0]
StartAction_aName__cocs8VenEd6zR8NNmb-sJg_78_59
ExternalCallAction_aName__T3QcgFhJEd6UHd_VhJH9SQ_45_36
ExternalCallAction_aName__SMQ6UFhJEd6UHd_VhJH9SQ_45_6
[0]
[0]
[0]
StopAction_aName__JdWO4WyAEd68zLmG_5OTdQ_45_39
StopAction_aName__LCybQV6hEd6D97fyAyMq6A_45_49
[0]
[0]
LAN_ExternalCallAction_aName__T3QcgFhJEd6UHd_VhJH9SQ_45_36_return
LAN_ExternalCallAction_aName__SMQ6UFhJEd6UHd_VhJH9SQ_45_6_return
[0]
[0]
StopAction_aName__QQIk0lhJEd6UHd_VhJH9SQ_45_36
StopAction_aName__N32e0VhJEd6UHd_VhJH9SQ_45_6
[0]
[0]
+
StopAction_aName__cocs8lenEd6zR8NNmb-sJg_78_59
[0]
+
+
StopAction_aName__7gTfQto5EdyCSNQKlT1LxA_45_36
StopAction_aName__7gTfQto5EdyCSNQKlT1LxA_45_6
[0]
[0]
(1)
ExternalCallAction_aName__jP7-UFenEd6zR8NNmb-sJg_78_59
[0]
(1)
StopAction_aName__WnPFwNo5EdyCSNQKlT1LxA_78_59
[0]
(1)
(1)
(1)
(1)
Scheduler_IBRSSystem_logout_789_Entry
Scheduler_IBRSSystem_login_789_Entry
WebServer_IHTTP_processOnlineView_4518_Entry
Scheduler_IBRSSystem_view_7840_Entry
Scheduler_IBRSSystem_report_7850_Entry
GraphicalReporting_IGraphicalReport_graphicalReport_414260_Entry
WebServer_IHTTP_processLoginLogout_4536_Processor
WebServer_IHTTP_processLoginLogout_456_Processor
UsageScenario_Loop_15_Processor
WebServer_IHTTP_processGraphicalView_4539_Processor
WebServer_IHTTP_processOnlineReport_4549_Processor
Scheduler_IBRSSystem_report_7859_Processor
Scheduler_IBRSSystem_logout_789_Task { }
Scheduler_IBRSSystem_login_789_Task { }
StartAction_aName__Mv7U0FhGEd6UHd_VhJH9SQ_78_9
StartAction_aName__MSbggFhGEd6UHd_VhJH9SQ_78_9
[0]
[0]
WebServer_IHTTP_processOnlineView_4518_Task
Scheduler_IBRSSystem_view_7840_Task
{ }
{ }
StartAction_aName__J6s5QGyAEd68zLmG_5OTdQ_45_18
StartAction_aName__WnPs0do5EdyCSNQKlT1LxA_78_40
[0]
[0]
Model Transformations in Non-Functional Analysis, S. Becker
Scheduler_IBRSSystem_report_7850_Task
GraphicalReporting_IGraphicalReport_graphicalReport_414260_Task
{ }
{ }
StartAction_aName__WnOesNo5EdyCSNQKlT1LxA_78_50
StartAction_aName__QM9uklelEd6zR8NNmb-sJg_4142_60
[0]
[0]
22.06.12
34
UI
Database
Server
MediaStore
Delegate
Work Order
Session
Work
Order
Session
Experiment: Results excerpt
l
king
Database
Cache
Media
Database
Process Control System
1 request/
sec
150 requests/
sec
quests/
Server 2
Message
Sender
Session
48 request/
sec
Server 3
Mfg
Session
JDBC Driver
HEINZ NIXDORF INSTITUT
JDBC
Connection Pool
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Business Reporting System
Server 1
Server 3
Tomcat
Webserver
Database
Server 4
Graphical
Reporting
Core
Graphics
Engine
Online
Reporting
Core
Online
Engine
C17
C28
C12
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
C13
C15
C14
C1
C16
C2
C18
C19
C20
C21
C22
C23
C24
C25
C26
C27
Server 2
Scheduler
Cache
User
Management
48 request/
sec
3 CPU cores
Solver
Mean RT pred. Diff in %
Solver RT
Simulation
5,22 s
N/A
591,5 s
SimQPN
5,92 s
13,3%
74,5 s
LQN
7,39 s
41,5%
4,0 s
Prototype
6,53 s
25,0%
972,9 s
HEINZ NIXDORF INSTITUT
Experiment: Results excerpt
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Media Store
1 user in closed workload
WebGUI
MediaStore
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Digital
Watermarking
1 request/sec
Media
Database
1 request/
sec
Process Control System
150 requests/
sec
Mean RT pred. Diff in %
Server 1
Simulation
SimQPN
Database
Cache
15 requests/
sec
Solver
Oracle WebLogic Server
MySQL
Database
Server
Application Server
C4
C5
C7
LQN
C9
Prototype
C11
1,33 s
Server 2
Server 3
N/A
C6
1,33 s
C8
1,28 s
C10
1,02
s
C1
0%C18
C12
C13
C15
C14
C3
C2
C17
C28
C20
-3,3%
C19
C21
C22
C23
C24
C25
C26
C27
-22,8%
C16
SPECjEnterprise
45 requests/
sec
Delegate
Work Order
Session
Work
Order
Session
Message
Sender
Session
Mfg
Session
48 request/
sec
Ora
Business Reporting
Server 1
Solver RTTomcat
Webserver
31,4 s
Server
Da
Server
Server 2
6,4 s
Scheduler
1,3 s
User
1034,7
s
Management
3 CPU cores
Graphical
Reporting
Online
Reporting
G
E
O
E
Interpretation
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Performing role: Skilled analysts, …
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
•  Take analysis model results
•  Translate them into statements about the SuS
•  Suggest improved designs
Example:
•  Analysis model: Service failure probability > 10%
•  Software model: Analyse single failure probabilities
•  Countermeasures
•  Introduce redundancy
•  Replication
•  Voting algorithms
•  …
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
37
HEINZ NIXDORF INSTITUT
Outline
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Motivation
Conclusions
& Outlook
Process
Trade-off
Analysis
Input Models
& Annotations
Model
Completions
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
38
HEINZ NIXDORF INSTITUT
Intro and Static Viewpoint
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
§  Dynamic web server example
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
§  Dynamic web service for querying the weather forecast of
the next 7 days “Sunshine2Go”
§  Reports weather conditions of next 7 days
for given location in the world
<<System>>
Sunshine2Go
Webserver
Model Transformations in Non-Functional Analysis, S. Becker
DBServer
22.06.12
39
HEINZ NIXDORF INSTITUT
Allocation of “Sunshine2Go”
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
§  Deployment is on local servers per component
§  Web server runs the web server component, e.g., Apache
§  Database server runs the DB, e.g., MySQL
Webserver Node
Webserver
Model Transformations in Non-Functional Analysis, S. Becker
DB Server Node
DBServer
22.06.12
40
HEINZ NIXDORF INSTITUT
Main usage scenario
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
§  Users query for the weather of a certain location and the
amount of days in the future
ws:Webserver
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
User
db:Database
getWeather(loc,days)
loop (0, days)
getDay(loc,d)
weather=getDay(loc,d)
return
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
41
Requirements
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
§  Initial use case
§  At the main business hours, users arrive with an mean
interarrival time of 3 sec and exponential distribution
(Poisson arrival process)
§  The mean response time of each call should not exceed
1sec
§  The utilisation of all resources should be below 80%
§  Additional use cases are interesting, but skipped here for
time reasons
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
42
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Measurements…
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
§  Service time at webserver CPU is exponentially distributed
with mean of 1WU
§  At the database for each request the CPU has to perform
2WU and the HDD has to execute 1Read
§  1WU of Webserver CPU = 0.01 sec
§  1WU of DB Server CPU = 0.01 sec
§  1WU of DB HDD = 0.03 sec
§  All CPU act approx. with processor sharing
§  All HDDs approx. with FCFS scheduling
§  Measurements indicate each step can fail with a
probability of failure of 0.001
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
43
HEINZ NIXDORF INSTITUT
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Enhancing the model using
UML MARTE
Using UML MARTE
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
44
HEINZ NIXDORF INSTITUT
Process
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Model using
UML2
Interpretation
& Feedback
Solve
Model Transformations in Non-Functional Analysis, S. Becker
Add quality
annotations
Transform
into analysis
model
22.06.12
45
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
UML MARTE
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
§  UML Profile for MARTE
Modelling and Analysis of RT Embedded Systems
§  Current Version: 1.0
§  Released in 2009
§  Maintained by OMG
§  Successor of UML SPT which it is supposed to replace
§  Uses UML QoS & FT to define quality characteristics
§  Slight overlap with SysML, supposed to be aligned
§  Extensive Standard, >700 pages
§  Defines several profile packages
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
46
HEINZ NIXDORF INSTITUT
MARTE Layout
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
MARTE foundations
<<profile>>
NFPs
<<profile>>
Time
<<profile>>
GRM
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
MARTE design model
<<profile>>
RTEMoCC
<<profile>>
GCM
<<profile>>
Alloc
MARTE analysis model
<<profile>>
SRM
<<profile>>
HRM
<<profile>>
GQAM
<<profile>>
SAM
<<profile>>
PAM
MARTE annexes
<<profile>>
VSL
<<profile>>
RSM
<<profile>>
MARTE_ModelLibrary
[http://www.omg.org/omgmarte/]
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
47
HEINZ NIXDORF INSTITUT
Main Usecase
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
<<GaPerformanceContext>>
{contextParams=in $days}
ws:Webserver
<<PaRunTInstance>>
{instance=ws}
User
db:Database
<<PARunTInstance>>
{instance=db}
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
getWeather(loc,days)
<<GaWorkloadEvent>>
{open(IntArrTime=
exp(0.3333,sec))}
loop (0, $days)
getDay(loc,d)
<<PaStep>>
{hostDemand=exp(1,WU)}
weather=getDay(loc,d)
return
<<PaStep>>
{hostDemand=exp(2,WU),
servHost=CPU}
Model Transformations in Non-Functional Analysis, S. Becker
<<PaStep>>
{hostDemand=exp(1,WU),
servHost=HDD}
22.06.12
48
HEINZ NIXDORF INSTITUT
Annotated Deployment
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
<<GaPerformanceContext>>
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Webserver Node
DB Server Node
Webserver
DBServer
CPU
<<GaExecHost>>
{processingRate=(100WU/
sec)}
CPU
<<GaExecHost>>
{processingRate=(100 WU/
sec)}
Model Transformations in Non-Functional Analysis, S. Becker
HDD
<<GaExecHost>>
{processingRate=(100/3
WU/sec)}
22.06.12
49
HEINZ NIXDORF INSTITUT
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Analysis Model
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
50
HEINZ NIXDORF INSTITUT
Process
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Model using
UML2
Interpretation
& Feedback
Solve
Model Transformations in Non-Functional Analysis, S. Becker
Add quality
annotations
Transform
into analysis
model
22.06.12
51
HEINZ NIXDORF INSTITUT
Queuing systems
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
§  Mature analysis model for computer system performance
§  Idea: Represent active resources as waiting queues
§  Graphical notation:
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
λ
µ
E.g. queues for
§  CPUs
§  Disks
§  Network
§  Reflects waiting times due to contention
§  Underlying formalism is Continuous Time
Markov Chains (CTMCs)
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
52
Queuing network
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
§  Queuing network
§ 
§ 
§ 
§ 
Directed graph
Nodes are service centre
Edges represent potential paths of jobs
Multiple job classes
•  Each having homogeneous service demands
•  Homogeneous routing probabilities
•  I.e., jobs in a class cannot be distinguished
§  Widely used
§  Offer decent performance prediction accuracy
§  Good modelling paradigm and usability
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
53
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Open network
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
§  Jobs enter net
at source nodes
§  Jobs leave net
at sink nodes
§  I.e., number of jobs
not defined upfront
§  Arrival process
characterised by inter-arrival time
§  Each queue is characterised in Kendall notation
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
54
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Closed network
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
§  Network of a fixed
number of jobs
...
§  Contains a set of terminals
§  Jobs circulate in network
§  Jobs delay in terminals
for a given think-time
interval
§  Sometimes also modelled by
IN and OUT ports which are connected in a loop
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
55
HEINZ NIXDORF INSTITUT
Routing in QNs
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
§  Jobs take probabilistic routes
§  Route specified via transitions probabilities
§  Given per class of jobs for multiclass networks
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Disk 1
0.25
CPU
Disk 2
0.75
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
56
HEINZ NIXDORF INSTITUT
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
QN for our example…
Highlighting the core ideas of the transformation step
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
57
Performance:
Derive QN
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Source
Arrival Rate=1/3 Arrivals/s
Exponential distribution
Sink
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
58
HEINZ NIXDORF INSTITUT
Derive QN
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Add a queue per GaExecHost
and their parameters
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
CPU WS
HDD DB
M/M/1/PS
µ = 1/100 Jobs/s
?/M/1/FCFS
µ = 3/100 Jobs/s
CPU DB
?/M/1/PS
µ = 2/100 Jobs/s
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
59
HEINZ NIXDORF INSTITUT
Derive QN
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Add routes and their
probabilities
CPU WS
HDD DB
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
1/7
6/7
CPU DB
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
60
HEINZ NIXDORF INSTITUT
Put it in a Solver
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
JSimGraph Model
System
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Normal Users Arrival
CPU_WS
CPU_DB
HDD_DB
Normal Users Departure
§  Results (according to jMVA)
§  Mean System RT = 0.5502 sec
§  Util CPU_WS = 0.33%
§  Util CPU_DB = 2.33 %
§  Util HDD_DB = 6.99 %
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
61
HEINZ NIXDORF INSTITUT
Process
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Model using
UML2
Interpretation
& Feedback
Solve
Model Transformations in Non-Functional Analysis, S. Becker
Add quality
annotations
Transform
into analysis
model
22.06.12
62
What-if analysis for arrival rate
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
•  At arrival rate 4.5 the DB
HDD (cyan) becomes the
bottleneck
•  The HDD queue starts to
overload
•  RT constraint from our
requirements is violated at
arrival rate of > 3.5 job/s
•  Starting from a rate of 2.5
jobs/s the system shows
overload effects
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
63
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Reliability
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
§  Common method is to create a Discrete Time Markov
Chain (DTMC), i.e., disregard time
§  Each step of the system becomes a state of the DTMC
§  In each state, system can fail according to a failure
probability. It reaches an absorbing failure state in case of
failure
§  Loop can be either unrolled or approximated by a
geometric distribution
§  PROFOD is probability of reaching the success state, i.e.,
the state after the last system action
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
64
Reliability:
Derive DTMC
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
§  Create a DTMC for our SuS. Here approx. of loop counts
§  PROFOD approx. 1,48% according to PRISM
{DBHDD}
.999
S3
.1427
.001
S2
.999
start
{DBCPU}
.8563
S5
{succ}
.001
S4
.001
{fail}
S1
{WS}
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
65
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
What if analysis
HEINZ NIXDORF INSTITUT
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
66
HEINZ NIXDORF INSTITUT
Outline
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Motivation
Conclusions
& Outlook
Process
Trade-off
Analysis
Input Models
& Annotations
Model
Completions
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
67
HEINZ NIXDORF INSTITUT
Motivation
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Architectural Model
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Client
Facade
[Becker, WOSP, 2008]
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
68
HEINZ NIXDORF INSTITUT
Motivation
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Architectural Model
Client
Facade
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Prediction
Transformation
Prediction Model
Prediction
Predicted
Time
2 sec
[Becker, WOSP, 2008]
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
69
HEINZ NIXDORF INSTITUT
Motivation
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Architectural Model
Client
Facade
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Prediction
Transformation
Implementation
Prediction Model
……..
……..
Execution/
Measurement
Prediction
Measured
Times
1,9 sec
Predicted
Time
2 sec
[Becker, WOSP, 2008]
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
70
HEINZ NIXDORF INSTITUT
Motivation
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Architectural Model
Client
Facade
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Prediction
Transformation
Implementation
Prediction Model
……..
……..
Execution/
Measurement
Measured
Times
1,9 sec
Prediction
Wanted:
correspondence
Predicted
Time
2 sec
[Becker, WOSP, 2008]
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
71
HEINZ NIXDORF INSTITUT
Motivation
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Architectural Model
Client
Facade
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Prediction
Transformation
Implementation
Prediction Model
……..
……..
Execution/
Measurement
Measured Times
(1,9/3/2,5/.. sec)
Prediction
Wanted:
correspondence
Predicted
Time
2 sec
[Becker, WOSP, 2008]
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
72
HEINZ NIXDORF INSTITUT
Motivation
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Architectural Model
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Client
Facade
Implementation
Prediction Model
……..
……..
Execution/
Measurement
Measured
Times
1,9 sec
Prediction
Wanted:
correspondence
Predicted
Time
2 sec
[Becker, WOSP, 2008]
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
73
HEINZ NIXDORF INSTITUT
Motivation
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Architectural Model
Client
Facade
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Needed:
Dependency
Implementation
Prediction Model
……..
……..
Execution/
Measurement
Measured
Times
1,9 sec
Prediction
Wanted:
correspondence
Predicted
Time
2 sec
[Becker, WOSP, 2008]
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
74
HEINZ NIXDORF INSTITUT
Motivation
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Architectural Model
Client
Facade
Palladio
Component
Model (PCM)
PCM2EJB
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Needed:
Dependency
Coupled
Transformations
Implementation
PCM2SimuCom
Prediction Model
……..
……..
SimuCom
Execution/
Measurement
Measured
Times
1,9 sec
Prediction
Wanted:
correspondence
Model Transformations in Non-Functional Analysis, S. Becker
Predicted
Time
2 sec
[Becker, WOSP, 2008]
[Becker et al., JSS, 2008]
22.06.12
75
HEINZ NIXDORF INSTITUT
Example cont.
Client Machine
Client
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Application Server
1ms
SOAP
0.1ms
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Business
Facade
WebServer
WebClient
Model Transformations in Non-Functional Analysis, S. Becker
0.1ms
RMI
Database Server
Data
Access
Database
2ms
0.8ms
Business
Logic
22.06.12
10
HEINZ NIXDORF INSTITUT
Example cont.
Client Machine
Client
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Application Server
1ms
1ms
0.1ms
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Business
Facade
WebServer
WebClient
Model Transformations in Non-Functional Analysis, S. Becker
0.1ms
0.5ms
Database Server
Data
Access
Database
2ms
0.8ms
Business
Logic
22.06.12
10
HEINZ NIXDORF INSTITUT
Connector Completion
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
IA
IA
Con1
Comp1
Comp2
<<annotate>>
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Legend
XOR-Relation
AssemblyConnector
Optional Feature
Required Feature
RemoteCall
LocalCall
Additional
Processing
Protocol
RMI
SOAP
Encryption
Model Transformations in Non-Functional Analysis, S. Becker
Compression
Authorisation
22.06.12
78
HEINZ NIXDORF INSTITUT
Connector Completion
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
IA
IA
Con1
Comp1
Comp2
<<annotate>>
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Legend
XOR-Relation
AssemblyConnector
Optional Feature
Required Feature
RemoteCall
LocalCall
Additional
Processing
Protocol
RMI
SOAP
Encryption
Model Transformations in Non-Functional Analysis, S. Becker
Compression
Authorisation
22.06.12
79
HEINZ NIXDORF INSTITUT
Connector Completion
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
IA
Con1
IA
Comp1
IA
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Comp1
<<ConnectorCompletion>>
Con1
IMiddleware-Sender
Client
Middleware
Model Transformations in Non-Functional Analysis, S. Becker
IA
Comp2
Comp2
IMiddleware-Receiver
Server
Middleware
22.06.12
80
HEINZ NIXDORF INSTITUT
Connector Completion
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
<<ConnectorCompletion>>
Con1
IA
IA
Comp2
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Comp1
IMiddleware-Sender
Model Transformations in Non-Functional Analysis, S. Becker
IMiddleware-Receiver
22.06.12
81
HEINZ NIXDORF INSTITUT
Connector Completion
IA
Marshal
IA
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
<<ConnectorCompletion>>
Con1
IA’
IA’
Con1'
Demarshal
IA
Comp2
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Comp1
IMiddleware-Sender
Model Transformations in Non-Functional Analysis, S. Becker
IMiddleware-Receiver
22.06.12
82
HEINZ NIXDORF INSTITUT
Connector Completion
IA
Marshal
IA
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
<<ConnectorCompletion>>
Con1
IA’
IA’
Con1'
Demarshal
IA
Comp2
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Comp1
IMiddleware-Sender
Model Transformations in Non-Functional Analysis, S. Becker
IMiddleware-Receiver
22.06.12
83
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
<<ConnectorCompletion>>
Con1'
IA’
IA’
Con1'’
Encrypt
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
IMiddleware-Sender
IA
IA
Marshal
IA’
Decrypt
IMiddleware-Receiver
<<ConnectorCompletion>>
Con1
IA’
IA’
Con1'
Demarshal
IA
Comp2
Comp1
IMiddleware-Sender
Model Transformations in Non-Functional Analysis, S. Becker
IMiddleware-Receiver
22.06.12
13
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
<<ConnectorCompletion>>
Con1'
IA’
IA’
Con1'’
Encrypt
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
IMiddleware-Sender
Comp1
IMiddleware-Receiver
<<ConnectorCompletion>>
Con1
IA’
IA’
IA
IA
IA’
Decrypt
Marshal
IA
Marshal
IMiddleware-Sender
Model Transformations in Non-Functional Analysis, S. Becker
Demarshal
IA
Comp2
Demarshal
IMiddleware-Receiver
22.06.12
13
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Con1'’
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Encrypt
Decrypt
<<ConnectorCompletion>>
Con1
IA
Comp1
IA
IA
Marshal
IMiddleware-Sender
Model Transformations in Non-Functional Analysis, S. Becker
Comp2
Demarshal
IMiddleware-Receiver
22.06.12
13
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
<<ConnectorCompletion>>
Con1
IA
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Comp1
IA
IA
Con1'’
Marshal
Encrypt
IMiddleware-Sender
Model Transformations in Non-Functional Analysis, S. Becker
Decrypt
Comp2
Demarshal
IMiddleware-Receiver
22.06.12
13
HEINZ NIXDORF INSTITUT
Deployed Completion
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
<<ConnectorCompletion>>
ClientRC
ServerRC
IA
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Client
IA
Marshal
NetComp
Encrypt
Decrypt
Demarshal
Server
Net
Client
Middleware
Model Transformations in Non-Functional Analysis, S. Becker
Server
Middleware
22.06.12
14
HEINZ NIXDORF INSTITUT
Deployed Completion
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
<<ConnectorCompletion>>
0.1ms
ClientRC
0.4ms
1ms
Encrypt
NetComp
0.4ms
0.1ms
IA
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Client
ServerRC
IA
Marshal
Decrypt
Demarshal
Server
Net
Client
Middleware
Server
Middleware
SOAP & Encryption
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
14
HEINZ NIXDORF INSTITUT
Deployed Completion
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
<<ConnectorCompletion>>
0.05ms
ClientRC
0.2ms
0.5ms
Encrypt
NetComp
0.2ms
0.05ms
IA
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Client
ServerRC
IA
Marshal
Decrypt
Demarshal
Server
Net
Client
Middleware
Server
Middleware
RMI & Encryption
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
14
HEINZ NIXDORF INSTITUT
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Model-Driven Completions:
Evaluation
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
91
HEINZ NIXDORF INSTITUT
Measurement Setting
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
0.215ms
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
? ms
Glassfish
Application
Server
Client
Linux 2.6
MySQL
AssemblyConnector
RemoteCall
LocalCall
Additional
Processing
Protocol
RMI
SOAP
Encryption
Compression
Model Transformations in Non-Functional Analysis, S. Becker
Authorisation
22.06.12
92
HEINZ NIXDORF INSTITUT
Results
E(Meas)
[ms]
E(Pred)
[ms]
Diff. [ms]
Diff. [%]
None
n/a
0.215
n/a
n/a
RMI
0.988
0.971
0.017
1.7
RMI
1.941
1.871
0.070
3.6
RMI
1.113
1.111
0.002
0.2
RMI
2.060
2.011
0.049
2.4
SOAP
1.329
1.283
0.046
3.5
SOAP
1.498
1.463
0.035
2.3
Protocol
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Auth
Enc
Prediction Error > 400%
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
93
HEINZ NIXDORF INSTITUT
Results
E(Meas)
[ms]
E(Pred)
[ms]
Diff. [ms]
Diff. [%]
None
n/a
0.215
n/a
n/a
RMI
0.988
0.971
0.017
1.7
RMI
1.941
1.871
0.070
3.6
RMI
1.113
1.111
0.002
0.2
RMI
2.060
2.011
0.049
2.4
SOAP
1.329
1.283
0.046
3.5
SOAP
1.498
1.463
0.035
2.3
Protocol
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Auth
Enc
Prediction Error < 3.6%
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
94
HEINZ NIXDORF INSTITUT
Outline
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Motivation
Conclusions
& Outlook
Process
Trade-off
Analysis
Input Models
& Annotations
Model
Completions
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
95
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Trade-off Analysis
Example: Q-ImPrESS
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
96
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Analytic Hierarchy Process
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
§  Decision support method developed by the mathematician
Thomas Saaty in the 1970s
§  General method for multi-criteria decision making (not only
software, but, e.g., also politics)
§  Works quantitatively with qualitative inputs
§  Goals
§ 
§ 
§ 
§ 
Make decisions rationally
Make decision preferences explicit and understandable
Support group/team decisions
Discover inconsistent decisions/preferences
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
97
HEINZ NIXDORF INSTITUT
Process Overview
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Gather
data
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
•  Goal
•  Criteria
•  Alternatives
Compare
•  Alternatives
•  Criteria
Interpret
Model Transformations in Non-Functional Analysis, S. Becker
•  Answer
question
•  Select
alternative
22.06.12
98
1st Step: Gather Data
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
§  Goal (here): „Which architectural alternative offers the best
trade-off among the quantitative properties?“
§  Criteria
§  Quantitative measurements/predictions on architecture
metrics
§  We can also include non-quantitative criteria if we like
§  E.g., response time, PROFOD, etc.
§  Exclude direct alternative costs, they are normally included
in step 3
§  Alternatives
§  Alternative architectures
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
99
HEINZ NIXDORF INSTITUT
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Alternatives and criteria
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Avg. Job / s
= 1 / Avg. RT
1 - PROFOD
Alternative 1
1 / 1s
0.99
3
Alternative 2
1 / 2s
0.9
2
Alternative 3
2 / 3s
0.9
5
Alternative 4
1 / 3s
0.9999
1
Values e.g.
from a QN
Model Transformations in Non-Functional Analysis, S. Becker
Ease of impl.
[5 = easy, 1 = hard]
Values e.g.
from a DTMC
Manual
input
22.06.12
100
Dominance & Pareto front
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
§  If an alternative A is better in all criteria values than an
alternative B, A is said to dominate B à B can be removed
from the list of alternatives
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
§  E.g. Alternative 1 dominates Alternative 2
§  The set of alternative which are not dominated by any
other alternative is called the Pareto-optimal set or
(if shown in figures) the Pareto-front
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
101
HEINZ NIXDORF INSTITUT
1st Step: Create a Hierarchy
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Best
Architecture
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Performance
Dependability
Maintainability
Response time
Reliability
Ease of
implementation
Throughput
Availability
Ease of
changes
Utilisation
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
102
2nd Step: Priorities
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
§  How to formalise priorities?
§  Naive approach: Let stakeholders assign direct weights
§  Problem
§  Decision theory research shows stakeholders give bad
weights when asked to do it directly
§  Test yourself: Which weight is your preference for
performance vs. reliability?
§  Solution
§  AHP uses pair-comparisons
§  Compare two sub-goals of a given parent goal
§  Judge it quantitatively
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
103
Priorities (cont.)
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
§  Use Scale from 1 to 9 (1-5 or 1-15 also used)
§ 
§ 
§ 
§ 
§ 
§ 
§ 
1 = equally important
3 = a bit more important
5 = more important
7 = much more important
9 = extremely more important
2,4,6,8 = intermediate values
Use 1/5 for „less important“ and so on
§  Advantages
§  Easier to judge and express
§  Pairwise comparison limits scope of comparison
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
104
HEINZ NIXDORF INSTITUT
Priorities example
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Derive a matrix
from answers
Performance
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Performance
Reliability
Ease of Impl.
Reliability
Ease of Impl.
1
3
1/3
1/3
1
1/9
3
9
1
Matrix has “symmetry” if answers are
consistent, AHP can also deal with
some inconsistency
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
105
HEINZ NIXDORF INSTITUT
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Consistency
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
§  Can be used to check consistency in the answers of the
decision makers
§  Let vhij denote matrix entry ij in a matrix
of the parent goal h
§  The comparison matrix is consistent if
h
h
h
vik
= vij
· vjk
⇥i, j, k
U (h)
holds.
(with U(h) contains the indices of the criteria or
alternatives)
§  For consistent matrices it is sufficient to determine the first
row, i.e., we have to determine |U (h)| 1 pair
comparisons
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
106
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Determining weights
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
§  Let wh denote the weights of subgoals used to determine
the value of the parent goal
§  wh should reflect the preferences of the decision maker as
given in the matrix
§  That is
wi
vij ⇡
wj
i.e. the weights should reflect the relative preferences
§  To ensure comparability with other goals normalise wh
|U (h)|
X
wi = 1
i=1
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
107
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Determining weights cont.
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
§  If matrix is consistent, take arbitrary column vector and
normalise
vij
wi = P|U (h)|
k=1 vki
§  Example: 2nd column = (3,1,9), i.e., it follows
3 1 9
w=( , , )
13 13 13
§  Otherwise find a solution with as little deviation as possible
à Heuristics exists
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
108
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Heuristic Solutions
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
§  Determine eigenvector belonging to the maximum
eigenvalue showed to be a good heuristic
§  Process
§  Solve characteristic equation det(V
Iµ) = 0
§  Pick maximum eigenvalue µmax
§  Determine the matching eigenvector by solving
(V
µmax I)x(µmax ) = 0
§  Let w = x(µmax )
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
109
Quantify inconsistency
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
§  There is a inconsistency index telling us the inconsistency
of the matrix
IK(V ) = (µmax |U (h)|)/(|U (h)| 1)
§  Problem IK(V) depends on the size of V
§  The larger V the more likely are violations of the
consistency constraint
§  Solution: Use randomly generated V‘s as reference to
normalise the metric. This is called ZI(|U(h)|)
§  Heuristic for too inconsistent matrices: If
IK(V )
> 0.1 then redo pair comparisons
ZI(|U (h)|)
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
110
Determine utility values
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
§  Numeric values (1 / RT, 1 – PROFOD)
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
§  Normalise values
§  Multiply normalised vector with weight
§  Ensure larger values correspond to more utility
§  Qualitative values (e.g. ease of implementation)
§  Again use pair comparisons
§  Determine utility in the same way as weights
§  Use normalised vector as utility
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
111
HEINZ NIXDORF INSTITUT
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Utility of our alternatives
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Avg. Job / s
= 1 / Avg. RT
1 - PROFOD
Ease of impl.
Alternative 1
0.5
0.34
3/9
Alternative 3
0.33333
0.311
5/9
Alternative 4
0.16666
0.345
1/9
[5 = easy, 1 = hard]
Maybe a better
scaling is needed
here??
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
112
HEINZ NIXDORF INSTITUT
Solution for our simple example
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
§  Solve (for Alternatives 1,3,4, i.e., removing the dominated)
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
0
0.5
@ 0.333
0.166
0.34
0.311
0.345
10
1
0
1
0.333
0.23
0.372
0.555 A @ 0.076 A = @ 0.485 A
0.111
0.694
0.141
§  Select Alternative with maximum weighted value
§  In this case, it is Alternative 3 with a value of 0.485
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
113
Discussion
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
§  Rank inversion
§  Adding alternatives may alter alternative ranking
§  Even for irrelevant alternatives
§  One reason is normalisation over the range of values
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
§  Number of pair comparisons
§  Min |U(h)|-1, Max |U(h)|(|U(h)|-1)/2
§  More comparisons allow to judge inconsistency
§  However, people get tired answering pair comparisons
§  Scale
§  Good scale for AHP determined in practice
§  However, no rationale for the scales used
§  Nevertheless, helpful in practice for small numbers of
alternatives
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
114
HEINZ NIXDORF INSTITUT
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Automated Quantitative Approach
Using Meta-Heuristics
[Slides with kind permission from Anne Koziolek]
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
115
HEINZ NIXDORF INSTITUT
Idea
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
C1
3 sec
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
A
Transform
Solve
Change component and deployment
2.5 sec
C2
A
Transform
Solve
[MKB10]
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
116
HEINZ NIXDORF INSTITUT
Idea
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
C1
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
A
Transform
Solve
3 sec
p(fail) 0.01%
$ 5700
Solve
2.5 sec
p(fail) 0.02%
$ 12000
Change component and deployment
C2
A
Transform
 Optimise multiple criteria at once
[MKB10]
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
117
HEINZ NIXDORF INSTITUT
Multi-criteria optimisation
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Architectural Candidate
5s
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Time
A
$40K
Costs
[MKB10]
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
118
HEINZ NIXDORF INSTITUT
Multi-criteria optimisation
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Generated & Evaluated
5s
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Time
A
$40K
Costs
[MKB10]
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
119
HEINZ NIXDORF INSTITUT
Pareto front
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Time
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
A
5s
B
3s
C
2s
Pareto-optimal
$20K
$33K
$40K
Costs
[MKB10]
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
120
HEINZ NIXDORF INSTITUT
Process
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Generic degrees of freedom
Initial candidate
1. Search problem instantiation
2. Evolutionary Optimisation
a
Evaluation of each candidate
Performance
Cost
others...
Set of candidates with QoS metrics
b
Selection: Choose candidates for next generation
Set of n best candidates
Set of candidates
System-specifc degrees of freedom
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Initial and
random
candidates
c
Reproduction: Generate new candidates
Crossover
Mutation
Tactics
QoS metrics
3. Present results
[MKB10]
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
121
Degrees of Freedom
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Software
Component selection
Middleware selection
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Component replication
Software configuration
Deployment
Allocation
Processing Rate
Number of Servers
[MKB10]
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
122
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Case Study
HEINZ NIXDORF INSTITUT
[MKB10]
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
123
HEINZ NIXDORF INSTITUT
Results
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
§  1235 candidates
§  58 Pareto optimal
§  8h running time
Response Time
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
" Component allocation
"   Processing rates
" Component selection
Cost
s
[MKB10]
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
124
HEINZ NIXDORF INSTITUT
Results
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
20
All candidates
Average Response Time
(Seconds)
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
18
Pareto-optimal candidates
16
Initial Candidate
14
12
10
RT: 1.34 s
POFOD: 5.2E-4
8 Cost: 69.83
Only four, but faster
6 servers
Different
Webserver
4
RT: 2.2 s
POFOD: 6E-4
Cost: 98
2
0
0
20
40
60
80
100
Costs (K$)
120
140
160
[MKB10]
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
125
Other Approaches
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
ArcheOpteryx
•  E.g., Allocation of Embedded
Components
•  Reliability and Cost
Deep Compass
•  Robocop Component
•  Schedulability and costs
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
126
HEINZ NIXDORF INSTITUT
Outline
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Motivation
Conclusions
& Outlook
Process
Trade-off
Analysis
Input Models
& Annotations
Model
Completions
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
127
Lessons Learned
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Lessons Learned
•  Motivation for Model-Driven QoS analysis
•  Process for model-driven analysis
•  Transformations & Refinements
•  Trade-off Analysis
Outlook
•  New input models and system types
•  Mechatronic systems
•  Self-adaptive systems
•  Automated selection of right abstraction level
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
128
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Questions, Comments, Feedback?
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
Palladio
Component Model
Jun.-Prof. Dr.-Ing. Steffen Becker
Heinz-Nixdorf-Institute, University of Paderborn, s-lab
Zukunftsmeile 1
33102 Paderborn, Germany
[email protected]
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
129
© Fachgebiet Softwaretechnik, Heinz Nixdorf Institut, Universität Paderborn
Further Reading
HEINZ NIXDORF INSTITUT
Universität Paderborn
Softwaretechnik
Jun.-Prof. Dr.-Ing. Steffen Becker
§  Model-Based Software Performance Analysis, Vittorio
Cortellessa, Antinisca Di Marco, Paola Inverardi
§  Performance Solutions: A Practical Guide to Creating
Responsive, Scalable Software, Connie Smith, Lloyd G.
Williams
§  The Art of Computer Systems Performance Analysis:
Techniques for Experimental Design, Measurement,
Simulation, and Modelling: Techniques for Experimental
Design, Measurement, Simulation and Modelling, Raj Jain
§  Queuing Networks and Markov Chains: Modeling and
Performance Evaluation with Computer Science
Applications, Gunter Bolch, Stefan Greiner, Hermann de
Meer
Model Transformations in Non-Functional Analysis, S. Becker
22.06.12
130