Review resources access policy, procedures,
rules and challenges:
The Italian experience and future
challenges
Antonia Ghiselli
INFN-CNAF
Workshop on eInfrastructures (Internet and Grids)
The new foundation for knowledge-base Societies
Roma, Accademia Nazionale dei Lincei
9 December 2003
– n° 1
Outline

Introduction:


INFN-Grid and the national research grid




INFN resource sharing experience in the past
Goals and Results
Italian-Grid present status

Resource access mechanism and management tools

production service :Management, operations and support organization
International Grid scenario: LCG and EGEE

Challenges: Multi-grids for multi-VOs

Multi–grids :definitions and issues
Conclusions
– n° 2
INFN Computing Resource sharing in the past

80th
user
TRENTO
MILANO

RJE to INFN resources by INFN
users
LNL
PAVIA
TRIESTE
FERRARA
GENOVA
PARMA
BOLOGNA

user
UDINE
PADOVA
TORINO
Resource sharing within a single
distributed community
(agreement between sites based
on common convenience )
Network
CNAF
PISA
FIRENZE
S.Piero
user
PERUGIA
LNGS
ROMA2
ROMA
L’AQUILA
user
LNF

user
Access policy agreement:
SASSARI


NAPOLI
low priority queues during the
night
Proxy logins mechanism
BARI
SALERNO
LECCE
CAGLIARI
COSENZA
PALERMO
CATANIA
LNS
VAX/VMS cluster
– n° 3
INFN Computing Resource sharing in the past

90th : Condor – INFN
collaboration
user
TRENTO
MILANO
LNL
PAVIA

Condor submit to INFN desktops
and workstations
user
UDINE
PADOVA
TORINO
TRIESTE
FERRARA
GENOVA
PARMA
BOLOGNA
Condor
on WAN
CNAF
PISA


FIRENZE
S.Piero
Users Resource sharing by INFN
users
Access policy agreement:
transparent access through CPU
cycle stealing
user
PERUGIA
LNGS
ROMA2
ROMA
L’AQUILA
user
LNF
SASSARI
NAPOLI
BARI
SALERNO

user
LECCE
~300 machines, still up.
CAGLIARI
COSENZA
PALERMO
CATANIA
LNS
– n° 4
INFN Computing Resource sharing in the past

1999
user
TRENTO
MILANO

Globus evaluation on WAN
user
UDINE
PADOVA
TORINO
LNL
PAVIA
TRIESTE
FERRARA

Preliminary grid tests to the
INFN-Grid project.
GENOVA
user
PARMA
BOLOGNA
CNAF
Globus
test
PISA
FIRENZE
S.Piero
user
PERUGIA
LNGS
ROMA2
ROMA
L’AQUILA
user
LNF
SASSARI
NAPOLI
BARI
SALERNO
LECCE
CAGLIARI
COSENZA
PALERMO
CATANIA
LNS
– n° 5
INFN-Grid – goals (started at 2000)
1. To promote computational grid technologies research & development:
Middleware
1. Through european and international projects
1.
DataGrid, DataTAG, GLUE
2. Internal R&D activities
2. To implement the INFN grid infrastructure
1. National layout: 20 sites
3. To set up the national Grid Infrastructure for the national research
community
1. FIRB: Grid.it
4. To participate to the implementation of the global Grid infrastructure for
the LHC community
1. LCG: Tier1 and n*Tier2
5. To set up the eInfrastructure for the European Research Area
1. EU FP6: EGEE, IG-BIGEST
– n° 6
INFN-Grid – collaborations and results

EU - Datagrid : middleware development

WMS = job submission to the Grid,






Virtual Organization authentication and authorization service: VOMS (VO
Membership Service, EDG/EDT)
EU – DataTAG : inter-grid Interoperability; EU-US collaboration within the
GLUE framework


CE and SE selection on the basis of job requirements specification, CPU load, CE-SE network
conditions…..
Support for interactive jobs
Job checkpointing
Support for parallel jobs
Grid Resources Information modeling: GLUE schema for Computing and Storage
Element

Authorization/authentication service : VOMS-VOX integration (EDT-Fnal/CMS coll.)

First WorldGrid demo by nov.2002 within IST2002 and SC2002 events

Grid monitoring system based on GLUE schemas extension
Italian Grid.it : Grid management and support infrastructure

First tools in production

R&D on Resource Utilization Policies
– n° 7
Italian – Grid now
(Site/resource map)
INFN
TRENTO
MILANO
UDINE
PADOVA
TORINO
LNL
PAVIA
TRIESTE
National
Grid
(Internet)
FERRARA
GENOVA
PARMA
BOLOGNA
CNAF
PISA
FIRENZE
S.Piero
PERUGIA
LNGS
ROMA2
ROMA
L’AQUILA
LNF
SASSARI
NAPOLI
BARI
SALERNO
CAGLIARI
COSENZA
PALERMO
CATANIA
LECCE
CMS T2 T2/3
Atlas T2
T2/3
Alice T2 T2/3
LHCb T2 T2/3
Babar
VIRGO
T2 (50-80 nodes)
T3 (10-15 nodes)
T1 Cnaf (~200)
grid.it resources
INFN (15-25 nodes)
INAF (5-10 nodes)
INGV (NEC computers),
BIO (tbd)
general purpose resources (8-15
nodes)
LNS
Tot. ~ 600 nodes , next year ~ 1000
– n° 8
Resource access policies: Basic grid
Authorization, authentication mechanisms
Security characteristics:


Login via X.509 certificates from PKI/Certificate Authorities (CA)
Single sign-on.


Delegation.


The user is not required to repeat login procedures on the grid more than once.
Once a user has successfully identified himself with the Grid, it is possible for grid
services to act on the behalf of the user as if they were the user himself.
User-based trust relationship.

All trust mechanism have the user’s credential at their core.


Integrated with local systems.


If a user wants to access farms A and B, there should be no need for farms A and B to
trust each other.
The grid security mechanism does not supplant the local authorization
mechanism, but instead work on top of it.
New membership concept: user belongs to a Virtual Organization
– n° 9
User: CA, VO and Resource Providers

CAs: Policies

and procedures
 mutual thrust
CA’s
CERN

Certificates are issued by a set of well-defined Certification
Authorities (CAs).
Grant authorization at the VO level.

Each VO has its own VOMS server.

Contains (group / role / capabilities) triples for each member of the VO.
RP’s evaluate authorization granted by VO to a user and map into
local credentials to access resources
CESNET
CNRS
GermanGrid
Grid-Ireland
INFN
cert-request
Authentication
Request
NIKHEF
NorduGrid
LIP
cert signing
Russian DataGrid
VOMS
pseudo
-cert
US–DOE Root CA
agreement
(map into Local credential)
cert/crl update
C=IT/O=INFN VOMS
pseudo
/L=CNAF
-cert
/CN=Pinco Palla
/CN=proxy
US-DOE Sub CA
CrossGrid
(administer user
membership, roles and
Capabilities)
Resource provider
DATAGRID-ES
GridPP
VO-Manager
Service
– n° 10
Resource access policies


Authentication/ authorization: coded and tested procedures and tools
New issue : resource sharing according to Service Level Agreement


first trials based on “grid level priority queues”
ongoing research on more sophisticated mechanisms based on accounting +
resource utilization Policies management
VO-users
(Requirements
Support)
Resource providers /
AA/SLA
Grid release
VO-managers
(VOMS and SLA
Control)
Grid management
organization
Grid operations / support
Certificate Authorities
Grid deployment planning
– n° 11
Italian Grid organization : integrates all the actors to
provide flexible and efficient grid computing service
Experiments (VOs)
GRID resources
Projects/owners
Grid Resource
Coordination
Coordination Committee
Service level
Agreement
Resource availability
Shared resources
Management
coordination
VO representatives,
Grid technical coord.,
Operations resp.
grid experts
•Deployment Planning
• resource Policy
application
•…….
Grid Technical coordination
release
Configuration
management
Central
management
Team
Operations
coordination
GridService
support
Experimemt or
research org.
support
VO User
Support for
support
New VO-users
VO admin
Site-man
Resource admin
VO admin
New VO admin
& support
User
Application
Release distribution, documentation
and porting
– n° 12
Tools for Operations

Software repository : release maintenance and distribution

Installation and configuration:


Release validation:


Configuration and automatic installation tools for the production infrastructure
sites
Integration/customization of middleware release with application specific
software
GRID Site and GRID service validation

Testing programs to verify and validate site and services installation

Site manager support

Grid services, VO services support and User support

Monitoring: GridICE



Based on automatic resource discovery from Grid Information System
Dynamic monitoring of Grid services, Grid resources and Jobs
Customized view for
Grid Operation Center operators, and site managers
VO-managers and Grid Users


– n° 13
0perations Portal






User
documentation
site managers
documentation
Software
repository
Monitoring
Trouble tickets
system
Knowledge
base
http://grid-it.cnaf.infn.it
– n° 14
Get your personal certificate
– n° 15
How to register to a VO
– n° 16
Monitoring tool
– n° 17
Grid services
User
Interface
Grid Monitoring
(GridICE)
VO server
ingv
Resource Broker
BDII
Information Index
INFN-Padova
INGV-Bologna
GIIS
GRIS1
GRIS
RLS
VO server
VO server
atlas
atlas
GRAM
Computing
Element
GIIS
GRIS1
GRIS
Storage
Element
GRAM
Computing
Element
Storage
Element
WorkerNode
WorkerNode
WorkerNode
WorkerNode
WorkerNode
...
WorkerNode
...
– n° 18
Grid Service monitoring
– n° 19
Outline

Introduction:


INFN-Grid and the national research grid




INFN resource sharing experience in the past
Goals and Results
Italian-Grid present status

Resource access mechanism and management tools

production service :Management, operations and support organization
International Grid scenario: LCG and EGEE

Challenges: Multi-grids for multi-VOs

Multi-grids: definitions and issues
Conclusions
– n° 20
International Grids scenario
 LCG
: First international experience on sharing resources between
national grids

Grid Resource sharing issues :

how to guarantee the committed CPU power and satisfy local needs

How to guarantee priorities on VO-owned resources

Different needs for different VOs (HEP experiments plans)

Management coordination

Support coordination
 EGEE
: project based on national grids interconnection for an
increased number of VOs


Not only middleware but mainly policies, service level agreement and
management coordination issues
Need to find a model …..
– n° 21
Grid access challenge: Grid and
Virtual Organisations
The
real problem at the basis of the grid idea is how to
implement a coordinated resource sharing on a
large scale for a multi-institutional and dynamic
virtual organisation.
-
From
computer sharing to grid sharing
From
multiple users to multiple VOs (INFN experiments
+ others research organizations)
– n° 22
Challenges: Capability to provide multiGrid computing service to Multi-VO
General scenario
VO services
and private
resources
VO services
and private
resources
Shared Resources
and Services
VO services
Shared Resources
and Services
Shared Resources
and Services
VO services
and private
resources
VO services
– n° 23
VO-Virtual Grid on top of Multi-Grids


International VO is a multi-institutional distributed user
community
Etherogeneous grid environment

Dedicated VO services

Dedicated resources

Shared resources with different policies
VO-User
VO - Virtual Grid
VO-User
RB
RB
VO-User
VO-monitoring
VOMS
Vo-RLS
Coordinated Vo-support
National and International Grids
Italian-Grid
EGEE
same middleware
shared resources
US-Grid
same core services
– n° 24
multi - grids : definitions and issues
 National

grid identity and authority boundaries
A coordinated set of shared resources and services providing defined
SLAs.

A single management and operations organization

Specific authorization, accounting and monitoring tools

A collection of user communities (VOs)
 Federation

Cooperating grids to provide services to the common VOs?


Which level of transparency to VO-users?
Which Interoperability Requirements:




of grids, what does’t mean?
common core services?
common or interoperable collective services? (level of service
interoperability)
Common Resource sharing policies?
What level of management/operations/support coordinations?
– n° 25
Conclusions
 Production
also:

grid does not mean only efficient, stable services but
A topology/organizational model capable to provide the most flexible
and efficient computing service to VO-users across multiple grids

Sufficient level of service quality (SLA)

Operations and support coordination

the minimum level of interoperability in order to allow VO virtual grid
configuration across multiple grids
– n° 26