The EPIKH Project
(Exchange Programme to advance e-Infrastructure Know-How)
GRID Science Gateway
Riccardo Rotondo ([email protected])
National Institute of Nuclear Physics
Africa 5 2011 – EPIKH Workshop
Stellenbosch, South Africa 10.06.2011
www.epikh.eu
Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011
1
Outline
•
•
•
•
Computationally intensive research
Grid access technologies
Science Gateway
The e-Collaboration environment
– The old paradigm
– Science Gateway architecture
– The new “brick” approach
• Use Cases:
– The CHAIN Science Gateway
– The SPECIAL Science Gateway
– The INDICATE e-Culture Science Gateway
• Conclusions
Riccardo Rotondo
Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011
2
Computationally intensive research
Riccardo Rotondo
Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011
3
Several scientific domain involved
Riccardo Rotondo
Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011
4
GRID at world scale
Riccardo Rotondo
Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011
5
But…using Grids is not straightforward 
Scripts
Type = "Job";
JobType = "MPICH";
MPIType = "MVAPICH2_PGI706";
CpuNumber = 16;
MPIGranularity = 4;
JDL
Executable = "flash2";
StdOutput = "mpi.out";
StdError = "mpi.err";
InputSandbox =
{"watchdog.sh","mpi.pre.sh","mpi.post.sh","flash.par","summers_den_
1e0.rates","flash2"};
OutputSandbox =
{"mpi.err","mpi.out","watchdog.out","flash_bubble.log","amr_log"};
Requirements = (other.GlueCEUniqueId == "unipa-ce01.pa.pi2s2.it:2119/jobmanager-lcglsf-hpc");
RetryCount = 0;
echo Staging Input Data \(Courtesy of European Space
Agency\);
#edg-rm --vo=gilda copyFile lfn:$1.N1 file://$PWD/$1.N1;
lcg-cp --vo=gilda lfn:$1.N1 file://$PWD/$1.N1;
echo Staging Application;
gunzip beam20.tar.gz;
tar xvf beam20.tar;
cd beam-2.0/bin;
echo Starting Application;
./pds2jpg-ASAR-run.sh $1;
mv $1-b*.jpg ../..
cd ../..
rm -fr beam-2.0;
rm -fr $PWD/$1.N1;
rm -fr $PWD/beam20.tar;
echo Input ENVISAT Product courtesy of European Space
Agency
touch
ENVISAT_Product_courtesy_of_European_Space_Agency
echo No Output Packaging;
echo Done!;
$ voms-proxy-info --all
subject : /C=IT/O=GILDA/OU=Robots/L=INFN
Catania/CN=MrBayes/CN=proxy/CN=proxy
issuer : /C=IT/O=GILDA/OU=Robots/L=INFN
Catania/CN=MrBayes/CN=proxy
identity : /C=IT/O=GILDA/OU=Robots/L=INFN
Catania/CN=MrBayes/CN=proxy
type
: proxy
strength : 1024 bits
path
: /tmp/x509up_u512
timeleft : 500:38:44
=== VO gilda extension information ===
VO
: gilda
subject : /C=IT/O=GILDA/OU=Robots/L=INFN Catania/CN=MrBayes
issuer : /C=IT/O=INFN/OU=Host/L=Catania/CN=voms.ct.infn.it
attribute : /gilda/Role=NULL/Capability=NULL
attribute : /gilda/generic-users/Role=NULL/Capability=NULL
attribute : /gilda/idl/Role=NULL/Capability=NULL
timeleft : 23:59:33
uri
: voms.ct.infn.it:15001
CLI
$ glite-wms-job-status https://gilda-lb-01.ct.infn.it:9000/1o4BVjqg2tJ4rNXTAIGAg
*************************************************************
BOOKKEEPING INFORMATION:
Status info for the Job : https://gilda-lb-01.ct.infn.it:9000/1o4BVjqg2tJ4rNXTAIGAg
Current Status: Done (Success)
Logged Reason(s):
- Job terminated successfully
Exit code:
0
Status Reason:
Job terminated successfully
Destination:
grid010.ct.infn.it:2119/jobmanager-lcgpbs-gilda
Submitted:
Tue Jun 29 15:34:40 2010 CEST
*************************************************************
GSI
Riccardo Rotondo
Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011
6
Grid Interface Evolution
• The way users access Grid resources has
continuously evolved towards simplicity and
transparency:
• Command Line
– Globus and gLite CLI
– Used by the enthusiastic and early adopter scientists
• GUI applications
– gEclipse, Grid2Win
– Good to expand the communities but difficult to maintain
• Web Interface
– GENIUS, P-GRADE
– Easier for new users but monolithic
• Science Gateways
Riccardo Rotondo
Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011
7
The GENIUS Grid Portal
“robot” certificates
on “e-tokens”
Riccardo Rotondo
Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011
8
Science Gateways
A framework of tools that allows scientists to run
applications with little concern for where the
computation actually takes place. This is similar to
cloud computing in which applications run as Web
services on remote resources in a manner that is not
visible to the end user. However, a science gateway
is usually more than a collection of applications.
Gateways often let users store, manage, catalogue,
and share large data collections or rapidly evolving
novel applications they cannot find anywhere else.
Training and education are also a significant part of
some Science Gateways
Riccardo Rotondo
Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011
9
Liferay (www.liferay.com)
• Highly-configurable, scalable, open source
portal framework;
• Compatible with JSR 168/286 standards
and based on modern web 2.0
technologies;
• Liferay services planned to be used:
– Portal;
– CMS & WCM;
– Collaboration and “social” software
10
Riccardo Rotondo
Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011
10
10
Grid Access enable: Portlets as bricks
• Portlets can interact with the Grid
e-Infrastructure
• Different approaches are available:
– Execute the Command Line behind the portal;
– Using API where available:
 Must be in Java or other languages supported by Liferay;
– Call REST services from Javascript code in the browser;
• Additional layers between Liferay and the Grid
can be necessary for some services;
• Each portlet can follow its own communication
method.
Riccardo Rotondo
Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011
11
11
GRID Science Gateway Architecture
2. Verify ACL
4. Grid
Credential
Exchange
1. Log In
3. Grid
Services
Request
GRID
5. Grid
Services
6. Results
7.
Download
data
Riccardo Rotondo
Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011
12
12
Robot Certificates in a nutshell
• Robot certificates have been introduced by several
Certification Authorities to allow users, who are not
familiar with personal certificates and do not belong
to any Vos, to access and use Grids.
– They are extremely useful, for instance, to automatize grid
service monitoring, data processing production, distributed data
collection systems, etc.;
– Basically, these certificates can be used to identify a person
responsible for an “unattended” service or process acting as
client and/or server.
Riccardo Rotondo
Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011
13
13
Robot certificates & e-Token
• In order to strong reduce the risks to have the portal
certificate compromised, the INFN CA has decided to
issue these special certificates on board of Aladdin
eToken PRO 32K smart cards.
• Each smart card can support several
(up to 32) robot certificates:
one for each application available
on the grid portal.
• Our Science Gateways fully supports robot
certificates !
Riccardo Rotondo
Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011
14
14
Liferay AT INFN-CT
Riccardo Rotondo
Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011
15
15
www.special-project.it
Riccardo Rotondo
Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011
16
16
The CHAIN Science Gateway
Riccardo Rotondo
Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011
17
17
The CHAIN Science Gateway
Riccardo Rotondo
Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011
18
18
The CHAIN Science Gateway
Riccardo Rotondo
Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011
19
19
The INDICATE e-Culture Science Gateway
(1/3)
http://www.indicate-project.eu
http://indicate-gw.consorzio-cometa.it
20
Riccardo Rotondo
Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011
20
20
The INDICATE e-Culture Science Gateway
(2/3)
Riccardo Rotondo
Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011
21
21
The INDICATE e-Culture Science Gateway
(3/3)
22
Riccardo Rotondo
Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011
22
22
Conclusions
• Conclusions
– Using different frameworks it is possible to build a Science
Gateway able to interact with Grid Services;
– Liferay is the most used framework and offers an easy,
customizable and user friendly interface;
– Thanks to robot certificates and X.509 standard the level of
security is kept high and the resources safe;
– The new “bricks” developed are being used in two projects
belonging from two distinct scientific domains.
• Future Work:
– While the role of Liferay as “front-end” is clear, the interactions
of the tools behind are still under development;
– Test the new Science Gateway in a production environment.
Riccardo Rotondo
Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011
23
23
Questions?
Thank you for your attention
Riccardo Rotondo
Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011
24
24
References
•
•
•
•
http://liferay.ct.infn.it
http://www.special-project.it
http://www.chain-project.it
http://indicate-gw.consorzio-cometa.it
Riccardo Rotondo
Stellenbosch, Africa 5 2011 - EPIKH Workshop, 10.06.2011
25
25