CRS4
A General Overview
Carlino Casari
[email protected]
ICHNOS PRESENTATION - 12.12.2008
1
CRS4
CRS4:
•Center for Advanced Studies, Research & Development
in Sardinia
•Createt in 1991 - Active since 1992
•About 150 researchers
•Annual budget: close to 9 M€
ICHNOS PRESENTATION - 12.12.2008
2
Mission Statement
CRS4 Mission:
•Start-up & Management of a Research & Development
knowledge center on advanced technologies:
• Trigger the development of the Information Society
in Sardinia (Diesis, ICT district…)
• Support to the innovation instances of the
Autonomous Sardinia Region
(CSR, SardIT….)
• Support to the start-up of new Hi-Tech enterprises
(VoL, US online, Tiscali….)
ICHNOS PRESENTATION - 12.12.2008
3
Funding
CRS4 Funding:
• EU Projects
• National Research Department Projects
• Regional Projects
• Industry Projects
ICHNOS PRESENTATION - 12.12.2008
4
Strategic Model
Spin-off
Start-up
Research
& Development
Spin-off
Start-up
“Service”
INDUSTRY
activities for
Autonomous
Sardinia Region
Spin-off
Start-up
ICHNOS PRESENTATION - 12.12.2008
5
STRATEGIC Programs
CRS4 Strategic Programs:
• ICT
• Energy & Environment
• Biotech
ICHNOS PRESENTATION - 12.12.2008
6
STRATEGIC Program
Further areas of interest:
• Extension of the technology transfer capacities to
a wider area respect to the local region
• International cooperation
ICHNOS PRESENTATION - 12.12.2008
7
Facilities
CRS4 Computing & Networking facilities:
• High Performance Computing Cluster (40 Tflops)
• High speed internal network
• High speed Regional Research Network (up to 80 Gb/s)
• High speed connection to the National Research Network
(10 Gb/s)
ICHNOS PRESENTATION - 12.12.2008
8
Network Infrastructure (Cybersar project)
INFN
AOB
RTR
Janna dark fibers
ICHNOS PRESENTATION - 12.12.2008
Network Infrastructure (Cybersar Project)
INAF–SRT
San Basilio
INAF
Capoterra
8xGbE
2x10GbE
5x10GbE punto-punto
CRS4
Pula
UNICA
Ingegneria
8xGbE
2x10GbE
5x10GbE punto-punto
8xGbE
2x10GbE
5x10GbE punto-punto
8xGbE
2x10GbE
5x10GbE punto-punto
Ttiscali
Sa Illetta
8xGbE
2x10GbE
5x10GbE punto-punto
10x10GbE
Remote
Supervision
ICHNOS PRESENTATION - 12.12.2008
40x1GbE
UNICA
INFN
Monserrato
New Computing Center Infrastructure
SAN
SWITCH
Server
node
Server
node
Server
node
Server
node
CLUSTER
RACK
Infiniband
Interconnect
Big-HPC1
10 GbE Link
Big-HPC2
CLUSTER
RACK with
Infiniband Nodes
1GbE Link
2.5Gb FC Link
Other HPC
VLANs
ICHNOS PRESENTATION - 12.12.2008
USER and
SERVICES
VLANs
HPC cluster Network
CLUSTER
RACK
CLUSTER
RACK IB
Nuovi apparati
Rete HPC Preesistente
• 1 Foundry BIRX16
–
•
Big-HPC1
1 Foundry BIRX8
–
–
•
BigHPC2
USER and
SERVICES
VLANs
HPC
network
24 porte 10GE
8 porte 10GE
48 porte 10/100/1000BaseT
10 switch accesso Force10, Extreme, foundry
–
–
20 porte 10GE
480 porte 10/100/1000BaseT
Nuovi apparati HPC
 2 Foundry BIRX16
 120 porte 10GE wire speed
 48 porte 10GE oversubscription
Caratteristiche tecniche per sistema
 Switching capacity
 data capacity disponibile 1.60 Tbps
 switch capacity totale 3.84 Tbps
 Packet forwarding capacity 1,142 Mpps
Gli enclosure dei cluster HP utilizzano dei Virtual connect con 10GE verso gli switch HPC
Foundry
ICHNOS PRESENTATION - 12.12.2008
HPC Cluster Configuration
CLUSTER GbE
RACK
CLUSTER IB
RACK
Infiniband
Interconnect
Big-HPC1
Big-HPC2
Cluster a bassa Latenza
 256 HP BladeBL460c con 2CPU QuadCore 5440 2.83 GHz, 2 HD SATA 250GB, 2 schede Gigabit Ethernet, 1
scheda Infiniband 4xDDr (16 enclosure totali)
 Switch infiniband Voltaire con 288 porte
 16 switch infiniband Mellanox, uno per enclosure
 64 porte 10GE dai Virtual connect verso l’infrastruttura di rete
 5 rack raffredati ad acqua
Cluster a media latenza
 128 HP BladeBL460c con 2CPU QuadCore 5440 2.83 GHz, 2 HD SATA 250GB, 2
 schede Gigabit Ethernet, 1 scheda Infiniband 4xDDr (8 enclosure totali)
 32 porte 10GE dai Virtual connect verso l’infrastruttura di rete
 2 rack raffredati ad acqua
Nodi di servizio
 16 HP BladeBL480 con 2CPU QuadCore 5440 2.83 GHz, 4 HD SAS a 146GB, 2 schede Gigabit Ethernet, 1
scheda Infiniband 4xDDr, 1 scheda FC Dualport 4Gbps (3 enclosure)
Potenza di calcolo totale 34,6 TFlops nei nuovi sistemi, circa 40 TFlops al CRS4
ICHNOS PRESENTATION - 12.12.2008
Storage system (800TB)
SAN
SWITCH
Server
node
Server
node
Server
node
Server
node
SISTEMA SAN Sun StorageTek 9990V equipaggiato
Capacità totale di spazio disco pari a 557TB di capacità RAW
Capacità in configurazione RAID 5 di 480TB
462 HDD FC da 400GB 10.000 rpm con connettività a 4Gb/s pari ad una capacità RAW di 180TB
373 HDD SATAII da 1TB 7.200 rpm pari ad una capacità RAW di 373TB
Capacità di throughput sostenuto SPC-1 nei dischi più veloci fino ad un max di 3.5Milioni di
IOPS (valore di picco) con blocchi di dati e capacità disco opportunamente dimensionati
Software di gestione per ST-9990V e director CISCO 9509
2 SAN Director CISCO 9509 da 72 porte ciascuno
ICHNOS PRESENTATION - 12.12.2008
Network Overall Picture

10 Gigabit Ethernet LAN Backbone
Internet

Sa Illetta (Cagliari)
10 Gigabit Ethernet Campus Backbone
2 Mbit/s

GARR
xWDM MAN Link to Cagliari
100 Mbit/s
Tiscali
CWDM
Mux/Demux
2 Mbit/s
router
G.652 fibers
(46 km)
public servers
front-end subnet (GARR)
front-end subnet (Tiscali)
nx1Gbps
WDM
firewall
Sardegna
Ricerche
DMZ CRS4
CRS4 internal subnets ( VLANs)
switch L2/L3
100M/1G/10G Ethernet
HPC
Vlans
CRS4
users
ICHNOS PRESENTATION - 12.12.2008
intranet servers
Lustre
storage
cluster
GPFS
NFS
storage storage servers
cluster
computing
clusters
CWDM
Mux/Demux