information - Agenda Catania

Grid User Interface
Giuliano Taffoni
Istituto Nazionale di Astrofisica
Information Technology Unit
INAF-SI
Marc-Elian Bégin - Demos - 1st EU reviewCorso Avanzato di Calcolo Parallelo e Grid Computing - 27 Sep Catania
Outline
• The User Interface
– Installation and Configuration
• UIPnP
– Glite3.0.0
– Combo
• The virtual UI
– Vmware solution
– QEMU
• UI and Apple
2
Marc-Elian Bégin - Demos - 1st EU reviewCorso Avanzato di Calcolo Parallelo e Grid Computing - 27 Sep Catania
What is a User Interface?
•WN is the real execution node
•WN is like a Slave
User access point to Grid
•Proxy credential
•Job submission
•Job Monitoring
•Command info site
CE
•Entry point of a queue in a
batch system
•CE is like a Master
3
Marc-Elian Bégin - Demos - 1st EU reviewCorso Avanzato di Calcolo Parallelo e Grid Computing - 27 Sep Catania
How to install a UI
•
•
•
•
A dedicated SL 306 Linux PC;
The UI in user space: UIPnP;
Egrid LiveCD;
The art of virtualization:
– Vmware
– Qemu
– Usermode linux
– XEN 3.0
4
Marc-Elian Bégin - Demos - 1st EU reviewCorso Avanzato di Calcolo Parallelo e Grid Computing - 27 Sep Catania
The problem!
• Grid MW is BSD license
• It is very difficult to find sources
• When you find, it is very difficult to find
dependencies
• They build a Grid is for dummies!
5
Marc-Elian Bégin - Demos - 1st EU reviewCorso Avanzato di Calcolo Parallelo e Grid Computing - 27 Sep Catania
Installation and Configuration
of the
User Interface
Standard method
6
Marc-Elian Bégin - Demos - 1st EU reviewCorso Avanzato di Calcolo Parallelo e Grid Computing - 27 Sep Catania
A UI workstation
• SL 30X
• Yam + yaim or however you like
• /opt/lcg/sbin/ig_install_node site.def
ig_UI
• NOTE:
7
– BDII_HOST=egee-bdii.cnaf.infn.it
– NTP_HOSTS_IP="pool.ntp.org"
– RB_HOST=”egee-rb-01.cnaf.infn.it”
– LFC_CENTRAL="lfcserver.cnaf.infn.it"
– PX_HOST=myproxy.cnaf.infn.it
Marc-Elian Bégin - Demos - 1st EU reviewCorso Avanzato di Calcolo Parallelo e Grid Computing - 27 Sep Catania
Installation process
• If the installation is performed successfully,
the following components are installed:
– gLite in /opt/glite
– Globus in /opt/globus
– GPT in /opt/gpt
8
Marc-Elian Bégin - Demos - 1st EU reviewCorso Avanzato di Calcolo Parallelo e Grid Computing - 27 Sep Catania
After the installation
• Check the DATE!!!
• Check the environment variables:
– GLOBUS_TCP_PORT_RANGE
– GLOBUS_LOCATION
– LCG_GFAL_INFOSYS
– LFC_HOST
– etc...
9
Marc-Elian Bégin - Demos - 1st EU reviewCorso Avanzato di Calcolo Parallelo e Grid Computing - 27 Sep Catania
Test the UI: prerequisites
• A UI does not need a host certificate;
• Any user of the UI needs his user certificate;
• Open the firewall:
– Globus tcp port range: 20000 25000
– Connection to RB, BDII, myproxy, LFC etc...
• DO NOT use a private network for the UI!!!!
10
Marc-Elian Bégin - Demos - 1st EU reviewCorso Avanzato di Calcolo Parallelo e Grid Computing - 27 Sep Catania
Test the UI
• As a user:
– Copy the usercert/key in
$HOME/.globus directory
– Initialize the proxy
– Test the connection:
11
•
•
•
•
•
Lcg-infosites –vo inaf ce
Lcg-infosites –vo inaf se
globusrun -a -r a.ce.fqhn
edg-gridftp-ls gsiftp://a.se.fqhn/tmp
lfc-ls /grid/inaf
Marc-Elian Bégin - Demos - 1st EU reviewCorso Avanzato di Calcolo Parallelo e Grid Computing - 27 Sep Catania
Some important configuration
files
• To point a new Resource Broker
check the attributes NSAddresses
and LBAddresses in the file
……
– /opt/edg/etc/VO/edg_wl_ui.conf
NSAddresses = "server3.gs.unina.it:7772";
– One file for each VO supported
LBAddresses = "server3.gs.unina.it:9000";
……
12
Marc-Elian Bégin - Demos - 1st EU reviewCorso Avanzato di Calcolo Parallelo e Grid Computing - 27 Sep Catania
Some important configuration
files
• To point at different BDII check the
variable LCG_GFAL_INFOSYS
…..
export LCG_GFAL_INFOSYS=server3.gs.unina.it:2170
….
The environment variables are set in /etc/profile.d
and configured during the installation
13
Marc-Elian Bégin - Demos - 1st EU reviewCorso Avanzato di Calcolo Parallelo e Grid Computing - 27 Sep Catania
The Default VO
• You can edit the following file to set the
default Virtual Organisation (VO):
>vi /opt/edg/etc/edg_wl_ui_cmd_var.conf
[
…..
LoggingDestination = "server3.gs.unina.it:9002";
# Default NS logger level is set to 0 (null)
# max value is 6 (very ugly)
NSLoggerLevel = 0;
DefaultLogInfoLevel = 0;
DefaultStatusLevel = 0;
DefaultVo = "inaf";
]
14
Marc-Elian Bégin - Demos - 1st EU reviewCorso Avanzato di Calcolo Parallelo e Grid Computing - 27 Sep Catania
Installation and Configuration
of the
User Interface
UIPnP
15
Marc-Elian Bégin - Demos - 1st EU reviewCorso Avanzato di Calcolo Parallelo e Grid Computing - 27 Sep Catania
The UIPnP
• It a tarball with all the UI middleware
components compiled;
• It is installed in user space;
• Need a Linux distribution;
• May be also a system installation (needs root
privileges)
16
Marc-Elian Bégin - Demos - 1st EU reviewCorso Avanzato di Calcolo Parallelo e Grid Computing - 27 Sep Catania
System requirements
• They claim it works with any
distribution: DO NOT BELIEVE
THEM!!!!
– Fedora core 2/3
– SL 30X/40X
– Suse 9.x (NOT 10)
– Mandriva
17
• Set properly time and date;
• Open the firewall;
• Install your certificate.
Marc-Elian Bégin - Demos - 1st EU reviewCorso Avanzato di Calcolo Parallelo e Grid Computing - 27 Sep Catania
Installation
• Download the SW:
– wget https://gridit.cnaf.infn.it/fileadmin/sysadm/UserInterface/UIPn
P-3.0.0.tar.gz
Untar in a directory
Enter the directory
Edit the configuration file UIPnP.conf
Export the variable UIPnP=install_directory
•
•
•
•
• Install it (source install.sh)
• More info in the README file
18
Marc-Elian Bégin - Demos - 1st EU reviewCorso Avanzato di Calcolo Parallelo e Grid Computing - 27 Sep Catania
And what happens...
• Install.sh does mainly:
– Downloads CA certificates, Signing Policy,
CRL
– Creates .globus directory
– Creates JobOutput directory
– Set all the relevant environment variables
– Set a cron job to update CRL periodically
19
Marc-Elian Bégin - Demos - 1st EU reviewCorso Avanzato di Calcolo Parallelo e Grid Computing - 27 Sep Catania
RB Environment Settings
To point at LCG Resource Broker
server3.gs.unina.it
from your UIPnPcomb, uncomment this value
for the
attribute NSAddresses and LBAddresses in the
following file
……
NSAddresses
= "server3.gs.unina.it:7772";
>vi
UIPnP/edg/etc/gilda/edg_wl_ui.conf
LBAddresses = "server3.gs.unina.it:9000";
……
20
Marc-Elian Bégin - Demos - 1st EU reviewCorso Avanzato di Calcolo Parallelo e Grid Computing - 27 Sep Catania
BDII Environment Settings
To point at BDII (Berkeley Database Information
Index)
from your UIPnP, edit server3.gs.unina.it:2170
as
value for variable LCG_GFAL_INFOSYS:
>vi UIPnP/Environment.sh
…..
export LCG_GFAL_INFOSYS=server3.gs.unina.it:2170
….
21
Marc-Elian Bégin - Demos - 1st EU reviewCorso Avanzato di Calcolo Parallelo e Grid Computing - 27 Sep Catania
Default VO settings
You can edit the following file to set the default Virtual
Organisation (VO):
>vi UIPnPcomb/edg/etc/edg_wl_ui_cmd_var.conf
[
…..
LoggingDestination = "server3.gs.unina.it:9002";
# Default NS logger level is set to 0 (null)
# max value is 6 (very ugly)
NSLoggerLevel = 0;
DefaultLogInfoLevel = 0;
DefaultStatusLevel = 0;
DefaultVo = "inaf";
]
22
Marc-Elian Bégin - Demos - 1st EU reviewCorso Avanzato di Calcolo Parallelo e Grid Computing - 27 Sep Catania
Installation and Configuration
of the
User Interface
Virtual
Servers
Virtual Servers
23
Marc-Elian Bégin - Demos - 1st EU reviewCorso Avanzato di Calcolo Parallelo e Grid Computing - 27 Sep Catania
The art of virtualization
•
•
•
•
•
24
Ops...but I use WindowsXP!
My distribution does not support UIPnP;
I do not have money to buy a new PC;
I have a MAC
I just want to play with the GRID...
Marc-Elian Bégin - Demos - 1st EU reviewCorso Avanzato di Calcolo Parallelo e Grid Computing - 27 Sep Catania
Virtual Machines
• XEN
– Pretty invasive but very useful
– http://www.cl.cam.ac.uk/Research/SRG
/netos/xen/
• UserModeLinux
– Fast but still linux
• QEMU
– Fast, multiplatform, FREE
• VMWARE
25
– Fast, Win+Linux, now free!
Marc-Elian Bégin - Demos - 1st EU reviewCorso Avanzato di Calcolo Parallelo e Grid Computing - 27 Sep Catania
Qemu
• QEMU is a generic and open source
processor emulator which achieves a
good emulation speed by using dynamic
translation.
• http://fabrice.bellard.free.fr/qemu
• Accelerator for linux/OpenBSD/WIN(?)
– 2 times slower
• Mac version
– http://kberg.ch/q/
– 10 times slower
26
Marc-Elian Bégin - Demos - 1st EU reviewCorso Avanzato di Calcolo Parallelo e Grid Computing - 27 Sep Catania
Vmware Player
• Free & Fast
• Needs a vmware image
– https://gilda.ct.infn.it/VirtualServices.html
• We are planning to provide an INAF
Vmware UI for INAF users (15days)
27
Marc-Elian Bégin - Demos - 1st EU reviewCorso Avanzato di Calcolo Parallelo e Grid Computing - 27 Sep Catania
Live UI
•
•
•
•
•
Knoppix live CD
UI 2.7.0
Developed by EGRID project
Very useful
Can be used to install a UI but I do
not suggest it
• There is a gilda version
– https://gilda.ct.infn.it/live-cd/
28
Marc-Elian Bégin - Demos - 1st EU reviewCorso Avanzato di Calcolo Parallelo e Grid Computing - 27 Sep Catania
THE
END
29