ࡱ> 3B{x7 {VbjbjUU 7|7|P3lT"T"T"T"%%%4%hz~% >l ,,,2 ZM O O O O O O $l s % " @s JuT"T",, JuJuJu1T",$,M JuM Ju JuN%Y!,$%@],2 `%BQ"Ey 0 = z t*zyJu%%T"T"T"T" DataGrid WP1 - WMS Software Administrator and User Guide (PM9 Release)  Document identifier:DataGrid-01-TEN-0118-0_1Date:  SAVEDATE \@ "dd/MM/yyyy" \* MERGEFORMAT 14/01/2002Work package:WP1Partner:Datamat SpADocument statusDRAFTDeliverable identifier: Abstract: This note provides the administrator and user guide for the WP1 WMS software delivered for PM9 release. Delivery SlipNamePartnerDateSignatureFromFabrizio PaciniDatamat SpA14/01/2002 Verified byStefano BecoDatamat SpA14/01/2002 Approved by  Document LogIssueDateCommentAuthor0_021/12/2001First draftFabrizio Pacini0_114/01/2002DraftFabrizio Pacini Document Change RecordIssueItemReason for Change0_1General updateTake into account changes in the rpm generation procedure. Add missing info about daemons (RB/JSS/CondorG) starting accounts Some general corrections FilesSoftware ProductsUser filesWord 97 FILENAME \* MERGEFORMAT DataGrid-01-TEN-0118-0_1-Document Acrobat Exchange 4.0DataGrid-01-TEN-0118-0_1-Document.pdf Content  TOC \o "1-3" 1. Introduction  PAGEREF _Toc535395863 \h 5 1.1. Objectives of this document  PAGEREF _Toc535395864 \h 5 1.2. Application area  PAGEREF _Toc535395865 \h 5 1.3. Applicable documents and reference documents  PAGEREF _Toc535395866 \h 5 1.4. Document evolution procedure  PAGEREF _Toc535395867 \h 6 1.5. Terminology  PAGEREF _Toc535395868 \h 6 2. Executive summary  PAGEREF _Toc535395869 \h 8 3. Build Procedure  PAGEREF _Toc535395870 \h 9 3.1. Required Software  PAGEREF _Toc535395871 \h 9 3.2. Build Instructions  PAGEREF _Toc535395872 \h 10 3.2.1. Environment Variables  PAGEREF _Toc535395873 \h 10 3.2.2. Compiling the code  PAGEREF _Toc535395874 \h 12 3.3. RPM Installation  PAGEREF _Toc535395875 \h 18 4. Installation and Configuration  PAGEREF _Toc535395876 \h 20 4.1. Logging and Bookkeeping services  PAGEREF _Toc535395877 \h 20 4.1.1. Required software  PAGEREF _Toc535395878 \h 20 4.1.2. RPM installation  PAGEREF _Toc535395879 \h 21 4.1.3. The installation tree structure  PAGEREF _Toc535395880 \h 22 4.1.4. Configuration  PAGEREF _Toc535395881 \h 23 4.1.5. Environment Variables  PAGEREF _Toc535395882 \h 23 4.2. RB and JSS  PAGEREF _Toc535395883 \h 25 4.2.1. Required software  PAGEREF _Toc535395884 \h 25 4.2.2. RPM installation  PAGEREF _Toc535395885 \h 27 4.2.3. The Installation Tree structure  PAGEREF _Toc535395886 \h 27 4.2.4. Configuration  PAGEREF _Toc535395887 \h 28 4.2.5. Environment variables  PAGEREF _Toc535395888 \h 32 4.3. Information Index  PAGEREF _Toc535395889 \h 34 4.3.1. Required software  PAGEREF _Toc535395890 \h 34 4.3.2. RPM installation  PAGEREF _Toc535395891 \h 34 4.3.3. The Installation tree structure  PAGEREF _Toc535395892 \h 35 4.3.4. Configuration  PAGEREF _Toc535395893 \h 35 4.3.5. Environment Variables  PAGEREF _Toc535395894 \h 36 4.4. User Interface  PAGEREF _Toc535395895 \h 37 4.4.1. Required software  PAGEREF _Toc535395896 \h 37 4.4.2. RPM installation  PAGEREF _Toc535395897 \h 38 4.4.3. The tree structure  PAGEREF _Toc535395898 \h 39 4.4.4. Configuration  PAGEREF _Toc535395899 \h 40 4.4.5. Environment variables  PAGEREF _Toc535395900 \h 41 5. Operating the System  PAGEREF _Toc535395901 \h 43 5.1. LB local-logger  PAGEREF _Toc535395902 \h 43 5.1.1. Starting and stopping daemons  PAGEREF _Toc535395903 \h 43 5.1.2. Troubleshooting  PAGEREF _Toc535395904 \h 44 5.2. LB Server  PAGEREF _Toc535395905 \h 45 5.2.1. Starting and stopping daemons  PAGEREF _Toc535395906 \h 45 5.2.2. Purging the LB database  PAGEREF _Toc535395907 \h 45 5.2.3. Troubleshooting  PAGEREF _Toc535395908 \h 46 5.3. RB and JSS  PAGEREF _Toc535395909 \h 47 5.3.1. Startig PostgreSQL  PAGEREF _Toc535395910 \h 47 5.3.2. Starting Condor-G  PAGEREF _Toc535395911 \h 47 5.3.3. Starting and stopping RB daemons  PAGEREF _Toc535395912 \h 47 5.3.4. Starting and stopping JSS daemons  PAGEREF _Toc535395913 \h 48 5.3.5. RB troubleshooting  PAGEREF _Toc535395914 \h 49 5.3.6. JSS troubleshooting  PAGEREF _Toc535395915 \h 49 5.4. Information Index  PAGEREF _Toc535395916 \h 49 5.4.1. Starting and stopping daemons  PAGEREF _Toc535395917 \h 49 6. User Guide  PAGEREF _Toc535395918 \h 50 6.1. User interface  PAGEREF _Toc535395919 \h 50 6.1.1. Security  PAGEREF _Toc535395920 \h 50 6.1.2. Common behaviours  PAGEREF _Toc535395921 \h 51 6.1.3. Commands description  PAGEREF _Toc535395922 \h 55 7. Annexes  PAGEREF _Toc535395923 \h 87 7.1. JDL Attributes  PAGEREF _Toc535395924 \h 87 7.2. Job Status Diagram  PAGEREF _Toc535395925 \h 87 7.3. Job Event Types  PAGEREF _Toc535395926 \h 89 7.4. wildcard patterns  PAGEREF _Toc535395927 \h 91 7.5. The Match Making Algorithm  PAGEREF _Toc535395928 \h 93 7.5.1. Direct Job Submission  PAGEREF _Toc535395929 \h 93 7.5.2. Job submission without data-accesss requirements  PAGEREF _Toc535395930 \h 93 7.5.3. Job submission with data-access requirements  PAGEREF _Toc535395931 \h 95 Introduction This document provides a guide to the building, installation and usage of the WP1 WMS software released for PM9. Objectives of this document Goal of this document is to describe the complete process by which the WP1 WMS software can be installed and configured on the DataGrid test-bed platforms. Guidelines for operating the whole system and accessing provided functionalities are also provided. Application area Administrators can use this document as a basis for installing, configuring and operating WP1 WMS software released for PM9. Users can refer to the User Guide chapter for accessing provided services through the User Interface. Applicable documents and reference documents Applicable documents [A1] Job Description Language HowTo DataGrid-01-TEN-0102-02-Document.pdf 17/12/2001 (http://www.infn.it/workload-grid/docs/DataGrid-01-TEN-0102-02-Document.pdf)[A2] DATAGRID WP1 Job Submission User Interface for PM9 (revised presentation) 23/03/2001 (http://www.infn.it/workload-grid/docs/20010320-JS-UI-datamat.pdf)[A3]WP1 meeting - CESNET presentation in Milan 20-21/03/2001 (http://www.infn.it/workload-grid/docs/20010320-L_B-matyska.pdf)[A4]Logging and Bookkeeping Service 0705/2001 (http://www.infn.it/workload-grid/docs/20010508-lb_draft-ruda.pdf)[A5]Results of Meeting on Workload Manager Components Interaction 09/05/2001 (http://www.infn.it/workload-grid/docs/20010508-WM-Interactions-pacini.pdf)[A6]Resource Broker Architecture and APIs 13/06/2001 (http://www.infn.it/workload-grid/docs/20010613-RBArch-2.doc)[A7]JDL Attributes - DataGrid-01-NOT-0101-0_4 17/12/2001 (http://www.infn.it/workload-grid/docs/DataGrid-01-NOT-0101-0_2.pdf) Reference documents [R1] Document evolution procedure The content of this document will be subjected to modification according to the following events: Comments received from Datagrid project members, Changes/evolutions/additions to the WMS components. Terminology Definitions CondorCondor is a High Throughput Computing (HTC) environment that can manage very large collections of distributively owned workstationsGlobusThe Globus Toolkit is a set of software tools and libraries aimed at the building of computational grids and grid-based applications. Glossary class-adClassified advertisementCEComputing ElementDBData BaseFQDNFully Qualified Domain NameGDMPGrid Data Management Pilot ProjectGISGrid Information Service, aka MDSGSIGrid Security Infrastructurejob-adClass-ad describing a jobJDLJob Description LanguageJSSJob Submission ServiceLBLogging and Bookkeeping ServiceLRMSLocal Resource Management SystemMDSMetacomputing Directory Service, aka GISMPIMessage Passing InterfacePIDProcess IdentifierPMProject MonthRBResource BrokerRCReplica CatalogueSEStorage ElementSI00Spec Int 2000SMPSymmetric Multi ProcessorTBCTo Be ConfirmedTBDTo Be DefinedUIUser InterfaceUIDUser IdentifierWMSWorkload Management SystemWPWork PackageExecutive summary This document comprises the following main sections: Section 3: Build Procedure Outlines the software required to build the system and the actual process for building it and generating rpms for the WMS components; a step-by-step guide is included. Section 4: Installation and Configuration Describes changes that need to be made to the environment and the steps to be performed for installing the WMS software on the test-bed target platforms. The resulting installation tree structure is detailed for each system component. Section 5: Operating the System Provides actual procedures for starting/stopping WMS components processes and utilities. Section 6: User Guide Describes in a Unix man pages style all User Interface component commands allowing the user to access WMS provided services. Section 7: Annexes Deepens arguments introduced in the User Guide section that are considered useful for the user to better understand system behaviour. Build Procedure In the following section we give detailed instructions for the installation of the WP1 WMS software package. We provide a source code distribution as well as a binary distribution and explain installation procedures for both cases. Required Software The WP1 software runs and has been tested on platforms running Globus Toolkit 2.0 Beta Release 21 on top of Linux RedHat 6.2. Hereafter are listed the software packages, apart from WP1 software version 1.0, that are required to be installed locally on a given site in order to be able to build the WP1 WMS on it. They are: Globus Toolkit 2.0 Beta 21 or higher (download at  HYPERLINK "http://datagrid.in2p3.fr/distribution/globus/alpha-15" http://datagrid.in2p3.fr/distribution/globus/beta-21) Python 2.1.1 (download at  HYPERLINK "http://datagrid.in2p3.fr/distribution/config/external.html" http://datagrid.in2p3.fr/distribution/config/external.html) Swig 1.3.7 (download at  HYPERLINK "http://datagrid.in2p3.fr/distribution/config/external.html" http://datagrid.in2p3.fr/distribution/config/external.html) Expat 1.95.1 (download at  HYPERLINK "http://datagrid.in2p3.fr/distribution/config/external.html" http://datagrid.in2p3.fr/distribution/config/external.html) MySQL Version 9.38 Distribution 3.22.32, for pc-linux-gnu (i686) (download at  HYPERLINK "http://datagrid.in2p3.fr/distribution/config/external_services.html" http://datagrid.in2p3.fr/distribution/config/external_services.html) Postgresql 7.1.3 ( HYPERLINK "http://datagrid.in2p3.fr/distribution/config/external_services.html" http://datagrid.in2p3.fr/distribution/config/external_services.html) Classads library CondorG 6.3.1 for INTEL-LINUX-GLIBC21 Perl IO Stty 0.02, Perl IO Tty 0.04 (download at  HYPERLINK "http://datagrid.in2p3.fr/distribution/config/external.html" http://datagrid.in2p3.fr/distribution/config/external.html ) Perl 5 (download at  HYPERLINK "http://datagrid.in2p3.fr/distribution/config/external.html" http://datagrid.in2p3.fr/distribution/config/external.html) gcc and c++ compilers egcs-2.91.66 or egcs-2.95.2 (mandatory for CondorG) GNU make version 3.78.1 or higher GNU autoconf version 2.13 GNU libtool 1.3.5 GNU automake 1.4 GNU m4 1.4 or higher RPM 3.0.5 sendmail 8.11.6 Build Instructions The following instructions deal with the building of the WMS software and hence apply to the source code distribution. Environment Variables Before starting the compilation, some environment variables related to the WMS components can be set or configured by means of the configure script. This is needed only if package defaults are not suitable. Involved variables are listed below: GLOBUS_LOCATION base directory of the Globus installation The default path is /opt/globus. MYSQL_INSTALL_PATH base directory of the MySQL installation The default path is /usr. EXPAT_INSTALL_PATH base directory of the Expat installation. The default path is /usr. GDMP_INSTALL_PATH base directory of the Gdmp installation The default path is /opt/edg. PGSQL_INSTALL_PATH base directory of the Pgsql installation. The default path is /usr. CLASSAD_INSTALL_PATH base directory of the Classad library installation. The default path is /opt/classads. CONDORG_INSTALL_PATH base directory of the Condor installation. The default path is /opt/CondorG. PYTHON_INSTALL_PATH base directory of the Python installation. The default path is /usr. SWIG_INSTALL_PATH base directory of the Swig installation . The default path is /usr/local. In order to build the whole WP1 package, all the environment variables in the previous list must be set. Instead for building the User Interface module, the environment variables that need to be set are the following: GLOBUS_LOCATION CLASSAD_INSTALL_PATH PYTHON_INSTALL_PATH SWIG_INSTALL_PATH EXPAT_INSTALL_PATH If you plan to build the Job Submission and Resource Broker module, variable to set are: GLOBUS_LOCATION MYSQL_INSTALL_PATH EXPAT_INSTALL_PATH GDMP_INSTALL_PATH PGSQL_INSTALL_PATH CLASSAD_INSTALL_PATH CONDORG_INSTALL_PATH Whilst the LB server and Local Logger modules, to be built need the following environment variables: GLOBUS_LOCATION MYSQL_INSTALL_PATH EXPAT_INSTALL_PATH Finally, the LB library module needs: GLOBUS_LOCATION EXPAT_INSTALL_PATH and the Information Index module only: GLOBUS_LOCATION Compiling the code After having unpacked the WP1 source distribution tar file, or having downloaded the code directly from the CVS repository, change your working directory to be the WP1 base directory, i.e. the Workload directory, and run the following command: ./recoursive-bootstrap At this point the configure command can be run. The configure script has to be invoked as follows: ./configure The list of options that are recognized by configure is reported hereafter: --help --prefix= It is used to specify the Workload installation dir. The default installation dir is /opt/edg. --enable-all It is used to enable the build of the whole WP1 package. By default this option is turned on. --enable-userinterface It is used to enable the build of the User Interface module with Logging/Client, Broker/Client, Broker/Socket++ and ThirdParty/trio/src submodules. By default this option is turned off. --enable-jss_rb It is used to enable the build of the Job Submission and Resource Broker modules with Logging/Client, Common, test, and ThirdParty/trio/src submodules. By default this option is turned off. --enable-lbserver It is used to enable the build of the LB Server service with Logging/Client, Logging/etc, Logging/Server, Logging/InterLogger/Net, Logging/InterLogger/SSL, Logging/InterLogger/Error, Logging/InterLogger/Lbserver and ThirdParty/trio/src submodules. By default this option is turned off. --enable-locallogger It is used to enable the build of the LB Local Logger service with Logging/Client, Logging/InterLogger/Net, Logging/InterLogger/SSL, Logging/InterLogger/Error, Logging/InterLogger/InterLogger, Logging/LocalLogger, man and ThirdParty/trio/src submodules. By default this option is turned off. --enable-logging_dev It is used to enable the build of the LB Client Library with Logging/Client and ThirdParty/trio/src submodules. By default this option is turned off. --enable-information It is used to enable the build of the Information Index module.By default this option is turned off. --with-globus-install= It allows specifying the Globus installation directory without setting the environment variable GLOBUS_LOCATION. --with-pgsql-install= It allows specifying the Pgsql installation directory without setting the environment variable PGSQL_INSTALL_PATH. --with-gdmp-install= It allows specifying the GDMP installation directory without setting the environment variable GDMP_INSTALL_PATH. --with-expat-install= It allows specifying the Expat installation directory without setting the environment variable EXPAT_INSTALL_PATH. --with-mysql-install= It allows to specify the MySQL installation directory without setting the environment variable MYSQL_INSTALL_PATH. --with-expat=