Linux software distribution

Standard software is installed via rpm packages; the software we package ourselves is either packaged as rpm packages in the uio software repo, or installed via this  system.

The system has three parts:

The build machine

Software packages are compiled on a machine with the desired OS/architecture. septimus.uio.no is the build machine for RHEL7/x86_64. If we wish to build for other architectures, we need separate build machines.

Build machines have the directories

/opt/uio/modules/packages/<package name>
/opt/uio/modules/modulefiles/<package name>

/opt/uio/modules/arch-indep/<package name>
/opt/uio/modules/src/<package name>

/opt/uio/modules/packages/:

These are the program packages. To build software with autoconf, you will use "./configure --prefix=/opt/uio/modules/packages ...". We assume the target (client) has the GNU utilities and miscellaneous common linux binaries.

/opt/uio/modules/modulefiles/:

We distribute (optional) modulefiles, corresponding to the above packages. For a package "foo" (/opt/uio/modules/packages/foo), the modulefiles are found under /opt/uio/modules/modulefiles/(type)/foo/ , the type is typically a directory named "software", "library", etc.  A similar name structure are transfered to the client. The same module files are used for all architectures.

/opt/uio/modules/arch-indep/:

Packages independent of client architecture so available for all clients

/opt/uio/modules/src/:

Source code

The distribution machine

We use geo-master (currently RHEL6 although arch is unimportant). The above directories are NFS mounted and made available as 

/geo-modules/(OS-arch)/opt/uio/modules/<x>

Here, (OS-arch) is a string like RHEL7-x86_64, RHEL8-x86_64, Ubuntu14.04-x86_64, or something like that, in general composed from the output from "lsb_release -ir". As a special case, if (OS-arch) is "ARCH-INDEP", the contents are assumed to be architecture-independent, and available for all clients. The script geodist.pl, running on geo-master, syncs the desired software packages via rsync push to the clients.

 

Clients signal what they want by creating
the corresponding directory under /opt/uio/modules/packages,
this is then populated by a dist script running on geomaster.

The client

The client needs to have modules installed and configured properly:

1) yum install environment-modules

2) update the correct module path to /usr/share/Modules/init/.modulepath (rsync from septimus)

3) If you wish users to have default modulefiles loaded when they login, create /usr/share/Modules/etc/rc file where the list of modulefiles to be loaded is specified (like sverdrup). This step is not mandatory and if you skip it, no modulefiles will be loaded by default.

 

Clients signal what they want by creating the corresponding directory under /opt/uio/modules/packages . This should be writeable by any user. A directory, once present, will be populated via geodist.pl. At present geodist.pl runs once an hour, it is possible we should call it via some Cfengine. 

Files in the packages are generally owned by the maintainer. If a file ./SETUP in the package is executable, owned by root, and wx-able only by root; the dist script will run it (as root)  on the client.  This is intended for simple tasks, like setting  up symlinks outside of the package tree.  NB: This is only possible to achieve as root on Septimus, all scripts should be audited by hanspv. 

 


 

Tags: linux software distribution
Published Oct. 29, 2015 2:25 PM - Last modified Oct. 29, 2015 3:21 PM