======================================================================

GUIDE FOR INSTALLING ``nsim``/``nmag`` all FROM SOURCES (big tar ball)
======================================================================

.. contents:: Table of Contents Installing Nmag all from big tarball

Installation requirements
-------------------------

Building ``nmag all`` will require approximatively 1.4 GB of harddisk space.

After compilation and issuing ``make clean``, ~550 MB will be
used. (The reason for this large footprint is that the ``nmag``
archive tries to make very few assumptions on pre-installed software
and hence has to bring along all the packages it depends on. This is
particularly relevant for computing clusters.)

Depending on hardware characteristics, compilation will usually take

between 15 minutes and 2 hours.

The following software packages have to be available on the build system:

- A sane build environment:

- The GNU compiler collection: ``gcc``, ``g++``, ...;

- ``m4`` (scripting language used by configure scripts that are generated by
``autoconf``);
- other GNU standard tools, such as ``make``, ``tar``, ``gzip``, ``bzip2``, ...;

- Libraries for fast linear algebra: ``blas``, ``lapack``;

- ``gawk`` (GNU awk);
- ``readline`` libraries and headers (for ``python``, ``ipython``, ``nsim_i``).
- zlib / libz (package zlib1g-dev on Ubuntu/Debian)

On Ubuntu 11.10 you can install these requirements using::

apt-get install g++ libblas-dev libreadline-dev \

make m4 gawk zlib1g-dev readline-common liblapack-dev

Download
--------

Download the tarball (with a name such as nmag-0.2.0.tar.gz) from

http://nmag.soton.ac.uk. (try `this directory
<http://nmag.soton.ac.uk/nmag/0.2/download/all>`__ to see available tar balls.

Installation
------------

Untar the archive::

$ tar xvzf nmag-0.2.0.tar.gz

At this point you should know that, once the package has been compiled,
you won't be able to move the nmag-0.2.0 directory to another location.
Doing so would make the package unusable, since absolute paths are hard-coded
inside the executables.
Therefore be sure to unpack the package into an appropriate directory.
Enter the directory just created by the tar command (which should be
named "nmag-0.2.0")::

$ cd nmag-0.2.0

and run ``make``::

$ make

This command will build ``nsim`` and a number of other required

packages.

All the software is installed inside the directory ``nmag-0.2.0``,

so root privileges are not required (see `Setting up the environment`_).

If all goes well, nsim can be started once the 'make' command has completed::

$ ./nsim/bin/nsim

If this produces a prompt::

>>>

then ``nsim`` was compiled successfully from source. (However, the libreadline-dev
libraries were missing, and thus the prompt is not an Interactive Python prompt.)

If::

$ ./nsim/bin/nsim

produces a prompt like::

In [1]:

then libreadline-dev has been found during the compilation, and the prompt
resembles the IPython prompt.

(The interpreter can be left, as usual, by pressing Control+d,

which will send an "End of File" command on its input stream.)

Setting up the environment

~~~~~~~~~~~~~~~~~~~~~~~~~~

In order to run ``nsim`` (and related programs such as ``ncol``,

``nmagpp``, etc.), the shell has to know the path where to find
these utilities. While it is possible to start ``nsim`` as
``./nsim/bin/nsim``, it is much more convenient in the long run
to adjust the shell's $PATH environment variable appropriately.

For the widespread ``bash`` shell, this can be done with these
commands::

$ cd nmag-0.2.0 # enter the main installation directory

$ export PATH=`pwd`/nsim/bin:$PATH
After these adjustments, ``nsim`` can be started with the command
``nsim`` without further path adjustments, regardless of the current
directory.

However, as this ``export`` command has to be issued within

very new shell, it may make sense to add it to the shell startup
command file ``.bashrc``.

Executing the command ``make hints`` in the ``nsim`` build directory

will give information on how to adjust ``.bashrc``::

$ cd nmag-0.2.0
$ make hints

will produce output similar to ::

To be able to launch nsim more easily, append the following line

to the end of your shell configuration file (for BASH users
this file is .bashrc and can be found on your home directory ~/.bashrc):
export PATH="/home/me/nmag-0.2.0/nsim/bin:$PATH"

Note that there are other shells which behave differently from bash,
such as in particular cshell or tcshell. For these in particular, the
command to set environment variables is ``setenv`` rather than
``export``.

Testing the software

--------------------

After successful completion, we recommend to run a suite of unit and

regression tests. If these pass (i.e. they do not fail), then
everything seems to work.

The test can be run by calling::

$ cd nmag-0.2.0
$ make check

This should take less than one minute to complete. If the output finishes with ::

======== tests finished: k passed, l skipped in XX.XX seconds ==========

(where k and l are integers), then all seems to be

well. Congratulations. While you can attempt to run the tests below,
this should not be necessary.

The l tests that have been skipped, take a bit more computation time (of the
order of 1 hour). To test them as well, one can use::

$ cd nmag-0.2.0
$ make checkslow

If, after completion of either of these checks, the last line reads::

======== tests finished: k passed, m failed, l skipped in XX.XX seconds ========

then m tests have failed. You will also get further debug output
following this summary line.

Other available test targets are ``make checkmpi`` and ``make

checkhlib``. Please read the corresponding sections in the manual
before running those -- they will fail unless the right environment has
been set up.

Common test failure

-------------------

The following messages are typical for the output of ``make check`` if the zlib
library was not present *during the compilation of nmag*. (This relates to the
pytables python module using hdf5 which in turn relies on zlib to read and write
compressed data.)

::

regression/nmesh/import/gmsh1_0/test_gmsh1_0_import.py[2] .F
../../../nsim/mpi/test_mpi_hello.py[2] ss
../../../unit/nmeshmirror/test_nmeshmirror.py[1] .
../../../unit/pycaml/test_refcount.py[1] .
../../../unit/pycaml/test_trial.py[2] ..
../../nmag/bigbar/mpi/test_slow_mpi_bigbar.py[2] ss
../../nmag/bigbar/nmag4/test_slow_bigbar.py[2] ss
../../nmag/bugs/twohmx/test_hlib_twohmx.py[1] s
../../nmag/demag_twomat/test_slow_bug_ticket175.py[1] s
../../nmag/nmagprobe/test_all.py[1] F
../../nmag/pymodules/test_python_modules_installed.py[1] .
../../nmag/restart/test_restart.py[1] .
../../nmesh/import/gmsh1_0/test_gmsh1_0_import.py[2] .F
../../../nsim/mpi/test_mpi_hello.py[2] ss
../../../../unit/nmeshmirror/test_nmeshmirror.py[1] .
../../../unit/pycaml/test_refcount.py[1] .
../../../unit/pycaml/test_trial.py[2] ..

<snip>

% (complib, default_complib), FiltersWarning )

HDF5-DIAG: Error detected in HDF5 (1.8.7) thread 0:
#000: H5Dio.c line 174 in H5Dread(): can't read data
major: Dataset
minor: Read failed
#001: H5Dio.c line 448 in H5D_read(): can't read data
major: Dataset
minor: Read failed
#002: H5Dchunk.c line 1729 in H5D_chunk_read(): unable to read raw data chunk
major: Low-level I/O
minor: Read failed
#003: H5Dchunk.c line 2760 in H5D_chunk_lock(): data pipeline read failed
major: Data filters
minor: Filter operation failed
#004: H5Z.c line 1094 in H5Z_pipeline(): required filter is not registered
major: Data filters
minor: Read failed
Closing remaining open files: /home/test/local/nmag-
0.2.0/nsim/tests/mesh/s3n1ellipsoid.nmesh.h5... done
make[3]: *** [run_dat.h5] Error 1
make[2]: *** [all] Error 2
 <snip>

#check that file name exists;

if not os.path.exists(filename):
E raise NmeshUserError,"file '%s' does not exist" % filename
> NmeshUserError: file 'sphere2.nmesh' does not exist

[/home/test/local/nmag-0.2.0/nsim/interface/nmeshlib/lib1.py:853]
- - - - - - - - test_gmsh_1_0_import_gzipped: recorded stdout - - - - - - - - -
root:2011-12-07 11:24:05,242 nmesh_exceptions.py 8 CRITICAL Exception
type: NmeshUserError
root:2011-12-07 11:24:05,243 nmesh_exceptions.py 9 CRITICAL Exception
msg : file 'sphere2.nmesh' does not exist

________________________________________________________________________________
======= tests finished: 52 passed, 4 failed, 16 skipped in 18.17 seconds =======
root:2011-12-07 11:24:05,664 shell.py 85 WARNING SystemExit with
exit_status=1
make[1]: *** [check] Error 1
make[1]: Leaving directory `/home/test/local/nmag-0.2.0/nsim/tests'

.. comment:

Testing the software

--------------------

After successful compilation, we suggest to run a small test to ensure

``nsim`` is working properly.
`gnuplot <http://www.gnuplot.info>`__ is required for producing the final
graph with the results. ``gnuplot`` is a plotting program;
many distributions provide an easy way to install it.
The script runs a basic simulation example, which should bring up
most unresolved problems::

$ cd nmag-0.2.0
$ cd nsim/tests/regression/nmag/bigbar
$ make check

The simulation should take 5-10 minutes, depending on the machine's

hardware. In the end, it will produce a graph that can be viewed with
the command::

$ gv results/results.eps

This graph shows the time evolution of the three components of the
mean magnetization for a Permalloy bar of size 30 nm x 30 nm x 100
nm. There should be good agreement between the ``nsim`` data and those
produced using ``OOMMF`` and ``Magpar`` for the same problem.

Removing the software

---------------------

When compilation has finished, unnecessary files can be removed with

the command ``make clean``. The command ``make uninstall`` will remove
all the files produced in the compilation phase and essentially leave
the bare source tree. (The name ``uninstall`` may seem strange at
first, as this way of building ``nsim`` does not install anything in
system directories such as ``/usr/lib``, but this is justified as we
recursively call ``make uninstall`` on all the packages ``nsim``
depends on which will be installed into a ``nmag-0.2.0`` subdirectory.)

To completely clean nmag from the system, it is sufficient to simply

remove the ``nmag-0.2.0`` directory::

$ rm -rf nmag-0.2.0

The important point to remember is that the ``nmag``/``nsim``

installation lives entirely within this directory and does not touch
anything outside it. (The directory /tmp may be an exception, though.)

Package and platform specific comments

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The packages which are more likely to be missing are ``gawk`` and
``readline``.

``gawk`` (GNU awk) is a very common program. Most operating system

distributions provide means to fetch and install a ``gawk`` package,
which often already comes pre-installed. (We need gawk to compile one
of the libraries [Petsc] that we depend on.)

``readline`` is commonly installed on linux systems, but often without

its development files. If the directory ``/usr/include/readline`` (or
``/usr/local/include/readline``) exists, readline development headers
should be available. Otherwise, it is likely that the readline library
is installed without its development files. On Debian-based Linux
distributions such as Ubuntu, one can issue the command
``aptitude install libreadline-dev``
(resp. ``apt-get install libreadline-dev``) to install the
missing package.

Troubleshooting issues such as ending up with a ``nsim_i`` executable

which does not show a coloured prompt and misses features such as tab
autocompletion are discussed in the section FAQ_.

The GNU compiler collection, as well as ``m4``, ``tar``, ``gzip``, ``bzip2``

should already be present if the machine already has been used for
some software development.

These are widespread developer tools and --- as for ``gawk`` --- the
operating system distribution should provide an easy way to install
them.

The same holds for the ``lapack`` and the ``blas`` linear algebra
libraries.

Debian/Ubuntu
+++++++++++++

A fresh installation on debian (etch) requires the following extra

packages to be installed:

``m4`` ``gawk`` ``zlib1g-dev`` ``g++`` ``libblas-dev``

``liblapack-dev`` ``readline-common`` ``libreadline-dev`` ``make``

for example through executing::

apt-get install readline-dev g++ libblas-dev libreadline-dev \

make m4 gawk zlib1g-dev readline-common liblapack-dev

Mac OS X
++++++++

On Mac OS X, fink_ can be used to install the additional tools needed by

``nsim`` (such as ``gawk``, for example).

Platforms that compile Nmag from source

---------------------------------------

The Nmag 0.1 software has been compiled successfully (and without intervention)
on the following systems:

- on Windows XP SP3 (Intel Core 2) through andLinux based on Ubuntu 9.04;

- IntelMac (=CoreDuo CPU in MacBook) Mac OS X (10.4) system;

- IntelMac (Intel Core 2 in iMac) Mac OS X (10.4.11) system;

- PowerPC Mac (MacMini, iMac) Mac OS X Tiger (10.4) system;

- MacBook Pro (Intel), Mac OS X Leopard (10.5) system;

- AMD Athlon 64 (running in 32bit mode) with Debian Linux (Etch);

- AMD Athlon 64 (running in 32bit mode) with Debian Linux (Lenny);

- Opteron machine running 64bit linux;

- Pentium 3 machine running 32bit linux;

- Pentium M machine running 32bit Ubuntu Linux (Gutsy);

- AthlonXP 2200+ running Ubuntu 7.04 (Feisty), and Ubuntu 7.10

(Gutsy), both 32 bit

- Athlon64 X2 4200+ Dual Core running Ubuntu 7.10 64bit

- IBM x440 system (with 8x1.9Ghz Intel Xeon processors) running

Red Hat Enterprise Linux AS release 3 (Taroon Update 2)

- Debian 32 and 64 bit with Pentium 4, E7200, and Q9300 processors

- AMD64 Athlon64 3200, OpenSuSE 11.0, 64bit, Kernel 2.6.25.20

- Dual Xeon quad core processors, 2.4 GHz, 24GB RAM, Red Hat Enterprise Linux
Desktop 5.3
 - Fedora 12

Platforms that needed a little help:

- Aspire One Netbook running the (Fedora based) Linux Linpus Lite
on intel Atom processor. The user reports that the 'diffutils'
package on the Linpus is broken and stopped the installation. He
downloaded the original diffutils from the GNU webpage, installed
this. Afterwards, Nmag compiled automatically from source (Dec 2008).

The Nmag 0.2 software has been compiled successfully (and without intervention)
on the following systems:

- Ubuntu 11.10 (Intel 32 and 64 bit)

- Ubuntu 11.04

- Mac OS X 10.6 (Snow Leopard)

- Mac OS X 10.7 (Lion)

.. _FAQ:

FAQ (Frequently Asked Questions)

--------------------------------

.. contents::
:local:

What is all this talk about ``nsim`` --- Isn't the package named ``nmag``?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

``nsim`` is a general purpose multiphysics simulation

package. ``nmag`` is a particular application based on it,
providing micromagnetism.

The major effort is to install ``nsim``. Once this is done

``nmag`` will ''just work''. It is for this reason, that there are
many references to ``nsim`` in this document.

What machines/operating systems does nsim compile on?

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This installation tool was designed to work on Linux platforms. It

does not (yet) work on MS Windows, but it has been successfully
used to compile ``nsim`` on Mac (both PowerPC and Intel
architectures) and may work under other flavours of unix OSes.

See `Platforms that compile nmag from source`_ for a list of systems on
which a successful compilation has been reported.

Why do I not have autocompletion and colours in ``nsim_i``?

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

You have just installed ``nsim``. And ``nsim_i`` works, but the
``nsim_i`` output is not coloured and the auto-completition is not
available either.

Probably ``nsim``, was compiled on a system where the readline library

(or its developer files) was not installed. ``nsim_i`` needs this
library for features such as command auto-completition and coloured
text.

One way to resolve this is to install the libreadline developer

files (which should in particular provide ``readline.h``),
as explained in the section `Installation requirements`_
and then recompile from scratch.

This is a short-cut to recompile just the python interpreter

(i.e. the readline-dependent part)::

$ cd nmag-0.2.0
$ rm -rf .deps_python_* python/
$ make .deps_python_install
$ make --dry-run | grep touch | sh

The last line is necessary to update the dependency files:

normally rebuilding ``python`` would cause all the other libraries
(mpich, petsc, etc.) to be rebuilt as well, the last line
is a trick to avoid this!

How can I recompile the ``nsim`` package, *together* with all the required
libraries*?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
~~~

Normally there should be no need to recompile the ``nsim`` package.

If for some reasons you want to do so, you can issue the following
commands::

$ cd nmag-0.2.0
$ make uninstall
$ make

Note that ``make uninstall`` will remove all the files produced by a
previous compilation of ``nsim``. So, all the programs and libraries
required by ``nsim`` will be rebuilt by these commands (this can take
some time).

Can I free some (unused) disk space after compiling nsim from source?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

You have compiled ``nsim`` from source and it occupies more about 1.4 GB
of space on your hard-drive. Some directories which were necessary for
the compilation can be removed after the compilation has finished. The
command is::

$ cd nmag-0.2.0
$ make clean

This will free about 0.9 GB.

Binary files in nmag-0.2.0/bin not working

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The installation of ``nsim`` produces a number of executables in the
``nmag-0.2.0/bin`` directory. When you try to run some of these, they do
not necessarily work properly. Do you need to be concerned that there
is something wrong with the installation?

No, probably everything is alright. The software is not installed

in the traditional unix directories (``/usr/bin``, ``/usr/lib``,
etc.) therefore the operating system needs to be instructed where
to find executables, libraries, etc.

``nsim`` does that by calling the shell script

``nmag-0.2.0/exports.bash``.

If (for some strange reason) you want to execute the binary files
under ``.../nmag-0.2.0/bin``, these definitions have to be pre-loaded
into the shell. For example, to start the Objective Caml interpreter
that is compiled as part of the ``nsim`` installation from source, you
could use::

$ cd nmag-0.2.0
$ . exports.bash
$ ./bin/ocaml

This may not be necessary, but is strongly suggested.

(Note that this is specific to the bash shell.)

I want to recompile only ``nsim``, how can I do this?

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This should not be necessary for normal users. If you are a developer
and you are extending the ``nsim`` OCaml core, then this is what you
need::

$ cd nmag-0.2.0
$ . exports.bash
$ cd nsim # enter the nsim source directory
$ make

These commands will only compile ``nsim``, without reconfiguring it.

If you want to reconfigure and recompile ``nsim`` use the following
commands::

$ cd nmag-0.2.0
$ rm nsim/config/configuration.inc
$ make

Note that the ``make`` command is used from the main ``nmag-0.2.0``
directory, not ``nmag-0.2.0/nsim``. These commands will configure
``nsim`` by running ``nmag-0.2.0/config/configure.py`` and passing to it
the proper command line options to specify the paths of the required
libraries.

I get an HDF5-DIAG error message when trying to run an example

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The error message reads::

HDF5-DIAG: Error detected in HDF5 library version: 1.6.5 thread 0.

 Back trace follows.

This error has been seen on a (misconfigured) linux system and is

caused by the hdf5 library believing to link to ``libz`` but
actually linking to a library that has the same name but different
content. In particular, it seems that Matlab provides it's own
``libz`` library (which has nothing to do with the standard
``libz``) and this library was earlier in the search path than the
actual ``libz``.

A workaround is to copy the right ``libz`` (a file like

``/usr/lib/libz.so.*``) to the ``nmag-0.2.0/lib`` directory. It
seems prudent to also copy the symbolic links to this
library, for example with::

$ cp /usr/lib/libz.so.* nmag-0.2.0/lib

Known issues
------------

On some Suse systems the installation failed, due to a weakness

of the petsc configuration process: petsc seems unable to link against
the lapack library. The problem was solved by adding an appropriate
flag to the ``LDFLAGS`` environment variable::

$ cd nmag-0.2.0
$ export LDFLAGS=-lgfortran
$ make

This is the same procedure as explained in the section Installation_

with just one more line to set the LDFLAGS variable.

.. _fink: http://finkproject.org/
.. _`nsim developers`: nmag@soton.ac.uk