You are on page 1of 24

FoulantProfiler

CAPTURE ENHANCE
STORE

Developed by GXS
Under the DC20B Enhancing Filtration Membrane Fouling
Data Collection for Water Research Project
June 2011 - March 2012

Funded by the Australian National Data Service (ANDS)

Version 1.0 (09 March 2012)

FoulantProfiler
User Manual
Introduction
Purpose
Confocal Laser Scanning Microscopes (CLSM) are used by materials scientists, biologists and
environmentalists for analysis of biological and organic samples. Filtration Membrane
Researchers use CLSM extensively for investigation of membrane fouling and evaluation of
the effectiveness of new anti-fouling surface treatments.
A Confocal Laser Scanning Microscope typically works together with the LAS-AF acquisition
software running on a workstation linked to the microscope. The acquisition software
provides control and configuration of the microscope equipment and the scanning process.
After scanning, the acquisition software outputs the data as a stack of images with different
channels of color and some experimental data (instrument data). It is common for these raw
data images to contain different types of noise which may result from a number of factors
such as flaws in the equipment, poor operational skills or poor sample preparation. Further
image processing is usually necessary to reduce or eliminate the noise contamination, but
this processing is tedious and requires specialist computer vision knowledge and skills for
success. Furthermore, the acquisition software does not provide any quantitative
measurements of foulant configuration.
FoulantProfiler, developed by Deakin University, is an ANDS Funded open-source image
processing/analysis software application that works with the acquisition software LAS-AF for
Confocal Laser Scanning Microscopes (CLSM) to capture single or multiple channels of
fluorescent images from filtration membrane contaminated by different organic foulants,
enhance the images and profile the foulants, and store those data in a central research data
repository.
Deakins development of the FoulantProfiler software application was based on the capture
of images from CLSM scans of filtration membranes, generated within Deakins biological
research centre, BioDeakin.
FoulantProfiler is designed to enable filtration membrane scientists and water researchers
or environmentalists with limited training and knowledge on computer vision or
programming to obtain better quality of images for improved visualization and to
quantitatively profile the foulants automatically. FoulantProfiler also aims to foster faster
and easier capture and storage of raw research data, processed data and metadata in the
centralised data storage.
In keeping with ANDS Funding guidelines and Open-Source Software principles, the
FoulantProfiler software application has been designed to easily allow customisation and
further development of the application to support different technical equipment and
acquisition software. As a result, it could be easily adaptable to a wide range of research
data capture environments, image analysis and quantification needs.

(Updated: 24 August 2015)

Page 1

FoulantProfiler
User Manual
Project Background
The Federal Government, via the Australian National Data Service (ANDS), has funded the
creation and development of the Australian Research Data Commons (ARDC) infrastructure,
supporting the discovery of, and access to, research data held in Australian universities,
publicly funded research agencies and government organisations for the use of research.
Under this program, funding is available for Data Capture projects which create
infrastructure within institutions to collect and manage data, and to improve the way
metadata about it is managed.
Deakin University was been awarded funding under the Data Capture initiative for the
ANDS Enhancing Filtration Membrane Data Collection for Water Research (DC20A)
project, for completion during the period June 2011 June 2012.
The primary objective of this project was to develop infrastructure within Deakins Confocal
Microscope Facility to support the ongoing data capture and management of filtration
membrane fouling data and metadata, supporting the discovery, reuse and sharing of this
research data by other researchers into the future.
This project was managed and implemented collaboratively with two other Deakin ANDS
Projects Seeding the Commons (SC19) and Crystal Orientation Data Collection for
Conversion to a General Data Type (DC20A). Collectively, these projects implemented the
ANDS funded, open-source system ReDBox as a solution for research metadata capture and
storage in Deakins Confocal Microscope Facility and Electron Microscope Facility.
This specific project developed a customised open-source system, FoulantProfiler, for
capture, profiling and storage of research data and metadata within the Confocal
Microscope Facility. The research data and metadata are linked via identification using the
Deakin Project ID.
In developing FoulantProfiler, this project re-used architecture and software components of
the CRYSTO software application developed by the Crystal Orientation Data Collection for
Conversion to a General Data Type (DC20A) project. This ensured an approach and
architecture for data capture that was consistent across both projects, ensured maximum
reuse and provided efficiencies in the development of the software applications.
This project also utilised infrastructure developed by the ANDS Seeding the Commons
project for metadata storage, harvest and discovery.

What will this software do?


FoulantProfiler works together with the existing Confocal Microscope equipment and the
existing image acquisition software LAS-AF in the Deakins Confocal Microscope Facility, to
capture the image data output from the acquisition software, enhance and profile the
images and store them in a central research data repository.
FoulantProfiler incorporates an automated image processing and analysis algorithm that
enhances the quality of CLSM images by transforming the color space from RGB to HSV,
denoising and filtering, then segmenting and distinguishing the different foulants and
computing the foulant configurations in 3-dimensional volume within the fouling layer.
FoulantProfiler provides a simple solution for the entry, capture and storage of Object-Level
metadata related to specific data-files. Some Object-Level metadata fields are extracted

(Updated: 24 August 2015)

Page 2

FoulantProfiler
User Manual
from the original data files, as output by the acquisition software package. Further ObjectLevel metadata fields can be specified on commencement of a data capture session, if
required. The Object-Level metadata is stored in a text file alongside the data files in the
central data repository, allowing for simple searching and filtering using the metadata.

What wont this software do?


The FoulantProfiler software application will not operate the microscope equipment or
replace the LAS-AF acquisition software. The software runs in the background while the
scans are executed and the acquisition software captures data.
The FoulantProfiler software application does not alter or remove the original data files
created by the acquisition software. The software creates new data based on the data in the
original data file. The original data file is maintained.
The FoulantProfiler software application does not provide capture, storage or harvest of
Collection-Level metadata for research data collections. The application provides a solution
for capture and storage of research data and Object-Level metadata only. FoulantProfiler is
designed to be used with a research metadata store such as ReDBox for the capture and
storage of Collection-Level metadata.
The FoulantProfiler software application will not interrupt research activities.

(Updated: 24 August 2015)

Page 3

FoulantProfiler
User Manual
System Architecture and Design
Target Audience:

Project Leader, Researcher, System Administrator

System Architecture
The following diagram shows the basic systems architecture required to support the
software application.
FoulantProfiler System Architecture
Local PC Workstation connected to Microscope

Networked Data Store

FoulantProfiler

Original Data Files

Data Capture

Data Processing/ &


Analysis
Acquisition Software
Confocal Laser
Scanning Microscope

Data
Storage

Processed Data

RESEARCH
DATA
STORE

Object -Level
Metadata

Server

Design Principles

The FoulantProfiler software application is based on the principle that all the data
files captured across the life of a Research Project will comprise the Research Data
Collections for that Project. A Research Project may generate a single or multiple
Research Data Collections.
The FoulantProfiler software application is based on the principle that all research
data collections will be stored in a centralised research data store, organised by
Project IDs. It is assumed that a centralised, networked and accessible data store will
be available for the storage of the data files created by the software application.
(Deakins Enhancing Filtration Membrane Fouling Data Collection For Water
Research project piloted the use of Deakins ISILON shares, which provide a secure,
supported, backed-up facility for the storage of research data collections. The ISILON
shares are web-enabled, supporting direct or managed access from Research Data
Australia (RDA) via the Deakin Research Online portal (DRO) to data collections
which have been released for open access.)

(Updated: 24 August 2015)

Page 4

FoulantProfiler
User Manual

The FoulantProfiler software application is not a metadata store and does not
provide capture, storage or harvest of Collection-Level metadata for research data
collections. FoulantProfiler is designed to be used with a research metadata store
such as ReDBox for the capture and storage of Collection-Level metadata.
(At Deakin University, ReDBox is implemented for the definition and storage of
Collection-Level metadata for research data collections. The research data files
stored in the Research Data Repository are linked to the Collection-Level metadata
via the Project Identifier (which acts as the Collection Identifier) within the data
repository, and via the via the URL of the collection as stored in the metadata.)
The following diagram represents how the software application could integrate with
an organisations metadata store in order to link the research data with the
collection metadata.
FoulantProfiler Architecture
Integration with a Metadata Store system
Local PC Workstation connected to Microscope

Networked Data Store

FoulantProfiler
Original Data Files

Metadata System
Metadata
Definition

Data Capture

Data Processing &


Acquisition Software

Analysis
Data
Storage

Confocal Laser
Scanning Microscope

Networked System

Processed Data

Collection
Metadata

RESEARCH
DATA
STORE

RESEARCH
METADATA
STORE

Object-Level
Metadata

Server

Data referenced via


URL in metadata
Data linked with metadata
by Collection identifier

System Components
The FoulantProfiler system has been developed with 3 primary functions, as follows:

Data Capture

Image Processing (Enhancement and Profiling)

Data Storage

These components have been developed as separate open-source software modules. This
modular nature supports the development of alternative data capture, data storage or
image conversion components to suit different equipment or research environments.
(Note that the Data Capture and Data Storage components re-use the software components
developed for the CRYSTO software application.)

(Updated: 24 August 2015)

Page 5

FoulantProfiler
User Manual
The components are described in detail below:
Data Capture
The Data Capture process is primarily a monitoring process that runs in the background
during a microscope scanning and research data capture session. The process runs on the
local PC workstation connected with the microscope, in parallel with the existing microscope
acquisition software application, (e.g., LAS-AF). The Data Capture process detects when any
research data files are created by the acquisition software and written to the specified data
folder on the local PC workstation.
It is important to note that acquisition software applications are typically commerciallyavailable, proprietary packages that do not allow customisation or direct integration with
the program code or output files created. Output files are typically generated in proprietary
format that cannot be changed. As a result, the FoulantProfiler software application has
been designed to operate separately from the acquisition process and to maintain the
integrity of the original data files.
The Data Capture process provides a simple data-entry solution for the identification of the
Project (and therefore the Collection) to which the research data captured during a session
will belong. The researcher can change the Project Identifier during a session by Starting a
New Session to allow for the scenario where a researcher captures data for different
Projects across the course of a data capture session.
The Data Capture process also provides a simple solution for the entry of Object-Level
metadata, such as Project ID, material type, fluorescent dye types, etc., related to specific
data-files. Required metadata fields can be specified, requiring the researcher to enter the
metadata at the commencement of each data capture session.
The Data Capture process then monitors the data (both images and instrumental data) with
certain formats (e.g., .Tiff, .JPG, .xml or .xsl) created by the acquisition software.
Image Processing
The Image Processing module runs between Data Capture and Data Storage, taking the data
files generated by the acquisition software application and automatically processing the
images, including transforming the original color space from RGB to HSV, enhancing the
quality of the images obtained and profiling the foulants, before finally outputting the
results to a new data file.
After the acquisition software is closed, the software automatically reads the created data
and passes the data to the Image Processing software component. The image analysis
software component automatically transforms the images RGB color space to HSV space,
enhance the qualities of the images in HSV space by denoising and filtering, then segment
and distinguish different foulants and compute the foulant configurations in 3-dimensional
volume within the fouling layer.

(Updated: 24 August 2015)

Page 6

FoulantProfiler
User Manual
In keeping with ANDS Funding requirements and Open-Source Systems development
principles, users are encouraged to further develop the software application by customising
or developing their own image processing/analysis algorithms and sharing them with the
Open-Source community. Alternatively, the FoulantProfiler software application could also
be installed without the Image Processing module to provide capture and storage capability
only.
The current version of the Image Processing module includes the extraction of some ObjectLevel metadata fields from the original data files. These metadata fields typically describe
parameters set within the acquisition software application for control of the microscope
scanning process or imaging. The metadata fields extracted will be dependent upon the
specific acquisition software applications.
Data Storage
The Data Storage process is simply a file management process that automatically stores the
output-files from the Image Processing module in a specified central file store.
Data files are stored according to the specified Project, identified by the Project Identifier,
with a separate folder created for each data capture session, identified by the Date and Time
of the session. All the data files captured during the session are stored within this session
folder. (The Data Capture process includes a simple method for changing the Project
Identifier during a session in the scenario where a researcher may capture data for different
Projects across the course of a data capture session.)
The transfer will include the raw data produced by instrument, exported data from the
acquisition software and processed data. The Data Storage process also stores any ObjectLevel metadata entered by the researcher or extracted from the data-files as a text-file
alongside the research data files in the data store. This file will be stored in the session
folder with the name METADATA.ini. This approach supports the development of filtering
and searching mechanisms to support discovery of research data files for re-use via the
meta-data.
The captured research data will also remain on local drive of the PC workstation connected
to the microscope equipment. Moving or deleting data from the local drive will not affect
the data already stored on network storage.

(Updated: 24 August 2015)

Page 7

FoulantProfiler
User Manual
Installation and Configuration
Target Audience:

System Administrator

System Requirements

Intel 1.3GHz or faster processor


Microsoft Windows XP Home, Professional, or Tablet PC Edition with Service Pack
3 (32 bit) or Service Pack 2 (64 bit); Windows Server? 2003 (with Service Pack 2 for
64 bit; Windows Server? 2008 (32 bit and 64 bit); Windows Server 2008 R2 (32 bit
and 64 bit); Windows Vista? Home Basic, Home Premium, Business, Ultimate, or
Enterprise with Service Pack 2 (32 bit and 64 bit); Windows 7 or Windows 7 with
Service Pack 1 Starter, Home Premium, Professional, Ultimate, or Enterprise (32 bit
and 64 bit)
256MB of RAM (512MB recommended)
100MB of available hard-disk space
1024x786 screen resolution recommended
Microsoft Internet Explorer 7, 8, 9; Firefox 3.6, 4.0 or 6.0; Chrome
Video hardware acceleration (optional)

Download and Install the Software


In accordance with the terms of the ANDS Funding, FoulantProfiler has been developed as
an Open-Source software application.
The install package can be downloaded from:
http://code.google.com/p/ands-enhancing-filtration-membrane/
Always download and install the latest version.
The software components are installed on the local PC connected to the Confocal
Microscope equipment, on which the LAS-AF acquisition software is also installed.
The easiest way to install is to download the all-in-one installation package, i.e.,
FoulantProfiler.rar and extract to your local drive. The all-in-one installation package
delivers all required binary files, dependencies and sample configuration files.
To perform a complete installation, download the latest version of both packages if there
are two or more versions available, then extract the files from both packages to the same
folder. It is highly recommended to install this software in folder C:\DC20B
Following installation, the application software will start automatically when the PC
workstation is started-up.

Setup Network Storage


Before commencing data acquisition with the FoulantProfiler software application, the
Network Storage must be setup properly, as described below:

(Updated: 24 August 2015)

Page 8

FoulantProfiler
User Manual

A dedicated network storage service is required for data storage via Server Message
Block (SMB), also known as Common Internet File System (CIFS) Protocol.
Project folders and project definition files must be setup on the storage service prior
the research activity, as follows:
o By default, each research project for which data may be captured must have
a corresponding folder already created in this storage service. The project
folders should be named with the Project Identifier, thereby allowing the
data storage process to correctly identify the location for storage of the data
files for the Project.
o If required, you may specify a particular sub-folder as the default path for
research data transfer. All research data will go into this sub-folder within
the Project folder. This approach can support the provision of both openaccess and restricted data storage areas within each Project folder on the
research data store.
o The software application will look for a project definition in your appointed
sub-folder. The definition needs to be stored in a file named PROJECT.ini.
Within this file these definition items are expected.
Item

Description

[project] project.name

The project name

[project] project.owner

Project leader and contact information

[metadata]

Assign technical (object-level) metadata to this


project. These metadata will be collected at the
beginning of each research activity.

All research data acquired during a research session, as well as the technical
metadata, will be organised into one single folder for that session. These are
named Session Folders on the network storage. Each Session Folder will be
assigned a unique name representing the timestamp when the activity conducted.
The session folders will be created automatically.
When setup has been completed as described above, the network storage is ready
to receive research data.

Configure the Software


Before commencing data acquisition, the FoulantProfiler software must be configured, as
described below:

A configuration file will be loaded from path config\default.ini by default. You may
store your configuration file anywhere on local drive, then pass on to this software
via command line parameters.

(Updated: 24 August 2015)

Page 9

FoulantProfiler
User Manual

Set up these items in your configuration file:


Item

Description

sys.encryption.salt

A random string to encrypt your password

sys.network.enabled

Enable (1) or disable (0) network storage

sys.network.smb.service

The SMB service path, like //host/service

sys.network.smb.username

Username to access SMB storage

sys.network.smb.password

Encrypted Password

sys.network.smb.domain

The workgroup for SMB storage

sys.network.smb.storage

The default path to store data on the SMB.


Variable %PID% will be replaced by project
id, %SID% for the session id. For
example %PID%/upload

sys.network.checking.interval

Time interval to update connection status (in


seconds)

sys.utils.smbclient.program

The path to smbclient tool

Setup Instrument and Acquisition Software


To acquire data with FoulantProfiler, your research instrument must be supported by the
software application. The current version of the software application supports Leica
Confocal Laser Scanning Microscope and its acquisition software LAS-AF.
If your instrument and microscope software is supported, download the scripts then extract
to installation folder, for example C:\DC20B. You will also need to follow the integration
instructions for the configuration of your instrument and acquisition software.
If your instrument or acquisition software is not supported yet, further developments or
customisations may become available for your instrument in the future. Alternatively, you
can create your own script file to enable your acquisition software as part of the ongoing
Open-Source development of this software application.
Refer to Add Support for Your Instrument section for script construction and induction.

(Updated: 24 August 2015)

Page 10

FoulantProfiler
User Manual
In keeping with Open-Source Systems development principles, users are encouraged to
further develop the software application by customising or developing their own image
processing processes and sharing them with the Open-Source community.

Upgrade & Uninstall


To upgrade the FoulantProfiler software application, download the latest version from
project website, extract and overwrite files in the install folder.
To uninstall, simply delete the folder the FoulantProfiler software has been installed into,
(note that installation was recommended in folder C:\DC20B).

(Updated: 24 August 2015)

Page 11

FoulantProfiler
User Manual
Software Usage
Target Audience:

Project Leader, Researcher, System Administrator

Helpsheets
A set of helpsheets have been created for the FoulantProfiler software application to
provide users with a step-by-step guide to the basic use of the system. These helpsheets are
available from:
http://code.google.com/p/ands-enhancing-filtration-membrane/w/list

Start the Software


The FoulantProfiler application runs in the background during all microscope data capture
sessions. It is configured to start automatically when the Windows operating system is
started.
For a step-by-step guide to starting the FoulantProfiler software and entering parameters,
refer to the Helpsheet:

FoulantProfiler Helpsheet Starting FoulantProfiler.

Start a New Research Session


The Project you specify in FoulantProfiler at the start of your microscope session will
determine where your research data is stored in the networked data repository. If you are
capturing data for several Projects during a single microscope session, you must start a new
Research Session in FoulantProfiler each time you start capturing data for a different
project.
For a step-by-step guide to starting a new research Session, refer to the Helpsheet:

FoulantProfiler Helpsheet Start a new Research Session.


Advanced Users
For advanced users, new research session can also be started from command line, with
parameters as following:
-c --config-file CONFIG_FILE_PATH
Optional. Use the defined configuration file. If not specified, will use file
config\default.ini
-e --equipment-id EQUIPMENT_ID
Optional. Start the specific research equipment. If not specified or invalid,
appoints the first definition of research equipment ID by default.

(Updated: 24 August 2015)

Page 12

FoulantProfiler
User Manual
Export data
When your research activity is completed, all data created may be exported. This is an
automated procedure and no interaction from the user is required.
The data exporting is configured in your instrument and acquisition software settings.

Processing/Analysis data
If required, your data will be processed by image processing/analysis scripts. This is an
automated procedure and no interaction from the user is required. The processing of data is
configured in your instrument and acquisition software settings.

Transferring to Network Storage


After each session, all your data will be transferred automatically to the network storage.
This is an automated procedure and no interaction from the user is required.
The transfer will include the raw data produced by instrument, exported data from the
acquisition software and processed (standardised) data. A text file will also be created to
record Object-Level metadata for your current session. This file will be stored in the Session
Folder with the name METADATA.ini.
Note:

If you choose the -Not a registered research project- option when


starting the session, Object-Level metadata will not be captured.

Your research data will also remain on local drive of the PC workstation connected to the
microscope equipment. Moving or deleting data from the local drive will not affect the data
already stored on network storage.
For the path of your research data, please consult your system administrator.
When data transfer is completed, the FoulantProfiler software application will idle and get
ready for the next research activity.
For a step-by-step guide to the export, processing and storage, refer to the Helpsheet:

FoulantProfiler Helpsheet Wheres my Data?

(Updated: 24 August 2015)

Page 13

FoulantProfiler
User Manual
System Processes and Workflows
Target Audience:

System Administrator, Software Developer

Equipment, Project & Session


The FoulantProfiler software application defines the concepts Equipment, Project and
Session.
Equipment
Equipment is defined as the specific knowledge related to the research instrument and the
associated software for data acquisition and processing.
This software needs to be aware of,

name of the instrument associated software


how to start the instrument associated software
what data format will be produced during acquisition, and where to find the data
files
what is the finishing flag of the data acquisition
whether the data need to be exported for further customised processing
whether the data need to be transferred to network storage

Equipment is defined in the configuration file. Name of supported equipment is available


via:
http://code.google.com/p/ands-enhancing-filtration-membrane/wiki/InstrumentSupported
Project
A Project is defined as a series of research activities conducted over a period of time which
generates a research data collection.
A list of Object-Level metadata fields may be defined as required for a Project and a default
value can be assigned for each. When the researcher commences a data capture session for
that Project, values will be requested for these required fields. These fields are defined in
the Project Profile.
Project profile is updated automatically on a daily basis. This software will look for a file with
name PROJECT.ini in the Project folder on the networked data repository.
Session
A Session is defined as a research activity conducted by a Researcher that captures research
data. A Session will involve Equipment as source of data, and attach the research data to a
specific Project data collection.
If Object-Level metadata fields have been defined as required, the Researcher operating the
instrument will be requested for the parameters of each item, at the beginning of a session.

(Updated: 24 August 2015)

Page 14

FoulantProfiler
User Manual
A session will also record a list of data files to be transferred to network storage, including
raw research data and any data files generated after the conversion process.
A Session exchange file is created during the activity. It is then uploaded to the Session
folder on network storage with a name METADATA.ini, together with all data files
acquired during the session.

System Processes
Several data streams are working together within the FoulantProfiler software application.

Project Update
This data stream will synchronise the project definitions from network server. The
FoulantProfiler software application will look for PROJECT.ini in each project
folder. Synchronisation happens every 24 hours. If no project definition stored on
local, the synchronisation will happen immediately when this software is started.

Data Acquisition and Processing


When a researcher starts the software application with an Equipment parameter,
the FoulantProfiler software application will start the associated acquisition
software application and record all new data files generated during the research
session. Data files will be exported and customised to the expected format. A file list
of these new data files will be produced in the session exchange file. When the data
acquisition and processing completed, the session will be queued for data transfer.
If supported, the FoulantProfiler software application will export the raw research
data acquired during the session to a particular format. This software will utilise the
instrument associated software to perform this action. As per the configuration, this
software may simulate keyboard and mouse operations to interact with the
instrument software.
Some equipment may also be configured to have the data processed with a
customised processing program. For instance, the data may be converted to another
format, or be enhanced for better quality. The processing program is a customised
tool and not necessarily from the instrument vendor.
Data Transferring to Network Storage
This data stream will start immediately when this software started. This data stream
will work on a queue of session exchange files. For each session, the list of research
data file, as well as the research exchange file itself will be transferred to network
storage.

(Updated: 24 August 2015)

Page 15

FoulantProfiler
User Manual
System Workflows
The life cycle of each internal process component is as follows:

1. The FoulantProfiler software application is launched when its shortcut is clicked.


Windows operating system is started.
2. When started, the FoulantProfiler software application will commence updating the
project profile on a time basis
3. The research requests to start a new session. A parameter is sent to the
FoulantProfiler software application indicating the equipment and instrument.
4. When a new Research session is started, the associated project and relative ObjectLevel metadata are collected. Also, the decision about whether image
processing/analysis is required can be made by typing Y for yes or N for no.
5. When new session started, the instrument associated software will be started and
managed. The FoulantProfiler software application starts to monitor for new
generated data file, optionally filtered by nominated file format (file extension
name) and directory path.
6. When the instrument associated software exits, the manipulating module of the
FoulantProfiler software application will execute a script to initiate the image
processing/analysis module, if required. When the processing program completed,
all data file will be organised into a session exchange file by the manipulating
software while it stops monitoring. Session exchange file is queued for transfer to
the centralized data store.
7. Alternatively, when no image processing/analysis required, the FoulantProfiler
software application will not execute the processing program and will directly
transfer the raw data, instrumental data, and object-level metadata to the
centralized data store.

(Updated: 24 August 2015)

Page 16

FoulantProfiler
User Manual
Extending and Re-Development
Target Audience:

Software Developer

In keeping with ANDS Funding requirements and Open-Source Systems development


principles, users are encouraged to further develop the FoulantProfiler software application
by customising or developing their own image processing/analysis algorithms and sharing
them with the Open-Source community.
If deploying with an instrument currently not supported, users can create a script file to
enable their specific acquisition software to comply with the integration standard.
The FoulantProfiler software application is Open-Source and can be further developed for
any specific requirements.

Technical Details
Compile Environment and Dependencies
The FoulantProfiler software application is developed with dependencies including

Microsoft C Runtime Library


provided by Microsoft, as components of the Windows operating system
GNUWin32
provides native ports in the form of runnable computer programs, patches, and
source code for various GNU and open source tools and software. Project website
http://gnuwin32.sourceforge.net
wxWidget toolkit and library, version 2.9 or after
a cross-platform GUI and tools library for GTK, MS Windows, and MacOS. Available
for download at http://www.wxwidgets.org
GNU Compiler Collection (GCC) or alternative
a compiler system produced by the GNU Project. Also tested with win32 port
MinGW (http://www.mingw.org) and TDM-GCC (http://tdm-gcc.tdragon.net)
AutoHotkey Basic
a free, open source macro-creation and automation software utility. Official website
http://www.autohotkey.com
SMBClient win32 port
the default storage is defined as a CIFS/SAMBA share. Smbclient is utilised for
interaction and communication with the storage server. Tested with two releases
available on http://leecutsco.de/lpackham/smbclient and
http://www.hagyroom.com/labs/smbclient (Japanese language)
Visual C++
Provides a platform for further development of image processing & analysis
algorithms

(Updated: 24 August 2015)

Page 17

FoulantProfiler
User Manual
Note:

These dependencies are only required for compilation and further


development of the software application. They are not required if
installing and using the standard version of the software application.

Compiling the Source Code


Source code for this software can be downloaded from the project website. More
information about how to checkout your copy of the source code is available via:
http://code.google.com/p/ands-enhancing-filtration-membrane/source/checkout
To compile the source code, install and setup the dependencies in your development
environment. Type the following command to start your compilation:
make compile
For further options, check the makefile in the source code.

Add Support for Your Instrument


The current version of the FoulantProfiler software application provides unique image
processing/analysis module for Leica confocal laser scanning microscope equipment and its
acquisition software package LAS-AF.
However, the FoulantProfiler software application can be easily customised to support
additional acquisition software packages or different microscope equipment. Users are
encouraged to further develop the software application by customising or developing their
own conversion processes and sharing them with the Open-Source community.
Check the project website to determine whether there is already a script providing support
for your instrument that has been contributed by the community. This information is
available at:
http://code.google.com/p/ands-enhancing-filtration-membrane/wiki/SoftwareInterfaceStandard

If your instrument is not supported yet, you will need to work out these settings

Instrument Configuration
Instrument Export Script (Optional)
Data File Processing Program

Instrument Configuration
The FoulantProfiler software application is a universal tool which reads the instrument and
research data. It has no knowledge about your instrument. The first step is to have your
instrument defined to the software application. The instrument configuration is created for
this purpose.
An instrument configuration is a series of settings in the system configuration file, by default
located in config/default.ini. You will need to provide settings for:

(Updated: 24 August 2015)

Page 18

FoulantProfiler
User Manual
Item

Description

equipment.EID.id

The id for this equipment

equipment.EID.name

The name for this equipment

equipment.EID.launcher.enabled

Enable or disable the launcher. If enabled, the


appointed program will be started.
1 for enabled or 0 for disabled

equipment.EID.launcher.program

Path to the software associated with this equipment

equipment.EID.monitor.extensions A list of file extensions separated by a comma (not


casesensitive).
Any file with this extension will be reported during
monitoring, and then exported, processed and
transferred to network storage.
Unless your Processing Program will manage to add
the output file in exchange file, otherwise add your
customised output file format extension here
equipment.EID.monitor.directories A list of file-paths to be monitored, separated by a
comma
equipment.EID.export.enabled

Enable or disable the export of data file from


instrument associated software.
1 for enabled or 0 for disabled

equipment.EID.export.script

The script to perform export action

equipment.EID.processing.enabled Enable or disable the processing of data.


1 for enabled or 0 for disabled
equipment.EID.processing.program The path to the image processing/analysis program
which will deal with the research data
This processing program will be executed with
particular parameters
equipment.EID.storage.enabled

(Updated: 24 August 2015)

Enable or disable the storage of data. If disabled no


data will be transferred.
1 for enabled or 0 for disabled

Page 19

FoulantProfiler
User Manual
equipment.EID.storage.flatfolder

Note:

Whether to create sub-folders on network storage or


not. If enabled all acquired data files will be stored in
the same folder. Files with same name will be
overwritten during transfer.
1 for enabled or 0 for disabled

Replace EID with the identity of your instrument. The identity is a unique
string composed of lower-case alphanumeric only (I.e.: letters a-z and
numbers 0-9). This identity is case sensitive and must be unique.

If more than one instrument is managed by your operation system, for example, multiple
instrument associated software installed on the same computer, you also need to add your
instrument identity to the system configuration as following
Item

Description

equipment.list

A list of instrument identities to be supported,


separated by a comma. By default, the instrument ID
is LAS-AF in this case.

Instrument Export Script


If export enabled, you need provide a script to setting item equipment.EID.export.script.
It is recommended to utilise the AutoHotkey Basic to perform the export action. Create your
script to simulate keyboard and mouse movements to operate the instrument associated
software. Refer to the script code in config/AutoHotKey.Scripts for sample.
When executed, session exchange file will be passed to your script as the only parameter.
Check section Session Exchange File for the interface standard.
Image Processing/Analysis Program
If enabled, the image processing/analysis program will automatically enhance image quality
through, filtering, denoising, segmentation and profile foulants configurations. This
processing program has an automatic workflow which requires no interaction.
The processing program is built with C++ language but it can also be built with any other
language or compile tool, not necessarily limited to the dependencies of this software.
The processing program is expected to have specific knowledge about the research data,
rather than the instruments, to perform data processing. It is expected to communicate with
the manipulating module seamlessly.
When executed, the image processing/analysis program will receive parameters as follows:

(Updated: 24 August 2015)

Page 20

FoulantProfiler
User Manual
--launched-from-manipulating-software --research-exchange-file FILE
where:

FILE will be replaced with the absolute path to a session exchange file.

The session exchange file will contain metadata of the session, including the project profile,
equipment information and technical metadata acquired from researchers. The session
exchange file will also contain a list of data files acquired during the session. The processing
program may produce new data files based on the file list. The new data file may be
monitored and recorded in the session exchange file, or be added to the session exchange
file by the processing program explicitly.
To indicate if the processing is successful or not, the processing program can return a status
code and message by putting two settings in the metadata section of session exchange file.

Item

Description

error.code

An error code. 0 for success and fail for otherwise

error.message

An error message. Only valid when error.code is nonzero

Note:

If the error.code stated in session exchange file is non-zero, current


session will be terminated and no data will be transferred. The data files
on the local disk will not be removed.

Users are encouraged to contribute their equipment settings and processing programs to
this project.

Extending to Other Network Storage Solutions


The network storage can be extended to protocols other than SMB/CIFS, such as
HTTP/HTTPS, FTP, rsync over SSH etc.
To extend, you need to implement relative interface exposed in DataFileStorage class. Refer
to source code file src/main/datafile_storage.cpp for more information.
Your implementation of DataFileStorage needs to perform tasks including

Synchronise project profile


Create relative sub-folders in network storage
Uploading local file to network storage

You need to re-compile the source code and generate a new binary execution file
afterwards.

(Updated: 24 August 2015)

Page 21

FoulantProfiler
User Manual

(Updated: 24 August 2015)

Page 22

FoulantProfiler
User Manual
Acknowledgement

This project is supported by the Australian National Data Service (ANDS). ANDS is supported
by the Australian Government through the National Collaborative Research Infrastructure
Strategy Program and the Education Investment Fund (EIF) Super Science Initiative.

Copyrights and Licenses


Copyright 2011-2012 Deakin University.
This program is free software: you can redistribute it and/or modify it under the terms of the
GNU General Public License as published by the Free Software Foundation, either version 3
of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY;
without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR
PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program.
If not, see <http://www.gnu.org/licenses/>.

(Updated: 24 August 2015)

Page 23

You might also like