A Novel Approach to Multi Modal Hybrid
Image Fusion Using Wavelet and Contourlet
Transform for Medical Diagnosis Applications
P C VIJAYA LAKSHMI1, T MAHEBOOB RASOOL2
2Associate Professor & HOD
1,2 Dept. of Electronics and Communication Engineering
Bharath college of Engineering and Technology For Women, C.K.Dinne,Kadapa
Abstract— Image fusion is used to enhance the quality
of images by combining two images of same scene obtained
from single or multiple modalities. In medical diagnosis
different types of imaging modalities such as X-ray, computed
tomography (CT), magnetic resonance imaging (MRI),
magnetic resonance angiography (MRA), PET scan etc.,
provide limited information where some information is
common, and some unique. This paper presents a hybrid
combination of Discrete Wavelet and Contourlet. For the
obtained fused images from above case, it is proposed to
compute performance metrics like Entropy, Peak signal to
noise ratio and mean square error and compare them with
existing methods as to come out with best combination of
transformations that yield highly informative fused image.
Proposed method will be simulated using MATLAB tool.
Index Terms— Fusion, wavelet, contourlet, Filter banks, Multi-
resolution, Multi-direction, Geometric image processing, MSE,
PSNR, Entropy.
I. INTRODUCTION
Medical image fusion is the process in which
complimentary information from multiple images from
single or multiple imaging modalities are combined to
improve the imaging quality so that the resultant fused
image will be more informative and suitable for processing
tasks thereby increasing their clinical applicability in
diagnosis and assessment of medical problems
In the recent years, Multimodal image fusion
algorithms and devices, has evolved as a powerful tool in
the clinical applications, of medical imagining techniques. It
has shown significant achievements in improving clinical
accuracy of diagnosis based on medical images. The main
motivation is to produce most relevant information from
different sources into a single output, which plays a crucial
role in medical diagnosis.
Medical imaging has gained significant attention due to
its predominant role in health care. Some of the different
types of imaging modalities used now-a-days are X-ray,
computed tomography (CT), magnetic resonance imaging
(MRI), magnetic resonance angiography (MRA), etc., These
imaging techniques are used for extracting clinical
information, which are although complementary in nature
most of the times, some are unique depending on the
specific imaging modality used.
For example,
X-rays: Is used to detect fractures, abnormality in bone
position
CT: Is used to provide more accurate information about
calcium deposit, air and dense structures like bones with less
distortion, acute bleeds and tumours. But it cannot detect
physiological changes.
MRI: Under strong magnetic field and radio-wave energy,
information about Nervous system, structural abnormalities
of soft tissue, muscles can be better visualized.
MRA: Is used to evaluate blood vessels and its
abnormalities.
PET: PET (positron emission tomography) offers
quantitative analyses, allowing relative changes over time to
be monitored as a disease process evolves or in response to a
specific stimulus by looking at blood flow, metabolism,
neurotransmitters, and radio-labelled drugs.
SPECT: Single positron emission computed tomography
provides functional and metabolic information. It helps to
diagnose and stage a cancer.
FMRI: Functional magnetic resonance imaging is a
functional neuro-imaging procedure using MRI technology
that measures brain activity by detecting changes associated
with blood flow
Hence, we can understand none of these modalities are
able to carry all relevant information in a single image.
Generally for a physician to analyze the condition of a
patient, in most of the cases he needs to study different
images like MRI, CT, PET, SPECT simultaneously, which
is time consuming. Hence the anatomical and functional
medical images are needed to be combined for a concise
view. For this purpose, the multimodal medical image
fusion has been identified as a source with better potential. It
aims to integrate information from multiple modalities to
obtain a more complete and accurate description of the same
object which facilitate in more precise diagnosis and better
treatment. Fused image provides higher accuracy and
reliability by removing redundant information.
The applications of image fusion are found in radiology,
molecular and brain imaging, oncology, diagnosis of cardiac
diseases, neuro-radiology and ultrasound. Multimodal
medical image fusion helps in diagnosing diseases, and also
cost effective by minimising storage to a single fused image
instead of multiple-source images.
 Wavelet decomposition can be implemented by two channel
II. BACKGROUND AND RELATED WORK filter bank shown in fig.1.

A. Image fusion techniques:

Image fusion method can be broadly classified into two
groups namely, Spatial domain and Transform domain fusion.
In spatial domain techniques, we directly deal with the image
pixels. The pixel values are manipulated to achieve desired
result. The fusion methods such as averaging, Brovey
method, principal component analysis PCA) and IHS based
methods fall under spatial domain approaches. The
disadvantage of spatial domain approaches is that they
produce spatial distortion in the fused image.
It does not give directional information and also leads to
spectral distortion, while the arithmetic combination will
Keep 1 column out of 2 (down sampling in columns) Keep row out of 2
lose original details as a result of low contrast of the fused
image. It becomes a negative factor while we go for further (down sampling in columns)
processing, such as classification problem, of the fused Fig. 1 Two level WT decomposition
image. The Discrete Wavelet Transform has the property that
This can be overcome by transform domain. It involves the spatial resolution is small in low-frequency bands but
the decomposition of the source image into sub-bands which large in high frequency bands. This is because the scaling
are then selectively processed using appropriate fusion function is treated as a low pass filter and the mother
algorithm. In frequency domain methods the image is first wavelet as high pass filter in DWT implementation. The
transferred in to frequency domain. It means that the Fourier wavelet transform decomposition and reconstruction take
Transform of the image is computed first. All the Fusion place column and row wise. Firstly row by row
operations are performed on the Fourier transform of the decomposition is performed and then column by column.
image and then the Inverse Fourier transform is performed This can be shown in fig.2.
to get the resultant image. Image Fusion applied in every
field where images are ought to be analyzed.
(GENERAL IMAGE FUSION BASED ON DWT DWT
comes under the classification of multi-scale
decomposition. This is used to map the wavelet transform to Fig. 2 Image decomposition using DWT
A. digital world. Filter banks are used to approximate
the behaviour of the continuous wavelet transform. The ILL(x, y) sub-band is the original image at the coarser
Double channel filter bank is used in discrete wavelet resolution level, which can be considered as a smoothed and
transform (DWT). The coefficients of these filters are sub-sampled version of the original image. Most
information of their source images is kept in the low
evaluated using mathematical analysis. The wavelet frequency sub-band. It represents the frequency usually
transform is used to identify local features in an image. contains slowly varying grey value information in an image
It also used for decomposition of two dimensional so called approximation.
(2D) signals such as 2D gray-scale image for multi- The ILH (x, y), IHL(x, y) and IHH(x, y) are sub-bands contain
resolution analysis. The available filter banks the detail coefficients of an image, which usually have large
decompose the image into two different components absolute values correspond to sharp intensity changes and
i.e. high- and low- frequency. When decomposition is preserve salient information in the image.
carried out, the approximation and detail components
can be separated 2-D Discrete Wavelet Transformation
(DWT) converts the image from the spatial domain to
transform domain. The image is divided by vertical
and horizontal lines and represents the first-order of
DWT, and the image can be separated with four parts
those are LL1, LH1, HL1 and HH1. Wavelet
Decomposition of Images:
Wavelet separately filters and down samples the 2-D data 1, 2, 3 - - - Decomposition Levels
(image) in the vertical and horizontal directions (separable H - - - - - High Frequency Bands
filter bank). The input (source) image is I(x, y) filtered by L - - - - - Low Frequency Bands
low pass filter L and high pass filter H in horizontal Fig. 3 Different levels of decomposition
direction and then down sampled by a factor of two
There are different levels of decomposition which are
(keeping the alternative sample) to create the coefficient
matrices IL(x, y) and IH(x, y) . The coefficient matrices IL shown in fig.3. After one level of decomposition, there will
(x, y) and IH(x, y) are both low pass and high pass filtered be four frequency bands, as listed above. By recursively
in vertical direction and down sampled by a factor of two applying the same scheme to the LL sub-band a multi-
to create sub bands (sub images) I LL(x, y) , ILH (x, y) , resolution decomposition with a desires level can then be
IHL(x, y), and IHH (x, y). achieved.
 The schematic diagram of wavelet- based image
fusion algorithm is shown in Fig.4. In wavelet image
fusion scheme, the source images I1(x, y) and I2(x, y), are
decomposed into approximation and detailed coefficients
at required level using DWT. The approximation and
detailed coefficients of both images are combined using
Fig.7 Laplacian Pyramid decomposition process
fusion rule ø. The fused image (If(x, y)) could be obtained
by taking the inverse discrete wavelet transform (IDWT) as:
If(x, y) = IDWT [ø {DWT (I1(x, y)), DWT (I2(x, y))}] (1)
Fused Wavelet
Registered Wavelet coefficients
Source images coefficients
Fig. 4 Multi-level image fusion using DWT
The fusion rule used in this paper is simply averages the
approximation coefficients and picks the detailed coefficient
in each sub band with the largest magnitude.
B. Procedure for implementation of wavelet transforms:
Step 1: The images to be fused must be registered to assure
that the corresponding pixels are aligned.
Step 2: These images are decomposed into wavelet
transformed images, respectively, based on wavelet
transformation. The transformed images with K -level
decomposition will finally have 3K+1 different frequency
bands, which include one low-frequency portion (ILL)
and 3K high-frequency portions (low-high bands, high-low
bands, and high-high bands).
Step 3: The transform coefficients of different portions or
bands are performed with a certain fusion rule.
a) Fuse approximate coefficients of source image
Advantages:
This new construction results in a flexible multi-
resolution, local, and directional image expansion using
contour segments, and thus it is named the contourlet
transform. It is of interest to study the limit behaviour
when such schemes are iterated over scale and/or direction,
which has been analyzed in the connection between filter
banks, their iteration, and the associated wavelet
construction. The general block diagram is shown below in
Fig.5.
 Laplacian Pyramid
 Directional Filter Bank
The overall result is an image expansion using basic
elements like contour segments, and thus is named
contourlet.In particular, contourlet have elongated supports at
various scales, directions and aspect ratios. This allows
contourlet to efficiently approximate a smooth contour at
multiple resolutions.In LP at each stage the original image is
multiplied with Gaussian filter by down sampling to obtain
Low pass version of the original image or blur image and then
the obtained low-pass image is separated from the original
image by up-sampling to obtain band-pass image (or High
Frequency components). This band-pass image is applied to
directional filter bank to obtain directionality information. The
decomposition process in Laplacian pyramid is implemented as
shown below fig.7.
This is the process that, Laplacian pyramids separates low
frequency and high frequency components. The obtained LF
Sub-band (Scaled) image is further decomposed until we get
the desired fused image.
A. Directional Filter Bank:
The High frequency components are given to the directional
filter bank which links point discontinuities into linear
structure.
Fig.8 Frequency partitioning in DFB
The High pass sub-band images are applied to Directional
filter bank to further decompose the frequency spectrum using
an n-level iterated tree structured filter banks as shown in
fig.8. By doing this we capture smooth contours and edges at
any orientation. Finally we combine the scaled information
with scaled multiplication. Since the directional
Fig:5 general block diagram
III. General image fusion based on Contourlet Transforms
1) Curved singularity representation.
2) Limited orientation (vertical, Horizontal and
Diagonal).
3) Absence of anisotropic element (isotropic scaling).
Time and frequency Localization.
Limitations:
6. Anisotropy: To capture smooth contours in images, the
representation should contain basis elements using a
variety of elongated shapes with different aspect ratios.
 b) Need for contourlet transform: Among these These frequency coefficients are fused together
desiderata, the initial three are effectively provided by based on certain fusion algorithms are reconstructed
separable wavelets, while the last two require new using the inverse CT. The schematic diagram of our
construction. Besides, a significant challenge in proposed methodology which is the hybrid image fusion
capturing geometry and directionality in images using wavelet-contourlet transform is shown in fig.9.
comes from the discrete nature of the data. For this
reason we construct multi-resolution and multi-
direction image development using non-separable
filter banks. using average method.
c) Fuse detail coefficients of source image using
Maxima method.
Step 4: The fused image in spatial domain is constructed by
performing an inverse wavelet transform based on the
combined transform coefficients from Step 3. The contours
of original images can be captured effectively with a few
coefficients by using Contourlet transform. Contourlet
transform provides multi-scale and multi directional
decomposition. It consists of two stages
A. Laplacian Pyramid: filter bank (DFB) was designed to
capture the high frequency (representing directionality) of
Fig:9 schematic for proposed fusion method
the input image, the low frequency content is poorly
handled. In fact, with the frequency partition low
frequency would “leak” into several directional sub-bands, till different frequencies and resolutions are obtained.
hence the DFB alone does not provide a sparse Thus a hybrid of wavelet and contourlet would lead to
representation for images. This fact provides another better results that could be used for medical diagnosis
reason to combine the DFB with multi- scale when compared with Existing methods those are
decomposition, where low frequencies of the input image individual results of wavelet and contourlet transform.
are removed before applying the DFB.
C. Flow chart
IV. PROPOSED FUSION
METHODOLOGY
A. Pre-processing stage:
Source images are necessary to align or register as perfectly
as possible prior to the main fusion process in order to
produce the best fusion results. Image registration process
can also be called as image alignment. The datasets of input
images are not aligned to each other, it is impossible to
yield best fusion results although fusion framework,
scheme and algorithm are optimum for maximum
counterlet transformation.
A. Decomposition -1:
After pre-processing these source images are first
decomposed using Wavelet Transform (WT) in order to
realize multi-scale sub band decompositions with no
redundancy. These sub-bands coefficients are
predominantly low and the high frequency sub-bands of
the image. Now, the obtained approximation and
detailed coefficients after application of appropriate
Step 8: To get final hybrid fused image in spatial
fusion rule are reconstructed using the inverse DWT.
domain we apply inverse contourlet transform.
The entire process carried out in this stage serves to
provide significant localization leading to a better
I. PERFORMANCE EVALUATION OF IMAGE
preservation of features in the fused image.
FUSION TECHNIQUES
B. Decomposition -2:
After reconstruction at Stage-1, fusion algorithm is
Quantitative performance of the various Hybrid Image
again applied at stage-2 in Contourlet domain. The
Fusion techniques in Transform domain can be done by
significance of such an approach is to overcome the
the following metrics Entropy Entropy measures amount
limitation of directionality in wavelets (in stage-1).
of information contained in an image. Higher entropy
Contourlet transform (CT) is applied in order to achieve
value of the fused image indicates presence of more
angular decompositions. After applying sub-band
information and improvement in fused image. If L
decomposition using CT, a set of coefficients are
indicates the total number of grey levels and p=
obtained (for both the images).
{p0, p1 ... pL-1} is the probability distribution of each
level, Entropy is defined as, i=0
 VIII. Comparison of Various Parameters of Proposed
Methods Have Been Listed In Tabular Forms and Related
Bar charts
Table 1: Fusion of CT and MRI of coronal section of lungs affected with
B. MSE tuberculosis.
The Mean Square Error (MSE) and the Peak Signal to
Noise
Ratio (PSNR) are the two error metrics used to compare
image compression quality. The MSE represents the
cumulative squared error between the compressed and the
original image, whereas PSNR represents a measure of the
peak error. The lower the value of MSE, the lower the
error.

C. PSNR
The PSNR block computes the peak signal-to-
noise ratio, in decibels, between two images. This ratio is
often used as a quality measurement between the original
and a compressed image. The higher the PSNR, the better
the quality of the compressed or reconstructed image

VII. RESULTS ANDANALYSIS

In this section we will check the effectiveness of the

proposed scheme that is hybrid of the DWT-Contourlet
transform. Various parameters such as Entropy, MSE and
PSNR are used to evaluate the effectiveness and compared
the performance metrics among the proposed method and
existed methods. We assume the source images to be in

perfect registration. Here we consider different source

images like CT, MRI, and PET of brain tumour, Skull,
MRI_T1, MRI_T2.
Table 2: Fusion of X-ray and MRI images of knee
Table 3: Fusion of CT and PET scan of Lung cancer
These bar charts and Tabular values give the
experimental values performed on different images we
discussed previously. From these we can observe that
curvelet contourlet transform has given better results for
which PSNR and Entropy values are of high values when
compared to other transformation techniques and the MSE
is less when compared with other methods, which satisfied
the conditions of better image quality that will obtain after
fusion.
IX. CONCLUSION
In this project, a hybrid technique for image fusion
using the conbination of DWT-Contourlet is being
simulated. The simulated results obtained for proposed
method for different combinations of medical images like
CT, MRI ,PET, and also for various input image sizes are
compared with existing techniques .
 In all cases , DWTContourlet based hybrid
technique is observed to be outsmarting than one which
provide best quality fused image when compared to other
fusion techniques in terms of PSNR, MSE, & Entropy. So
it is best suited for better medical diagnosis.

REFERENCES

[1] M. N. Do and M. Vetterli, “The contourlet

transform An efficient directional Multi-resolution
image representation”, IEEE Transactions on
Image Processing, vol. 14, No.12, pp. 2091–2106,
2005.

[2] Jyoti Agarwal1, Sarabjeet Singh Bedi,

”Implementation of hybrid image fusion technique
for feature enhancement in medical diagnosis,
Springer, Human centric Computing and
Information Sciences, page 1-17, (2015) 5:3.

[3] S. Yang, M. Wang, L. Jiao, R. Wu, and Z. Wang,

“Image fusion based on a new contourlet packet,”
Inf.Fusion, vol. 11, no. 2, pp. 78–84, 2010.

[4] J. Nunez, X. Otazu, O. Fors, A. Prades, V. Pala

and R. Arbiol, “Multiresolution-based image
fusion with additive wavelet decomposition,”
IEEE Transactions on Geo-science and Remote
sensing, vol. 37, no. 3, 1999,pp. 1204-1211.
[5] Sweta Mehta, Bijith Mara, “CT and MRI image
fusion using curvelet transform,” ISSN: 0975 –
6779, nov 12 to oct 13, volume – 02, issue – 02,
page 848-852.
[6] Navneet kaur, Madhu Bahl, Harsimran Kaur,
“Review On: Image Fusion Using Wavelet and
Curvelet Transform,” IJCSIT, Vol. 5 (2), page
2467-2470, 2014.
[7] R. J. Sapkal, S. M. Kulkarni, “Image Fusion based
on Wavelet Transform for Medical Application,”
IJERA, Vol. 2, Issue 5, September- October 2012,
pp.624-627.
[8] S. Ibrahim and M. Wirth, “Visible and IR Data
Fusion Technique Using the Contourlet
Transform”,International conference on
computational science and engineering, CSE 09,
IEEE, vol. 2, pp. 42-47, 2009.
[9] Miao Qiguang, Wang Baoshul, “A Novel Image
Fusion Method Using Contourlet Transform,”
IEEE trans. geosci. remote sens., vol. 43, no. 6,
pp. 1391-1402, June 2005.
[10] Huang, Junfeng Gao, Zhongsheng Qian “Multi-
focus Image Fusion Using an Effective Discrete
Wavelet Transform Based algorithm
measurement”, SCIENCE REVIEW, Volume 14,
No. 2, 2014,102-108.