Professional Documents
Culture Documents
978-1-4244-5612-3/09/$26.00 2009
c IEEE 553
Authorized licensed use limited to: UNIVERSIDADE DO MINHO. Downloaded on January 29, 2010 at 06:54 from IEEE Xplore. Restrictions apply.
knowledge, these techniques have not been applied to JG to this model, the evolution of the UCS over time is given by
data, namely to prediction of UCS, which is a complex the following expression:
geotechnical engineering problem due to the heterogeneous
(1)
nature of the soil and the large number of the parameters
involved. The adopted values for the coefficients in (1) were: q -
2. MATERIALS AND METHODS strength at age t (MPa); fcm - 28 day strength for each studied
formulation (MPa); s = 0.2 to cement CEM I 42.5R and
2.1 Jet Grouting Data CEM II 42,5R and s = 0.25 to cement CEM IV/A (V) 32.5R;
The dataset includes 175 results of UCS tests, derived and t - age of the respective formulation in days.
from 35 JG laboratory formulations and 9 input parameters,
which are referred as the more influent parameters in UCS TABLE II. SUMMARY OF THE SOIL CLASSIFICATIONS
prediction [17][18]: N of
% % % %
Water/Cement ratio - W/C; Soil Classification
Sand Silt Clay OM
samples
Coefficient related to the cement type (0.25 for CEM prepared
IV 32.5R and 0.20 for CEM I 42.5R and CEM I A Lean clay (CL) 39.0 33.0 27.0 8.3 10
42.5R) - s; B Organic lean clay (OL) 6.0 57.0 37.0 1.8 5
Age of the mixture - t; C Fat clay (CH) 7.0 53.0 40.0 3.2 85
The relation between the mixture porosity and D Silty clay (CL-ML) 25.0 52.5 22.5 0.4 20
volumetric content of cement - n/(Civ)d; E Lean clay (CL) 0.0 55.0 45.0 3.9 15
Percentage of cement of the mixture - %C; F Silty clay (CL-ML) 32.5 43.5 24.0 1.2 20
Percentage of sand - %Sand; G Lean clay (CL) 10.5 48.5 41.0 1.0 20
Percentage of silt - %Silt;
Percentage of clay - %Clay;
Percentage of organic matter - %OM.
The main statistics of the numerical parameters used are
presented in Table I. The dataset compiled resulted of a
study accomplishment in University of Minho with the goal
of analyzing the influence of several parameters in UCS of
JG materials. The soils used in the preparation of JG
laboratory formulations come from seven field works. The
geotechnical soil properties were evaluated using laboratory
tests and the respective soil classifications are presented in
Table II. While all of the soils were classified as inorganic
fine soil, they have different percentages of sand, silt, clay
and organic matter. The cement types used were CEM I
42.5R, CEM II 42.5R and CEM IV/A 32.5R. Fig. 1 plots the Figure 1. The histogram for the UCS of the JG laboratory formulations
histogram of the target variable, showing a log-normal
distribution shape. 2.2 Predictive Models
TABLE I. SYNOPSIS OF THE NUMERICAL PARAMETERS (INPUTS) We apply three different DM techniques to estimate the
UCS of JG material over time, which is a regression task.
Standard We adapted ANN, SVM and FN. Despite of high complexity
Soil Parameter Minimum Maximum Mean
deviation of first two models, it is still possible to extract knowledge in
W/C 0.68 1.12 0.88 0.16 terms of input variable importance [20].
s 0.20 0.25 0.21 0.02 ANN mimics some basic aspects of brain functions [21],
t (days) 3.00 56.00 22.00 19.00 which processes information by means of interaction among
n/(Civ)d 48.83 74.26 62.59 7.26 several neurons [22]. We will adopt the most popular ANN
Clay %C 24.19 73.98 47.44 0.15 model, the multilayer perceptron, which contains only
% Sand 0.00 39.00 13.47 11.54 feedfoward connections, with one hidden layer with H
% Silt 33.00 57.00 50.49 5.49 processing units. A network with H = 0 is equivalent to the
% Clay 22.50 45.00 35.87 7.74 multiple regression model. By increasing H, more complex
% OM 0.40 8.30 2.71 1.81 mappings can be performed, yet an excess value of H will
overt the data, leading to a loss of generality. To overcome
Nowadays, there are several models to estimate the UCS this difficult, a grid search was used to choose
of concrete over time [6][19]. However the EC2 analytical is best H value [9]. We adopted an internal 5-fold cross
the most popular. Thus, despite this model is defined for validation over the training set and then selected the network
concrete material, it was tested for the JG laboratory with the lowest validation error. Next, this ANN was
formulation, due to the similarity of the materials. According
554 2009 World Congress on Nature & Biologically Inspired Computing (NaBIC 2009)
Authorized licensed use limited to: UNIVERSIDADE DO MINHO. Downloaded on January 29, 2010 at 06:54 from IEEE Xplore. Restrictions apply.
retrained with all training data. The general model of the selection of the initial topology is normally based on the
ANN is: properties of the problem at hand, in ANN, several
topologies are considered and one is chosen using an optimal
(2) criterion. The initial topology in functional networks can be
further simplified using functional equations and its neural
where Wj,i represents the weight of the connection from functions can be multidimensional and set during the
neuron j to the unit i (if j=0, then it is a bias connection), o learning phase. FN incorporates different neural functions,
normally functions from a given family, such as polynomial
corresponds to an output unit, f is a logistic function , or exponential, and they are not restricted to be a linear
and I is the number of input neurons. combination of inputs. Furthermore, neurons outputs can be
The SVM was initially proposed for classification tasks connected, which is not the case of standard ANN.
(i.e., to model a discrete labeled output). After the The structure of a FN consists in: a layer of input storing
introduction of the -insensitive loss function, it was possible units; a layer of output storing units; one or several layers of
to apply SVM to regression tasks [23]. SVM has theoretical processing units, which evaluate a set of input values,
advantages over ANN, such as the absence of local minima coming from the previous layer and delivers a set of output
in the learning phase, i.e., the model always converges to the values to the next layer; none, one or several layers of
optimal solution. The main idea of the SVM is to transform intermediate storing units, and a set directed links, that
the input data into a high-dimensional feature space by using connect units in the input or intermediate layers to neuron
a nonlinear mapping . Then, the SVM finds the best units, and neuron units to intermediate or output units.
hyperplane within the feature space. The transformation When working with FN, several steps are necessary to be
depends on the kernel function adopted. The Gaussian kernel set. The first step is to define the initial topology of the
is the most popular one, presenting less parameters than network, based on problem in hand. Next, the architecture
other kernels, and will be adopted in this work: using functional equations and the equivalence concept
(3) needs to be initialized, and then checked the uniqueness
Under this setup, performance of the regression is condition of the desired architecture. Third, using the
affected by three parameters: the parameter of the kernel, available data, the learning procedure (i.e. training
C a penalty parameter, and the width of a -insensitive algorithm) is realized by considering the combinations of
zone. To reduce the search space, the first two values will be linear independent functions, , for all s to
set using the heuristics of [24]: C=3 (for a standardized approximate the neuron functions, that is:
output) and , where
2009 World Congress on Nature & Biologically Inspired Computing (NaBIC 2009) 555
Authorized licensed use limited to: UNIVERSIDADE DO MINHO. Downloaded on January 29, 2010 at 06:54 from IEEE Xplore. Restrictions apply.
The ANN and SVM models were training using the R RMSE = 0.88 and a correlation closely to unit (R2 = 0.91,
statistical tool [29] and the RMiner library [30], which Fig. 3). However, despite its good accuracy, to apply
facilitate the application of DM techniques in R and the successful EC2 analytical model it is necessary to know the
formulation and resolution of the FN implemented in the free UCS of each formulation over the 28 day time-frame of the
version of the GAMS [31]. cure, which implies waiting and carry out laboratory tests to
determine the UCS at this age. Furthermore, an inadequate
2.3 Evaluation value of fcm leads to significant errors in the predictions.
In order to evaluate the performance of the prediction A high performance was obtained for both ANN and
models and choose the best model, we consider three SVM. Despite ANN presented the best performance (Fig. 3),
commonly used evaluative measures: Mean Absolute Error it was not considered further since the importance attributed
(MAD), Root Mean Absolute Error (RMSE) and Coefficient by this model to each parameter is not in agreement with the
of determination (R2): empirical knowledge of JG technology (Fig. 4). According to
SVM, we can say that the key parameters to predict the UCS
(7) of JG over time are: the time of cure, the percentage of
cement of the mixture and the relation n/(CIV)d. It can also be
verified that the properties of soil, rather the amount of
organic matter and silt, are also relevant to evaluate the
behavior of the soil cement mixtures over time. When
(8)
comparing SVM and EC2, the former is better, as shown in
Fig. 3. In contrast with EC2, the SVM allows prediction of
the UCS in the initial stages of a project. However, despite
this advantage and its high predictive accuracy, the SVM
model is very complex and difficult to understand by
(9)
humans.
After we train the FN, from (5), using the Leave-On-Out
where denotes the desired value, the predicted value, estimation method and the minimization problem (6), we
and represent the mean of these variables. Lower values of obtain the performance shown in Fig. 2, which represents the
MAD and RMSE correspond to a higher predictive capacity, relation between the measures and predicted Jet Grouting
while the R2 should be close to the unit value. If it is UCS values, given by the expression:
acceptable to commit a few extreme errors, then the MAD is
the most adequate measure; else the RMSE measure should
be used. (10)
After fitting of a model, it becomes necessary to verify its
future performance, i.e. to measure the generalization
The high performance obtained is corroborated by the MAD,
capability in unseen data. We adopted the Leave-One-Out
RMSE and R2 error metrics (see Fig. 3). The values of the
scheme for such purpose. This method is especially suited
two first are 0.58 MPa and 0,75MPa respectively, which are
when the dataset is small (e.g. lower than 100 examples).
small values and the correlation presents a value of 0.92,
Under Leave-One-Out, sequentially one example is used to
which is very close to the unit value (the perfect prediction).
test the model and the remaining data is used for tting the
Moreover, as can be seen in Fig. 2, the number of predictions
model. Under this scheme, all data is used for training and
above of ideal prediction (black line in Fig. 2) is
testing. Yet, this method requires around N (the number of
approximately equal to number of all predictions below the
data samples) times more computation, since N models are
same line, revealing that the model does not underestimate or
tted. The final generalization estimate is evaluated by
overestimates the predictions. The importance of each FN
computing the MAD, RMSE and R2 metrics for all N test
parameter was also computed using a sensitivity analysis
samples.
procedure [20]. Similar to the SVM model, the amount of
To understand better the behaviour of the JG material,
cement and the relation n/(Civ)d are the most important
the influence of each parameter was quantified. To do this,
parameters in prediction of the UCS JG laboratory
we applied sensitivity analysis procedure [20][17]. This
formulation. It is still possible observe that the properties of
process determines the most important variables in a system
soil, rather the amount of organic matter and silt, are also
by successively holding all but one input constant and
relevant (as in SVM model).
varying the other over its range of values to observe its effect
When compared to SVM, the FN model is more
on the system. A high variance observed in the outputs
interesting since its mathematical expression is simple and
denotes a high input relevance.
understandable, while the obtained predictive accuracy is
3. RESULTS still high. In addition, we can yet apply FN model in the
initial stage of a project, what is impossible with EC2
The quality of the predictions obtained by applying the analytical model. Table III summarizes the advantages and
EC2 analytical model confirms the good performance of the disadvantages of the proposed approach.
predictions, presenting small values of MAD = 0.60 and
556 2009 World Congress on Nature & Biologically Inspired Computing (NaBIC 2009)
Authorized licensed use limited to: UNIVERSIDADE DO MINHO. Downloaded on January 29, 2010 at 06:54 from IEEE Xplore. Restrictions apply.
TABLE III. COMPARISSON OF THE PREDICTIVE MODELS ACKNOWLEDGMENTS
Model R2
Applied in Easy to interpret The authors would like to thank the interest and financial
project stage by humans support by Tecnasol-FGE and Tiago Valente for the dataset
EC2 0.91 gathered from the laboratory formulations.
ANN 0.94
SVM 0.93 14
0.94
0.93
0.92
0.91
formulations were also identified: the amount of cement and
0.88
Error - MAD and RMSE
1 1
the relation n/(Civ)d and the properties of soil. It should be
0.75
0.73
0.67
stressed that tested models are only valid for the conditions 0.8 0.8
0.60
0.58
0.54
Error - R2
0.48
found in the dataset used in this study (e.g., clay soil type). (MPa) 0.6 0.6
The best solution, the FN model, can give a valuable
contribution in terms of improving the construction process 0.4 0.4
of JG columns and reducing the costs of laboratory 0.2 0.2
formulations. In face of the results obtained, we intent apply
similar Data Mining (DM) techniques, to predict the Young 0 0
modulus of JG laboratory formulations, as well as to MAD RMSE R2
estimate the final diameter of JG columns and its mechanical Performance measures
properties (UCS and Young modulus).
Figure 3. Comparison of the performance between the four models: EC2,
SVM, ANN and FN
2.93%
n/(Civ)^d 19.96%
30.30%
12.15%
%C 23.73%
33.09% ANN
1.95%
t 4.25%
28.78%
1.08%
s 7.62%
Parameter
2.34%
0.38%
W/C 3.76%
12.05% SVM
2.56%
%OM 3.61%
8.38%
25.90%
%Clay 0.25%
0.93%
4.65%
%Silt 2.80%
16.78%
FN
48.41%
%Sand 1.20%
0.15%
Importance
Figure 4. Comparison of the importance of each parameter in the four models: EC2 analytical model, SVM, ANN and FN models
2009 World Congress on Nature & Biologically Inspired Computing (NaBIC 2009) 557
Authorized licensed use limited to: UNIVERSIDADE DO MINHO. Downloaded on January 29, 2010 at 06:54 from IEEE Xplore. Restrictions apply.
REFERENCES [16] Lai S., Serra M. Concrete Strength Prediction by means of Neural
Network. Constrruct Build Mater (1997); 11(2), pp. 93-98.
[1] GmbH, Keller Holding. The soilcret - Jet Grouting Process. Keller
Publications. 2005. [17] Shibazaki, Mitsuhiro. State of Pratice of Jet Grouting. Grouting and
Ground Treatment. New Orleans (2003), pp.198-217.
[2] Falco, Joo F., Alexandre L. Pinto, and Francisco D. Pinto. Cases
histories of ground improvement solutions using jet-grouting. 2000. [18] Gazzarrini, Paolo, Matt Kokan, e Stephen Jungaro. Case History of
Jet Grouting in British Columbia. Underpinning of CN Rail Tunnel in
[3] Padura, A. Barrios, J. Barrios Sevilla, and J. Garca Navarro. Study North Vancouver. The Grout Line (2005), pp. 47-54.
of the soil consolidation using reinforced jet grouting by geophysical
and geotechnical techniques: "La Normal" building complex [19] Jalali, Sad, e L.M.V. Santos. Early Quality Control and Early
Formwork Removal whith setting Time. 1996.
(Granada). Construction and Building Materials (2008),
doi:10.1016/j.conbuildmat.2008.07.011 [20] Kewley, R., Embrechts, M and Breneman, C. Data Strip Mining for
the Virtual Design of Pharmaceuticals with Neural Networks. IEEE
[4] Croce, P., and A. Flora. Analysis of single-fluid jet grouting.
Transactions on Neural Networks, 11(3), 2000, pp.668679.
Gotechnique ,Vol. 51 (2001), pp. 905 - 906.
[5] Kirsh, Fabian, and wolfgang Sondermann. Ground Improvement and [21] Kenig, S., Ben-David, A., Orner, M., and Sadeh, A. 2001. Control of
its Numerical Analysis. Istanbul, 2001. properties in injection molding by neural networks. Engineering
Applications of Artificial Intelligence, 14, pp.819-823. doi: J O.
[6] Normalisation, Comit Europen the. Eurocode 2: Design of concret 1016IS0952-1976(02)00006-4.
structures - Part 1-1: General rules and rules for buildings. Bruxelas:
CEN, 2004. [22] Perzyk, M., and Kochanski, A.W. 2001. Prediction of ductile cast
iron quality by artificial neural Networks. Journal of Materials
[7] D. Rumelhart, G. Hinton, and R. Williams. Learning Internal Processing Technology, 109, pp.305-307. doi: 10. 10 16/S0924-
Representations by Error Propagation. In D. Rulmelhart and J. 0136(00)00822-0.
McClelland, editors, Parallel Distributed Processing: Explorations in
the Microstructures of Cognition, vol. 1, pp.318-362, MIT Press, [23] Smola A. and Scholkopf B. A tutorial on support vector regression.
Cambridge MA, 1986. Statistics and Computing (2004), pp.14:199222.
[8] C. Cortes and V. Vapnik. Support Vector Networks. Machine [24] V. Cherkassy and Y. Ma. Practical Selection of SVM Parameters
Learning, 20(3):273297, 1995. and Noise Estimation for SVM Regression. Neural Networks, 17(1),
pp. 113126, 2004.
[9] T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical
Learning: Data Mining, Inference, and Prediction. Springer-Verlag, [25] Castillo, E., A. Cobo, J. M. Gutirrez, and E. Pruneda. Functional
NY, USA, 2001. Networks with Applications: a neural-based paradigm. Springer,
1998.
[10] Castillo, Enrique, Jos Manuel Gutirrez, Ali S. Hadi, and Beatriz
[26] El-Sebakhy, Emad A., Kanaan Abed Faisal, T. Helmy, F. Azzedin,
Lacruz. Some Applications of Functional Networks in Statistics and
Engineering. Technometrics 43 (2001), pp.10-24. and A. Al-Suhaim. Evaluation of breast cancer tumor classification
with unconstrained functional networks classifier. In Proceeding of
[11] El-Sebakhy, Emad A. Software reliability identification using the 4th ACS/IEEE international conference on computer. (2006),
functional networks: A comparative study. Expert systems with pp.281-287.
applications 35 (2009), pp. 4013-4020.
[27] Li, Chunguang, Xiaofeng Liao, Zhongfu Wu, and Juebang Yu.
[12] Castillo, Enrique, Ali S. Hadi, Beatriz Lacruz, and Rosa E. Pruneda. Complex functional networks. Mathematics and Computers in
Semi-parametric nonlinear regression and transformation using Simulation (2001), pp.355-265.
functional network. Computational Statistics & Data Analysis, 2008,
pp. 2129-2157. [28] Zhou, Yong Quan, Deng Xu He, and Zheng Nong. Problems,
Application of Functional Network to Solving Classification. World
[13] Betanzos, Amparo Alonso, Enrique Castillo, Fontenla Oscar Romero, Academy of Science, Engineering and Technology. 2005. 390-393.
and Noelia Snchez Marono. Shear Strength Prediction using
Dimensional Analysis and Functional Networks. European [29] R Development Core Team. R: A language and environment for
Symposium on Artificial Neural Networks. Bruges (Belgium), 2004, statistical computing. R Foundation for Statistical Computing,
pp.251-256. Vienna, Austria, ISBN 3-900051-00-3, http://www.R-project.org,
(Accessed July 27, 2009).
[14] Rezania, Mohammad, and Akbar A. Javadi. A new genetic
programming model for predicting settlement of shallow [30] Cortez, Paulo. RMiner: Data Mining with Neural Networks and
foundations. Canadian Geotechnical, December 2007, pp.1462- Support Vector Machines using R. In R. Ra jesh (Ed.), Introduction
1473. to Advanced Scientic Softwares and Toolboxes, In press.
[15] Yusuf E. Artificial Neural Networks Approach for Swell Pressure [31] GAMS, Development Corporation. On-line Documentation.
versus Soil Suction Behavior. Canadian Geotechnical, 2007, Welcome to the GAMS Home Page. Juny 15, 2009.
pp.1215-1223. http://www.gams.com/docs/document.htm (accessed Juny 27, 2009)
558 2009 World Congress on Nature & Biologically Inspired Computing (NaBIC 2009)
Authorized licensed use limited to: UNIVERSIDADE DO MINHO. Downloaded on January 29, 2010 at 06:54 from IEEE Xplore. Restrictions apply.