MAXIMUM LIKELIHOOD APPROACH TO STATE ESTIMATION IN ONLINE PIPELINE

MODELS
Jason P. Modisette
ATMOS International, Inc
3100 E. Miraloma Ave. Suite D
Anaheim, California 92806
Email: jpmodisette@gmail.com
ABSTRACT
Hydraulic models of pipelines driven by SCADA usually
have many more pressure and ow measurements available that
are needed as boundary conditions of the underlying model.
These extra measurements can be used to improve the estimate
of the true state of the pipeline and to give the hydraulic model
some resistance to measurement noise. The process of merging
the available SCADA into a form the model can use is called
state estimation.
A new technique for state estimation based on a maximum-
likelihood estimate of the state of the pipeline constrained by the
underlying physics will be presented. This method, which will
be referred to as Maximum Likelihood State Estimation (MLSE),
will be justied by comparison with two traditional methods of
state estimation in use in the industry, the approach of break-
ing the pipeline up into multiple models with pressure bound-
aries and then rectifying the ows, and the Equal Error Fractions
(EEF) method of Van der Hoeven [1].
An example of the application of this approach to a real-
world pipeline will also be presented, with performance data as
well as some of the deciencies that were found and how they
were corrected.
1 Introduction
A hydraulic model of a single pipe solves two coupled rst-
order nonlinear PDEs enforcing the conservation of mass and
the conservation of momentum. Regardless of the solution tech-
nique, this system of PDEs requires the initial state of the entire
system, i.e. the values of pressure and ow at every point in
the pipeline, to be specied as a boundary condition at the start
of the simulation, and further requires either a pressure value, a
ow value, or something equivalent, be specied at the two ends
of the pipe for every subsequent time.
The upstream and downstream boundary values are usually
given as pressure and/or ow rate because these are the mea-
surements that are usually available. The initial state is typically
either provided by making a guess and then letting the model cor-
rect it by running for a while in steady conditions, or by explicitly
solving the steady state. In an online model, i.e. one which tries
to model the current state of an existing pipeline, these boundary
conditions are typically drawn from pressure and ow measure-
ments. However, a real pipeline will have many more pressure
and ow measurements in the system than it needs for use as
boundary conditions. The extra measurements could just be
ignored, but the accuracy of the model can be improved by mak-
ing use of all available measurements; this process is called state
estimation.
Note that the coupled PDEs only have enough degrees of
freedom to admit two boundary conditions, so it is not possi-
ble to simply use all of the available measurements directly as
boundary conditions. Thus any state estimation technique must
either apply the extra information from the additional measure-
ments via some sort of process external to the PDE solution, or
must effectively come up with some form of the boundary con-
ditions that somehow include the effects of the extra meters.
A successful state estimation technique should make an ac-
curate estimate of the true state of the system when all meters
1 Copyright c 2012 by ASME
Proceedings of the 2012 9th International Pipeline Conference
IPC2012
September 24-28, 2012, Calgary, Alberta, Canada
IPC2012-90664
are functioning properly. It should also produce as little error
as possible in conditions of data loss or uncorrected erroneous
boundary condition measurements; and should ideally require
minimal user conguration beyond that required to congure the
underlying pipeline model. While the accuracy of the estimated
state of the pipeline can be measured using simulated data, the
other requirements are pretty fuzzy. It would also be appealing,
although not strictly necessary, if the differences between esti-
mated and real states of the system had the same form as the
errors that would be produced by erroneous meter values, rather
than violating basic physics: its easier to analyze the results of
the model if they can be relied on the conserve mass, etc.
In this article we will only consider semi-implicit nite dif-
ference models for solving the hydraulic PDEs and steady-state
nite difference models for solving the steady state ODEs. The
results presented herein should translate to other solution tech-
niques that are based on simultaneous solution of a system of
equations, such as other sorts of nite difference models. All of
the results presented here for articial pipelines use isothermal
models, to avoid unnecessarily complicating the analysis. The
results for real pipelines use coupled thermal models. Experi-
ence has shown that the actions of the hydraulic state estimation
are independent of the details of the thermal model. The possi-
bility of state estimation in the thermal model itself will also be
addressed.
There is no industry-wide accepted standard method for
state estimation; different model vendors use different tech-
niques. The objective of this paper is to describe and demonstrate
a new simple and robust state estimation technique called Max-
imum Likelihood State Estimation (MLSE). MLSE replaces the
full set of PDEs and associated boundary conditions with an op-
timization problem in which the PDEs are treated as constraints
and the objective function being optimized effectively replaces
the upstream and downstream boundary conditions during the
model run. This technique has been successfully used on a large
subsea gas pipeline network for several years; these results will
demonstrate the usefulness and effectiveness of the technique.
During this period the interaction of the MLSE technique with
model tuning was explored, and it was found that it was neces-
sary to adjust the automatic tuning techniques to accommodate
MLSE. These required changes will also be explained.
2 Existing State Estimation Techniques
The MLSE technique is the latest of several state estimation
techniques used in the pipeline simulation industry in conjunc-
tion with nite difference models. In this section we will describe
the two most popular existing techniques, the Pressure-Pressure
method and the Equal Error Fractions (EEF) method, and enu-
merate some of the deciencies of each. In the next section we
will describe the new MLSE method and how it corrects these
deciencies.
The performance of the three techniques, pressure-pressure,
EEF, and MLSE, was compared for artical gas and liquid
pipelines in a previous work [2], in which it was found that the
MLSE method was resistant to the deciencies described below.
2.1 The Pressure-Pressure Method
Assuming one already has a functioning ofine model, the
easiest way to performsome sort of state estimation is to run that
using some subset of the available measurements as boundary
conditions, and then after each model step is complete, somehow
integrate the remaining measurements into an estimate of the ac-
tual state. This has the benet that the actual model code doesnt
need to be modied at all from its ofine form; the additional
measurements are included after the fact.
If the model is run with pressure boundary conditions, then
the output of the model will include ow rates at the two ends
of the pipe. These can then be compared with the ow meter
readings; if the ow meters are reading higher than the model,
this suggests that the actual ow rate is higher than the modeled
ow rate, and vice versa. Given estimates of the errors in the
ow meter readings and the errors in the modeled ows, we can
then come up with an overall best guess of the ow in the system.
If we call the modeled ow at location i Q
i
and the ow meter
reading at meter j Q
j
, then we can solve for a consensus ow
at

Q
j
that minimizes the weighted sum
pipes

Q
i
Q
i

2
+
meters

Q
j
Q
j

Q
j

2
(1)
in which the rst summation represents the difference be-
tween the consensus ows and the ows calculated from the
pressure-pressure model at each pipe end, and the second term
represents the difference between the consensus ows and the
ow meter values.
Minimizing this sum with respect to the vector of consen-
sus ows

Q
i
produces a system of linear equations which can
be solved exactly on each time step. In a network, we can ap-
ply some additional knowledge here: for instance, that at a split
the consensus ows must add up to zero. This sort of thing can
be included in the above optimization problem using the method
of Lagrange Multipliers, which leaves the problem still exactly
solvable. This consensus ow can then be taken to be the real
ow since, unlike the model ow Q
i
, it conserves mass at junc-
tions between model regions. This property is necessary if we
are to use the calculated ows for composition tracking, which is
a common feature required of online models.
Note that one could also implement this method using ow
boundary conditions, but since a steady state model cant be
solved purely on ows that would require some special treatment
2 Copyright c 2012 by ASME
for steady state. In addition, if the ow meters dont add up to a
net inow of zero, the model will eventually either drain or blow
up the pipe unless some preprocessing is applied to force them to
sum to zero. Thus the author has not encountered this approach
except using pressure boundary conditions.
The pressure-pressure method is simple but has some po-
tential problems: if theres a small mid-line delivery (small
compared to the mainline ow rate), the hydraulic model will
be much more stable if a ow boundary condition is used there:
errors in the ow will cause smaller errors in the pressure, but er-
rors in the pressure in the event a pressure boundary is used will
cause large errors in the ow. However, if the model is run with
a ow boundary at some points, theres no direct way to use any
pressure data that may be available there in this scheme.
In the case of a pipeline (or section) operating at a small frac-
tion of capacity, the errors in the pressure meters can be signi-
cant compared to the total pressure drop between meters, mean-
ing that the ows calculated by a pressure-pressure model may
be complete nonsense.
Yet another problem with this method is that it requires the
user to come up with a relative weighting of the accuracy of the
ow meters as compared to the accuracy of the ows computed
by the model driven by pressure meters; this latter term has to in-
clude both the errors arising fromthe pressure meters themselves
and those arising from unknown or incorrectly known character-
istics of the pipeline system. This can be quite difcult to esti-
mate; it the authors experience it is often estimated by guess-
work.
The consensus ows calculated by this method may ex-
hibit some strange properties if the ow meters are out of cali-
bration, that is, if for instance the ow meter at the origin is read-
ing high but the terminus ow meter is accurate. In that case the
consensus ows can show a continuous packing of the line that
goes on forever. This can be a problem because in a complex
pipe network that has been split up into many pressure-meter-
bounded regions, the actual model solution ows are not going
to be conserved at mid-line pressure meters, such as those at me-
tered pump or compressor stations. If the model is being used
to drive a batch tracker, it will generally be more appropriate to
use the consensus ows for that since they can be generated in
a manner that ensures they are conserved where they should be;
however, in that case something will have to be done about this
possible eternal unpacking. So even though the pressure-pressure
method is initially easy to implement, it can lead to some addi-
tional complexities.
Finally, it is not aesthetically pleasing to the author (or, in
the authors experience, to the users of the software) that the ow
rates computed by this approach do not correspond to the com-
puted pressure drops unless the system is perfectly tuned up.
Simple hydraulic tuning under the pressure-pressure method
can be done by examining the differences between the consen-
sus ows and the modeled ows in the pipes; if the consensus
ows are higher than the modeled ows the pipes resistance to
ow can be reduced a bit, and vice versa. This would correct
for a steady state error in the pipes resistance if that error were
constant across all ows; or instead of the resistance, the pipe
roughness or diameter or some other property could be adjusted.
Thus the pressure-pressure method of state estimation, while
simple to implement, has the undesirable properties that it can be
unstable in the presence of small mid-line deliveries, and that the
consensus ows which are its best estimate of the true ow
in the system behave badly in the presence of imperfectly tuned
ow meters. MLSE, as we will see in the next section, does not
have either of these problems.
2.2 The Equal Error Fractions Method
Unlike the pressure-pressure method, the Equal Error Frac-
tions or EEF approach allows co-located pressure and ow mea-
surements to be used on the same footing.
If a pressure and ow meter are located in the same spot,
then there is only one possible set of values for the modeled pres-
sure and ow rate such that the modeled pressure disagrees with
the measured pressure exactly as much as the modeled ow rate
disagrees with the measured ow rate. Its compelling from a
physical standpoint to only treat meters in the same spot on the
same footing (as opposed to treating meters all over the pipeline
on the same footing, as is done in the maximum likelihood ap-
proach) because the meters at the two ends of a pipe really arent
interchangeable: if a very accurate pressure and ow meter are
located at the upstream end and a very inaccurate pressure and
ow meter are located at the downstream end, the model still
requires a downstream boundary condition, so the less accurate
meters must be used.
In the case of a single pipe with pressure and ow meters at
either end, the EEF approach has the boundary conditions:
P
i
P
i

P
i
=
Q
i
Q
i

Q
i
(2)
at the upstream end and
P
i
P
i

P
i
=
Q
i
Q
i

Q
i
(3)
at the downstreamend. Here we use the same notation as be-
fore for modeled and measured pressures and ows.
P
i
and
Q
i
represent errors in the measurements, so that as these values
become large that measurement is given relatively less weight
and the other measurement at that location is favored.
3 Copyright c 2012 by ASME
Qualitatively, EEF is somewhat more complicated to imple-
ment than the PP method but really it just requires the replacing
of pressure or ow boundary conditions in the model with the
special EEF boundary condition above.
However, for more complicated pipelines and congurations
of meters, EEF requires that some different boundary conditions
be derived in a customized manner: a mid-line delivery, a mid-
line compressor or pump station with suction and discharge pres-
sure metering and ow metering, etc., are all treated differently.
This can make EEF difcult to implement for arbitrary congu-
rations of meters.
Hydraulic tuning can be performed under the EEF method
by dening regions of pipes bounded by pressure and owmeters
and evenly distributing a eld of pressures and ows in some
manner that agrees as well as possible with the measurements;
the differences in each pipe between these pressures and ows
and the model solution can then be used to drive tuning.
The MLSE method provides similar robustness and accu-
racy to the EEF method for cases of a straight pipe network [2],
while not requiring special mathematics for each new topology
of pressure and ow meters and devices in the way EEF does, but
instead automatically providing state estimation for any topology
in a manner guaranteed to be a best estimate (as long as instru-
ment errors are normally distributed). This will be demonstrated
in the next section.
3 Maximum Likelihood Method
The maximum-likelihood method is a way of treating all of
the pressure and ow measurements in the system on the same
footing, while still enforcing the physics of the model. This
method has the nice property that one set of equations will work
for any pipeline conguration: there are no custom cases re-
quired for special congurations of ow or pressure meters like
there are in EEF. The maximum-likelihood method produces the
most likely estimate of the state of the pipeline under the assump-
tions that (1) the errors in the measurements are all normally dis-
tributed with standard deviation
P
i
and
Q
i
, (2) the underlying
pipe model captures the physics correctly, and (3) the errors in
the measurements are not correlated between time steps. In this
section we will describe the method and show how it corrects the
deciencies of the EEF and pressure-pressure methods described
above.
In this approach, the normal model equations other than the
boundary conditions are treated as constraints on the minimiza-
tion of the quantity

P
i
P
i

P
i

2
+

Q
j
Q
j

Q
j

2
(4)
using the method of Lagrange multipliers. The minimized quan-
tity, called the objective function, consists of a term representing
the net sum-of-squares error between the modeled and measured
pressures and one for the modeled and measured ows, weighted
by the respective meter errors. So if there are N model equations
which can be written as f ({P, Q}) = 0, then the full minimized
function becomes

P
i
P
i

P
i

2
+

Q
j
Q
j

Q
j

2
+

k
f
k
({P, Q}) (5)
which is minimized with respect to the Ps, Qs, and s. The
new term enforces the model PDEs and other model equations -
excepting the boundary conditions, which have effectively been
replaced by the original objective function - as per the method of
Lagrange Multipliers. Note that if the original model equations
are linear in the Ps and Qs (or, as is the case with our implicit
model, linearized in the Ps and Qs) then this function is quadratic
in all the variables, and so can be minimized exactly by solving
a linear system.
Unlike in the pressure-pressure method, having ow meter
drift does not present any problems for the maximum-likelihood
approach. Although initially the model may pack or unpack the
line a bit trying to honor the miscorrelated meters, it will reach a
point where any further unpacking (for example) will cause the
pressures to drift too far from the measured pressures, and so it
will end up accepting an error in the ow meters in order to keep
the linepack at a reasonable value. This will be seen below. On
the other hand, this method does not gain any benet in its state
estimation from the known fact that over a long period the net
ow in or out of the line has to be, on average, zero.
Maximum-likelihood results can be used to drive tuning in
exactly the same manner as the EEF results can, as was described
in the previous section.
All of the methods presented here exhibit a common weak-
ness: they treat each model solution time step in isolation from
all the other steps. This is a problembecause it throws away some
information: how the errors the model is seeing are correlated
from step to step. This information might be useful in tuning,
specically in distinguishing one type of error (say, a pressure
meter that has drifted) from another (say, a gas pipe which has
some liquid holdup and thus higher resistance to ow than ex-
pected.) However, using it is beyond the scope of the current
work.
4 Behavior of a Model with MLSE
Next we will examine MLSEs results in several common
circumstances seen on pipelines, with the goal of demonstrating
that it produces the best possible results.
4 Copyright c 2012 by ASME
4.1 Balancing Two Differing Measurements of the
Same Pressure
It often happens that two or more pressure meters are effec-
tively measuring the same pressure. MLSE automatically com-
bines the measurements into a single more accurate measure-
ment. This can be most easily seen by considering a model of a
single point with two pressure measurements reading P
1
and P
2
respectively and nothing else, which solves for the actual pres-
sure P at that point. If the errors on the two meters are
P
1
and

P
2
, then the objective function is
O.F. =

PP
1

P
1

2
+

PP
2

P
2

2
(6)
which has an extremum (a minimum) at P = P
1
+ (1 )P
2
where
=
(
P
2
)
2
(
P
1
)
2
+(
P
2
)
2
(7)
If the two errors on the pressure meter are equal, this clearly pro-
duces the desired result of simply averaging them, and in general
if the errors are normally distributed then the overall likelihood
of pressure P is proportional to the exponential of negative Equa-
tion 6, so minimizing that function maximizes the probability,
and this Equation 7 gives the most likely value for P.
This result can be declared with what an EEF-like approach
would give for P in this simple system:
PP
1

P
1
=
P
2
P

P
2
(8)
the solution of which is P =P
1
+(1 )P
2
where
=

P
2

P
1
+
P
2
(9)
This is not the most likely value for P given normally distributed
errors, but it does have the same behavior in the case that the two
errors are equal, and approximately the same behavior if one of
the errors is much larger than the other.
4.2 Balancing a Pressure and a Flow Measurement at
an Inlet or Outlet
If we consider a simple point-to-point pipeline with both
pressure and ow measurements at both the inlet and outlet,
-80
-60
-40
-20
0
20
40
60
80
0 2000 4000 6000 8000 10000 12000
M
e
t
e
r

e
r
r
o
r

(
n
o
m
in
a
l
s
t
a
n
d
a
r
d

d
e
v
ia
t
io
n
s
)
Elapsed time (s)
downstream pressure error
downstream flow error
0
FIGURE 1. Downstream-end pressure and ow differences between
model and measured values for a simple articial gas pipeline using
generated data
the solution of the steady-state or transient pipeline model cou-
pled with MLSE will result in some tradeoff between the av-
erage measured ow and the ow rate that would be produced
by the given pipe and uid properties at the measured pressure
drop. Unless the model is congured to perfectly represent the
pipeline, which never happens in practice, there will be some
disagreement between the modeled and measured ows and pres-
sures.
Generally, at least during small transients, experience has
shown that the ratio of ow offset to pressure offset for colo-
cated meters remains pretty constant. This can be seen in Figure
1 for a the ow and pressures at one end of a gas pipeline un-
dergoing continuous small transients; this behavior is typical (in
the absence of hydraulic tuning, which will be discussed further
below.)
If an online model using MLSE is initialized from a steady-
state solution using MLSE, there can be a signicant startup tran-
sient even if nothing is changing in the system. This is discussed
in detail below.
4.3 Handling a Lost Pressure Measurement
Ideally there will either be a status coming from SCADA or
some preprocessing that checks bounds and rate-of-change that
will identify when a pressure meter has gone bad, so the model
wont try to follow it. In these circumstances the MLSE algo-
rithm would ideally continue to make the best estimate of the
true state of the line, meaning that it wont create any sudden
transients if the other meters in the area dont suggest there are
any sudden transients.
If the term involving the bad meter is simply dropped from
the objective function, or if its error is instantaneously increased
to a large number, this can result in a ow surge around the meter
as the transient model tries to better match other meters in the
5 Copyright c 2012 by ASME
1.3e+06
1.4e+06
1.5e+06
1.6e+06
1.7e+06
1.8e+06
1.9e+06
2e+06
10600 10800 11000 11200 11400 11600 11800 12000
P
r
e
s
s
u
r
e

(
P
a
)
Elapsed time (s)
downstream model pressure (sudden loss)
downstream model pressure (gradual loss)
downstream measured pressure
FIGURE 2. Downstream-end measured pressure when a pressure me-
ter goes bad. The metered value, the model result when the meter goes
away suddenly, and the model result when the meter weight is relaxed
gradually are shown.
3.48
3.5
3.52
3.54
3.56
3.58
3.6
3.62
3.64
3.66
3.68
10600 10800 11000 11200 11400 11600 11800 12000
S
t
a
n
d
a
r
d

f
lo
w

(
m
^
3
/
s
)
Elapsed time (s)
downstream model flow (sudden loss)
downstream model flow (gradual loss)
downstream measured flow
FIGURE 3. Downstream-end measured owrate when a pressure me-
ter goes bad. The metered value, the model result when the meter goes
away suddenly, and the model result when the meter weight is relaxed
gradually are shown.
region. However, as we see in Figures 2 and 3, the behavior of the
system with a gradual reduction in meter weight by increasing

P
i
over an hour (the dashed plots) is only marginally smoother
than what happens with instantaneous loss of the meter (the solid
plots). The situation when a ow meter is lost suddenly is much
more severe, as we will see in the next section.
4.4 Handling a Lost Flow Measurement
This is handled by MLSE in a similar manner to the loss of
a pressure meter. If the ow meter is collocated with a pressure
meter, the weight is slowly relaxed to zero to avoid a surge. The
results of a sudden removal of the ow meter from the objective
function as compared to a 10-minute slow relaxation is shown
1.46e+06
1.48e+06
1.5e+06
1.52e+06
1.54e+06
1.56e+06
1.58e+06
1.6e+06
10600 10800 11000 11200 11400 11600 11800 12000
P
r
e
s
s
u
r
e

(
P
a
)
Elapsed time (s)
downstream model pressure (sudden loss)
downstream model pressure (gradual loss)
downstream measured pressure
FIGURE 4. Downstream-end pressure when a ow meter goes bad.
The measured value, the model result when the meter goes away sud-
denly, and the model result when the meter weight is relaxed gradually
are shown.
3.6
3.7
3.8
3.9
4
4.1
4.2
4.3
10600 10800 11000 11200 11400 11600 11800 12000
S
t
a
n
d
a
r
d

f
lo
w

(
m
^
3
/
s
)
Elapsed time (s)
downstream model flow (sudden loss)
downstream model flow (gradual loss)
FIGURE 5. Downstream-end ow rate when a ow meter goes bad.
The model result when the meter goes away suddenly and the model
result when the meter weight is relaxed gradually are shown.
in Figure 4 (downstream pressures) and Figure 5 (downstream
ows), for the same simple liquid pipeline with generated data
that was used in the previous example.
In this case the ow surge caused by the sudden loss of
a ow meter over one step is very visible, causing the ow to
briey shoot up by 20% and also causing some associated model
instability (the solid line in Figure 5). This problem goes away
when the ow meter is turned off gradually over 10 minutes (the
dashed line in Figure 5).
4.5 Speed
Model speed is very important to the user, so it is vital that
the state estimation technique not signicantly slow down the
model. Neither the pressure-pressure nor EEF techniques pro-
6 Copyright c 2012 by ASME
vide any signicant slowdown as compared to an ofine model,
and in this section we will demonstrate that MLSE doesnt slow
down the model either.
MLSE roughly doubles the size of the model solution matrix
as compared to an ofine model of the same system. The sys-
tem goes from N constraint equations representing physical laws
(and a few boundary conditions) and N variables in an ofine
model to the same N variables plus approximately N more La-
grange multipliers for what were formally N constraints. There
will generally be slightly fewer constraints in the online model
than there were in the ofine model of the same system since the
boundary conditions are not needed as constraints, and neither
are equations for any pumps, compressors, or regulators, but the
boundary conditions are always going to be a tiny minority of
the constraint equations on any real-world pipeline, so there will
be about 2N variables in the MLSE online model of a pipeline
whose ofine model had N variables. Since the model equations
for the MLSE system are derivatives of the objective function
with respect to those 2N variables, there are now 2N equations
as well.
The matrices produced by discretized solutions of nite dif-
ference pipeline systems are very sparse, with a only O(N) of the
elements lled, and they can therefore be inverted with sparse
matrix techniques in O(N) time. The matrix elements are either
trivial derivatives of the objective-function terms in Equation 5
that draw the pressures and ows at meters towards their mea-
sured values, or are members of the same set of matrix elements
that appear in an ofine model of the same system. In this au-
thors experience, nite difference pipeline models spend most
of their CPU cycles generating matrix elements - thats where
all of the equations of state, etc., come in - and so an online
model using MLSE will complete one timestep in about the same
time it takes an ofine model of the same system to complete
one timestep. MLSE itself doesnt end up requiring a noticeable
amount of overhead either in time or storage.
Performing a direct comparison of an online and ofine
model using live data is problematic because theres no ofine
that will be running exactly the same scenario. To compare the
speed of MLSE with an ofine run using traditional boundary
conditions, we ran an ofine liquid simulation of a shutdown and
startup scenario on a real-world transmission line with 4 pump
stations and 550 km of pipe. We used the results of this model
to generate simulated data for the lines 42 pressure meters and
21 ow meters and reran the simulation using MLSE and those
meters. We forced both the online and ofine model to use the
same xed 10-second time step, ensuring that the two runs were
simulating the same thing to the greatest degree possible. The
ofine model completed 3 hours of simulation (1080 transient
model steps plus an initial steady-state solution) in 1100.9 sec-
onds, while on the same computer hardware the online run of
the same system completed the same steps in 1233.9 seconds.
(In real-world usage the ofine model would generally be able
to take longer and thus fewer time steps when nothing was hap-
pening in the line, while the online model would still have to
take a time step for every SCADA scan, so we would expect the
ofine model performance to be better than seen in this com-
parison.) This shows that MLSE doesnt produce a signicant
performance degradation over the performance of the equivalent
ofine model with no state estimation.
5 State Estimation of Temperature
In addition to the hydraulic equations, most pipeline models
also solve a PDE enforcing conservation of energy for temper-
atures in the line. The energy part of the model can be solved
simultaneously with the mass and momentum parts, or on alter-
nating steps in a leapfrog manner [3]. One typical form of the
equation is Equation (16-25) from [4]. Since temperature me-
ters also have errors just like pressure and ow meters, and since
more temperature meters are usually present in the system than
are needed as boundary conditions for the energy conservation
PDE, it would be nice if we could somehow use an MLSE-like
technique to improve the accuracy of the pipeline thermal model
by taking into account the other meters.
The energy equation requires a boundary condition at the
upstream end, which in an online model will be taken from tem-
perature meter readings. There are often also temperature mea-
surements present elsewhere in the line, which suggests that it
might be possible to perform state estimation using these extra
meters. However, temperature fronts only propagate through the
line at the speed of the uid, which means it can take hundreds or
thousands of model steps for a temperature change to get from
an upstream meter to a downstream meter. On top of that, the
thermal contact with the environment often results in the temper-
ature front decaying to nothing by the time it traverses a pipe [5].
This means that a state estimation technique based on the as-
sumption that the previous step results are correct and nal, such
as the presente MLSE approach, cant use use both an upstream
and a downstream temperature meter: the downstream temper-
ature meter would be at best providing a second opinion on an
upstream temperature measurement that was locked in hundreds
of steps earlier. If there were two temperature meters that effec-
tively measured the temperature at the same point, a maximum
likelihood technique could be used to combine them into a single
more accurate measurement, but this situation rarely occurs.
Downstream temperature measurements can still be used to
tune the thermal properties of the pipe and environment, such as
the temperature of the medium outside the pipe, or, for buried
pipes, the soil thermal conductivity (which might not be well-
known because it varies for dry or wet soil). However, a direct
state estimation technique like MLSE utilizing downstream tem-
perature meters does not appear to be possible.
7 Copyright c 2012 by ASME
6 Lessons Learned
6.1 The Start-Up Transient
It quickly became apparent when running MLSE with real
data that there was a transient surge occurring after startup that
hadnt been present in the ofine model. This happened on both
liquid and gas systems. The transient didnt occur on test systems
where an ofine model was used to generate simulated online
data with no errors, however.
MLSE treats the physical conservation law equations as con-
straints on the solution. During the steady-state model solution
used to initialize the online model, ow conservation means that
the ow all up and down a straight pipe must be the same: in
steady state, there cant be any buildup of uid over time. How-
ever, on the rst transient step, there can be some buildup of uid.
This means that the best match the MLSE model can manage to
measured pressures and ows is going to be different (worse) in
steady state than it will on the rst transient step, even if the real
pipeline is truly in a steady state during startup.
For example, if a straight pipe has ow meters at the start
and end that are reading different values because one of them
has drifted, and if the pressure meters are given large
P
i
values
because the user wants the model to follow the ow meters to the
best extent possible, then the steady state MLSE model solution
will pick a single ow rate equal to the weighted average of the
upstream and downstream ow meter readings. However, once
the transient run starts, the MLSE model can now exactly match
both the upstream and downstream ow meters. Because this is
either going to cause a net inow or outow, the pressure in the
whole line will either start increasing or decreasing. Eventually,
it will have deviated far enough from the pressure meters that
even though they have been assigned large error values the cur-
rent pressure error will be equivalent to the necessary ow error
at either end to get back to a balanced ow, and then the line will
remain in that state.
This is not too much of a concern because online models
should ideally not need to restart very often and therefore should
only spend a minimal fraction of operating time with a steady-
state initialization in the recent past. Still, it would be nice to
minimize the violence of that startup transient.
As mentioned above, theres no startup transient if the pres-
sure and ow meters are all in perfect agreement with the model.
If, starting from the rst transient step, the model applies off-
sets to all of the meter readings such that the readings plus the
offsets agree exactly with the steady-state model pressures and
ows, and then linearly relaxes these offsets to zero over some
settle-in time that is long compared to the time the system takes
to reach hydraulic equilibrium (we nd that an hour is usually
sufcient) then the model will at least avoid having big surges
bouncing around the system, instead transitioning from the so-
lution of the steady-state MLSE model to the steady-state of the
MLSE transient model in a gradual, linear fashion.
We will now examine the startup transient in a simulated gas
pipeline. This is 30 km pipe with a 3 bar pressure drop along it
and a ow rate of 1 std m
3
/s. The pipeline has pressure and ow
meters at both the inlet and outlet. An error has been added to
the downstreampressure meter so that it reads 0.5 bar lower than
it should, i.e. so the meters says theres a 3.5 bar pressure drop
even though the ow rate is what the model would predict for a
3 bar pressure drop. The model results for this never produce a
pressure error of more than a minuscule 20 Pa (0.0002 bar) or
a ow error of more than 0.0001 std m
3
/s - theres almost no
startup transient.
The situation is much worse when the model is confronted
with ow meters that dont agree with each other. In reality this
often happens due to meter drift. In that case during the steady
state solution the mass equation, which is treated as a constraint
on the MLSE solution, forces the model to nd a balance be-
tween the disagreeing meters so the net ow into the network
is zero. Once the transient model starts, however, the model is
no longer so constrained: it can meet all of the disagreeing ow
measurements and still conserve mass by allowing the line to
pack or unpack.
We can see this in Figures 6 and 7: this is the same gas
pipeline as above, but this time with the upstream ow meter
reading 2.5% high and the downstream ow meter reading 2.5%
low, and the pressure measurements reading the correct 3 bar
pressure drop along the line. The downstreammodeled and mea-
sured pressure and ow are shown for the rst day of simulation.
Meter offsets are applied as described above to make the meters
initially in perfect agreement with the model results on the rst
transient step, and then the offsets are relaxed in a linear manner
to zero over the rst hour of simulation. The downstream ow
drops over this hour and then corrects itself back towards the true
measured value over the rest of the rst day, while the line packs
up. (A similar thing is happening at the upstream ow meter but
with the opposite sign.)
We see here that its important to have some sort of tuning
outside of MLSE to take care of this type of ow-meter-drift in-
duced error, because the MLSE ends up with a solution that has
an overfull pipeline as seen in Figure 7 even after a day of simula-
tion. MLSE and the model by themselves will never correct this
overfull pipeline unless some additional automatic tuning cor-
rects the ow meter drift.
6.2 Interactions with Tuning
One popular feature of online pipeline models is the ability
to automatically tune pipe roughnesses and meter offsets so that
eventually the model will end up in agreement with all the me-
ters. This can still be done in conjunction with MLSE, but since
the model results follow all of the conservation laws there isnt
any measure of error local to a pipe that directly tells the tuning
algorithm what it should do.
Its possible to tune pipe roughness, or pipe efciency (de-
8 Copyright c 2012 by ASME
0.97
0.975
0.98
0.985
0.99
0.995
1
1.005
1.01
0 10000 20000 30000 40000 50000 60000 70000 80000
D
o
w
n
s
t
r
e
a
m

f
lo
w

(
s
t
d

m
^
3
/
s
)
Elapsed time (s)
downstream measured flow
downstream modeled flow
FIGURE 6. Downstream-end measured and modeled ow in a gas
pipeline when the upstream ow meter reads 2.5% higher than it should
and the downstream ow meter reads 2.5% lower than it should
5.4e+06
5.5e+06
5.6e+06
5.7e+06
5.8e+06
5.9e+06
0 10000 20000 30000 40000 50000 60000 70000 80000
D
o
w
n
s
t
r
e
a
m

p
r
e
s
s
u
r
e

(
P
a
)
Elapsed time (s)
downstream measured pressure
downstream modeled pressure
FIGURE 7. Downstream-end measured and modeled pressure in a gas
pipeline when the upstream ow meter reads 2.5% higher than it should
and the downstream ow meter reads 2.5% lower than it should
ned as the ratio of the true standard ow at a given frictional
pressure gradient to the standard ow predicted at that pressure
gradient by the model), or pressure and ow meter offsets to get
the pressure meters into agreement with the modeled pressures
at the same time the ow meters agree with the modeled ows.
These different tuned quantities can all be adjusted to achieve
agreement between the model and the meter readings; the cur-
rent author nds it more satisfying to tune a pipe property (such
as efciency) since there are likely to be signicant unknowns
about the pipe: the roughness, effective diameter changes due to
scale buildup in liquids or liquid holdup in wet gas lines, etc. So,
for the current article we will only consider tuning the pipe ef-
ciency, which appears in the equation for the frictional pressure
gradient as :
d P
d x
|
friction
=
f vv
2D
2
where D is the inner diameter, f is the Moody friction factor, v
is the uid velocity, and
d P
d x
is the pressure gradient. The tunable
efciency essentially just adds an adjustable scale factor to the
frictional part of the pressure drop.
In order to tune for a pipe, we need to know what the pres-
sure gradient and ow would be in that pipe if the model were in
perfect agreement with the meters as compared to the modeled
values. However, the measured pressures are only directly avail-
able at the meters, not everywhere. We can however dene
a new tuned pressure everywhere that matches the pressures
at the pressure meters as closely as possible (it will match them
exactly, except in cases where multiple pressure meters are effec-
tively measuring the same pressure and coming up with different
results due to noise or meter drift) and which also matches the
pressure drops across the pipes as closely as possible given the
rst requirement.
These tuned pressure values everywhere can be calculated
as the solution of an unconstrained quadratic minimization prob-
lem. The objective function to be minimized has a heavily-
weighted part containing the weighted (by meter error) sum-of-
squares differences between the measured and modeled values,
plus a more lightly-weighted term that drags the pressure drop
across each pipe towards zero with a weight proportional to the
modeled pressure drop magnitude; this second set of terms has
the result of distributing tuned pressure drops proportionally to
the existing modeled pressure drops. We will indicate the tuned
pressure at location i with a star as P

i
.
The objective function is thus
O.F. =W
meters

i
P
i

P
i
+
pipes

i up
P

i dn

2
P
i up
P
i dn
where W is much greater than 1.
The action of the second summation may not be immedi-
ately clear. It can be demonstrated through an example: if we
consider a system of two pipes, where the model says the rst
pipe has a pressure drop of 1 MPa and the second pipe has a pres-
sure drop of 2 MPa, with measurements of 1 MPa at the down-
stream end and 7 MPa at the upstream end (so model and mea-
surement disagreeing by 2x about the overall pressure drop), the
heavily-weighted rst summation will force P

at the upstream
and downstream ends to pretty much match the measurements,
leaving only two terms in the right-hand summation involving
P

mid
,
9 Copyright c 2012 by ASME
O.F.(P

mid
) =

7 P

mid

2
2
+

mid
1

2
1
(10)
If we then solve for stationariness of this objective function
with respect to P

mid
by setting its rst derivative with respect to
P

mid
equal to zero, we nd
P

mid
7 MPa
2 MPa
+
P

mid
1 MPa
1 MPa
= 0 (11)
or P

mid
equals 3 MPa, thus preserving the ratio between the
modeled pressure drops in the upstream and downstream sec-
tions in the tuning pressures. This objective function term also
has the same effect of preserving ratios between pressure drops
for more complicated networks. The tuning pressures represent
a best guess at what the real pressures in the system would be if
the pressure meters were all exactly right, and the only thing nec-
essary to correct the model pressures to the measured pressures
was adding a constant offset.
With these tuned pressures in hand, we can compute for each
pipe section the efciency that would be required to produce the
tuned pressure drop at the current ow. If we then tune the pipe
efciency towards this efciency, eventually (if the assumption
about the formof meter errors, that they are all just constant pres-
sure meter offsets, is correct) the model results and the measured
results will match up exactly. The slowness of tuning is nec-
essary because we havent accounted for hydraulic transients,
but those will average out to nothing over long periods. Nor
have we accounted for static head, so if the pipe goes over el-
evation changes the modeled and measured pressures need to be
corrected everywhere to a baseline elevation before applying the
above algorithm.
MLSE doesnt interfere with another common type of online
model automatic tuning, the tuning of ow meter offsets so that
the net inow into the system is equal to zero (with the aim of
compensating for ow meter drift over long time periods). This
can be driven directly by the difference between the measured
and modeled ow at the meter. The global enforcement of con-
servation of mass ensures that over long periods the average total
ow into or out of the network will equal zero.
7 Results from a Live System
The MLSE technique has been installed on a large pipeline
network, so we can examine how the system performs in the real
world. This is the Gassco gas network, largely under the North
Sea, which connects several offshore platforms to Europe. The
1
10
100
1000
0.01 0.1 1 10 100
N
u
m
b
e
r

o
f

in
s
t
a
n
c
e
s
Pressure error magnitude (standard deviations)
number of instances of pressure error magnitude
FIGURE 8. Histogram of deviations between pressure meter reading
and modeled pressure at that point, in units of pressure meter
P
. Log-
log scale.
network includes 7800 km of pipe, 132 pressure meters, and 128
ow meters. Meters are distributed less densely than they are
in a typical landbound pipe network because of the expense of
getting data back from the bottom of the North Sea.
The online model running on this network is using MLSE
as described above, coupled with ow meter offset tuning and
roughness tuning. The model takes 1-minute time steps, which
is the SCADA scan rate. The model is congured to run as 18
different independent subnetworks.
We have collected statistics on two hours of typical opera-
tion of the network, capturing how far off each pressure and ow
meter was from the model values on each model step. The ow
meters on this network are believed to be quite reliable, and are
typically congured with
Q
of about 1% of ow while pres-
sure meters are typically congured with
P
of 10 kPa. Figure 8
shows a histogram of deviation between measured and modeled
pressure at pressure meters in units of the meters
P
, and Figure
9 shows the analogous histogram of standard ow deviations at
ow meters. The plots are on a log/log scale.
The model manages to keep almost all of the ow meter
readings within one
Q
i
or better of the measured value and the
pressure meter readings within about 10
P
i
of those measure-
ments.
During this two hours, the network was in fairly steady op-
erating conditions. One subsection, the Heimdal-Draupner sys-
tem, underwent a signicant transient during which it dropped to
about 60% of its original ow and then remained at the new ow.
This subsection is a point-to-point single pipeline 164 km long
with pressure and ow meters at either end. The upstream pres-
sure and ow meter deviations in units of
P
and
Q
respectively
are shown in Figure 10 and the downstream values in Figure 11.
Note that even though this was a large fractional change in ow,
the pipe is operating at a fraction of its capacity in this data set.
The short-timescale variations of the dowmstream pressure
10 Copyright c 2012 by ASME
1
10
100
1000
10000
0.01 0.1 1 10
N
u
m
b
e
r

o
f

in
s
t
a
n
c
e
s
Flow error magnitude (standard deviations)
number of instances of flow error magnitude
FIGURE 9. Histogram of deviations between ow meter reading and
modeled ow at that point, in units of ow meter
Q
. Log-log scale.
-0.0003
-0.0002
-0.0001
0
0.0001
0.0002
0.0003
0.0004
0.0005
0 500 1000 1500 2000 2500 3000 3500 4000 4500
E
r
r
o
r

m
a
g
n
it
u
d
e

(
s
t
a
n
d
a
r
d

d
e
v
ia
t
io
n
s
)
Elapsed time (s)
downstream flow error
downstream pressure error
0
FIGURE 10. Downstream-end ow and pressure deviation in units of

Q
and
P
, respectively for Heimdal-Draupner during transient event
-0.0014
-0.0012
-0.001
-0.0008
-0.0006
-0.0004
-0.0002
0
0.0002
0.0004
0 500 1000 1500 2000 2500 3000 3500 4000 4500
E
r
r
o
r

m
a
g
n
it
u
d
e

(
s
t
a
n
d
a
r
d

d
e
v
ia
t
io
n
s
)
Elapsed time (s)
upstream flow error
upstream pressure error
0
FIGURE 11. Upstream-end ow and pressure deviation in units of

Q
and
P
, respectively for Heimdal-Draupner during transient event
deviations track the ow deviations in a similar manner to what
was visible in Figure 1 for an articial pipeline. There is also a
slow longer-term change visible, which is likely caused by the
hydraulic efciency tuning.
These results demonstrate that the MLSE method is able to
keep a real-world network with widely separated metering loca-
tions running in a manner consistent with the entire set of meters,
which is the desired behavior of a state estimation method.
8 Conclusions
The new MLSE technique presented above can provide ac-
curate state estimation. It corrects some of the deciencies of the
pressure-pressure state estimation technique, in that it enforces
the physical equations like conservation of mass everywhere, and
also of the equal error fractions technique, in that it provides a
more correct tradeoff between pressures and ows with normally
distributed errors and it can automatically handle any congura-
tion of pressure and ow meters and other devices. In spite of the
double matrix size as compared to an ofine model, MLSE pro-
vides only slightly worse performance. This is possible because
of the sparseness and ease of generation of the added parts of the
matrix being inverted as compared to an ofine model.
The new method can work in conjunction with pipe ef-
ciency tuning and with ow meter offset tuning using the tech-
niques described above. It can gracefully handly the loss of pres-
sure or ow meters during a transient run, and can keep a real
pipeline model in good global agreement with pressure and ow
meter readings.
Nomenclature
pipe efciency factor
Lagrange multipliers used in constrained minimization
P measured pressure
Q measured standard volume ow
uid density

P
error in measured pressure

Q
error in measured ow in standard volume units
D pipe inner diameter
f Moody friction factor
P model-computed pressure
Q model-computed standard volume ow
v uid velocity
W extra weight factor applied to meters in meter-tuning ob-
jective function
x linear distance along the pipeline
REFERENCES
[1] van der Hoeven, T., 1987. Gas network state estima-
tion with the equal error fractions method. In PSIG An-
11 Copyright c 2012 by ASME
nual Meeting (October 1987), Pipeline Simulation Interest
Group.
[2] Modisette, J. P., 2009. State estimate in online models.
In PSIG Annual Meeting (May 12 - 15, 2009, Galveston,
Texas), Pipeline Simulation Interest Group.
[3] Modisette, J. P., 2002. Pipeline thermal models. In PSIG
Annual Meeting (October 23 - 25, 2002, Portland, Oregon),
Pipeline Simulation Interest Group.
[4] Wylie, E. B., and Streeter, V. L., 1993. Fluid Transients in
Systems. Prentice Hall.
[5] Nicholas, E., 2011. The impact of the pipe and ground on
pipeline temperature transients. In PSIG Annual Meeting,
(May 24 - 27, 2011, Napa Valley, California), Pipeline Sim-
ulation Interest Group.
12 Copyright c 2012 by ASME