Coherent Optical Receiver

K. M. Mehta

Georgia institute of Technology

Abstract: Coherent detection of optical signal has once again gained the attention of research
community because of its high sensitivity and ability to compensate linear transmission impairments.
Researchers have demonstrated polarization multiplexed transmission system compatible with 100-
Gb/s Ethernet standard with these receiver. This article discusses about basics of coherent reception
and digital signal processing involved.We also various types of receiver architectures and recent
demonstrations/experiments.

1. Introduction
A detection system in which phase coherence of the optical carrier plays an important role is
called coherent detection system. These systems are well known in the context of radio and
microwave communication but until recent its use in commercial systems has been hindered by
the additional complexity, due to the need to track the phase and the polarization of the incoming
signal. For optical communication, intensity modulated direct detection(IM/DD) systems remained
dominant in term of commercial deployment. The drive for higher performance and high data rate
in optical fiber systems has renewed interest in coherent detection.
Coherent detection has following advantages over direct detection[1][2]:
• Receiver is sensitive to the phase as well as the amplitude of the optical wave, therefore
linear impairments,including chromatic dispersion and polarization-mode dispersion, can be
compensated quasi-exactly using electronic filters.
• Higher data rate of the order of 111 Gbps can be achieved exploiting all parameters (inten-
sity,angle,polarization) of modulation.
• Coherent detection offers an inherent ultra narrow optical filtering capability useful for dense
wavelength-division multiplexing
• We can have up to 20 dB improvement in receiver sensitivity when compared with IM/DD
systems.
• Some nonlinear impairments, such as intra-channel four-wave mixing and nonlinear phase
noise, can be compensated partially.
Importance in case of long-haul communication systems: For a long-haul optical fiber system
it is required to transmit the highest data throughput over the longest distance without signal
regeneration. Given constraints on the bandwidth imposed by optical amplifiers, it is important to
maximize spectral efficiency. Also given constraints on signal power limited by fiber nonlinearity,
we need a power efficient system which minimizes the required average transmitted energy per
bit. Coherent detection with polarization multiplexing achieves both goals as in-phase (I) and
quadrature-phase (Q) signals in the two orthogonal field polarizations allows information to be
encoded in all the available degrees of freedom.
Importance of Digital signal processing: In this paper we will discuss various digital signal
processing techniques. A DSP-based receiver is highly advantageous because adaptive algo-
rithms can be used to compensate time-varying transmission impairments. Advanced forward
error-correction coding can also be implemented. Moreover, digitized signals can be delayed,
split and amplified without degradation in signal quality[3].

Vol. , No. , Page 1

 Fig. 1. Schematic of coherent detection scheme

This paper is organized as follows: In Section 2, we review basics of coherent detection and
various receiver architectures . In Section 3, we discuss about digital signal processing algorithm
for carrier phase recovery, equalization and detection. In Section 4, we discuss various demon-
strations and research experiments and finally we conclude the discussion in Section 5.

2. Basics of Coherent detection

The basic idea behind coherent detection is combining the optical signal coherently with a locally
generated(with the help of narrow line width laser, called as Local oscillator/LO) optical field before
it falls on the photo detector.This is shown in Fig. 1.
We use following complex notations to represent optical field associated with signal and local
oscillator.As and ALO are also called as low pass equivalent or complex envelopes.

Es = As exp [−i (ω0 t + φs )] (1)

ELO = ALO exp [−i (ωLO t + φLO )] (2)

Since a photo detector responds to the optical intensity, the optical power incident at the photo
detector is given by:

2
P = K |Es + ELO | (3)

where K is a constant of proportionality. Using (1) , (2) and (3) we get

p
P (t) = Ps + PLO + 2 Ps PLO cos (ωIF t + φs − φLO ) (4)

where

Ps = KA2s , PLO = KA2LO (5)

and

ωIF = ω0 − ωLO (6)

ωIF is also known as intermediate frequency.There are following two types of coherent receiver
based on value of ωIF :
• Homodyne receiver.
• Heterodyne receiver.

Vol. , No. , Page 2

 2.1. Homodyne receiver
For Homodyne receiver the local oscillator frequency ωLO is selected to coincide with the signal
carrier frequency ω0 so that ωIF = 0. In this case passband to baseband conversion happens in a
single stage.Photo current of diode, I(I = RP , where R is the detector responsivity) in this case
is given by:
p
I(t) = R (Ps + PLO ) + 2R Ps PLO cos (φs − φLO ) (7)

Typically, Ps  PLO and consider the case in which the local-oscillator phase is locked to the
signal phase so thatφs − φLO . The homodyne signal is then given by
p
I(t) = 2R Ps PLO (8)

The main advantage of homodyne detection can be seen from Eq.(8) If we note that the signal
current in the direct-detection case is given by Idd(t) = RPs (t). Denoting the average optical
power by P¯s , the average electrical power is increased by a factor of 4PLO /Ps with the use of
homodyne detection. Since PLO can be made much larger than Ps , the power enhancement can
exceed 20 dB. Although in this case shot noise is also enhanced, homodyne detection improves
the signal-to-noise ratio (SNR) by a large factor.
A disadvantage of homodyne detection is its phase sensitivity.In practice, both φs and φLO
fluctuate with time in a random manner. However, their difference φs − φLO can be forced to
remain nearly constant through an optical phase-locked loop. In addition,exact matching of the
transmitter and local-oscillator frequencies is required.

2.2. Heterodyne receiver

In the case of heterodyne detection the local-oscillator frequency ωLO is chosen to differ form the
signal-carrier frequencyω0 such that the intermediate frequency ωIF is in the microwave region
(FIF ∼ 1 GHz).
Photo current in this case is given by:
p
I(t) = R (Ps + PLO ) + 2R Ps PLO cos (ωIF t + φs − φLO ) (9)

Since PLO  Ps in practice, the direct-current (DC) term is nearly constant and can be removed
easily using bandpass filters. The heterodyne signal is then given by the following alternating-
current (ac) term:
p
Iac (t) = 2R Ps PLO cos (ωIF t + φs − φLO ) (10)

Similar to the case of homodyne detection, the local oscillator amplifies the received signal by a
large factor, thereby improving the SNR. However, the SNR improvement is lower by a factor of 2
(or by 3 dB) compared with the homodyne case. This reduction is called as “heterodyne detection
penalty”. Because of the ac nature of Iac , the average signal power is reduced by a factor of 2
2
when Iac is averaged over a full cycle at the intermediate frequency.
The advantage gained at the expense of the 3-dB penalty is that the receiver design is consid-
erably simplified and the line width requirements are quite moderate. These features makes the
heterodyne detection scheme quite suitable for practical implementation.

2.3. SNR performance

Now we will calculate SNR of the receiver current which will quantify advantage of coherent
reception.The receiver current fluctuates because of shot noise and thermal noise. Total variance
σ 2 of current fluctuations is obtained by adding the two contributions. Thus,

σ 2 = σs2 + σT2 (11)

Vol. , No. , Page 3

 where,

σs2 = 2q (I + Id ) ∆f, σs2 = (4KB T /RL ) FN ∆f (12)

Here, I = resultant photo current,Id = Dark current,∆f = electrical bandwidth of receiver, KB

= Boltzman’s constant,T = temperature in Kelvin,RL input resistance of the receiver and FN is
noise figure of the receiver. Thus SNR in case of heterodyne receiver is given by:

2
I 2R2 P̄s PLO
SN R = ac2 = (13)
σ 2q (RPLO + Id ) ∆f + σT2
where R is responsivity of the photo detector.The above equation assumes PLO  Ps , moreover
PLO can be made large enough such that the receiver noise is dominated by shot noise.In this
condition dark current contribution can be assumed to be negligible.Thus final equation for SNR
is given by:

RP̄s
SN R ≈ (14)
q∆f
The use of coherent detection allows one to achieve the ‘‘shot-noise limit” even for PIN receivers
whose performance is generally limited by thermal noise. Moreover, in the case of avalanche photo
diode (APD) receivers, this limit is realized without adding any excess shot noise.

2.4. Modulation formats

In single-mode optical fibers, the optical field has three physical attributes that can be used to
carry information: Intensity,Phase (including frequency),Polarization.
Depending on which of the three quantities is used for information transport, coherent re-
ceiver has different architecture.Intensity and phase modulation formats have been widely used
in RF/optical communications. Encoding information onto the polarization of light (Pol-SK) is very
special type of modulation scheme in optical communication.It has advantage of increase spectral
efficiency but it requires an active polarization management at the receiver, because of random
polarization changes in optical fiber. Two different schemes used with polarization are:
1) Polarization-multiplexing: transmitting two different signals at the same wavelength but in
two orthogonal polarizations,
2) Polarization-interleaving: transmitting adjacent WDM channels in alternating polarizations to
reduce coherent WDM crosstalk or nonlinear interactions between the channels.
Using all the three attributes for modulation significantly reduces symbol rate/ baud rate.
Popular modulation schemes used with coherent receiver are:
• Binary Differential Phase Shift Keying (DPSK)
• Differential Quadrature Phase Shift Keying (DQPSK): RZ or NRZ, Pol-Mux or single polar-
ization.

2.5. Sensitivity degradation of coherent receiver

There are various sources which contribute to sensitivity degradation of coherent receiver. Let
us discuss them one by one. An important source of sensitivity degradation is the phase noise
associated with the transmitter laser and the local oscillator.A receiver design approach which
solves the phase-noise problem by designing special receivers having two or more photo detectors
whose outputs are combined to produce a signal that is independent of the phase difference
φIF = φs − φLO known as “phase-diversity” receivers.
Relative intensity noise (RIN) of the local oscillator is another source of sensitivity degradation.A
solution to the intensity-noise problem is offered by the “balanced coherent” receiver made with
two photo detectors.

Vol. , No. , Page 4

 Fig. 2. Schematic of Phase diversity balanced detection

polarization-mismatch is the third major contributor.Several schemes have been developed

for solving the polarization-mismatch problem.The most commonly used approach solves the
polarization problems by using a two-branch receiver with the difference that the two branches
process orthogonal polarization components. Such receivers are called ‘‘polarization-diversity
receivers”.The technique of polarization diversity can be combined with phase diversity to realize
a receiver that is independent of both phase and polarization fluctuations of the signal received.
Let us discuss these receiver architectures in detail:

2.5.1. Phase-diversity balanced receiver

Fig. 2 shows the configuration of a phase-diversity coherent receiver.The optical signal is mixed
with local oscillator light in a 90◦ hybrid. The local oscillator path is longer by a quarter cycle(90◦ )
in the lower arm than the upper arm, which means that the balanced photo detector in the upper
arm effectively sees an in-phase beat product while the one in the lower arm sees a quadrature
beat product.The photo detectors at the outputs of the 90◦ hybrid each see the signal and LO
adding as electric field. The powers at the two balanced outputs of the upper arm are
2 2
P1a (t) = |As (t)| + |ALO (t)| + 2Re{As (t)A?LO (t)exp (iωIF t + iφs − iφLO )} (15)

2 2
P1b (t) = |As (t)| + |ALO (t)| − 2Re{As (t)A?LO (t)exp (iωIF t + iφs − iφLO )} (16)
and the balanced photo detector output is, therefore, proportional to

∆P1 (t) = P1a (t) − P1b (t) ≈ 4Re{As (t)A?LO (t)exp (iωIF t + iφs − iφLO )} (17)
The DC term is eliminated completely during the subtraction process when the two branches
are balanced (each branch receives equal signal and local-oscillator powers). More importantly,
the intensity noise associated with the DC term is also eliminated during the subtraction process
since intensity fluctuations in the two branches are perfectly correlated and cancel out. The quarter
wave shift in LO path of the lower arm of the 90◦ hybrid means that the balanced photo detector
in the lower arm responds to

∆P2 (t) = 4Re{−iAs (t)A?LO (t)exp (iωIF t + iφs − iφLO )} (18)

∆P2 (t) = 4Im{As (t)A?LO (t)exp (iωIF t + iφs − iφLO )} (19)
Hence, the electric field envelope of the signal can be obtained by:
 
exp (−iωIF t − iφs + iφLO )
As (t) = (∆P1 (t) + i∆P2 (t)) (20)
4A?LO
The frequency difference and phase difference must be estimated from ∆P1 and ∆P2 to deduceAs
, which would be discussed in the later part of the paper.

Vol. , No. , Page 5

 Fig. 3. Schematic of Pol-Mux QPSK receiver

Fig. 4. Typical Digital signal processing blocks

2.5.2. Pol-Muxed QPSK receiver

Fig. 3 shows a typical architecture of Pol-mux QPSK receiver.The receiver composes of a po-
larization beam splitter (PBS), two 90◦ hybrids and 4 detectors. Note that this receiver does
not use balanced detectors to reduce the associated cost. We can write similar expressions
for outputs xi ,xq ,yi ,yq as we had derived in the previous case, where X and Y represents two
states of polarization.The input polarization state of the received signal is not controlled, and an
arbitrary mix of each transmitted polarization state is incident on the photo detectors whereas
polarization controllers are required for the LO.Main advantage of this type of receiver is that
we are able to recover information from in-phase and quadrature phase components of both
orthogonal polarization. Thus, xi ,xq ,yi ,yq have complete information of the incoming signal.

3. Digital signal processing and electronic equalization

The output signals from optoelectronic converter is filtered and then simultaneously sampled.The
sampling can be synchronous or asynchronous.When the signal envelope is modulated in the
return-to-zero (RZ) waveform, the intensity of RZ pulses can be used as the sampling clock for
synchronous sampling. For asynchronous sampling, the signal must be sampled at 1.5 to 2 times
the symbol rate (‘‘Nyquist rate”) and then re-sampled to keep one sample per symbol. Sampling
becomes challenging when symbol rates are of the order of tens of MHz. Such a receiver require
very high speed analog-to-digital converters (ADCs). The sampled signals xi (n),xq (n),yi (n),yq (n),
where n denotes the number of the sample, are then processed with DSP circuits.Typical DSP
blocks are shown in Fig. 4. As the digitized waveform retains full information of the electric field, it
enables us to compensate for linear and nonlinear transmission impairments. We will now discuss
each of these blocks in detail:

3.1. Clock recovery and re-timing

This is an important signal processing block in case of asynchronous sampling, which enables
us to achieve symbol period synchronization. One of the methods of clock recovery is to use
power envelope of the received signal. The power envelopeP is calculated from the square of

Vol. , No. , Page 6



the received signal components P = x2i + x2q + yi2 , yq2 . The incoming signal is divided in blocks
of length M and the power signal for the mth block is then filtered to get sampling phase error.
The sampling phase error is then used to resample the data using an interpolator.

3.2. Equalization
Electronic equalization is used to compensate for linear transmission impairments and recover
the polarization multiplexed data. This usually involves a filtering operation[4]. Equalization can
be partitioned into two distinct stages.
• Chromatic Dispersion compensation
• Polarization dependent impairment (Polarization rotation and PMD)compensation

3.2.1. Chromatic dispersion compensation

Now we will discuss design of a typical dispersion compensating filter[5]. In absence of fiber
nonlinearity, chromatic dispersion of the envelope of a pulse may be modeled as,

∂A(z, t) Dλ2 ∂ 2 A(z, t)

=j (21)
∂z 4πc ∂t2
where z is distance of propagation, A(z, t) is complex envelope of the optical signal,λ is
wavelength and D is dispersion coefficient. Taking Fourier transform and solving for A(z, ω) we
get,

−jDλ2 2
 
A(z, ω) = A(0, ω)exp ω z (22)
4πc
This can be viewed as LTI system with frequency response of

−jDλ2 2
 
H(z, ω) = exp ω z (23)
4πc
and thus frequency response of dispersion compensating filter is given by Hc (z, ω) = 1/H(z, ω).Such
a filter can be implemented using FIR or IIR system(as in case of [6]). Let us discuss design of
FIR filter. Time domain impulse response of Hc (z, ω) is given by:
r
−jπct2
 
jc
hc (z, t) = exp (24)
Dλ2 z Dλ2 z
It can be seen that this impulse response is of infinite duration, non-causal and since it passes all
frequencies, for a finite sampling frequency aliasing will occur.By truncating this to finite length by
using a Kaizer window we can solve these problems. Length of the window(which is proportional
to N taps of an FIR filter) can be calculated using following criterion as specified in [5]

Dλ2 z Dλ2 z
− ≤t≤ (25)
2cTs 2cTs
where Ts is sampling interval.Filter coefficients of N tap FIR filter are given by
r
jcTs2 πcTs2 2
 
ak = exp −j k (26)
Dλ2 z Dλ2 z
where

Dλ2 z
     
N N
− ≤k≤ and N = 2 +1 (27)
2 2 2cTs2
This filter can be implemented in FPGA using shift registers or with FFT algorithm.

Vol. , No. , Page 7

 3.2.2. Polarization dependent impairment compensation
These impairments are usually modeled as ‘‘Jones matrix”.The Jones matrices are the operators
that act on the Jones Vectors,which describes the polarization of light.[7] In general, this matrix
is not unitary and will be frequency dependent.Unlike chromatic dispersion this impairment is not
constant and may vary rapidly with time, thus requiring adaptive computations.This equalization
can be achieved by implementing a Butterfly structure filter for MIMO system. Assume xp and yp
are two input streams in two orthogonal polarization and x0 and y 0 are output of the module.They
are related by following system of equations:
M
X −1
x0 (k) = Hxx
T T
Xp + Hxy Yp = hxx (m)xp (k − m) + hxy (m)yp (k − m) (28)
m=0

and
M
X −1
0 T T
y (k) = Hyx Xp + Hyy Yp = hyx (m)xp (k − m) + hyy (m)yp (k − m) (29)
m=0

where hxx ,hxy ,hyx ,hyy are all adaptive filters of length M .Various methods have been suggested
for calculating coefficients of these filters but we will look into ‘‘Constant Modulus Algorithm”(CMA)
suggested by Dominique[8].This algorithm exploits the constant modulus property of QPSK mod-
ulated signal. Let us assume unit amplitude of the modulated signal.The equalizer will try to
minimize mean square error x and y given by:
2 2
x = 1 − |x0 | and y = 1 − |y 0 | (30)
giving following four criterion:



d 2y d 2y



d 2x d 2x
=0, =0, =0, =0 (31)
dHxx dHxy dHyx dHyy
To determine optimum tap weights stochastic gradient algorithm is used.In the initialization stage
all tap weights are set to zero with the exception of central tap of hxx and hyy . Once the equalizer
has converged it moves to decision directed mode using DD-LMS algorithm. The filter coefficients
are now adapted as:
2
µ d |x |
Hxx → Hxx − = Hxx + µx x¯p (32)
2 dHxx
where x¯p is complex conjugate of xp and µ is a convergence parameter. and x = dx − x0 with
dx being the symbol closest to x0 . Similarly filter coefficients of Hxy ,Hyx ,Hyy are calculated on
the fly.

3.3. Carrier recovery/Phase estimation

There are various phase estimation algorithm studied and experimented[9]. Here is a list of few
of them:
• MAP(Maximum a posteriori) phase estimator.
• Decision directed phase estimator.
• Power law- average phase estimator
We will discuss power law -average phase estimator(’’Viterbi-and-Viterbi”) in detail[10]. Since
the line-width of lasers used as the transmitter and LO typically ranges from 100 kHz to 10 MHz,
assuming a homodyne receiver, the optical carrier phase φn (t) = φs (t) − φLO varies much more
slowly than the phase modulation, whose symbol rate is in order of tens of Gsymbol/s.Therefore, by
averaging the carrier phase over many symbol intervals, it is possible to obtain an accurate phase

Vol. , No. , Page 8

 Fig. 5. Phase estimator

Fig. 6. Architecture of a processing unit of phase estimator

estimate, as shown in the following. Coherent detection using such a method for carrier phase
estimation will offer significant improvement in BER performance over differential detection[11].
The equalized signal samples(x0 (n) = Equalized (xi + jxq )) and (y 0 (n) = Equalized (yi + jyq ))
in both the polarization are processed with the DSP circuit shown in Fig. 5.Let us call x0 (n) or
y 0 (n) by E(n) for explaining the algorithm. The processing module consists of a demultiplexing
unit, several processing units (PUs) that estimate the phase offset, and a remultiplexing unit. The
stream of E(n) are demultiplexed into M blocks of N complex samples (as shown in Fig. 5)
before being sent to the PUs.When ASE noise is the dominant noise N should be as large as
possible. The core processing functions residing in the PUs can demodulate the data in parallel
and independently from each other, which allows this architecture to support high symbol rates.
A schematic of a PU is shown in Fig. 6.E(i)(i = 1, ..., N ) to the fourth power cancels the
QPSK phase modulation θs (= 0, π/2, π, π/2), sinceE 4 ∝ exp[j(4θs )]. The complex amplitudes E 4
are summed, so that the phase is averaged over the entire block. The phase of the resulting
complex amplitude is divided by 4, leading to a phase correction for the k th block θ0 (k) that lies
between π/4 and π/4. The phase correction θ0 (k) is, thus, given as

"N #
0 1 X
4
θ (k) = arg E (kN + i) (33)
4 i=1

This phase error can be subtracted form each symbol of the block to get modulated symbol
back.It may be noted that differential phase encoding solves the problem of phase ambiguity as
in this case absolute phase is not relevant and the information is encoded in phase difference of
two consecutive symbols.

Vol. , No. , Page 9

 3.4. Slicer and Decoder Unit
It consists a of a slicer (threshold decoder) and digital decision logic which maps the received
symbols onto binary 1 and 0. This block will perform differential decoding if transmitted signal is
differentially encoded.

4. Demonstrations and conclusion

Recently, many researchers have shown long-haul (upto 2375km) transmission using a coherent
receiver with data rates of the order of 111 Gbit/s.[12] It is also possibility of use existing in-
frastructure designed for 10 Gbps system for 100 Gbps transmission. 2 Tbits/s DWDM transmis-
sion has also been demonstrated. Penalty free compensation of chromatic dispersion and other
impairments have open up the possibility of employing such systems cost effectively[3].Nortel
has introduced commercially available 100G solution, already deployed and carrying in-service
customer traffic in Verizons European network.Alcatel Lucent and Kazakh telecom have also
demonstrated 100Gbps transmission trial over the existing fiber optic network. Various vendors
have started investing in design and manufacturing of 100 G components.Thus a coherent receiver
along with digital signal processing can be seen as the system for future long-haul communication.

References
[1] G. P. Agrawal, Fiber-Optic Communications Systems, Third Edition. John Wiley and Sons, 2002.
[2] D. J. F. B. J. M. K. Ezra Ip, Alan Pak Tao Lau, “Coherent detection in optical fiber systems,” Optical express, vol. 16,
no. 2, 2008.
[3] M. G. Taylor, “Coherent detection method using dsp for demodulation of signal and subsequent equalization of
propagation impairments,” IEEE Photonics Technology Letters., vol. 16, no. 2, pp. 674–676, 2004.
[4] R. I. K. P. B. Seb J. Savory, Giancarlo Gavioli, “Electronic compensation of chromatic dispersion using a digital
coherent receiver,” Optical express, vol. 15, no. 5, 2007.
[5] S. J. Savoryl, “Digital filters for coherent optical receivers,” Optical express, vol. 16, no. 2, 2008.
[6] G. Goldfarb and G. Li, “Chromatic dispersion compensation using digital iir filtering with coherent detection,” IEEE
Photonics Technology letters, vol. 19, no. 13, pp. 969–971, 2007.
[7] Jones calculus. http://en.wikipedia.org/wiki/Jones calculus.
[8] D. N. Gorald, “Self-recovering equalization and carrier tracking in two-dimensional data communication system,” IEEE
transactions on communication, vol. 28, no. 11, 1980.
[9] M. G. Taylor, “Phase estimation methods for optical coherent detection using digital signal processing,” Journal of
Lightwave Technology, vol. 27, no. 7, pp. 901–914, 2009.
[10] S. T. Kazuro Kikuchi, Dany-Sebastien Ly-Gagnon and K. Katoh, “Coherent detection of optical quadrature phase-shift
keying signals with carrier phase estimation,” Journal of Lightwave Technology, vol. 24, no. 1, pp. 12–21, 2006.
[11] K. K. Dany-Sebastien Ly-Gagnon and K. Kikuchi, “Unrepeated optical transmission of 2o=gbit/s quadrature phase-
shift keying signal over 210-km transmission using homodyne phase-diversity receiver and digital signal processing,”
Electronic letters, vol. 41, no. 4, 2005.
[12] C. R. S. F. et al., “Coherent equalization and plomux-rz-dqpsk for robust 100-ge transmission,” Journal of Lightwave
Technology, vol. 26, no. 1, pp. 64–71, 2008.

Vol. , No. , Page 10