Delivarable D4.A - Annex To D4.1 and D4.2 MUCAPS Prototype and Experimentation

Seventh Framework STREP No.
317846
D4.A
Public
Socially-aware Management of
New Overlay Application Traffic with
Energy Efficiency in the Internet
European Seventh Framework Project FP7-2012-ICT- 317846-STREP
Deliverable D4.A
(Annex to D4.1 and D4.2)
MUCAPS Prototype and Experimentation
The SmartenIT Consortium

Universitt Zrich, UZH, Switzerland
Athens University of Economics and Business - Research Center, AUEB, Greece
Julius-Maximilians Universitt Wrzburg, UniWue, Germany
Technische Universitt Darmstadt, TUD, Germany
Akademia Gorniczo-Hutnicza im. Stanislawa Staszica w Krakowie, AGH, Poland
Intracom SA Telecom Solutions, ICOM, Greece
Alcatel Lucent Bell Labs, ALBLF, France
Instytut Chemii Bioorganicznej PAN, PSNC, Poland
Interoute S.P.A, IRT, Italy
Telekom Deutschland GmbH, TDG, Germany
Copyright 2015, the Members of the SmartenIT Consortium

For more information on this document or the SmartenIT project, please contact:
Prof. Dr. Burkhard Stiller
Universitt Zrich, CSG@IFI
Binzmhlestrasse 14
CH8050 Zrich
Switzerland
Phone: +41 44 635 4331
Fax: +41 44 635 6809
E-mail: info@smartenit.eu
Version 1.0
Page 1 of 30
D4.A
Seventh Framework STREP No. 317846

Public
Document Control
Title:
D4.M MUCAPS Prototype and Experiments
Type:
Public
Editor(s):
Sabine Randriamasy, Frdric Faucheux
E-mail:
sabine.randriamasy@alcatel-lucent.com, frederic.faucheux@alcatel-lucent.com
Author(s):
Sabine Randriamasy, Frdric Faucheux
Doc ID:
D4.A-v1.0
AMENDMENT HISTORY
Version
Date
Author
Description/Comments
V0.1
2015/10/19
Sabine Randriamasy
First version
V0.2
2015/10/30
Frdric Faucheux
Revised version after comments by Roman Lapacz and Jeremias

Blendin
V1.0
2015/10/30
Sabine Randriamasy, Frdric

Faucheux
Completed version after fixing formal issues
Legal Notices
The information in this document is subject to change without notice.
The Members of the SmartenIT Consortium make no warranty of any kind with regard to this document,
including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose. The
Members of the SmatenIT Consortium shall not be held liable for errors contained herein or direct, indirect,
special, incidental or consequential damages in connection with the furnishing, performance, or use of this
material.
Page 2 of 30
Version 1.0
D4.A
Public
Table of Contents
1
Executive Summary .................................................................................................. 5
Introduction ............................................................................................................... 6
MUCAPS and VITALU in the SmartenIT Architecture............................................. 7

3.1
MUCAPS ..............................................................................................................................................7
3.1.1
Integrating MUCAPS with the SmartenIT architecture..................................................................8
3.1.2
MUCAPS implementation and execution: entities and interfaces .................................................9
3.2
Using VITALU to Evaluate MUCAPS .................................................................................................10
MUCAPS Prototype Design and Instrumentation ................................................. 11

4.1
MUCAPS Prototype Design................................................................................................................11
4.2
MUCAPS Prototype Set-up and Configuration...................................................................................11
4.2.1
TinyDNS......................................................................................................................................11
4.2.2
Service handling..........................................................................................................................12
4.2.3
MACAO Block .............................................................................................................................13
4.2.3.1 Newconfig_initial.cfg................................................................................................................13
4.2.3.2 Mappingconfig.cfg ...................................................................................................................13
4.2.3.3 user_endpoint_config.cfg ........................................................................................................13
4.2.3.4 Metric_weights.cfg...................................................................................................................13
4.2.3.5 Newconfig.cfg ..........................................................................................................................14
4.2.4
ALTO Server ...............................................................................................................................15
4.2.4.1 Network-map.txt ......................................................................................................................15
4.2.4.2 Pidcosts.txt ..............................................................................................................................15
4.3
Prototype Instrumentation for Evaluation and Assessment ...............................................................16
4.3.1
QoE performance measurement points and KPIs ......................................................................17
4.3.2
The VITALU QoE analyzer..........................................................................................................17
Experiments Definition and Set-up for MUCAPS.................................................. 19

5.1
MUCAPS Experiments .......................................................................................................................21
5.1.1
Functional tests on the DIG command........................................................................................21
5.1.1.1 Parameters, Measurements, and Metrics ...............................................................................22
5.1.1.2 Functionality test documentation .............................................................................................23
5.1.2
Performance tests with QoE analysis .........................................................................................24
5.1.2.1 Parameters, Measurements, and Metrics ...............................................................................24
5.2
MUCAPS Showcase ..........................................................................................................................24
5.2.1
MUCAPS Showcase Scenario 1 Video Streaming on a High Resolution PC ..........................25
5.2.1.1 Scenario Topology...................................................................................................................25
5.2.1.2 Showcase Scenario.................................................................................................................26
5.2.2
MUCAPS Showcase Scenario 2 Video Streaming on a Mobile Phone....................................26
5.2.2.1 Scenario Topology...................................................................................................................26
5.2.2.2 Showcase Scenario.................................................................................................................26
Summary and Conclusions .................................................................................... 28
References ............................................................................................................... 29
Abbreviations........................................................................................................... 30
Version 1.0
Page 3 of 30
D4.A

Public
(This page is left blank intentionally.)
Page 4 of 30
Version 1.0
D4.A
Public
1 Executive Summary
The SmartenIT project has selected two core mechanisms which are DTM and RBHORST for full prototyping and experimentation in WP4. These mechanisms provide a full
traffic management and transportation solution for the EFS and OFS scenarios. Multipartner specifications of these mechanisms and their implementations started in Q1 2014
within SmartenIT. In Q1 2015, it was decided to add MUCAPS to the achievements of
SmartenIT in an Annex deliverable. Contrary to DTM and RB-HORST, MUCAPS is not a
fully fleshed multi-partner developed mechanism, but was proposed as an add-on to
optimize application traffic management based on a transport network layer aware
paradigm that is being standardized at the IETF as the ALTO (Application Layer Traffic
Optimization) protocol, specified in RFC 7285. MUCAPS has been defined and prototyped
by ALBLF.
MUCAPS revises the initial Endpoints selection done by the application overlay by adding
awareness on the underlying network topology and on transport costs through information
not available by traditional end to end on-line measurement mechanisms. The selection
criteria used in MUCAPS are an abstraction of end-to-end performances on application
paths. This abstraction relies itself on an ISP-centric abstraction of the Internet topology,
provided to applications via the IETF ALTO protocol and one of its extensions supporting
several network cost metrics and constraints combining several logical operators and
metrics, see [1]. MUCAPS also integrates the access technology used by the device
receiving the application resource in its decision making.
Besides MUCAPS, ALBLF had been developing a QoE measurement and analysis tool
called VITALU that measures a set of QoE specific metric values at devices receiving
video files and devices and derives a Video Quality Score (VQS) built in a similar way to a
Mean Opinion Score. VITALU is in a pre-product stage and represents an effort of about 6
person years. VITALU has been customized within the SmartenIT project to support
measurements done on mobile phones and further analysis functions, candidates to the
QoE analysis entity pictured in the SmartenIT architecture.
The purpose of this annex is to complement, for MUCAPS, the test-bed definition provided
in D4.1 and experimentation definitions provided in D4.2 that were provided for the multipartner implemented solutions DTM and RB-HORST.
Version 1.0
Page 5 of 30
D4.A

Public
2 Introduction
MUCAPS has been defined and prototyped by ALBLF as an add-on mechanism to further
optimize application traffic management based on a transport network layer aware
paradigm that is being standardized at the IETF as the ALTO (Application Layer Traffic
Optimization) protocol, specified in RFC 7285.
MUCAPS stands for "MUlti-Criteria Application endPoint Selection" and has been
specified in Section 4.11 of Deliverables D2.2 [4] and D2.4 [5]. MUCAPS addresses the
EFS scenario, in particular the use cases Locality in UNaDa and Service and content
placement for users. The MUCAPS added value to application traffic management and
SmartenIT is a further optimization of overlay application traffic. To this end, MUCAPS
revises the initial Endpoints selection done by the application overlay by adding
awareness on the underlying network topology and on transport costs through information
not available by traditional end to end on-line measurement mechanisms. The selection
criteria used in MUCAPS are an abstraction of end-to-end performances on application
paths. This abstraction relies itself on an ISP-centric abstraction of the Internet topology,
that is available to applications via the IETF ALTO protocol, documented in [1]. MUCAPS
leverages proposed ALTO protocol extensions, extending the base protocol set of ALTO
metrics and allowing ALTO transactions with multiple metrics and constraints combined
with multiple logical operators allowing richer compromises. So MUCAPS can help out any
SmartenIT mechanism that supports applications having multiple candidate resources
locations and that have no awareness of the underlying network.
The MUCAPS design integrates an extension to the ALTO protocol called Multi-Cost
ALTO that enables ALTO Clients to request path costs on several metrics jointly and also
allows clients to place constraints combining several metrics and logical operators. MultiCost ALTO is now an IETF ALTO WG item, see [1]. MUCAPS also integrates awareness
of the access technology used by a device receiving the application resource.
In the remainder of this document, section 2 complements D4.1 with the prototype set-up
and configuration for MUCAPS and section 3 complements D4.2 with the experiments
definition and Set-up.
Page 6 of 30
Version 1.0
D4.A
Public
3 MUCAPS and VITALU in the SmartenIT Architecture

While MUCAPS main outline is presented first, the use of VITALU to evaluate MUCAPS is
described thereafter.
3.1
MUCAPS
The ALTO protocol is integrated in the SmartenIT paradigm: the ALTO Server stores and
provides abstracted transport network information to requesting ALTO Clients associated
to applications. However, the ALTO Client alone cannot automatically decide what
information to request and how to process it once received. MUCAPS builds the decision
making around the ALTO Client. It assists the choice of the location from which to
download content. Locations can be various entities including Video Server, Data Center,
uNaDa and Peers. Content covers video streams as well as computing resources for
virtualized or distributed applications. MUCAPS is suitable for applications having the
choice among several feasible Application Endpoints (AEP) and supports multi layer and
multi party incentive decision making by involving the following aspects in the AEP
selection:
Topology distance: number of hops (inter or intra AS), impacting delay,
Routing cost: reflecting the ISP policy in terms of AS peering agreements,
ISP preferences in terms of resources availability (e.g., path bandwidth).
An ALTO Client has no intelligence to choose the AEP selection metrics and the ranking
method, as ALTO is mainly a transport means for abstracted network information.
The initial selection of overlay Endpoints, referred to in MUCAPS as Application Endpoints
is usually produced by gathering functions such as DNS Servers and Clients, DHTs, Peer
Trackers and referred to in MUCAPS as Application EndPoint Gathering (AEPG)
functions. Figure 1 illustrates how MUCAPS is involved in a content downloading scenario,
as detailed in SmartenIT deliverables D2.2 [4] and D2.4 [5]. In this figure, the term
MACAO stands for Multi Access and Cost ALTO. Basically, the MACAO Client block does
(or performs) the following actions:
decides for which metrics an ALTO Client must request values,
decides which weight is applied to the metrics, and then
uses the obtained values to rank the AEPs.
MUCAPS comprises three parts: a MACAO Service Client (MARC) that requires from the
MACAO Service interface (MARS) an additional ranking of the initial AEP selection. The
MARC is hooked to an AEPG that may call it for enhanced ranking and hands it from it the
initial list of AEPs, identified in the current implementation, by their IP addresses. The
MARS is just an interface to the MACAO block that does the actual selection and ranking.
The MACAO block with its MARS is entirely deployed in the provider network. Finally, the
MARC and the MARS communicate via a simple IP Socket carrying a light message. The
message typically includes: a list of AEP IP addresses, the MUCAPS service ID pointing
here to AEP Ranking and the number of AEPs to rank.
Version 1.0
Page 7 of 30
D4.A

Public
Figure 1 Involvement of MUCAPS components (MACAO, MARC and MARS) in a content

downloading scenario.
3.1.1 Integrating MUCAPS with the SmartenIT architecture
The impact of MUCAPS on the architecture is shown in Figure 2 with following
modifications:
The ALTO component in the SBox is replaced by the set {MACAO block, MARS
interface and ALTO Server}. The ALTO Client is now in the MACAO block.
There is no more need to have an ALTO Client in end systems involved in overlay,
that is: Data Centers, uNaDas and End User Entities. Instead, the ALTO Client
there is replaced by the MARC that sends to the MARS in the SBox, MUCAPS
requests, that is requests to re-order the candidate AEP list wrt ALTO based
network information.
The MARC is invoked by the Overlay Manager or the application of EUEs and
communicates with the MARS located in the SBox via a simple IP socket.
An alternative mapping of MUCAPS to the SmartenIT architecture is also described in

deliverable D3.3. It consists in replacing the ALTO (Client) component in the Overlay end
systems by the MARC, thus keeping the current interfaces established with
MUCAPS/ALTO replacing ALTO. This setting must however assume that the MARC can
be hooked with an AEPG function located in the Overlay Manager of the Data Center and
the uNaDa or in some appropriate place in the End User Entity.
The following SmartenIT architecture extensions are needed:
A placeholder is needed in the EUE for an AEPG that could call the MARC.
Interfaces should now be established between the MUCAPS/ALTO block and the
MARCs.
Page 8 of 30
Version 1.0
D4.A
Public
Figure 2 Entities and components involved in MUCAPS.

3.1.2 MUCAPS implementation and execution: entities and interfaces
In the current MUCAPS deployment, the MARC is located in the ISP network. This is the
case for instance when the ISP wants to ensure that the DNS selection of the application
network is compliant to its policy in resources usage. In that case, the ISP captures the
response of its DNS Resolver, processes it with the MACAO module and sends the
MACAO selection to the DNS Resolver that sends it to the DNS Client. All this is done in a
transparent way for end users and applications.
As illustrated in Figure 3, the MACAO functional block has an external interface called
MARS for MACAO Request handling Server allowing it to communicate with a MARS
Client (called MARC) hooked to the AEPG.
The network-side deployment enables a lightweight transaction between MUCAPS and
the ISP DNS resolver by: adding to the MUCAPS block a MARS service interface,
becoming thus a MACAO Request Server (MARS). This deployment includes the following
steps:
1. The application client queries the DNS server for the list of content servers to
download/upload from.
2. The DNS Server,
2.1. selects the MACAO Service Endpoint Ranking
Version 1.0
Page 9 of 30
D4.A

Public
2.2. hands to its embedded MARC its query result, that is the list of the candidate
Application Endpoint IP addresses, together with the IP address of the User
Endpoint, that is the DNS Client associated to the user device,
3. sends this light information to the MARS IF via an established IP Socket, as a query to
perform the multi-cost ranking of the AEPs,
4. The MUCAPS block performs the ranking and the MARS sends the reordered list of
AEPs to the MARC.
The MARC passes it to the AEPG/DNS Server that sends it to the requesting AEPG/DNS
Client.
Figure 3: MUCAPS entities and interfaces: MARC integrated in a network located AEPG,
here a DNS server.
3.2
Using VITALU to Evaluate MUCAPS
VITALU uses and analyses PCAP (Packet CAPture) messages either produced by tools
such as: wireshark, tshark, tcpdump, tpacketcapture, or that it captures by launching a
tshark instance, or gets on a non rooted Android mobile phone via tpacketcapture (with all
related issues)
The procedure to follow is to use disjoint networks for media download and PCAP sending
(e.g. WiFi for video and Ethernet for PCAP sending ) to avoid interferences between pcap
and media flow, or simply (and probably better) to wait until the end of the media flow and
then transfer the captured PCAP to the VITALU machine for analysis.
Page 10 of 30
Version 1.0
D4.A
Public
4 MUCAPS Prototype Design and Instrumentation

The MUCAPS test-bed has been designed to:
Evaluate the ability of MUCAPS to revise the initial AEP selection whenever it was
appropriate,
Evaluate the impact of the MUCAPS selection on the QoE.
A MUCAPS intervention is qualified as appropriate whenever the initial AEP selection

does not optimize ISP costs and other metrics that impact the application QoE. The
applicable ISP is the one hosting the UEP. The other metric considered is a bandwidth
score, rating the bandwidth performance of the end to end path between an AEP and the
UEP. All metrics and their values are assumed to be set by the ISP.
In the present evaluation, the initial AEP selection is assumed to be done by the ISP DNS
resolver.
4.1
MUCAPS Prototype Design
The design of the prototype for MUCAPS with the involved hardware is shown in Figure 4.
Figure 4 MUCAPS prototype and involved hardware
4.2
MUCAPS Prototype Set-up and Configuration
4.2.1 TinyDNS
TinyDNS, a part of djbdns software package, is a popular open source DNS server.
It has been modified in order to include the support of Macao API.
Version 1.0
Page 11 of 30
D4.A

Public
In order to install the customized version of tinydns, the following command must be run:
cd /home/alto/workspace/tinydns
./install-tinydns-alto.sh
tinydns-conf tinydns dnslog /etc/tinydns @IP_DNS_Server
where @IP_DNS_Server: represents the IP address of DNS Server or of host.

The next step is to fill the tinydns database with the IPs of the different video servers. This
can be done using the following commands:
./add-ns alto 10.201.65.165
# add dns server
./add-host video.alto 10.201.70.1

make # recompilation to create file data.cdb
The first commands modify a file called /etc/tinydns/root/data that contains the
link between the IP addresses and the domain names. It can also be modified manually.
Here is an example of the data file:
.alto:10.201.65.165:a:259200
=video.alto:10.201.70.1:10
=video.alto:10.201.70.2:10
=video.alto:10.201.170.133:10
=server1.alto:10.201.70.1:10
=server2.alto:10.201.70.2:10
=server3.alto:10.201.170.133:10
The last three entries have been added in order to be able to address the three servers
individually. After the data file is modified, the tinydns-data command should be run in
order to validate the changes in the running tinydns server.
Another file to consider is /etc/tinydns/env/IP. It contains the IP address at which the
tinydns server listens. If it is set to a specific IP (like 10.201.65.165 in the example) it will
only listen from this address. As the server is using a different IP address for LAN access,
Wi-Fi and Cellular, SmartenIT needs the tinydns server to be able to listen to all of these
addresses. The simpler way to achieve this behavior is to put the IP address 0.0.0.0 in
the file /etc/tinydns/env/IP. By doing this tinydns will answer dns queries coming
from any network interface. Tinydns needs to be restarted after changing this file.
4.2.2 Service handling
In order to automate the launching of tinydns it can be used as a linux service. In order to
achieve this the following commands can be issued:
ln -s /etc/tinydns /service/
sudo svscan /service &
Page 12 of 30
Version 1.0
D4.A
Public
svc -u /service/tinydns
svstat /service/*
netstat -anpe | grep 53 # check that dns service is launched on
port 53.
4.2.3 MACAO Block

When no MACAO server is launched, the tinydns server returns the results directly from
its internal database. When the server is launched, the tinydns transmits the results from
the internal database to the MACAO block so that it sorts the results depending on the
configuration.
In order to install and run the server, the following commands should be run in root mode
cd ~/workspace/MACAO2012/ALTO-CLIENT-BLOCK/ALTO_Application_server/
unset http_proxy
./server
# remove proxy if there is.
# launch the MACAO block
The Macao block reads the following configuration files.

4.2.3.1 Newconfig_initial.cfg
This file indicates is ALTO should be used or not and if the Automatic mode is enabled or
not.
Status = 1; # 0=ALTO OFF, 1=ALTO ON
Metric_Automatic = 1; # 0=AUTO mode OFF, 1=AUTO mode ON
4.2.3.2 Mappingconfig.cfg
This file stores the IP addresses of the destination endpoint and identifies the type of
server (video, audio or other) depending of its IP address :
vidservers = [ "10.201.170.133", "10.201.70.101", "10.201.70.102"
];
audservers = [ "10.201.70.5", "10.201.70.6", "10.201.70.7" ];
otherservers = [ "10.201.80.8", "10.201.180.8", "10.201.222.2" ];
4.2.3.3 user_endpoint_config.cfg
This file is used to recognize on which link the client IP address (i.e. the address issuing
the DNS request) is connected.
An example of its content is the following:
LAN = [ "10.201.70.102", "10.201.70.103", "10.201.70.14" ];
WiFi = [ "10.201.70.52", "192.168.1.145", "192.168.1.104" ];
Cellular = [ "10.201.70.10", "192.168.42.208" ];
4.2.3.4 Metric_weights.cfg
This file contains the weights associated to each metric that can be considered, and for
each of the network interfaces.
Version 1.0
Page 13 of 30
D4.A

Public
The syntax is as follows:

LAN :
{
routingcost = 1.0;
hopcount = 1.0;
propadelay = 1.0;
bandwidth = 1.0;
};
WiFi :
{
routingcost =2.3;
hopcount = 4.0;
propadelay = 1.0;
bandwidth = 1.0;
};
cellular :
{
routingcost = 3.0;
hopcount = 2.0;
propadelay = 3.0;
bandwidth = 3.0;
};
4.2.3.5 Newconfig.cfg
This file is used to modify the weights of the different metrics depending on the chosen
configuration (Routing Cost, Routing Cost + BW Cost).
Its syntax is the following
nummetrics = 2; # number of metrics to use
elements = (
{
costtype = 1;
costmode = 1;
weights = 1.0;
},
{
costtype = 2;
costmode = 1;
weights = 1.0;
} );
Page 14 of 30
Version 1.0
D4.A
Public
service = 1;
topN = 0;
4.2.4 ALTO Server

The ALTO server answers to the MACAO requests by giving the costs of the different
metrics for the source and destination IPs considered.
The software was originally designed to use Routing Cost and Hop Count as metrics, but
the decision was later changed to Routing Cost and Bandwidth Score. However, the
software was not updated and the name hopcount is still used in configuration files, but
is now used to represent Bandwidth Score.
4.2.4.1 Network-map.txt
This file allows identifying the equipments access type by their IP address.
The sample file used during the experiments is the following:
###### UEP-PIDS ######
## LAN
mypidLAN
10.201.65.165/18
## WIFI
mypidWiFi
192.168.1.0/24
## 3G
mypid3G
192.168.42.0/24
###### AEP-PIDS ######

mypid101
10.201.70.1/32
mypid102
10.201.70.2/32
mypid133
10.201.170.133/32
4.2.4.2 Pidcosts.txt
This file defines the costs between two equipments depending on the access type.
An example of use is the following:
##### COSTS for routingcost #####
type=routingcost
setall=999
## LAN
mypidLAN,mypid101,7
mypidLAN,mypid102,11
## WIFI
mypidWiFi,mypid101,7
Version 1.0
Page 15 of 30
D4.A

Public
## 3G
mypid3G,mypid101,7
mypid3G,mypid102,11
mypid3G,mypid133,60
##### COSTS for hopcount/BWScore #####
type=hopcount
setall=1
## LAN
mypidLAN,mypid101,8
## WIFI
## 3G
mypid3G,mypid101,8
mypid3G,mypid102,18
mypid3G,mypid133,20
defaultpid,defaultpid,0
4.3
Prototype Instrumentation for Evaluation and Assessment
MUCAPS aims at jointly optimizing ISP transport costs and path bandwidth usage while
maintaining or improving end user QoE. To do so, MUCAPS must
Appropriately choose the AEP selection metrics wrt the type of application,
Use the ISP defined values on the selection metrics, stored in the ISP managed ALTO
Server,
Appropriately set the selection metric weights according to the access type of the UEP,
Rank the candidate AEPs according to the above cited elements
Once the UEP connects to the AEP selected with the MUCAPS support, the resulting
impact must be measured in terms of
User QoE as measured by a fully fleshed QoE measurement and analysis tool,
Theoretical MUCAPS inferred ISP costs based in the ISP ALTO information
Page 16 of 30
Version 1.0
D4.A
Public
4.3.1 QoE performance measurement points and KPIs

The following key performance indicators are considered to evaluate the performance of
MUCAPS. The measurement points are summarized in Table 1.
Table 1 Key performance indicators and measurement points for MUCAPS
KPI
Cross-Layer
(CLU)
End user mobile

devices
End-user PC
Utility
UEP
X
Video Quality Score
Directed to PC
Start time delay
Directed to PC
NFRz, DFRz (number Directed to PC

and
duration
of
freezes
Media Bit Rate
Directed to PC
The Cross-Layer Utility (CLU) is the scalar value that MUCAPS outputs on which its final
ranking is based. It can be measured at the UEP or any device that receives the MUCAPS
response information, including ALTO responses and MUCAPS vector based ranking. It is
a theoretical value that assesses the MUCAPS influence on the AEP choice.
The rest of the KPIs are measurements, done by the VITALU QoE analyzer on PCAP files
associated to video streams.
4.3.2 The VITALU QoE analyzer
VITALU has been developed in ALBLF and transferred to an Alcatel-Lucent Business
Unit. The primary goal of the tool is to evaluate/monitor the QoE for root cause of
multimedia delivery issues. VITALU handles multiple streaming protocols such as HTTP
Live, RTSP, RTP Multicast, or Flash HTTP. VITALU is part of service offer portfolio for
operator customers.
VITALU has the following main functions for each monitored video flow:
Collect/Analyze IP network info (Packet Lost, Jitter, Delay, Timestamp, Sequence

number),
Compute video quality metrics ( Video Score, Freezes, Frame Errors)
Provide frame by frame analysis, with jitter buffer simulation for RTP protocol
The metrics measured by VITALU include: DateTime Capture (s), Date, Clip Name,
Resolution, Operator, Session ID, RTT (Avg, Std, Max), Video Access Time (s), Video
Duration (s), Video Bitrate (bps), VQS Avg (Real), VQS Avg (Theor.), VQS Std (Real),
VQS Std (Theor.), Frame Error (%), Freeze (%), Nb of Freezes, Freeze Duration (Avg) (s),
Freeze Duration (Max) (s), Nb of Error Frames, Nb. of Packets, Packets Retransmitted
(%), Nw Video Jitter (Avg) (s), Nw Video Jitter (Max) (s), Nw Bitrate (Avg) (bps), Nw Bitrate
(Std) (bps), Late & Err. Frame Ratio (%), Frame Rate (Avg) (fps), Frame Rate (Std) (fps),
Nb of Frame Resolutions, Frame Resolution (px).
Version 1.0
Page 17 of 30
D4.A

Public
SmartenIT uses 6 metrics derived from the above measurements list which are: Media Bit
Rate, Start Time, RTT, VQS, Freeze Nb, Freeze Duration (avg), Freeze Duration (max).
VITALU is also able to evaluate these metrics on partially captured streams, if the source
video file is provided.
For this experiment the VITALU tool was used to automatically assess the file captured on
the UEP (after transmission of the captured file to the VITALU server). The results are
then stored to a .csv file that can be analyzed offline.
Page 18 of 30
Version 1.0
D4.A
Public
5 Experiments Definition and Set-up for MUCAPS

The end-user focused (EFS) scenario aims at providing increased QoE and ISP costs and
network resources efficiency. In particular, this goal is achieved by applying an in-network
optimization of application endpoint selection that takes interests of both ISPs and
application resources consumers into account. In this context, MUCAPS revises the initial
endpoint selection done by applications by directly involving of ISP-defined network costs
and performance rating together with the access-type of the end-user device in its final
decision. The MUCAPS intervention is transparent to the user, as the functionality is
embedded in the ISP network. In the SmartenIT use case, MUCAPS is hooked to the ISP
DNS resolver.
The EFS experiments type for MUCAPS will be: prototype validation. Their goal is to
validate its set of functionalities, evaluate its performances and benefits to end-users and
ISPs on a set of sample configurations.
Functional tests will validate the ability of the VLC video reader to integrate MUCAPS
when searching the video, the ability of MUCAPS to catch context information like End
User Endpoint (UEP) access, Application Endpoint (AEP) localization by tinyDNS,
establish a connection with the ALTO Server and the MARS. All this for three UEP
access types: LAN/FTTH, WiFi/ADSL, 3G
The performance evaluation will be done by the QoE analyzer VITALU developed by
ALBLF. During a video stream, TcpDump (or other network analyzer software such as
Wireshark) will be used to capture the PCAP files.
Figure 5 shows the topology of the MUCAPS experiments. Figure 6 shows this topology
mapped to the prototyping environment.
Figure 5 Topology of the MUCAPS prototype experiments
Version 1.0
Page 19 of 30
D4.A

Public
Figure 6 MUCAPS topology mapped to the evaluation environment

Two types of experiments are being conducted
Functional tests by:

o Launching a dig command on the UEP, to validate the prototype,
o Leading a video with VLC (for human visual assessment) or WGET (for automated
tests) on an end-user system that is either a PC or a mobile phone, to see
whether the MUCAPS processing was launched via VLC or WGET.
Impact tests by:

o Capturing the PCAP files generated by TCPDUMP of Wireshark,
o Computing and analyzing the QoE metric values and resulting VQS.
For LAN tests, a web proxy had to be used in order to be able to access external video
servers on the internet. In this case the name resolution was done on the web proxy
server and SmartenIT had to change its configuration so it uses our tinydns server.
For Wi-Fi tests, the PCs had a direct internet connection.
For 3G tests, the mobile phone was used as a usb-modem for the PC hosting the video
player: this was done because SmartenIT has to use a specific DNS server and that is not
possible with an unrooted smartphone. The link to the internet was through the 3G/4G
access of the smartphone but the video player and DNS resolution was performed on a
linux PC.
Some of the problems are linked to the use of a fake name server for our experiments. In
a near future ALBLF plans to use a real name server in order to be able to use the default
DNS configuration and be able to use it directly on unrooted smartphone for example.
In addition, the following Table 2 provides the hosts list and the services used to run the
MUCAPS experiments.
Page 20 of 30
Version 1.0
D4.A
Public
Table 2 Hosts and the services running on them for MUCAPS experiments
Host
Services
Comment
isp1-PC1
UEP receiving the video on a PC
isp1-Phone
UEP receiving a video on a mobile phone
isp1-PC2
tinyDNS+MACAO
isp1-PC3
ALTO Server
isp1-PC4
VITALU QoE Analyzer
isp1-AEP1
Video Server 1 in ISP1
high BW Score, moderate

Routing Cost
isp1-AEP2
Video Server 2 in ISP1
Low BW Score, low Routing

Cost
isp2-AEP3
Private Video Server 3 in ISP2
High BW Score, high Routing

Cost
Table 3 provides the IP addresses of the Hosts listed in Table 2 that are involved in
MUCAPS-assisted video streaming. The QoE analyzer is not listed as it performs off line
analysis. AEP3 is a private server that only has a public address, which will for
confidentiality not be given in the SmartenIT documentation. Addresses of hosts other
than LAN addressed are partially masked for the same reasons.
Table 3 Hosts involved in the MUCAPS experiments and their addresses
Host
PC1-UEP
LAN
VPN
PUBLIC
LAN 172.alu.vpn.uep
WiFi 192.168.pc1.uep
Phone-UEP
3G 1982.198.mph.uep
PC2-DNS+MACAO
172.alu.vpn.165
PC3-AOS
172.alu.vpn.153
AEP1
10.201.70.1
172.alu.vpn.142
sp1.pub.srv.113
AEP2
10.201.70.2
172.alu.vpn.51
sp1.pub.srv.111
spB.prv.srv.37
AEP3
Table 4 provides the ISP defined metric costs, for each AEP class.
5.1
MUCAPS Experiments
The following tests are defined to validate functionality and estimate performance.
5.1.1 Functional tests on the DIG command
The dig command performs a DNS name resolution taking as input the domain name from
the URI of the requested video, composed of the domain name and video file name.
Version 1.0
Page 21 of 30
D4.A

Public
The dig command shows how the name resolution is performed by the VLC reader when
the URI of the requested video is entered, using a system library function such as
gethostbyname.
On a PC, it can be launched as a standalone command with optionally the address of the
desired DNS resolver.
The functional tests consist in launching a dig command on a PC and on a VLC reader
and verify the following elements:
Response with MUCAPS OFF,
Response with MUCAPS ON and both selection metrics RC and BW activated, for all
3 access types.
The functionality tests ensure that the MUCAPS functionality works as expected when the
dig command is triggered by VLC.
The assumption to see the MUCAPS changing the initial selection is that the initial
endpoint selection does not jointly optimize the ISP routing cost and path bandwidth
performance.
5.1.1.1 Parameters, Measurements, and Metrics
ALTO Values for the 3 AEPS
The AEPs have been mapped to 3 classes depending on their end2end path
performances on RC and BWS that is specified in the ALTO Server. MUCAPS performs a
multi-objective vector based ranking of the AEPs, which is more reliable. To better
illustrate the abilities of MUCAPS to do so, the 3 classes of AEP are built so as to be
mutually non-dominated in terms of performances. The RC and BWS values on the AEPs
used for the experiments are the values of the AEP classes utilized for the theoretical
evaluation described in D2.5. The three AEP classes are characterized as follows:
AEP1 Class C1: moderate cost, high path bandwidth: RC = 11, BWS = 18
AEP2 Class C2: low cost, low path bandwidth: RC = 7, BWS = 8
AEP3 Class C3: high cost, high path bandwidth: RC = 60, BWS = 20
Setting ALTO BWS and RC values

Although ALTO values are supposed to be specified by the ISP, the BWS values are set
so as to be mapped to the downlink rates measured in France with different access types.
Sources [2] and [3] provide access-dependent value ranges (min-max) for e2e downlink
path bandwidth, based on measurements performed from 2013 to today in France.
SmartenIT, thus, considers the following ranges and access types:
FTTH [45-160] Mb/s
ADSL [6-9] Mb/s
Cell 3G: [4-7] Mb/s,
In the AOS used by MUCAPS, BWS is a score with values in [1, 20]. In order to mitigate to
effect of FTTH (Fiber-to-the-Home) performance, the mapping is done so as to mitigate
the effect of FTTH performance. This approach took FTTX as a mixture of FTTH, FTTB
(Fiber-to-the-Basement), and TTS (Fiber-to-the-Street).
Page 22 of 30
Version 1.0
D4.A
Public
Likewise, RC is completely dependent of the ISP policy. SmartenIT nevertheless assumes

that RC may be influenced by factors such as OPEX and peering agreements.
The resulting mapping was, therefore, adopted as a basis for example AOS values.
Table 4: Typical values for metrics depending on access type
ACCESS
BW Range
BWS median
RC basis
Cell 3G
[4-7] Mbs
6.7
15
WIFI/ADSL
[4-9] Mbs
7.2
FTTX
[45-140+] Mb
17.5
11
The ALTO values of the AEPs have, therefore, been chosen as defined in Table 5.
Table 5: ALTO values for each AEP
RC
BWS
AEP1
11
18
AEP2
AEP3
60
20
Finally, Table 6 recalls weights associated to those metrics depending on the UEP access
type.
Table 6: Weights of metrics depending on access type
FTTX
WiFi/ADSL
3G
RC
2.3
BWS
5.1.1.2 Functionality test documentation

The results of functional tests on the dig command will be documented in 2 different ways:
A screen copy of the result of a dig command will be provided, as an illustration
Results of dig commands will be provided in terms of RC, BWS and CLU value
associated to the AEP selected in the different modalities of MUCAPS activation.
MUCAPS test modalities: for each UEP access type

o G1: MUCAPS OFF
o G2: MUCAPS ON + RC
o G3: MUCAPS ON + BWS
o G4: MUCAPS ON + (RC, BWS)
Version 1.0
Page 23 of 30
D4.A

Public
5.1.2 Performance tests with QoE analysis

The performance tests quantify the impact of MUCAPS on the QoE
5.1.2.1 Parameters, Measurements, and Metrics
Deployment Infrastructure
The deployment infrastructure is the one depicted in Figure 6. The location of the different
elements is as follows:
The ALTO Server (AOS), AEP1, AEP2 are located in the ALBLF premises,
The UEP and MACAO blocs are located in remote premises of the Paris area and
connect to AOS, AEP1 and AEP2 via a VPN
Server AEP3 is a private server belonging to another ISP
Assuming the initial selection is AEP3 the tests groups have been defined as follows.
Video sessions (2 types wrt length, resolution, internals such as movements).
With UEP in different access types: LAN/FTTX, wifi/ADSL, cellular 3G
With different devices (PC, Phone)
AEP types: video public server, private server
QoE Measurements with VITALU:
On each video session: the QoE metrics will be measured
MUCAPS test modalities: for each video class and each UEP access type, the tests
will only consider:
G1: MUCAPS OFF
G4: MUCAPS ON + (routingcost, BWscore)
For each test group Gi, the QoE performance is measured.

The QoE results are compared and impact is derived.
5.2
MUCAPS Showcase
MUCAPS can be showcased following the 2 scenarios proposed below: Scenario 1 as

Video Streaming on a High Resolution PC and scenario 2 as Video Streaming on a
Mobile Phone.
When the ALBLF VPN is accessible to the UEP, the experimental topology as described
in Figure 7 is suitable.
If the ALBLF VPN is not accessible to the UEP and to minimize the involved hardware
MUCAPS can be showcased using the topology depicted in Figure 8.
The showcase topology could even be reduced to having PC1 and PC2 merged so as to
host the whole (UEP, tinyDNS, MACAO, AOS) set as all its elements can be assumed to
belong to the same ISP.
Page 24 of 30
Version 1.0
D4.A
Public
Figure 7 Initial MUCAPS showcase topology
Video
Server
AEP3/ISP2
VM
VM
tinyDNS
resolver
MACAO
VM
PC2/ISP1
ALTO Server
eth
eth
WLAN
Video
Server
AEP1/ISP1
eth
Video
Server
AEP2/ISP1
UEP
VLC
PC 1/ISP1
VLC
PHONE
RAN
Figure 8 MUCAPS showcase topology when the UEP is not accessing its VPN
5.2.1 MUCAPS Showcase Scenario 1 Video Streaming on a High Resolution PC
5.2.1.1 Scenario Topology
The MUCAPS showcase topology on LAN is illustrated in Figure 9.
Version 1.0
Page 25 of 30
D4.A

Public
Figure 9 MUCAPS showcase topology on LAN

5.2.1.2 Showcase Scenario
This scenario goes through the following steps
1. UEP on a PC requests a video V1 without MUCAPS
a. DNS selects AEP3
b. Transfer at high routing cost (RC), high path BW score (BWS)
2. UEP requests V1 with DNS assisted by MUCAPS+RC
a. DNS-MACAO points to AEP2
b. Transfer at low cost but low quality
3. UEP now requests V1 with DNS assisted by MUCAPS + (RC, BW)
a. DNS-MACAO points to AEP1
b. Transfer at moderate RC AND high BWS
5.2.2 MUCAPS Showcase Scenario 2 Video Streaming on a Mobile Phone
5.2.2.1 Scenario Topology
The MUCAPS showcase topology on 3G is illustrated in Figure 10.
5.2.2.2 Showcase Scenario
This scenario goes through the following steps and follows showcase of scenario 1.
1. UEP on a mobile phone gets V1 on AEP1
2. It gets a bad QoE
a. because the mobile phone cannot sustain the HR bitrate
3. UEP requests video V1 with DNS assisted by MUCAPS + (RC, BW)
a. DNS-MACAO takes 3G-access type into account and points to AEP3
b. UEP gets a good QoE because AEP3 delivers at a sustainable bitrate
c. Transfer at low RC
Page 26 of 30
Version 1.0
D4.A
Public
Figure 10 MUCAPS showcase topology on 3G
Version 1.0
Page 27 of 30
D4.A

Public
6 Summary and Conclusions

This deliverable complements the SmartenIT Deliverables D4.1 and D4.2 by adding the
respective details of the MUCAPS mechanism. In particular, this annex here
Details how MUCAPS can be integrated in the SmartenIT Architecture
Presents the MUCAPS prototype set-up and how it is configured during the showcase
Analyses the results of experiments
Defines the main showcase scenarios.
The main outcomes from these experiments is that it was possible to build a prototype that
is able to automatically direct an user to the best suiting server depending on the type of
service requested (e.g. video, audio, web), and the type of access of the user (LAN, Wi-Fi,
3G). The use of an offline analyzer also shows an improvement in the quality experienced
by the user thanks to MUCAPS mechanism rather than using static routing tables.
The 2 different showcase scenarios help to understand different configurations possible
with MUCAPS, taking into account different metrics and showing which choice will give the
best result. Each of the metrics is attributed a different weight depending on the chosen
access type, hence providing different results. The showcase, thus, allows to achieve the
impact of each of the metrics and of the access type on the chosen server for a specific
type. The selected application for the demo is video streaming, as the different quality
levels can be easily distinguished and show that the choice of the server will impact the
quality experienced by the user.
In conclusion, thanks to the MUCAPS prototype used in the experiments and showcases,
the benefits linked to this mechanism can be evaluated both objectively and subjectively.
The deployment of this mechanism can bring advantages inside the SmartenIT solution or
independently and proves to be a very good solution in terms of network performance and
user experience.
Page 28 of 30
Version 1.0
D4.A
Public
7 References
[1]
draft-ieft-alto-multi-cost-01: Multi-Cost ALTO, (work in progress) S. Randriamasy

and W. Roome and N. Schwan, October 19th 2015, https://tools.ietf.org/html/draft-ietfalto-multi-cost-01
[2]
"Baromtre des connexions Internet mobiles en France mtropolitaine Deuxime

trimestre
2015
",
nPerf
report,
3
Juillet
2015,
http://media.nperf.com/files/publications/2015-07-03_Barometre-connexions-mobilesnPerf-2015-T2.pdf
[3]
"Observatoire
des
dbits
Internet",
(1/05/2013
http://www.ariase.com/fr/vitesse/observatoire-debits.html
[4]
The SmartenIT project: Deliverable D2.2: Report on Definitions of Traffic

Management Mechanisms and Initial Evaluation Results; October 2013.
[5]
The SmartenIT project: Deliverable D2.4: Report on Final specifications of Traffic

Management Mechanisms and Evaluation Results; October 2014.
Version 1.0
to
09/2015),
Page 29 of 30
D4.A

Public
8 Abbreviations
ADSL
Asymmetric Digital Subscriber Line
AEP
Application Endpoint
AEPG
Application Endpoint Gathering
ALTO
Application Layer Traffic Optimization
AOS
ALTO Server
AS
Autonomous System
BWS
BandWidth Score
CLU
Cross-Layer Utility
DTM
Dynamic Traffic Management
EFS
End-user Focused Scenario
EUE
End User Entity
FTTB
Fiber-to-the-Basement
FTTH
Fiber-to-the-Home
FTTS
Fiber-to-the-Street
FTTX
Fiber-to-the-X
IETF
Internet Engineering Task Force
ISP
Internet Service Provider
MACAO
Multi Access and Cost ALTO
MARC
MACAO Service Client
MARS
MACAO Service interface
MUCAPS
MUlti-Criteria Application endPoint Selection
OFS
Operator Focused Scenario
OPEX
Operational Expenditure
PCAP
Packet CAPture
QoE
Quality-of-Experience
QoS
Quality-of-Service
RC
Routing Cost
RFC
Request for Comments
SmartenIT
Socially-aware Management of New Overlay Application Traffic with Energy

Efficiency in the Internet
UEP
End User Endpoint
uNaDa
User Nano Data Center
VQS
Video Quality Score
Page 30 of 30
Version 1.0

Delivarable D4.A - Annex To D4.1 and D4.2 MUCAPS Prototype and Experimentation

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Delivarable D4.A - Annex To D4.1 and D4.2 MUCAPS Prototype and Experimentation

Uploaded by

Copyright:

Available Formats

Seventh Framework STREP No.

The SmartenIT Consortium

Copyright 2015, the Members of the SmartenIT Consortium

Seventh Framework STREP No. 317846

D4.M MUCAPS Prototype and Experiments

Sabine Randriamasy, Frdric Faucheux

Sabine Randriamasy, Frdric Faucheux

Revised version after comments by Roman Lapacz and Jeremias

Sabine Randriamasy, Frdric

Completed version after fixing formal issues

Seventh Framework STREP No. 317846

Executive Summary .................................................................................................. 5

MUCAPS and VITALU in the SmartenIT Architecture............................................. 7

MUCAPS Prototype Design and Instrumentation ................................................. 11

Experiments Definition and Set-up for MUCAPS.................................................. 19

Summary and Conclusions .................................................................................... 28

Seventh Framework STREP No. 317846

(This page is left blank intentionally.)

Seventh Framework STREP No. 317846

Seventh Framework STREP No. 317846

Seventh Framework STREP No. 317846

3 MUCAPS and VITALU in the SmartenIT Architecture

Topology distance: number of hops (inter or intra AS), impacting delay,

Routing cost: reflecting the ISP policy in terms of AS peering agreements,

ISP preferences in terms of resources availability (e.g., path bandwidth).

decides for which metrics an ALTO Client must request values,

decides which weight is applied to the metrics, and then

uses the obtained values to rank the AEPs.

Seventh Framework STREP No. 317846

Figure 1 Involvement of MUCAPS components (MACAO, MARC and MARS) in a content

An alternative mapping of MUCAPS to the SmartenIT architecture is also described in

Seventh Framework STREP No. 317846

Figure 2 Entities and components involved in MUCAPS.

Seventh Framework STREP No. 317846

Using VITALU to Evaluate MUCAPS

Seventh Framework STREP No. 317846

4 MUCAPS Prototype Design and Instrumentation

Evaluate the impact of the MUCAPS selection on the QoE.

A MUCAPS intervention is qualified as appropriate whenever the initial AEP selection

MUCAPS Prototype Design

Figure 4 MUCAPS prototype and involved hardware

MUCAPS Prototype Set-up and Configuration

Seventh Framework STREP No. 317846

where @IP_DNS_Server: represents the IP address of DNS Server or of host.

# add dns server

./add-host video.alto 10.201.70.1

Seventh Framework STREP No. 317846

4.2.3 MACAO Block

# remove proxy if there is.

# launch the MACAO block

The Macao block reads the following configuration files.

Seventh Framework STREP No. 317846

The syntax is as follows:

Seventh Framework STREP No. 317846

4.2.4 ALTO Server

###### AEP-PIDS ######

Seventh Framework STREP No. 317846

Prototype Instrumentation for Evaluation and Assessment

Rank the candidate AEPs according to the above cited elements

Seventh Framework STREP No. 317846

4.3.1 QoE performance measurement points and KPIs

End user mobile

Video Quality Score