Table of Contents

Introduction..........................................................................................................................2
1. IPSec and Implementation of Security............................................................................3
2. IPSec’s Robustness and Scalability.................................................................................7
3. Limitations of IPSec........................................................................................................8
4. Best Practices of IPSec Configuration and Management................................................9
Conclusion.........................................................................................................................10
Bibliography .....................................................................................................................12
Introduction

Today, the communication between networks, that are being established, have a

strong need of good security mechanisms in order to ensure the security, integrity,

confidentiality and authenticity between two hosts or two networks. The most common

services of IPSec implementation are VPN (virtual private networking) services that can

be used over existing networks e.g. internet, can provide the secure transfer of sensitive

data over public networks. (Sheila Frankel et al., 2005).

The reality that the Internet is deficient in security is still undeniable. So to solve

this issue researchers are trying to increase the network security at each layer by

designing a range of security protocols. The designed protocols include PGP, S/MIME,

SET which are specifically designed to secure the application layer; SSL/TLS are

designed to work on the transport layer. In this race, IPSec is also a security standard

proposed by the IETF, that concerns with the security on the network layer, processes

data packages on the IP packet layer, makes available security services such as access

control, data source authentication, integrity, data confidentiality etc. (Liangbin Zheng,

Yongbin Zhang, 2009)

The fundamental idea behind the specification of IPSec is to provide security

utilities, authentication of the source, content integrity and confidentiality, at the IP

(Internet Protocol) level that exists o network layer. This necessitates a higher-level

management protocol, Internet Key Exchange (IKE), to establish security association (the

context and parameters) for choosing cryptographic keys and performing mutual

authentications, making safe data transfer, possible. The data transfer through IPSec uses
one or both of two other protocols. First is, Authentication Header (AH) that provides

source authentication and data integrity. Second protocol is Encapsulating Security

Payload (ESP), that provides data confidentiality and authentication. (Heng Yin, Haining

Wang, 2007)

Structure of the report is as follows: 1st section describes the IPSec standard and

implementation of security in the network using IPSec. 2nd section elaborates the

robustness and scalability of IPSec standard with respect to other standards. 3 rd section

describes some of the limitations of IPSec implementation. 4th section highlights some

best practices that have been observed as accelerating network communication and

providing a better security against attacks. And the last section summarizes the

conclusions.

1. IPSec and Implementation of Security

IPSec that basically stands for Internet ProtocolSecurity defines a fundamental,

low level mechanism for secure communication between two hosts or networks for use

with the Transmission Control Protocol/Internet Protocol (TCP/IP), which is the

protocol being used on the Internet and on other private networks such as LANs or

intranets. It was mainly designed for the new IPv6 standard but can optional be used with

IPv4. (Ralf Spenneberg, 2003)

According to (Sheila Frankel et al., 2005), IPSec standard is based on a set of

protocols to implement the security in network layer. These protocols include: two

security protocols, Authentication Header (AH) and Encapsulating Security Payload

(ESP), Internet Key Exchange (IKE) protocol, IP Payload Compression Protocol

(IPComp), which is used optionally. IPSec protocols work together in various

combinations to provide protection for communications. A brief description of each

protocol is given below:

Authentication Header (AH)

AH is basically used by IPSec to provide integrity protection for packet headers

and data, but it is not designed to encrypt them as compared to ESP (discussed below)

which can provide encryption and integrity protection for packets but as compared to AH,

ESP cannot make the outermost IP header, secure, as AH can. Though, this protection is

not needed in most cases.

Encapsulating Security Payload (ESP)

The frequency of the usage of ESP is much more than AH because it facilitates

more encryption capabilities, as well as other operational advantages over AH. For a

VPN, which requires confidential communications, ESP is the natural choice. ESP also

allows encryption-only and authentication-only configurations, but using such schemes is

usually not recommended because it is insecure. Unlike Authentication Header (AH),

ESP does not provide security for IP packet header. Though, in Tunnel Mode, where the

entire original IP packet is encapsulated with a new packet header added, ESP supports

the protection of the whole inner IP packet which also includes the inner header, at the

same time as the outer header remains insecure

Internet Key Exchange (IKE)

 IPSec uses IKE to agree IPSec connection settings. The main purpose is to

authenticate the endpoints to each other and also specifying the security parameters of

IPSec-protected connections. It is used for setting up the encryption keys and managing,

updating, and deleting communication channels that are protected by IPSec.

IP Payload Compression Protocol (IPComp)

IPSec uses IP Payload Compression Protocol (IPComp) optionally, to compress

packet payloads before encrypting them. This protocol will increase the overall

communication performance between a pair of communicating hosts by compressing the

packet payloads.

IPSec works in two modes, with each mode providing its own functionality.

These modes are tunnels mode and transport mode (Sheila Frankel et al., 2005, Ralf

Spenneberg, 2003). Both modes are described below:

Tunnel mode

When working in tunnel mode, security gateways are needed to provide support

for tunnel mode connections. Client machines can use the tunnels provided by the

gateways for routing purpose. The client machines do not require any IPSec processing,

they just have to perform their usual tasks such as routing things to gateways.

Transport mode
 To work in transport mode using IPSec implementation Host machines (as

opposed to security gateways) must also support transport mode. In this mode, the host

performs its own IPSec processing and routes some packets by means of IPSec.

Implementation of security through IPSec

According to (Liangbin Zheng, Yongbin Zhang, 2009), IPSec implements the

security in a network by maintaining the security associations. Security Association (SA)

works as the basis for IPSec, which determines the security parameters that will be used

in communication to make it secure, such as IPSec security protocol, hash function,

encryption algorithm and encryption key. Security Association is typically specified by a

unique triple (security parameter index, destination IP address, security protocol).

Security Associations are materialized in pairs, one in each of the communication peers.

These associations are determined after the negotiation between the communicating hosts

in the networks. To store these security associations, special Security Association

Database (SAD) is designed. Additionally, IPSec also maintains a Security Policy

Database (SPD). Every network interface that is secured by IPSec, possesses a pair of

Security Policy Database and Security Association Database, which cooperates with

processing inbound and outbound IP packets. One Security Association Database entry is

equivalent to a Security Association, whereas, one entry in the Security Policy Database

depicts a security policy. When data is sent to the destination host, the corresponding

policy in Security Policy Database is retrieved, if the recorded action is to “apply” the

data transfer (as specified in the security policy), then corresponding Security

Associations are retrieved according to the Security Association pointer. In case, if the
Security Association does not exist in the Security Association Data base, then a new

Security Association is created and stored into the database. Once Security Association

has been retrieved from the database, the data packets are processed with the security

protocol and authentication encryption algorithm specified in the Security Association.

Then the processed data packets are sent to the IP of destination host. The receiver side

discovers the Security Association according to the Security Parameter index parameter

in the datagram, and verifies if retransmission of data is required. Otherwise, the data is

decrypted and authenticated with the protocol specified in the Security Association.

2. IPSec’s Robustness and Scalability

According to (Ole Martin Dahl, 2004), IPSec is really a robust and scalable

standard for providing network security. it is basically designed for IPv6 but also scalable

with IPv4. IPSec offers security directly on the IP network layer and secure everything

that is put on top of the IP network layer. IPSec protocol has also been established as an

Internet standard for quite some time and has been confirmed to be a safe and trusted

mechanism to provide the security in communications in a network or between the

networks. IPSec also allows us for the use of nested tunnels i.e. if a user must move

across two or more secure gateways the tunnels can be double encrypted.

(GTE Internetworking, 1999) have also highlighted some prominent features of

IPSec, that make this protocol more robust as compared to other security standards. IPSec

allows for transparency as One of IPSec’s noticeable strong points lies in the integration

of encryption and authentication methods with robust and full-featured key exchange

Algorithms and protocol negotiation features to provide security against vulnerabilities

on network layer. IPSec is complete package including both, a tunneling technology and

a security technology. It enhances robustness as using tunneling without encryption

facilitates no security against many forms of attack. Tunneling for an organization may

not be just concerned with securing external routers from dealing with internal addresses.

It may also be adopted for hiding those addresses from attackers beyond the firewall.

Now days, because of many powerful attacker tools, security mechanisms that perform

no authentication of the source and destination of every IP packet may provide worst

results than no authentication at all. IPSec real strength lies in the fact the as compared to

other standards, it combines tunneling, authentication, and encryption in a package that

provide the organizations with a secure route between private networks, or into a network

from a trusted host, while traveling right through a public network such as internet. IPSec

is a scalable security standard and also promises for interoperability i.e. its spans all the

vendors and platform same as IP do.

3. Limitations of IPSec

Despite of IPSec’s strengths over other security standards, it also has some

limitations that may degrade the performance of network, implementing the IPSec

standard. (HP Networking, 2001) define some limitations that, specifically, IPSec/9000-

secured systems in a network usually have. These are:

• When an IPSec/9000 system stops working and the system had already created

ISAKMP (Internet Security Association and Key Management Protocol) Security

Accusations with peer IPSec systems, the peers will not be capable of using any
 existing ISAKMP and IPSec Security Accusations to start communication with

the peer system that has just restarted.

• When the IPSec Security Associations are configured to be shared betweens

peers, the peer system can not initiate any communication with the restarted

system which is using same IPSec Security Associations. But existing Security

Association have to be expired for this purpose.

In addition, IPSec security standard have some limitations in general. These are:

• IPSec is not able to provide the same end-to-end security for the systems that are

working at higher levels. IPSEC supports the encryption of an IP connection

between two machines, but it is not applicable for higher level security such as

encrypting messages between users or between applications.

• IPSec does not provide support for the stoppage of Denial of Service attacks.

• IPSec does not provide protection against analyzing the unencrypted headers of

encrypted packets such as source and destination’s gateway addresses and packet

size etc. This information can be acquired by attackers with some intelligent tools.

4. Best Practices of IPSec Configuration and Management

IPSec has been designed as a standard to provide the security in communications

within and between the networks. Researchers have put great efforts to use this standard

in the most efficient manner to make the communications more secure and safe.

(Yunhe Zhang et al., 2009) have proposed a strategy to configure the IPSec

standard for achieving best communication performance. Their strategy is based on IPSec
Thumbnail Protocol (ITP) to speed up IPSec communication. According to them,

communication speed can be accelerated by caching data segments of the original IP

packet and constructing ITP Thumbnail packet to transfer. They have also shown the

validity of their proposed strategy by implementing an ITP prototype system on Linux

platform and have evaluated it in the test environment. The experimental results have

shown a great improvement in IPSec’s communication performance.

(Liangbin Zheng et al., 2009) have proposed to use a dynamic pre-shared key

generation mechanism that may keep the system away from the harm due to the crack of

the pre-shared key in IKE protocol . The new practice involves the method that generates

the pre-shared key dynamically before deciding the security associations. So the new pre-

shared key will be generated every time when the security association is created.

Generating the pre-shared key dynamically before the security association creation,

allows for two way authentication. If the authentication through the shared key is not

successful, then security associations can not be established. So configuring IPSec

Standard in this way can effectively defend against the DoS attacks.

Conclusion

This report presents a brief introduction of a security standard, called, IPSec and its

capabilities in ensuring the secure communication in the network. IPSec basically uses a

combination of protocols such as Authentication Header (AH), Encapsulating Security

Payload (ESP), Internet key exchange (IKE) and IP Payload Compression Protocol

(IPComp), which is used optionally. Each protocol plays its part in improving the

security, integrity and confidentiality of communication by using different algorithms for

encryption and authentication. IPSec is usually implemented by maintaining security

associations which are stored in security association database and are retrieved according

to the actions specified in the security policies that are stored in security policy database.

Though IPSec provides a better, scalable and robust mechanism for ensuring the security

in communications, as compared to other standards, but it also have some limitations as it

can not resist DoS attacks. However, there are some strategies that have been proposed

and are being followed to improve the effectiveness of IPSec standard. These practices

ensure better performance of IPSec in speeding up the communication as well as

protecting it against DoS attacks while using IPSec standard.

Bibliography

Liangbin Zheng, Yongbin Zhang, 2009. An Enhanced IPSec Security Strategy,

International Forum on Information Technology and Application, pp. 499-502

Heng Yin, Haining Wang, 2007. Building an Application- Aware IPSec Policy System,
IEEE/ ACM Transactions on Networking, volume 15.

Yunhe Zhang, Zhitang Li, Song Mei, Ling Xiao, Meizhen Wang, 2009. A new Approach
for Accelerating IPSec Communication, in the proceedings of International Conference
on Multimedia Information Networking and Security, pp. 482-485.

William Stallings, 1995, Network and Internet Security: Principles and Practice, IEEE
Computer Press, ISBN 0-02-425483-0.

Sheila Frankel, Karen Kent, Ryan Lewkowski, Angela D. Orebaugh, Ronald W. Ritchey,
Steven R. Sharma, 2005. Guide to IPSec VPNs [online] available at:
csrc.nist.gov/publications/nistpubs/800-77/sp800-77.pdf [accessed: 9th November 2010]

Ralf Spenneberg, 2003. IPSec HOWTO [online] available at: www.IPSec-

howto.org/IPSec-howto.pdf [accessed: 9th November 2010]

Ole Martin Dahl, 2004. Limitations and Differences of using IPSec, TLS/SSL or SSH as
VPN- Solution [online] available at: olemartin.com/projects/VPNsolutions.pdf [accessed:
9th November 2010]

GTE Internetworking, 1999. IPSec VPNs with Digital Certificates: The Most Secure and
Scalable Approach to Implementing VPNs [online] available at:
www.firstnetsecurity.com/library/gte/GTE%202.pdf [accessed: 10th November 2010]

HP Networking, 2001. Installing and Administering IPSec/9000 [online] available at:

http://docs.hp.com/en/J4255-90011/J4255-90011.pdf [accessed: 10th November 2010]