CPUG: THE CHECK POINT USER GROUP

How to Debug

IKE and IPSEC

An Abbreviated Guide By Warren Verbanec

Your Host
Warren Verbanec UCDavis graduate Silicon Valley Local Two years in Nokias Product Line Support group

Hello Everybody!

Goals for tonight

Eat free food Win raffle Discuss how Check Point implements IKE and IPSec Review available troubleshooting tools Provide practical examples

First, a bit of review

What is IKE and IPSEC (in our context)?
Generally speaking, IKE is a method for securely exchanging encryption ciphers that will be used in a later encrypted session IPSec is an overall term used to describe encrypted data communication over IP, using the keys exchanged with IKE

This is Part One

IKE and IPSec is a huge topic. Theres no feasible way to cover it in of an hour Therefore, Part One will provide a basic introduction to only the concepts required to work with Check Point debugging Part Two will provide hands-on examples of troubleshooting the application Part Two will be presented next month

Whats the big deal?

These are complex protocols ratified by multiple international bodies Lots of configurable parameters Generally define a framework for security, while allowing for advances in cryptography
MD5 collision demonstates need for expandability http://eprint.iacr.org/2004/199.pdf

Thus, debugging and configuration is hard

How does encryption work?

In its basic form, cryptography entails the addition of two messages to make a new, unreadable message Message1 + Key Message = Cipher If you know the Key Message, you can decode the cipher by subtracting the Key information from the cipher Computers are good at this, and can do it very quickly- say, on a packet by packet basis But this means that you have to keep the secret key safe This makes the process difficult- generally speaking, if you lose the key to the bad guys, ALL information ever encrypted with that key is now readable

Basic crypto applications

Algorithms: DES, AES, 3DES Used for actual reversible encryption non-entropic, reversible operations Requires a unique secret key for the encryptor and decryptor Hashes: SHA-1, MD5 Used to generate a unique mathematical summary value for a given dataset Entropic, non-reversible operation Used to authenticate a data set Can be combined with a secret key value to create a custom Hash- ensures that your hash was created by someone you trust.

How does SSL differ?

SSL is based on PKI, which uses public/private key pairs- using entirely different math Designed to enable secure transfer of data (like a temporary crypto key) to someone you dont necessarily trust IKE/IPSec does not use PKI, as it is inherently less safe- and designed for e-commerce use Actually, PKI-like key exchange is used in some limited ways in IKE, but the core of IPSec is not based on public/private key exchange A discussion of PKI is beyond the scope of this presentation A good PKI tutorial is:
http://www.opengroup.org/messaging/G260/pki_tutorial.htm

That being said

PKI uses public/private key pairs Anything encrypted with the private key is readable with the public key Anything encrypted with the public key is readable with the private key Anything encrypted with the public key is NOT readable with the public key The core of IPSec uses a single key- anything encrypted with it is readable only with the same key Diffie-Hellman key exchange, and PKI certificates are used in IKE and IPSec, but in a limited way by Check Point The extent to which public/private key exchange is used in IPSec will be discussed later

So, how does a Hash fit in?

A cryptographic hash is used to derive a mathematical summary value for a set of data Data hashed with a particular algorithm generates a unique output value This value has a one-to-one correlation with its data set- this ensures that if a data set is altered, its hash value will change This is good for ensuring data integrity

Diffie-Hellman is key!
Remember, the problem is not just encrypting the messages- its keeping your keys safe in the long term
This is accomplished by renegotiating keys often in IPSecthis compartmentalizes the encryption and data exchange This means that secret keys must be exchanged often

Diffie-Hellman key exchange defines how to use public/private key pairs to transport your secret keys D-H group numbers define the strength of the public/private key encryption used- Check Point just added new Group support in HFA 55_10

And now, IPSec!

There are three parts to IPSec:
AH- authentication header- provides session security at a sophisticated level by checking data integrity and protecting against replay attacks ESP- encapsulating security payloadprovides the bulk data encryption method IKE- handles the exchange of secret keys used in the prior two categories

AH
In the operational mode used in VPNs, AH wraps an IP packet (header and all) in an encryption envelope, then adds a new IP header This process is performed at a VPN gateway, and is undone at the terminating gateway at the other end of the secure tunnel AH uses IP Protocol 51- so its not UDP or TCP AH is not too relevant to the Check Point world

More AH
AH has several fields in its header:
Security Parameter Index is a numeric identifier that specifies a particular logical connection
This SPI is tracked on the gateways along with the encryption parameters associated with it (hash algorithm, bulk encryption algorithm, other parameters)

Sequence number field is used to track individual packets

Optionally used to protect against replay attacks

ESP
ESP is used for the bulk encryption
Its basically an algorithm-encrypted packet inside a PKI signature wrapper for authenticity ESP uses IP protocol 50 for the transport- this is what you commonly see in packet traces of tunnel traffic Has a SPI field, like AH, as well as the optionally utilized sequence number in its header ESP is the core method for bulk VPN data transmission with Check Point

ESP again
A new term: Each logical session that utilizes a unique SPI is referred to as a security association or SA And to clarify: AH is used by the encryption stack to verify data integrity, while ESP is used to perform the actual transport of the encrypted data You will generally see a single ESP packet for each encrypted packet inside the tunnel

IKE
IKE is the glue that binds ESP and HA It is the protocol that handles the initial key exchanges between gateways on either side of a VPN tunnel It defines the parameters utilized for an SA The number of parameters that can be defined by the IKE process is staggering- but Check Point only uses a small subset

SAs
SAs are the heart of debugging a VPN tunnel If you can understand the IKE initialization process, you will be able to track where individual SAs are breaking SAs are unidirectional (why is this never mentioned anywhere?) Remember: the SPI is the actual number we are referring to when we look at SAs

Understanding IKE Better

IKE is a collective term for ISAKMP and Oakley IKE establishes its own logical SA for its key exchanges but this one is BIDIRECTIONAL IKE uses UDP port 500, although you can force the use of TCP NOTE: IKE generally cannot be NATted, as the IP addresses used by each participating gateway are tracked, and NAT looks like a replay attack But this only applies to NAT of the IPSec traffic itself (post-encryption). Traffic can be NATted by the firewall prior to being encrypted However, many problems can occur while defining NATted encryption domains

The guts of key exchange

Sending Gateway determines a packet needs to be encrypted Sending Gateway opens an IKE session with the Receiving gateway- this step defines the IKE SA Diffie-Hellman key exchange uses hashing of a certificate or shared secret to authenticate each gateway, and sets up a public/private data exchange channel Sending and Receiving Gateways exchange protocol settings, algorithm settings, and secret keys using PKI A new IPSec SA is defined for the ESP tunnel, and data begins to be transferred New term: Selector- a logical construct similar to a route, that allows the gateway to determine if an inbound packet is to be encrypted and passed over a particular SA

IKE Phases
Phase One is used to actually to the work of exchanging and negotiating the parameters that will be used
Can be done in the full Main Mode way, or an abbreviated Aggressive Mode, where some encryption security steps are skipped. Aggressive mode is not recommended, as it doesnt really save you much time (IKE is done irregularly)

Phase Two (aka Quick Mode) is used to negotiate the SAs that will be used for later communication
Quick Mode does not mean the same thing as aggressive mode

IKE Parameters
Check Point gateways require the following information to be set for each tunnel:
Bulk encryption algorithm for the ESP session Hash algorithms used in the IKE authentication Diffie-Hellman group to be used What the authentication source will be: Certificate or Shared Secret Other miscellaneous stuff (SA definitions on a pernetwork basis, etc..)

IKE packets
When you sniff IKE, youll usually see:
Six packets for Phase One Main Mode Three for the forbidden Aggressive Mode Three or Four packets for Phase Two

These steps are computation-intensive, and so they take a while An aside: what is Perfect Forward Secrecy? Nothing you need.

Tunnel Test?
At the end of IKE establishment, vpnd attempts to send some ICMP traffic across the tunnel. If the packet does not arrive, or if the IP addresses are mangled (not encrypted when sent, etc..), the gateway will report tunnel test failed This often fails due to NAT or encryption domain issues What is an encryption domain? The set of network addresses that are defined to be available on one side of a particular tunnel

A final word on SAs

SAs have a lifetime. The protocol specification itself recommends several ways of doing this. Some vendors (Cisco) have calculated lifetimes on a amount of traffic across the tunnel basis. Check Point and most everyone else does it on a time since session establishment basis The point of having a lifetime is to force renegotiation of the secret keys on a regular basis- thus increasing security Any gateway that participates in an IPSec session can manually end its own SAs

A bit more explanation..

Certificates are part of PKI, but you can use them to authenticate a gateway in IKE This is a limited utilization of PKI- and the certificates are typically not public CA generated Certificates are generally a better method of security, as they have a set expiration date, and can be tracked centrally at a CA Also, certificates are a larger data set from which the authentication hash is generatedthus increasing security

Its 2AM
So, your VPNs downwhat do you do? I personally have a bit of a flowchart I follow, with increasing levels of interference in network operations Thats the real trade-offwhat do you reboot, and what will it effect? Often, config is the culprit, but CP is notorious for VPN bugs (although better in R55)

Bugs? What Bugs?

R55_10-14 FireWall-1 When a large database was employed, vpnd utlilized 100% of the cpu and then exited during: initializaton policy install, simulaneous IKE with many DAIP objects.

So..What to do?
Check Point provides several tools for debugging VPNs CLI commands: vpn debug ikeon is the most valuable This generates IKE.elg vpn debug on generates vpnd.elg IKE.elg is the most important- and Check Point provides a tool for translating its gibberish: Ikeview This is part of the infoview package available on their support site to CSPs

Ikeview
Breaks down Phase One and Two on a perpacket basis Useful for seeing mismatches in configuration Be sure to use the latest version of Infoview (3.5.3x) available from Check Point You will need to be a CSP to get access- talk to your Sales rep if you are an enterprise customer

Out of time? Uh-Oh

Perform vmstat and ps aux to see if vpnd is hogging resources If so, HUP vpnd, or reboot both gateways on either side of the tunnel The vpn tunnelutil command allows you to view and clear SAs manually from Check Points tables IKE_SA_table for IKE Inbound_SPI and Outbound_SPI for IPSEC Tunnelutil is downloadable for early FW-1 versions

Next Time
IKE.elg example Tunnelutil example Packet trace example Logging example What about ClusterXL? SecureRemote/Client debug Reporting an issue to Check Point

This is the next time

Where do we begin?
Start with some graphical examples of IKE and IPSEC Look at some ike.elg and vpnd.elg files See two firewalls set up a tunnel Look at some packet traces Review some relevant sk articles Look at an actual troubleshooting process

IPSEC in Depth
IPSEC: RFC2401-2409, 2451, etc What does it do?
Encapsulation (optional) Encryption (optional) Authentication Integrity Protection Replay Protection Key Management

IP Confidential
Data is signed by the sender in an unforgeable way More accurately, forging wouldnt work, as signatures are verifiable against the creator The Key management portion of IKE provides session negotiation and establishment, and sessions can be re-keyed automatically Authentication can be performed in many ways

The key is the selector

A selector is a combination of parameters that allows a gateway to define how it wishes to deal with inbound traffic Selectors include:
IP address or address range A particular IP protocol number (UDP, OSPF) A particular IP port number (500, 5000, etc)

Based on how a packet matches a selector, the gateway will protect (encrypt), drop, or bypass the packet Aka the pass, punt, or play decision

Lets get to the authentication

ESP is used by Check Point almost exclusively Heres how it looks in Tunnel Mode:
IP Header

IP DATA
ESP trailer ESP Auth

New IP Header ESP Head Old IP Head

IP DATA

Authenticated and Encrypted

What does the header look like?

Heres a picture:
NEW IP HEADER Security Parameter Index Sequence Number Initialization Vector Encrypted IP Header UDP header (or whatever) DATA Data Padding Trailer: padding, pad ln ESP Authentication
Encap. Header ESP Header ESP Header

ESP Header

ESP Trailer

Why padding? Some Algorithms (DES) require specific block sizes for Cipher Block Chaining, which speeds encryption.

Initialization Vector?
In order to prevent similarities in your cipher, its a good idea to mix some data from the last packet into the current packet. This prevents the same input from giving you the same output all the time (easy to break) The Initialization Vector is a chunk of the prior packets data that is fed into the next packets data to jumble the output

OK- now for the meat..

IKE: RFC 2401 Uses SAs to track conversations Three things are tracked by ike:
Authentication algorithm and keys Encryption algorithm and keys Lifetimes for the encrypted conversations A selector is also defined in order to specify what traffic the SA applies to

More details:
You dont really have to use IKE:
Enter many large ugly numbers Keep track of them and keep them secret Pass them from site to site Change them secretly

Have fun!

Down to the packet level:

As we mentioned before, there are two modes of operation: Main mode and Quick Mode Main mode authenticates and decides on the encryption algorithms Quick Mode actually defines the tunnels, exchanges the keys, and establishes the SAs Remember: IKE SAs are bidirectional, AH/ESP SAs are unidirectional

Main Mode
Key exchange uses Diffie Hellman method: public/private key pairs generated on the spot are used to initiate secure communication The initial packets of Main Mode describe HOW D-H will be used (encryption strength, etc). Not worth going into the math now, but assume that DH is secure enough to pass the keys used for later communication Once the secure keys are passed over the D-H link, then symmetric (non public/private) algorithms like 3DES or AES are used to pass the secure traffic

Nuts and Bolts

Once the DH parameters are exchanged, then the key generation begins Long computations to generate pseudorandom numbers, which are used as temporary keys At this point, a secure channel has been created, keys have been defined, and the gateways can now enjoy a secure link but.. NO AUTHENTICATION HAS TAKEN PLACE

Authentication
After the DH establishment, after secure key generation and exchange, we tell eachother who we are Can be done with hashed passwords, certificates, or raw public keys Check Point only supports certs or passwords The hashing method used for the passwords is set and negotiated between the gateways Now that weve gone through secure channel generation and authentication, we can set up some SAs

QUICK MODE!
Four packets
First packet: a bunch of crypto stuff from initiator Second packet: a reply with more crypto stuff from the recipient Third packet: Essentially an ACK from the initiator Fourth packet: a second ACK to let the initiator know its ok to start transmitting

QM part 2
Hash type, SA type (ESP), IP information (encryption domains/selectors)

Hash type, SA type (ESP), IP information (encryption domains/selectors)

ACK HASH

Optional return HASH

A bit more on selectors

Defines netmask granularity (0 to 32 bit) Can define port granularity The selector will show Protect for all traffic in the tunnelotherwise it wouldnt be there!

Ending the tunnel..

IKE can use a Delete message to kill an SA, or vendors can use extensions of the Notify message to describe error states Check Point uses Notify often SAs can be killed after a time or volume-based parameter elapses Can cause vendor issues

IkeView (again!)

RFCs: 2408/2409
2408: provides ISAKMP framework
2409 IKE Rfc Both provide ascii header framework

Case Study- Bogus.com

User is having problems getting point to point vpn to establish between gateways running traditional mode and simplified mode vpn configs. First thought: run vpn tu (both ways) to force manual tunnel establishment, while debugging. This should force a tunnel entry in the debugs- ensure that certificates are really being used Determined that pre-shared secret was enabled on one of the gateways

 Test1-fw can establish a VPN using Simplified to Traditional with Bgs-Cluster. Duplicating the rule base for two active VPN's, bgs-Argentina-fw and bgsSydney-fw gives the same error as I had received in the past. "Main Mode Validation timed out. Certificate, 0=fwman.smz23x". The initial phase from the remote firewall reports IKE: Mode completion. One thing noticed is that comparing bgs-Test1-fw and bgs-Sdyney-fw is that after the inital exchange of keys are made, Sydney attempts to connect to the management station using service FW1_ica_services serveral times and then the Validation time out is recorded and no attempt from the bgs-cluster to connect to the management station at all.. When test1-fw initiates the exchange of keys, the Text1-fw and the bgsCluster connect to the management station using the FW1_ica_service.

More troubleshooting..
1. Translation between new firewall (Sydney r55) and Lowell. Was not taking place - correct NAT rule, but no IP Addresses! (Changed global properties - NAT Manual UNCHECK) Fixed. *Specified UNCHECK translate client side / Manual NAT 2. FP3 cannot use preshared's with R55.

Current issues
Current issue: Vpnd maxes out and restarts
Fixed in HFA 11 for R55 and HFA03 for R55P Reason: Cyclic group reference ps -auxw:
USER root PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND 541 81.9 12.5 21596 31636 ?? R 2:02PM 0:41.24 vpnd 0 (vpn)

Cyclic what?
Suppose you have remote user groups, and that you have nested groups within groups (never a good idea)
Group A:
Subgroup 1 Subgroup 2 Subgroup 3

Subgroup 2:
Subgroup P Subgroup Q Group A

Thank You!