You are on page 1of 108

Troubleshooting HP Networks

Learner Guide Version 10.41

Copyright 2010 Hewlett-Packard Development Company, L.P.

The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein. This is an HP copyrighted work that may not be reproduced without the written permission of HP. You may not use these materials to deliver training to any person outside of your organization without the written permission of HP. Troubleshooting HP Networks Learner Guide Rev 10.41

Contents
Module 1: Troubleshooting Methodologies and Practices ................................. 1 - 1 Troubleshooting Methodology ................................................................ 1 - 2 Problem Solving Methodology ............................................................... 1 - 4 Identification and Analysis .................................................................... 1 - 6 Hypothesis and Validation..................................................................... 1 - 8 Implementation and Verification ............................................................ 1 - 10 Summary ........................................................................................... 1 - 1 1 Module 2: Layer 1 (Physical Layer) Troubleshooting and Problem Resolution ....... 2 - 1 Its the cable ..................................................................................... 2 - 2 Physical Layer Symptoms ....................................................................... 2 - 3 Module 3: Layer 2 (Data Link Layer) Troubleshooting and Problem Resolution ..... 3 - 1 Switching ............................................................................................ 3 - 2 VLANs ................................................................................................ 3 - 3 Switch VLAN port types ........................................................................ 3 - 4 Link Aggregation.................................................................................. 3 - 9 LACP Link Aggregation Control Protocol .............................................. 3 - 14 Configurable LACP States ..................................................................... 3 - 14 Static vs. Dynamic Link Aggregation ...................................................... 3 - 15 Spanning Tree .................................................................................... 3 - 16 Basic IRF Concepts .............................................................................. 3 - 21 How IRF simplifies networks ..................................................................3 - 23 Lab 4: VLAN Switching ....................................................................... 3 - 29 Module 4: Layer 3 (Network Layer) Troubleshooting and Problem Resolution ...... 4 - 1 Forwarding between VLANs .................................................................. 4 - 2 VRRP Basics......................................................................................... 4 - 5 OSPF Basics ........................................................................................ 4 - 7 External and internal Border Gateway Protocol (BGP) .............................. 4 - 12 Network Address Translation (NAT) ....................................................... 4 - 14 Static and Dynamic NAT ...................................................................... 4 - 16 Lab 5: Layer 3 Practice and Tools .......................................................... 4 - 17 Lab 6: OSPF Routing Issues .................................................................. 4 - 18 Lab 7: Addressing Issues ...................................................................... 4 - 19 Lab 8: Inter-VLAN and Routing ............................................................. 4 - 20 Module 5: Layer 4 (Transport Layer) Troubleshooting and Problem Resolution ..... 5 - 1 Troubleshooting TCP/UDP ..................................................................... 5 - 2 Firewalls.............................................................................................. 5 - 7 Firewall types....................................................................................... 5 - 9 Network address translator (NAT) ......................................................... 5 - 1 1 Module 6: Layer 5 (Application Layer) Troubleshooting and Problem Resolution.. 6 - 1 QoS process flow ................................................................................. 6 - 2 802.1p traffic prioritization .................................................................... 6 - 8 Traffic marking by an end station .......................................................... 6 - 1 1 Retaining priority between VLANs ......................................................... 6 - 12 Normal priority data traffic ................................................................... 6 - 14 Lab 10: Quality of Service.................................................................... 6 - 15
Rev 10.41

Troubleshooting HP Networks

Module 7: Troubleshooting an End-to-End Complex, Integrated Multi-Protocol Network .................................................................................................... 7 - 1 Lab 1 Final lab ................................................................................... 7 - 2 1:

ii

Rev 10.41

Troubleshooting Methodologies and Practices


Module 1

No network or networking technology operates smoothly all of the time. Every network technician will be required at some time to troubleshoot issues in network configuration and performance. This module introduces basic techniques for network troubleshooting. After completing this module, you will be able to:

Describe a framework for basic network troubleshooting

Rev 10.41

1 1

Troubleshooting HP Networks

Troubleshooting Methodology
Network troubleshooting benefits from having:

Methodology

A discipline for evaluating, analyzing and investigating problem conditions Includes determining the scope of the problem, developing a hypothesis, testing it out, and if successful, implementing a resolution Familiarity with network devices, how they operate and how they are managed Technical tools that may be useful for investigating and verifying problems; from CLI commands and protocol analyzers Good Q&A skills Over time, applying a methodology and the technical tools helps develop your own library of problem recognition capabilities and yields a more efficient problem resolution process

Skill sets

Experience

The basics of troubleshooting any kind of networking trouble might be succinctly stated as keep eliminating obvious causes until the real cause presents itself. But understanding what this means requires a systematic approach and real discipline when attempting to identify causes from symptoms and apply the right fixes or workarounds. Troubleshooting is a skill that all networking professionals learn by trial and error. But skipping some of the more painful or obvious errors can make your learning somewhat less trying than it might be otherwise. The most important characteristic to cultivate when solving problems is calmness. If you can keep a clear head when things fail or start degrading seriously, youll be better able to assess your situation and better equipped to solve whatever problems you discover.

Methodology
Development of problem solving techniques is often an on-the-job acquisition process. Few of us can expect much along the lines of formal network troubleshooting training in our job positions for a number of reasons. These reasons may include:

The relatively fast pace of the day-to-day job tasks and challenges yields little time to pursue formal training on troubleshooting aspects such as technical tools like a protocol analyzer.
Rev 10.41

1 2

Troubleshooting Methodologies and Practices

Few business environments provide the luxury of a test lab and the time to hone your skills where a progression of test problems can be examined, worked through, and resolutions tried out.

In the absence of a more ideal situation, a problem solving methodology can increase the effectiveness of support staff by standardizing the approach used to some extent. With a fairly modest amount of discipline, network technicians can improve their problem resolution efficiency in terms of the effort needed and the number of other people that must be directly involved.

Skill Sets
There are a variety of skill sets that can enhance a network technicians success in problem solving. Some of these skills are purely technical in nature. For instance, it is important to understand the fundamentals of how network devices operate and how they are managed. Having proficiency in reading logs or interpreting a protocol analyzer display are examples of having familiarity with the potential tools you may need to call upon from your toolbox. Other skills are much less technical, but still very important. As part of the problem investigation process, a network technician may need to talk with various levels of staff. The staff may include non-technical end-users and business unit managers to software and hardware vendor support people. Having sufficient interpersonal skills coupled with good investigative reporter-like skills can expedite the isolation of a problem and eliminate the noise that often conceals the real problem. Proactive IT support groups tend to spend time on developing procedures and tools to facilitate problem resolutions. Some examples of technical tools used by the network technicians are:

Device logsArchived instances of the logs as well as the current one may provide hints of where the problem may be. At the very least, familiarity with a log files typical contents helps you differentiate normal from abnormal situations. Device statistics and status informationBeing able to determine the health of a system or the network is important for gathering the vital signs. This type of information can include anything from port statistics and CPU utilization to network reachability results. Protocol analyzerAlthough this may not be a frequently used tool, it can be invaluable for examining what conversations are or are not occurring between communicating devices.

A problem solving methodology that is refined over time can be very beneficial to network technicians. Being methodical and learning from the macro and micro levels of mistakes can help network technicians improve problem recognition capabilities and yield a more efficient application of a problem resolution process.

Rev 10.41

1 3

Troublesh hooting HP Networks

Prob blem So olving Methodology y

Figure 1

A problem solving methodology is a process for managi problem resolution. m y s ing Although there is no one specific model that may be use for all pr h c t eful roblem situations, a general framework can provide guidelines a help ens c and sure efficienc cy forts made to solve a pro oblem. App plying a meth hodology ca improve the an in the eff probabili of a succe ity essful resolution. This g raphic illustr rates the framework for a general problem solv p ving method dology that h many ap has pplications, including in todays contemporar network environments ry s. e o m methodology outlined he y ere. The step ps There are six steps to the problem solving m must be executed in order startin with iden ng ntification. T rules of t The the methodo ology state that if a step fails, you m ust return to the preced o ding step abo ove or possib return to the top leve step. bly el The six st teps are:

Iden ntificationU Understand and docume the prob a ent blem from bo a user an oth nd technical perspe ective. Some etimes it is p possible to lo sight of w ose what the pote ential problem is before searching fo a cause wh we don consider or hen t mult tiple perspec ctives. Anal lysisEvalua the situat ate tion by inves stigating usi problem resolution ing m tools product do s, ocumentatio and user input. on
Rev 1 10.41

1 4

Troubleshooting Methodologies and Practices

HypothesisDevelop possible resolutions based on the analysis and document a possible resolution. This documentation may be fairly informal, but it is important to be able to explain it in writing. Doing so can reveal a hypothesis that is unclear and for which a possible resolution may not be plausible. ValidationRun a validation process to prove or disprove the hypothesis. This may not be particularly feasible, for example, if you have no test lab equipment to try out your hypothesis. At the very least, performing a walkthrough of the hypothesis in an articulate manner with other team members may help. ImplementationDevelop an implementation plan along with a back-out plan, just in case, and then implement the resolution. For example, have a backup configuration and software image readily available. VerificationVerify the success or failure of the implementation. If it fails, implement the back-out plan.

Rev 10.41

1 5

Troublesh hooting HP Networks

Iden ntification and Analy d ysis

Figure 2

The first step of the six-step methodology is identificatio which is a observati s on, an ion process. Try to observe everything, not just the apparen problem, a avoid nt and assuming something Because network trou bleshooting primarily in g g. n g nvolves evaluatin and resolv ng ving connect tivity issues, the general procedure begins with an l analysis of symptoms to determine the scop e of the issu o s ue. For exam mple, it is imp portant to de etermine wh hether the pr roblem is aff fecting a single host, a group of host or the ent network . If many ho are affec ts, tire osts cted, determ mine what they have in common. For instance, if a host can communicate with local e hosts, bu not remote hosts, verify connectiv with its d ut vity default gatew way. If all ho osts in the sam VLAN ca communic me an cate with loc hosts, bu not remote hosts, the cal ut e issue may be a logica problem with the defa gateway or a physical problem al w ault y concerning connectiv with the default gate vity e eway. Altho ough the def fault gateway y performs Layer 3 forw s warding on behalf of loc hosts, the communication with the cal eir default gateway is do using La g one ayer 2 addre ssing. The identification pro ocess consists of doing t tasks that ca include: an

Docu umenting th physical se he ettings. The specifics wi of course vary e ill depe ending on th problem scenario, bu t some exam he s mples are the following: e
Rev 1 10.41

1 6

Troubleshooting Methodologies and Practices

What client, server and network device hardware and software are in use? What is the network topology between the client and server? Where are the applications and services located? Determine the effects the problem has on the user/customer and the business.

Developing a problem definitionDocument probable failures. Prioritizing the problemPrioritize based on defined user/customer policies. Is this a problem that must be investigated immediately or can it wait until you can assemble a strategy using the problem solving methodology?

Step 2 is analysis. Analysis is the process of isolating the problem with the objective to narrow down the different possibilities.

The analysis process considers such factors as the following: Does the system work without the problem Previous changes to the system Something new, such as networking equipment, that may have been introduced Any changes to peripheral equipment that may have been made Whether the hardware or software is being used correctly

With the scope of the problem having been narrowed down, that can help suggest the type of network troubleshooting tools you may want to use to test probable causes. For example, the problem investigation may involve using simple network reachability tools, such as traceroute or ping, or examination of the logs of multiple switches, or even use of a protocol analyzer.

Rev 10.41

1 7

Troublesh hooting HP Networks

Hypothesis and Validation s V

Figure 3

Step 3 is hypothesis. The hypoth hesis step inv volves the ev valuation of the information acquired from the an d nalysis step t determine a number of probable to causes. Some thi ings to keep in mind are e:

Wha is the tech at hnical reason for the bus iness proble n em? You need a valid dation proced dure for the hypothesis to be usable Although e. r at he ng your intuition may prove to be correct a times, in th business world, relyin on th primarily makes it di hat y ifficult for m anagement to feel confi ident about the proc cess. Even ntual resolution of the pr roblem could create side effects, som that are not d e me imm mediately obv vious.

Validation, step 4, typ pically involv experime ves entally deter rmining whe ether the hypothes is reasona sis able. It incre eases the co nfidence lev that the p vel problem will in fact be re esolved after implement r tation of a po otential solu ution.

1 8

Rev 1 10.41

Troubleshooting Methodologies and Practices

The validation step involves:

Testing each hypothesis until you validate a probable cause with a high degree of certainty. The objective is not necessarily to be 100% sure, but to balance the time criticality of resolving a problem with the information you have available. If validation fails for all probable causes you developed, then you may need to return to the problem definition phase and start over. Despite what may appear to be time wasted, you will likely have improved your awareness of the problem situation and will have some additional facts to use when you attempt to redefine the problem.

Rev 10.41

1 9

Troublesh hooting HP Networks

Implement tation and Verificatio a on

Figure 4

Impleme entation, step 5, requires planning fo installation of some fo of system p or n orm m or netwo fix or modification alo with pre ork ong eparation for failure. If a an implementation fails, you must be able to re store the sys , b stem to a pr revious stable state. The planning involve es:

Deve elopment of a specific im f mplementati on plan. Deve elopment of a verificatio process to prove the i f on o implementation was succ cessful. Deve elopment of a back-out plan to ensu the imple f ure ementation can be remo oved, if it fails. It should also addres how to ha d ss andle side ef ffects.

Verification, step 6, is the process of proving the implem s s mentation wa successful as and dete ermining that any side ef t ffects are acc ceptable. If verification fails or side effects ar unaccepta re able, the bac ck-out plan d developed in the implem n mentation phase is executed. ccessful com mpletion, the user or cust e tomer must be informed and the d Upon suc problem resolution should be do s ocumented i n a trouble l log. Lack of f documen ing ntation can lead to lengt resolutio for recurri problem thy on ms.

1 10

Rev 1 10.41

Troubleshooting Methodologies and Practices

Summary

Network troubleshooting benefits from having a methodology, skill sets and experience General problem solving methodology consists of six steps:

Identification: Develop a problem statement Analysis: Narrow the scope Hypothesis: Define procedures to validate Validation: Test probable causes Implementation: Make changes with back-out plan ready Verification: Ensure that changes resolve problem without side effects

Rev 10.41

1 11

Troubleshooting HP Networks

This page left blank intentionally.

1 12

Rev 10.41

Layer 1 (Physical Layer) Troubleshooting and Problem Resolution


Module 2

In this module, various layer 1 problems will be discussed. The technologies include:

Cable / Link problems Link Errors

Rev 10.41

2 1

Troubleshooting HP Networks

Its the cable

Figure 2.1: Cables

Some of the most common Layer 1 problems can be isolated to the cable. Common physical layer problems:

Bad Cables can be terminated improperly or have physical breaks in one or more conductors, etc. Mis-wired cables can be terminated in the wrong order. A common symptom here is that a cable works with 10 or 100Mbps links but not 1 Gig links because of the extra conductors required for Gigabit. It is also common to have fiber links mis-wired so that transmit is connected to transmit and receive is connected to receive. Interference is mostly a problem with unshielded copper cables. This can be due to running data cable alongside power cable. Wrong cable types could be using a CAT3 cable with a Gigabit link or a multimode fiber cable with transceivers that require single mode, etc.

2 2

Rev 10.41

Layer 1 (Physical Layer) Troubleshooting and Problem Resolution

Physical Layer Symptoms


These are some common symptoms of layer 1 issues:

No link Link on one end only Errors on link

To troubleshoot these issues, the switches port counters and event logs can be very useful.

A-Series commands
display interface <INT-ID> display interface brief or display brief interface display logbuffer reverse

E-Series commands
show interfaces <INT-ID> show interfaces brief log r

Here are some examples of these commands.


[4800G]display interface GigabitEthernet 1/0/2 GigabitEthernet1/0/2 current state: UP IP Packet Frame Type: PKTFMT_ETHNT_2, Hardware Address: 00225782-fec2 Description: GigabitEthernet1/0/2 Interface Loopback is not set Media type is twisted pair Port hardware type is 1000_BASE_T 1000Mbps-speed mode, full-duplex mode Link speed type is autonegotiation, link duplex type is autonegotiation Flow-control is not enabled The Maximum Frame Length is 1522 Broadcast MAX-pps: 3000 Unicast MAX-ratio: 100% Multicast MAX-ratio: 100% Forbid jumbo frame to pass PVID: 1 Mdi type: auto Link delay is 0(sec) Port link-type: access Tagged VLAN ID : none Untagged VLAN ID : 1 Port priority: 0 Peak value of input: 279 bytes/sec, at 2000-04-26 12:09:54 Peak value of output: 78 bytes/sec, at 2000-04-26 12:09:59 Last 300 seconds input: 1 packets/sec 115 bytes/sec 0%
Rev 10.41

2 3

Troubleshooting HP Networks

Last 300 seconds output: 0 packets/sec 78 bytes/sec 0% Input (total): 916 packets, 136158 bytes 186 unicasts, 79 broadcasts, 651 multicasts Input (normal): 916 packets, - bytes 186 unicasts, 79 broadcasts, 651 multicasts Input: 0 input errors, 0 runts, 0 giants, 0 throttles 0 CRC, 0 frame, - overruns, 0 aborts - ignored, - parity errors Output (total): 199 packets, 35587 bytes 146 unicasts, 10 broadcasts, 43 multicasts, 0 pauses Output (normal): 199 packets, - bytes 146 unicasts, 10 broadcasts, 43 multicasts, 0 pauses Output: 0 output errors, - underruns, - buffer failures 0 aborts, 0 deferred, 0 collisions, 0 late collisions 0 lost carrier, - no carrier [4800G]display brief interface The brief information of interface(s) under route mode: Interface Link Protocol-link Protocol type Main IP NULL0 UP UP(spoofing) NULL -Vlan1 UP UP ETHERNET 16.1.1.50 The brief information of interface(s) under bridge mode: Interface Link Speed Duplex Link-type PVID GE1/0/1 DOWN auto auto access 1 GE1/0/2 UP 1G(a) full(a) access 1 GE1/0/3 DOWN auto auto access 1 ---- More ---[4800G]display logbuffer reverse Logging buffer configuration and contents:enabled Allowed max buffer size : 1024 Actual buffer size : 512 Channel number : 4 , Channel name : logbuffer Dropped messages : 0 Overwritten messages : 0 Current messages : 166 %Apr 26 13:54:59:803 2000 4800G LLDP/2/CREREM:Port GigabitEthernet1/0/2 (IfIndex 9437185):Created new neighbor, chassis ID: 001c-2e96-8900, port ID: 1. %Apr 26 13:54:58:908 2000 4800G MSTP/2/PFWD:Instance 0's GigabitEthernet1/0/2 has been set to forwarding state! %Apr 26 13:54:58:907 2000 4800G IFNET/4/UPDOWN: Line protocol on the interface Vlan-interface1 is UP %Apr 26 13:54:58:907 2000 4800G IFNET/4/LINK UPDOWN: Vlan-interface1: link status is UP %Apr 26 13:54:58:873 2000 4800G IFNET/4/LINK UPDOWN: GigabitEthernet1/0/2: link status is UP %Apr 26 13:54:56:209 2000 4800G IFNET/4/UPDOWN: Line protocol on the interface Vlan-interface1 is DOWN ---- More ----

2 4

Rev 10.41

Layer 1 (Physical Layer) Troubleshooting and Problem Resolution

E3500yl# show interfaces 23 Status and Counters - Port Counters for port 23 Name : MAC Address : 001c2e-968929 Link Status : Up Totals (Since boot or last clear) Bytes Rx : 1,821,092 304,614 Unicast Rx : 1626 Bcast/Mcast Rx : 10,253 Errors (Since boot or last clear) FCS Rx : 0 Alignment Rx : 0 Runts Rx : 0 Giants Rx : 0 Total Rx Errors : 0 Others (Since boot or last clear) Discard Rx : 0 Unknown Protos : 0 Rates (5 minute weighted average) Total Rx (bps) : 5,001,008 Unicast Rx (Pkts/sec) : 0 B/Mcast Rx (Pkts/sec) : 0 Utilization Rx : 00.50 % E3500yl# show interfaces brief Status and Counters - Port Status Port ----1 2 3 4 5 6 -- MORE | Intrusion MDI Flow Bcast Type | Alert Enabled Status Mode Mode Ctrl Limit ----- + ------ ------- ------ ------- ----- ---- ----1000 | No Yes Up 1000FDx MDI off 0 1000 | No Yes Down 1000FDx Auto off 0 1000 | No Yes Down 1000FDx Auto off 0 1000 | No Yes Down 1000FDx Auto off 0 1000 | No Yes Down 1000FDx Auto off 0 1000 | No Yes Down 1000FDx Auto off 0 --, next page: Space, next line: Enter, quit: Control-C

: Bytes Tx Unicast Tx Bcast/Mcast Tx : Drops Tx Collisions Tx Late Colln Tx Excessive Colln Deferred Tx : Out Queue Len : 0 : Total Tx (bps) : 3,010,520 Unicast Tx (Pkts/sec) : 0 B/Mcast Tx (Pkts/sec) : 0 Utilization Tx : 00.30 % : : : : : 0 0 0 0 0 : : 1938 : 503

E3500yl# log -r Keys: W=Warning I=Information M=Major D=Debug E=Error ---- Reverse event Log listing: Events Since Boot ---I 10/22/10 17:52:38 00561 ports: port 1 Applying Power to PD. I 10/22/10 17:52:38 00560 ports: port 1 PD Detected. I 10/22/10 17:52:36 00076 ports: port 1 is now on-line I 10/22/10 17:52:35 00565 ports: port 1 PD Removed. I 10/22/10 17:52:34 00561 ports: port 1 Applying Power to PD. I 10/22/10 17:52:34 00560 ports: port 1 PD Detected. I 10/22/10 17:52:31 00565 ports: port 1 PD Removed. I 10/22/10 17:52:30 00077 ports: port 1 is now off-line -- MORE --, next page: Space, next line: Enter, quit: Control-C

Rev 10.41

2 5

Troubleshooting HP Networks

Troubleshooting no link
Step 1: Determine if one or two fibers are in use. BX (bi-directional) transceivers use only one fiber for both transmit and receive. There are two "flavors" of BX transceiver. One is a "D" (downstream), the other is a "U" (upstream). You must connect a "D" to a "U". You cannot connect a "D" to a "D", and you cannot connect a "U" to a "U". Is this a BX transceiver link? Action: If BX, try using the other "flavor" (D or U). Or try a connection to a nearby device, ensuring D connects to U. Step 2: Roll (swap) transmit and receive fibers at only one place; for BX ensure "D" connects to"U". Does link come up? Step 3: If no link after rolling the fibers, try connecting to a nearby device with crossover fiber. NOTE: Fiber must be "crossover", meaning transmit at one end connects to receive at the far end. Many fiber patchcords are mis-labeled. Do not rely on color-coding of strain relief, or "A" and "B" labels on the patchcord, to determine if patchcord is crossover. (Those can be wrong.) Instead, use manufacturer's lettering on outside of fiber to identify which strand is which. With connector nub facing up on each end, and with each connector pointing the same direction, be sure lettering is on left at one end, and on right at other end, as shown here:

With both connectors facing same direction, crossover fiber has lettering on left fiber at one end, and lettering on right fiber at other end.

Does link come up? Action: If no link occurs using crossover fiber to nearby device with known-good transceiver, then validate with physical inspection that this is a genuine HP transceiver.

2 6

Rev 10.41

Layer 1 (Physical Layer) Troubleshooting and Problem Resolution

Troubleshooting Errors on link


HP switches keep per-port statistics (counters) that help us diagnose problems on the link or on the network. In addition to "normal" errors like an occasional bad packet received (with incorrect FCS/CRC for example), HP switches alert users to abnormal or "excessive" errors. "Excessive" errors and FFI FFI (Find, Fix, Inform) is a feature of HP switches that informs the user when the switch detects a large number of errors in a short period of time, with specific parameters defined by the HP Switch Lab. The feature was originally called "Fault-finder", and is a good indicator of problems on the link or network. Here are the FFI messages and explanations from the "Help" text in the menu-based event log. Description is what the switch detected. Possible causes are documented, as are user Actions to resolve the problem. Too many undersized/giant packets Description: A device on this port is transmitting packets shorter than 64 bytes or longer than 1518 bytes (longer than 1522 bytes if tagged), with valid CRCs. Possible Causes: A misconfigured NIC or a malfunctioning NIC, NIC driver, or transceiver. Actions: a. b. c. d. Check the NIC for a misconfiguration. Update the NIC driver software. Replace the malfunctioning NIC or transceiver. Check for a short-circuit in the cable path connected to this port.

Excessive jabbering Description: A device on this port is incessantly transmitting packets ("jabbering" is detected as oversized packets with CRC errors). Possible Causes: A misconfigured NIC, or a malfunctioning NIC or transceiver. It could also be caused by a short-circuit in the network cable path. Actions: a. b. c. d. Check the NIC for a misconfiguration. Update the NIC driver software. Replace the NIC or transceiver. Check for a short-circuit in the cable path connected to this port.

Excessive CRC/alignment errors Description: A high percentage of data errors was detected on this port. Possible Causes: Faulty cabling or topology, half/full duplex mismatch, a misconfigured NIC, or a malfunctioning NIC, NIC driver, or transceiver.
Rev 10.41

2 7

Troubleshooting HP Networks

Actions: a. If this port is 100Base-T, make sure the cable, connectors, punch-down blocks, and patch panels connecting to the port are Category 5 or better. Verify the correctness of the installation using a Category 5 test device. Check the directly-connected device for mismatches in half/full duplex operation (half duplex on the switch and full duplex on the connected device, or the reverse). Update the NIC driver software. Verify that the network topology conforms to IEEE 802.3 standards. Replace or relocate the cable. Also check wiring closet components, transceivers, and NICs for proper operation.

b.

c. d. e.

Excessive late collisions Description: Late collisions (collisions detected after transmitting ~64 bytes) were detected on this port. Possible Causes: An overextended LAN topology, half/full duplex mismatch, or a misconfigured or faulty device connected to the port.
Actions:

a. b.

Verify that the network topology conforms to IEEE 802.3 standards. Insert bridges or switches, if needed, to extend the network topology. Check the directly-connected device for mismatches in half/full duplex operation (half duplex on the switch and full duplex on the connected device). If this port is 100Base-T, make sure the cable connecting to that port is Category 5 or better. Check for faulty cabling, transceivers, and NICs.

c. d.

High collision or drop rate Description: A large number of collisions or packet drops have occurred on the port. Possible Causes: An extremely high level of traffic on this port, half/full duplex mismatch, a misconfigured or malfunctioning NIC or transceiver on a device connected to this port, or a topology loop in the network. Actions: a. Use a network monitoring device or application to determine the traffic levels on the affected segment. If needed, consider subdividing that segment with switches or bridges, or moving high-traffic devices to their own switch ports.

2 8

Rev 10.41

Layer 1 (Physical Layer) Troubleshooting and Problem Resolution

b.

Check the directly-connected device for mismatches in half/full duplex operation (half duplex on the switch and full duplex on the connected device). Check for a misconfigured NIC or transceiver (such as a transceiver configured for "loopback test" or "SQE test"). Verify that there are no topology loops in your network. If not enabled, you may also enable spanning.

c. d.

Excessive broadcasts Description: An excessively high rate of broadcast packets was received on the port. This degrades the performance of all devices connected to this switch. Possible Causes: This is usually caused by a network topology loop, but can also be due to a malfunctioning device, NIC, NIC driver, or software application. Actions: a. b. c. Verify that there are no topology loops in your network. Find and correct any malfunctioning devices or NICs on the segment. Find and correct any malfunctioning applications on devices on the segment.

Rev 10.41

2 9

Troubleshooting HP Networks

This page left blank intentionally.

2 10

Rev 10.41

Layer 2 (Data Link Layer) Troubleshooting and Problem Resolution


Module 3

In this module, various layer 2 technologies will be reviewed and common problems will be discussed. The technologies include:

Layer 2 switching VLANs Link Aggregation Spanning Tree IRF

Rev 10.41

3 1

Troubleshooting HP Networks

Switching

Figure 3.1: Switching

Todays switches forward frames in two ways. They flood frame and they switch frames. Frames are flooded if their destination is unknown. That is, the destination doesnt have an entry in the MAC address table. This is also the biggest difference between hubs and switches. Hubs do not maintain a MAC address table. When the destination address is known, then a frame is only forwarded towards that destination. This has the effect of reducing traffic on a network because traffic is not sent out on all links.

3 2

Rev 10.41

Layer 2 (Data Link Layer) Troubleshooting and Problem Resolution

VLANs

Virtual LAN A logical broadcast domain VLANS are used to divide a network segment to smaller sub networks to :

Reduce the overhead of layer 2 broadcast. Increase security. Improve management of network infrastructure

VLANs are created through software configuration. Type of VLANS

Port-based VLANs MAC address-based VLANs Protocol-based VLANs IP-subnet-based VLANs Policy-based VLANs

A virtual LAN (VLAN) is a collection of network nodes that are logically grouped together to form a separate broadcast domain. A VLAN has the same general attributes as a physical LAN, but it allows all nodes for a particular VLAN to be grouped together, regardless of physical location. One advantage of using VLANs is design flexibility. VLANs allow individual users to be grouped based on business needs. Connectivity within a VLAN is established and maintained through software configuration. The list above is a partial list of supported VLAN types. A-Series switches also support Voice VLANs and policy-based VLANs, which are used with 802.1X authentication. This security technology is covered in the Accredited Systems Engineer (ASE) certification track.

Rev 10.41

3 3

Troubleshooting HP Networks

Switch VLAN port types

Access ports:

Belong to one VLAN Port is untagged Carry multiple VLANs on a single physical link VLANs are 802.1Q tagged The native VLAN is untagged Belongs to multiple VLANs Multiple VLANs can be untagged and tagged Typically used for IP phone connection Also in conjunction with protocol VLANs, IP subnet VLANs

Trunk ports:

Hybrid ports:

A-Series switches
By default, VLAN 1 is the native VLAN. To define a trunk:
interface gi 1/0/1 port link-type trunk port trunk permit [all | vlan ids] port trunk pvid vlan [id] (Defines Native VLAN.)

In this case, VLAN 1 will be tagged if still carried. The undo port trunk permit vlan 1 command undoes VLAN 1 assignment. Control plane info, including BPDU and LLDP frames, is sent untagged. To configure multiple ports, define port groups:
[switch] port-group manual [port-group-name] [switch] group-member [port names] [switch] port link-type [trunk | hybrid | access]

Access ports are ports that belong to a single VLAN and the traffic is sent and received untagged. There are two methods to define access ports. Add access ports to VLAN for PCs
[SW-A]vlan 100 [SW-A-vlan100]port gigabitethernet 1/0/1 to gig 1/0/20

OR in interface configuration mode, set interface as an access port in VLAN 100


[SW-A]interface gi 1/0/1 [SW-A-GigabitEthernet1/0/1]port link-type access [SW-A-GigabitEthernet1/0/1]port access vlan 100 3 4
Rev 10.41

Layer 2 (Data Link Layer) Troubleshooting and Problem Resolution

Use these commands to view VLAN membership.


display vlan [vid] display vlan all

Hybrid Ports
Hybrid ports are used mostly for IP phones. Hybrid ports can be assigned to multiple VLAN as tagged or untagged. To set Hybrid ports using a port group:
[SW]port-group manual phones-1 [SW-port-group-manual-phones-1]group-member gi 1/0/11 to gi 1/0/20 [SW-port-group-manual-phones-1]port link-type hybrid

To set Data VLAN 100 as the native VLAN:


[SW-port-group-manual-phones-1]port hybrid PVID 100

Note: Hybrid port is still part of VLAN 1. Then Remove hybrid port from VLAN 1
[SW-port-group-manual-phones-1]undo port hybrid vlan 1 untagged

To set VLAN 200 as voice VLAN:


[SW-port-group-manual-phones-1]voice vlan 200 enable

This makes VLAN tagged on port and auto-QoS if Phone SNMP OUI Is detected. Voice VLAN command will dynamically:

Allocate the voice vlan as a tagged vlan with auto qos if a predefined phone SNMP OUI is detected. Add OUI with voice OUI command at system view.

Hybrid ports can be set as untagged in one or more VLANs. Here is an example of configuration on a hybrid port to use with protocol VLAN:
[SWA]vlan 2 [SWA-vlan2]Description IP and ARP VLAN [SWA-vlan2]protocol-vlan mode ethernetii etype 0800 [SWA-vlan2]protocol-vlan mode ethernetii etype 0806 [SWA-vlan2]vlan 3 [SWA-vlan2]Description Novell IPX VLAN [SWA-vlan3]protocol-vlan ipx llc [SWA-vlan3]interface gigabit 1/1/1 [SWA-gigabit1/1/1]description Access port Separate IP and IPX traffic [SWA-gigabit1/1/1]port link-type hybrid [SWA-gigabit1/1/1]undo port hybrid vlan 1 [SWA-gigabit1/1/1]port hybrid vlan 2 3 untagged
Rev 10.41

3 5

Troubleshooting HP Networks

[SWA-gigabit1/1/1]port hybrid protocol-vlan vlan 2 all [SWA-gigabit1/1/1]port hybrid protocol-vlan vlan 3 all [SWA-vlan3]interface gigabit 1/1/23

Hybrid ports can be assigned to multiple VLAN as tagged or untagged. To set Hybrid ports using a port group
[SW]port-group manual phones-1 [SW-port-group-manual-phones-1]group-member gi 1/0/11 to gi 1/0/20 [SW-port-group-manual-phones-1]port link-type hybrid

Ts set Data VLAN 100 as the native VLAN


[SW-port-group-manual-phones-1]port hybrid PVID 100 Note Hybrid port is still part of VLAN 1. Then Remove hybrid port from VLAN 1. [SW-port-group-manual-phones-1]undo port hybrid vlan 1 untagged

To set VLAN 200 as voice VLAN


[SW-port-group-manual-phones-1]voice vlan 200 enable

This makes VLAN tagged on port and auto-qos if Phone SNMP OUI Is detected.
[SWA-gigabit1/1/23]description Trunk port Separate IP and IPX traffic [SWA-gigabit1/1/23]port link-type trunk [SWA-gigabit1/1/23]port trunk permit vlan 2 3

Trunk Ports
On trunk 802.1Q ports: one VLAN at most is untagged, all other VLANs are tagged To configure the trunk interfaces & allow the VLANs:
[SW-A]interface gi 1/0/23 [SW-A-GigabitEthernet1/0/23]port link-type trunk [SW-A-GigabitEthernet1/0/23]port trunk permit vlan 100 200 [SW-A]interface gi 1/0/24 [SW-A-GigabitEthernet1/0/24]port link-type trunk [SW-A-GigabitEthernet1/0/24]port trunk permit vlan all

List trunk ports:


[SW-A]display port trunk Interface GE1/0/23 GE1/0/24 PVID 1 1 VLAN passing 1, 100, 200 1, 100, 200

On edge switches you can set the uplinks as trunk ports carrying all VLANs:
3 6
Rev 10.41

Layer 2 (Data Link Layer) Troubleshooting and Problem Resolution

port link-type trunk port trunk permit vlan all Note Do not confuse trunk ports with the link aggregation ports that are called trunk ports on HP E-Series switches.

On distribution/core switches, set exactly what VLANs should be carried on downlinks to edge switches:
port link-type trunk port trunk permit vlan 100 200 Note VLAN 1 is set by default.

To change the native VLAN to VLAN 99


[SW-A-GigabitEthernet1/0/23]port trunk PVID 99

This forces the interface to be tagged on VLAN 1. If VLAN 1 is not desired on port, remove it
[SW-A-GigabitEthernet1/0/23]undo port trunk permit vlan 1

List trunk ports


[SW-A]display port trunk Interface GE1/0/23 GE1/0/24 PVID 99 99 VLAN passing 99, 100, 200 99, 100, 200

VLAN 1 is not necessary on A-Series switches. For example: BPDUs for STP, LLDP or LACP are sent untagged whatever is the setup of VLANs on the link. BPDUs are accepted by a receiving switch because their destination MAC address matches the list of MAC addresses on the ports. In other words, because the protocols (LLDP, STP, LACP) are enabled on port and global levels.

Rev 10.41

3 7

Troubleshooting HP Networks

E-Series switches
E-Series switches do not use the same terminology as the A-Series. On E-Series devices, VLAN membership is configured from the VLAN context with the tagged and untagged commands. A port can be considered to be a VLAN trunk port if it is assigned to more than one VLAN. Similarly, a port can be considered to be an access port if it is only assigned to one VLAN for untagged traffic. To configure a port to be an untagged member of a VLAN (access port):
E-Series(config)# vlan 100 E-Series(vlan-100)# untagged a1-a12

To configure a port to be a tagged member of a VLAN (trunk port):


E-Series(config)# vlan 100 E-Series(vlan-100)# tagged a1-a12 E-Series(vlan-100)# vlan 200 E-Series(vlan-200)# tagged a1-a12 E-Series(vlan-100)# vlan 5 E-Series(vlan-5)# untagged a1-a12 (This is optional)

To configure a port to be a voice VLAN:


E-Series(config)# vlan 100 E-Series(vlan-100)# voice

3 8

Rev 10.41

Layer 2 (Data Link Layer) Troubleshooting and Problem Resolution

Link Aggregation
Link aggregation is called trunking on HP E-Series switches. E-Series switches support two trunking methods:

HP Port TrunkingHP has supported port trunking since its first offering of switches in the mid-1990s. The original HP port trunking technology remains an option on ProCurve switches. HP port trunking is the default on E-Series switches. For proper trunk operation, all links in the same trunk group must have the same speed, duplex, and flow control Link Aggregation Control Protocol (LACP)The IEEE standard for link aggregation. HPs implementation of LACP supports both active and passive configuration of trunking.

These link-aggregation methods impose a similar set of requirements and restrictions. However, LACP imposes an additional restrictionthe links must operate in fullduplex mode. This is rarely a concern because trunks consist of point-to-point links between switches, and these links will usually negotiate up to full duplex operation. HP port trunking does not have this requirement. Both methods for port trunking share one important limitation in the area of load sharingthey are static methods. They do not adjust to reflect traffic volume on the links or evaluate an individual conversation to determine which link would be best at a given moment. Instead, all methods distribute the conversations evenly across all links with the expectation that the load generally is balanced. The benefits of trunking are always best realized in the presence of many source and destination points on each side of the trunk.

Configuring Port Trunking on E-Series Devices


To enable static port trunking from the CLI, you use the trunk command. At the global configuration level, issue the trunk command followed by a list of the ports that will be aggregated, a name for the trunk, and the type of trunk (HP trunk or LACP). The ports need not be contiguous, although the example above shows four contiguous ports. A list of ports is separated by commas, for example: trunk a1,a7,b1,b24 trk1 LACP.
Note The 2500 series switches support only one trunk. If the trunk is statically defined, it will be named Trk1.

The trunk configuration must be performed on both sides of the trunk before the redundant links are connected.
Edge_1(config)# trunk ? [ethernet] PORT-LIST Specify the ports that are to be added to/removed from a trunk. Edge_1(config)# trunk c1,c2 ?
Rev 10.41

3 9

Troubleshooting HP Networks

trk1 trk2 ...

Trunk group 1 Trunk group 2

Edge_1(config)# trunk c1,c2 trk1 ? trunk Do not use any protocol to create or maintain the trunk. lacp <cr> Edge_1(config)# trunk c1,c2 trk1 lacp

Use IEEE 802.1ad Link Aggregation protocol.

The trunk command is used to create an HP port trunk or LACP port trunk trk1, trk2, etc. are fixed label names for trunks On the 8100fl series, trunks are referred to as Link Aggregation Groups

Configuring Link Aggregation on A-Series Devices


Static Link Aggregation
1. Create VLAN 10 and aggregate interface 1, and assign the aggregate interface to VLAN 10.

<DeviceA> system-view [DeviceA] vlan 10 [DeviceA-vlan10] quit [DeviceA] interface bridge-aggregation 1 [DeviceA-Bridge-Aggregation1] port access vlan 10 [DeviceA-Bridge-Aggregation1] quit

2.

Assign ports GE4/0/1 through GE4/0/3 to link aggregation group 1 and VLAN 10 one at a time.

[DeviceA] interface gigabitethernet 4/0/1 [DeviceA-Gigabitethernet4/0/1] port link-aggregation group 1 [DeviceA-Gigabitethernet4/0/1] port access vlan 10 Warning: This port is a member of the link aggregation group. If configuration of the whole group is required to be modified, please configure it under the aggregation interface view. Otherwise, this operation may interrupt network traffic.Continue?[Y/N]: y [DeviceA-Gigabitethernet4/0/1] quit [DeviceA] interface gigabitethernet 4/0/2 [DeviceA-Gigabitethernet4/0/2] port link-aggregation group 1 [DeviceA-Gigabitethernet4/0/2] port access vlan 10 Warning: This port is a member of the link aggregation group. If configuration of the whole group is required to be modified, please configure it under the aggregation interface view. Otherwise, this operation may interrupt network traffic.Continue?[Y/N]: y 3 10
Rev 10.41

Layer 2 (Data Link Layer) Troubleshooting and Problem Resolution

[DeviceA-Gigabitethernet4/0/2] quit [DeviceA] interface gigabitethernet 4/0/3 [DeviceA-Gigabitethernet4/0/3] port link-aggregation group 1 [DeviceA-Gigabitethernet4/0/3] port access vlan 10 Warning: This port is a member of the link aggregation group. If configuration of the whole group is required to be modified, please configure it under the aggregation interface view. Otherwise, this operation may interrupt network traffic.Continue?[Y/N]: y [DeviceA-Gigabitethernet4/0/3] quit

3.

Configure Device A to perform load sharing based on source and destination MAC addresses for link aggregation groups.

[DeviceA] link-aggregation load-sharing mode source-mac destination-mac

Dynamic Link Aggregation


1. Create VLAN 10 and aggregate interface Bridge-aggregation 1, configure the link aggregation mode as dynamic, and assign the aggregate interface to VLAN 10.

<DeviceA> system-view [DeviceA] vlan 10 [DeviceA-vlan10] quit [DeviceA] interface bridge-aggregation 1 [DeviceA-Bridge-Aggregation1] link-aggregation mode dynamic [DeviceA-Bridge-Aggregation1] port access vlan 10 [DeviceA-Bridge-Aggregation1] quit

2.

Assign ports GE4/0/1 through GE4/0/3 to link aggregation group 1 and VLAN 10 one at a time.

[DeviceA] interface gigabitethernet 4/0/1 [DeviceA-Gigabitethernet4/0/1] port link-aggregation group 1 [DeviceA-Gigabitethernet4/0/1] port access vlan 10 Warning: This port is a member of the link aggregation group. If configuration of the whole group is required to be modified, please configure it under the aggregation interface view. Otherwise, this operation may interrupt network traffic.Continue?[Y/N]: y [DeviceA-Gigabitethernet4/0/1] quit [DeviceA] interface gigabitethernet 4/0/2 [DeviceA-Gigabitethernet4/0/2] port link-aggregation group 1 [DeviceA-Gigabitethernet4/0/2] port access vlan 10 Warning: This port is a member of the link aggregation group. If configuration of the whole group is required to be modified, please configure it under the aggregation interface view. Otherwise, this operation may interrupt network traffic.Continue?[Y/N]: y [DeviceA-Gigabitethernet4/0/2] quit
Rev 10.41

3 11

Troubleshooting HP Networks

[DeviceA] interface gigabitethernet 4/0/3 [DeviceA-Gigabitethernet4/0/3] port link-aggregation group 1 [DeviceA-Gigabitethernet4/0/3] port access vlan 10 Warning: This port is a member of the link aggregation group. If configuration of the whole group is required to be modified, please configure it under the aggregation interface view. Otherwise, this operation may interrupt network traffic.Continue?[Y/N]: y [DeviceA-Gigabitethernet4/0/3] quit

3.

Configure Device A to perform load sharing based on source and destination MAC addresses for link aggregation groups.

[DeviceA] link-aggregation load-sharing mode source-mac destination-mac

Load Sharing Mode


1. Create VLAN 10.
<DeviceA> system-view [DeviceA] vlan 10 [DeviceA-vlan10] quit

2.

Create aggregate interface Bridge-aggregation 1, configure the source MACbased load sharing mode for the link aggregation group, and assign the aggregate interface to VLAN 10.

[DeviceA] interface bridge-aggregation 1 [DeviceA-Bridge-Aggregation1] link-aggregation load-sharing mode source-mac [DeviceA-Bridge-Aggregation1] port access vlan 10 [DeviceA-Bridge-Aggregation1] quit

3.

Assign ports GE4/0/1 and GE4/0/2 to link aggregation group 1 and VLAN 10.

[DeviceA] interface gigabitethernet 4/0/1 [DeviceA-Gigabitethernet4/0/1] port link-aggregation group 1 [DeviceA-Gigabitethernet4/0/1] port access vlan 10 Warning: This port is a member of the link aggregation group. If configuration of the whole group is required to be modified, please configure it under the aggregation interface view. Otherwise, this operation may interrupt network traffic.Continue?[Y/N]: y [DeviceA-Gigabitethernet4/0/1] quit [DeviceA] interface gigabitethernet 4/0/2 [DeviceA-Gigabitethernet4/0/2] port link-aggregation group 1 [DeviceA-Gigabitethernet4/0/2] port access vlan 10 Warning: This port is a member of the link aggregation group. If configuration of the whole group is required to be modified, please configure it under the aggregation interface view. Otherwise, this operation may interrupt network traffic.Continue?[Y/N]: y 3 12
Rev 10.41

Layer 2 (Data Link Layer) Troubleshooting and Problem Resolution

[DeviceA-Gigabitethernet4/0/2] quit

4.

Create aggregate interface Bridge-aggregation 2, configure the destination MAC-based load sharing mode for the link aggregation group, and assign the aggregate interface to VLAN 10.

[DeviceA] interface bridge-aggregation 2 [DeviceA-Bridge-Aggregation2] link-aggregation load-sharing mode destination-mac [DeviceA-Bridge-Aggregation2] port access vlan 10 [DeviceA-Bridge-Aggregation2] quit

5.

Assign ports GE4/0/3 and GE4/0/4 to link aggregation group 2 and VLAN 10.

[DeviceA] interface gigabitethernet 4/0/3 [DeviceA-Gigabitethernet4/0/3] port link-aggregation group 2 [DeviceA-Gigabitethernet4/0/3] port access vlan 10 Warning: This port is a member of the link aggregation group. If configuration of the whole group is required to be modified, please configure it under the aggregation interface view. Otherwise, this operation may interrupt network traffic.Continue?[Y/N]: y [DeviceA-Gigabitethernet4/0/3] quit [DeviceA] interface gigabitethernet 4/0/4 [DeviceA-Gigabitethernet4/0/4] port link-aggregation group 2 [DeviceA-Gigabitethernet4/0/4] port access vlan 10 Warning: This port is a member of the link aggregation group. If configuration of the whole group is required to be modified, please configure it under the aggregation interface view. Otherwise, this operation may interrupt network traffic.Continue?[Y/N]: y [DeviceA-Gigabitethernet4/0/4] quit

Rev 10.41

3 13

Troubleshooting HP Networks

LACP Link Aggregation Control Protocol


Link Aggregation Control Protocol (LACP) is another option for creating port trunk groups on HP switches. LACP is defined by the IEEE standard 802.3ad. LACP was standardized to allow a switch to automatically recognize coterminous, full duplex, same-speed links between itself and another LACP-compliant switch. Although LACP can automatically recognize links that are capable of aggregation, the activation of an LACP trunk requires some configuration. You cant simply connect four links between the same two switches and expect them to act as a trunk. When using dynamic LACP, you must define the trunk on one side, which is known as the active side. The active side sends Bridge Protocol Data Units (BPDUs) across every link that has LACP defined statically. Although a complete description of the fields in the BPDU is beyond the scope of this course, a few BPDU fields relevant to dynamic operation are worth noting. They are:

A system identifier, which is the switchs MAC address. A priority value, which is a permutation of the MAC address. A port identifier, which contains a port number.

When a switch receives BPDUs through multiple passive LACP ports that have the same system identifier, it knows that those ports are linked to the same switch. If the links are the same speed, the switch sends BPDUs to the active partners on the other side of the links, and the two switches agree to load share across the group of links. Passive LACP ports only speak when spoken to; a passive LACP port sends BPDUs only after it has received BPDUs from a connected switch.

Configurable LACP States


HP switches offer three possible options for LACP configuration:

Passive Active Disabled - (default state)

LACP is configured on a per-port basis. When a port is configured for a passive LACP state, it will be blocked for approximately five seconds when the switch is initialized. This is appropriate for ports that are linked to active LACP partners because it provides the ports with time to discover the LACP topology before forwarding any traffic. However, this delay can be unacceptable for normal switch operation. Consequently, HP recommends that LACP remain in the default state of disabled for all ports that will not participate in dynamic link aggregation. If you define a trunk using the trunk command described earlier in this module, the no lacp command is automatically executed and included in the configuration for
3 14
Rev 10.41

Layer 2 (Data Link Layer) Troubleshooting and Problem Resolution

the ports specified in the trunk commands port list. Static and dynamic port trunking cannot be simultaneously active on the same port. Finally, is the case of 802.1X (Port-Based Access Control) being configured on a Port. To maintain security, LACP is not allowed on ports configured for 802.1X authenticator operation. If you configure port security on a port on which LACP (active or passive) is configured, the switch removes the LACP configuration, displays a notice that LACP is disabled on the port(s), and enables 802.1X on that port.

Static vs. Dynamic Link Aggregation


One important advantage of dynamic link aggregation is its ability to recognize and use trunk standby links. When two switches detect more than four coterminous, same speed links, they aggregate the four links with the lowest port numbers. The remaining links are used as standby links. While dynamic LACP is the only way to set up standby links in a trunk, its disadvantage is that in certain circumstances it can give you less control. The primary disadvantage of static link aggregation is its lack of support for standby links. Switches configured for static link aggregation cannot automatically detect new members of the trunk group and, therefore, cannot use standby links. On the other hand, static aggregation enables administrators to retain more control of the operation of the trunk ports.

Rev 10.41

3 15

Troubleshooting HP Networks

Spanning Tree

Figure 3.1: Spanning tree

Multiple Spanning Tree Protocol (MSTP) enables the configuration of VLAN-aware Spanning Tree topologies. As described in IEEE 802.1S, multiple spanning trees allow frames assigned to different VLANs to follow different data routes within administratively established regions of the network. In this way, MSTP enables the configuration of Multiple Spanning Trees within a physical topology, which provides significant improvement in the utilization of redundant links. Furthermore, the standard notes that an MST configuration probably will provide simple and full connectivity for frames even in the presence of administrative errors in the allocation of VLANs to Spanning Trees. MSTP should not be confused with another VLAN-aware Spanning Tree protocol known as Per VLAN Spanning Tree (PVST). In PVST configurations, a separate Spanning Tree instance is created for each VLAN. BPDUs are transmitted with tags that identify the STP instance and VLAN ID to which they belong. While this enables the use of redundant links if you apply priorities and costs intelligently, it can be a CPU-intensive process if there are many VLANs. MSTP, on the other hand, enables the creation of multiple Spanning Tree instances that are specifically mapped to VLANs. It is not necessary to literally have a one-toone correspondence between Spanning Trees and VLANs. In this way, MSTP combines the best of two extremesthe single Spanning Tree configurations of STP and RSTP and the Spanning Tree per VLAN configuration of PVST.

MSTP Features

MSTP is the default protocol when Spanning Tree is enabled MSTP allows for multiple instances of a redundant path for a set of VLANs within the bridged network


3 16

Each Spanning Tree instance has its own Root Bridge Traffic is distributed across redundant links
Rev 10.41

Layer 2 (Data Link Layer) Troubleshooting and Problem Resolution

MSTP follows the same basic principles as STP and RSTP

Compatible and interoperable with STP and RSTP Emulates STP and RSTP behaviors when encountering switches that do NOT support MSTP

Because MSTP implements the same basic principles as the earlier Spanning Tree protocols, it is completely interoperable and compatible with STP and RSTP. Furthermore, MSTP will emulate STP and RSTP behaviors when encountering devices that do not support MSTP. MSTP is the latest iteration of Spanning Tree, and is the default Spanning Tree protocol on most switches. Check the release notes or manuals for a specific switch to determine its default.

Comparing RSTP, PVST and MSTP

Table 2.1: Comparing RSTP, PVST and MSTP

Before the release of the MSTP standard, the only IEEE-standardized way to combine VLANs and Spanning Tree was to resolve loops within the topology without regard to VLAN configuration. Cisco Systems Inc. developed PVSTand later PVST+to enable the configuration of VLAN-aware Spanning Trees. PVST enables administrators to configure Bridge and Port Priority settings and path costs so that any two paths between a pair of switches can both be used. With PVST enabled, some Spanning Tree instances will take one path while other instances take another path. However, each of the Spanning Tree instances is separately configured, which results in more overhead than the simpler

Rev 10.41

3 17

Troubleshooting HP Networks

RSTP solution. Furthermore, the scalability of PVST is limited because of the increased CPU utilization described earlier in this module. MSTP, on the other hand, enables the configuration of fewer Spanning Tree instances, typically between 1 and 16, with each VLAN mapped to the appropriate instance.

Spanning Tree for Instance 1

Figure 3.3: Multiple spanning tree (1)

With MSTP, Spanning Tree instances are associated with VLAN IDs, not with individual links. Because a separate Root Bridge is elected for each MST instance, each instance uses a different set of links as the active path. As with STP and RSTP, backupor Blocking Stateports are not used in the primary active path, but they enable the quick restoration of connectivity in the event of link failure. In the graphic above, Edge_1 was elected as the Root Bridge for MST Instance 1, which resulted in the topology shown. Instance 1 includes VLANs 2 to 10. The next slide illustrates the Spanning Tree topology for MST Instance 2.

3 18

Rev 10.41

Layer 2 (Data Link Layer) Troubleshooting and Problem Resolution

Spanning Tree for Instance 2

Figure 3.4: Multiple spanning tree (2)

In the diagram above, Edge_2 has been elected as the Root Bridge for MST Instance 2. Instance 1 includes VLANs 1 to 20. 1 Because of this election, the state of the physical links is different than in MST Instance 1, shown on the previous slide.

MST Regions

A group of switches that collectively define multiple Spanning Tree instances is known as an MST region Each switch can belong to only one region All switches in a region must have identical configuration attributes:

Alphanumeric configuration name Configuration revision number

Associations between VLANs and Spanning-Tree instances

A switch defines a region boundary if it receives BPDUs from:

A switch with different configuration attributes, or An STP or RSTP switch

MST Instances Within a Single Switch

When MST is initially enabled, the default conditions are as follows:


3 19

Rev 10.41

Troubleshooting HP Networks

Each switch defines its MAC address as its configuration name and 0 as its configuration revision number All of the VLANs defined on a switch belong to the Internal Spanning Tree (IST) instance

To cause the switch to interact correctly with other switches in the MST region, you must define common configuration attributes Any VLAN not explicitly mapped to a user-defined instance remains associated with the IST

VLAN 1 is often associated with the IST

Immediately after MSTP is enabled, all the VLANs configured on a switch are part of the Internal Spanning Tree (IST), which is an RSTP instance that exists within the MST region. As you add new instances and associate them with VLANs, the VLANs are removed from the IST. However, the IST remains in place, even if no VLANs are explicitly mapped to it. In most cases, user-defined VLANs are associated with user-defined instances configured identically on all switches in the MST region. The default VLAN (VLAN ID 1) remains associated with the IST. This provides an important benefit: if the VLAN-toinstance mappings are misconfigured, you can still access the switch because the ISTs association with VLAN 1 ensures that connectivity is not completely disrupted.

3 20

Rev 10.41

Layer 2 (Data Link Layer) Troubleshooting and Problem Resolution

Basic IRF Concepts

Figure 3.5: IRF concepts

The devices that form an IRF virtual device are called IRF member devices. A member device assumes the role of master or slave. An IRF stack contains only one master, which manages the IRF virtual device. All other members operate as slaves and as backups for the master. When the master fails, the IRF virtual device automatically elects a new master from one of the slaves. Master and slaves are selected through the role election mechanism. The details of the role election mechanism will be covered later in this module. A logical IRF port is a logical port dedicated to the internal connection of an IRF virtual device. These ports cannot act as access, trunk or hybrid ports. An IRF port is effective only when it is bound to a physical IRF port. Physical ports used for connecting members of an IRF virtual device are called physical IRF ports. Typically, an Ethernet port or optical port forwards frames to the network. When a physical port is bound to an IRF port, it acts as a physical IRF port and forwards data traffic such as IRF-related negotiation frames and data traffic among members. As shown in the figure above, an IRF stack can have a daisy chain topology or a ring topology. A ring connection is more reliable than the daisy chain connection. In a daisy chain topology, the failure of one link can cause the IRF virtual device to partition into two independent IRF virtual devices, which can disrupt connectivity as well as IRF functioning. The failure of a link in a ring connection results in a daisy chain connection, and does not affect IRF services.

Rev 10.41

3 21

Troubleshooting HP Networks

IRF application scenario: Increasing port density

Figure 3.6: IRF increases port density

IRF provides a simple, cost-effective solution to the issues that arise when use population exceeds the available network ports. With IRF deployed, you can add new members to your virtual IRF device, adding port density with minimal configuration of the new switches.

IRF application scenario: Expanding system processing capabilities

Figure 3.7: IRF expands system processing capabilities

When the forwarding capability of the core switch cannot satisfy users needs, you can add a switch to form an IRF stacking system with the original core switch. If the forwarding capability of one switch is 64 Mpps, the forwarding capability of the whole stack system is 128 Mbps after another switch is added. Note that this increases the forwarding capability of the entire stacking system, not a single switch.
3 22
Rev 10.41

Layer 2 (Data Link Layer) Troubleshooting and Problem Resolution

IRF application scenario: Expanding bandwidth

Figure 3.8: IRF expands bandwidth

You can increase the uplink bandwidth of an edge switch by adding another switch to form a stacking system with the existing edge switch. You can configure multiple physical links of the member devices as an aggregation group to increase the bandwidth of the link to the core switch. In the IRF configuration in the above Figure , four links (two from each switch) are aggregated to double the bandwidth from the edge to the core. Adding a second edge switch without IRF would add more throughput to the core, but the bandwidth would be divided between the edge switches and their corresponding clients. To the core switch, the number of edge switches does not change. The original edge switch will back up the current configurations to the newly added switch in batches, having minimal effect on network planning and configuration.

How IRF simplifies networks

Figure 3.9: IRF simplifies networks (1) Rev 10.41

3 23

Troubleshooting HP Networks

This network topology provides redundant links between the edge and the distribution layer. MSTP is required to prevent loops introduced by these redundant links. VRRP is a protocol for providing router redundancy. For each of the two segments in the configuration shown, one router in the distribution layer acts as the master and does the actual routing and the other acts as a backup. If the master fails, the backup can take over the routing. In enterprise networks, VRRP is often combined to add Layer 3 redundancy to the Layer 2 redundancy provided by MSTP.

Figure 3.10: IRF simplifies networks (2)

In this solution, all four of the distribution layer switches are combined into one IRF stack. All of the switches have the same routing table and can route packets received from the edge switches. The IRF master will run the routing protocol for the entire virtual device. When configured as an IRF stack, the distribution layer switches now act as a single virtual switch. Loops can still occur, however between an edge switch and the IRF virtual switch. In order to retain the redundant links between the edge and distribution layers, the redundant links can be combined in a link aggregation, creating a single logical link that spans two physical devices in the IRF virtual switch. Advantages of this topology The IRF topology is simpler to configure and maintain than the MSTP/VRRP solution. In the IRF implementation, the virtual switch is configured as if it were a single device. If the same switches were running MSTP and VRRP, each switch would need a distinctly different configuration to ensure the correct election of MSTP Root Bridge and VRRP Master. Furthermore, each switch would need to be configured separately for all routing and switching functions.

Architecture: Operational Planes


Plane Functions

3 24

Rev 10.41

Layer 2 (Data Link Layer) Troubleshooting and Problem Resolution

Management Control

Management interfaces (console, Telnet, SNMP, FTP, TFTP, etc.) Internal/hardware monitoring: temperature, fan status, module and power management, etc. File system including: Configuration File Layer 2 protocols: LACP, RSTP, MSTP Layer 3 Protocols: RIP, OSFP, BGP, ISIS, etc. Routing Table ACLs and QoS Policies FIB (Forwarding Information Base) and Local ACLs and QoS Policies Frame/packet forwarding and handling

Forwarding

Modern Switches and Routers segregate their functions into different groups called operational planes or simply planes.

Operational Planes in Standalone Switches

Figure 3.11: Operational planes in standalone switches

Modern Switches and Routers segregate their functions into different groups called operational planes or simply planes. The most common planes are:

Control Plane: this group includes all internal monitoring and control functions related to power, temperature, and hardware state in general. Management plane: this functional group is where the user interface is located and where and all protocols run, for example STP in Layer 2 and OSPF in layer 3.

It is in this plane that the routing table is built. Functions in this plane are software based to allow for upgrades.

Forwarding Plane: this group of functions includes L2 and L3 forwarding, packet filtering and QoS policies.
3 25

Rev 10.41

Troubleshooting HP Networks

It is in this plane that the routing table is actually used. Functions in this plane are hardware based because of speed requirements.

Operational Planes in IRFv2

Figure 3.12: Operational planes in IRFv2

In stackable switches, the distribution of these planes is simple: a general purpose CPU runs the management and control planes and one or two ASICs are in charge of actual packet processing and forwarding. In the case of chassis, the management and control plane are centralized in SRPUs (Switching and Routing Processing Units) and the forwarding plane is distributed in two or more LPUs (Line Processing Units). All chassis have the option of installing two SRPUs for redundancy.

Operational Planes in IRFv2


When connecting several units to form an IRF, the management and control planes of one of the units become active and those of the other units stay in standby. In the case of chassis, today only two of them can be connected in an IRF. If each has 2 SRPUs, one of these SRPUs is going to become active and the other three will stay in standby. In other words, an IRF system acts like a chassis with centralized management and control planes and a distributed forwarding plane.

3 26

Rev 10.41

Layer 2 (Data Link Layer) Troubleshooting and Problem Resolution

IRF-ports

Figure 3.13: IRF ports

To build an IRF-stack its member devices must be connected. This connection requires the configuration of IRF-ports. An IRF-port is a logical entity composed of one or more standard 10GbE ports. In other words, physical 10 GbE ports are bound to an IRFport. By allowing the configuration of standard 10GbE ports as IRF ports, HP offers the possibility of having:

Local IRF-stacks, in which all members are in the same room Geographically distributed IRF-stacks

Important: IRF-port 1 can only be connected to IRF-port 2 of the next device in the IRF-stack. By allowing the configuration of regular 10GbE ports as IRF ports, H3C offers the possibility of having:

Local IRF systems, in which all members are in the same room and Geographically distributed IRF systems, for Data Center redundancy. with CX4 and XFP ports, CX4 local connection cables can be used with SFP+ ports special IRF cables can be used

Local connections can be built using inexpensive copper cables:


In both cases, cables of 50, 100 and 300cm are available. For geographically distributed IRFs, the 10GbE technology required will depend on the distance.

IRF Member ID

Devices forming an IRF-stack must have a different IRF Member ID. This number is equivalent to the slot number in a chassis.

Rev 10.41

3 27

Troubleshooting HP Networks

Switches A5120 and A5500 support dynamic Member ID allocation: when there is a member-id collision one of the devices changes it Member ID automatically. In all other A-Series switches the Member ID must be configured manually. This step is the first step required when building an IRF-stack By default: IRF Member ID = 1

3 28

Rev 10.41

Layer 2 (Data Link Layer) Troubleshooting and Problem Resolution

Lab 4: VLAN Switching


Lab 4 is design to ensure you can use a structured troubleshooting methodology to resolve VLAN switching problems. There are three trouble tickets in this lab. Refer to your lab guide for instructions on how to do this lab.

Rev 10.41

3 29

Troubleshooting HP Networks

This page left blank intentionally.

3 30

Rev 10.41

Layer 3 (Network Layer) Troubleshooting and Problem Resolution


Module 4

In this module, various layer 3 technologies will be reviewed and common problems will be discussed. The technologies include:

IPv4 Routing and Addressing Inter-VLAN Routing VRRP OSPF iBGP/eBGP NAT

Rev 10.41

4 1

Troubleshooting HP Networks

Forwarding between VLANs

Figure 4.1: forwarding between VLANs

As is shown in the example above, IP address 10.1.2.1 with the 24-bit mask (255.255.255.0) defines a range of local IP addresses between 10.1.2.0 and 10.1.2.255. When using this mask, the first 24 bits of the IP address are recognized as the "network" portion; the addresses of all the hosts in this range have the same value in the network portion.

Layer 3 forwarding - host to router

Figure 4.2: Layer 3 forwarding

The router has traditionally been a tool for interconnecting networks. As a layer 3 device, it uses layer 3 information to make forwarding decisions and requires that
4 2
Rev 10.41

Layer 3 (Network Layer) Troubleshooting and Problem Resolution

each interface leads to a different network. The diagram above illustrates layer 3 forwarding. When Host 1 wants to talk to Host 2, it first determines whether Host 2 is local to its own network. Host 1 uses its own IP address and mask to determine the range of addresses that are local. In the example above, Host 2 is not in the same address range as Host 1. The local range of Host 1 is 10.1.2.0 10.1.2.255. Since the intended destination is remote, Host 1 sends the traffic to the MAC address of its configured default gateway, which is a local router interface. All traffic destined for address ranges other than the local network are directed toward the default gateway. While Host 1 maintains an ARP cache that contains information about local hosts, including the default gateway, it has no knowledge of layer 2 addresses on the other side of the router.

Layer 3 forwarding router to host

Figure 4.3: Forwarding router to host

A router is not transparent to end stations; IP hosts are configured with a local router's address as a default gateway and they send to the router all traffic destined for hosts on other networks or subnetworks. The router performs a lookup operation on the packet's destination IP address against the entries in a routing table or cache. A successful lookup returns an outbound interface. The router performs an ARP cache lookup operation to resolve the layer 2 address of the destination IP host. In the slide below, the destination host is on a network that is directly connected to the router. If the destination network is not directly attached to the router, it sends the packet to another router that leads toward the destination network.
Rev 10.41

4 3

Troubleshooting HP Networks

The router encapsulates the outbound IP datagram in a new layer 2 header and forwards it to Host 2. Unlike the switched frame, which is forwarded without modification, a routed frame is always changed by the router.

E-Series Enable routing between VLANs

Figure 4.4: Routing between VLANs

To forward IP traffic between VLANs on the HP 5400zl switch, you need to add the global configuration level command: ip routing. When you enable routing, the IP addresses that are defined within the context of the VLANs are used as router interfaces that provide default gateway service for end stations The members of the VLANs may be tagged and/or untagged ports. Note that in the diagram above two of the ports on the routing switch, ports C1 and C2, lead to switches that support two port-based VLANs. Although these VLANs completely overlap from the perspective of the 5400zl switch, they have two different IP addresses; each VLAN has its own IP address that is within the range of the hosts in that VLAN. Also note that two ports on the switch, ports C3 and C4, lead to layer 2 switches whose ports are all within the same VLAN. Although there are multiple physical ports within this VLAN, there is only one IP address assigned to the group of ports. All of the hosts within the address range 10.1.4.0/24 will use the same IP address (10.1.4.1) as their default gateway. Also note that, although none of the layer 2 switches have active ports in VLAN 1, they do have an IP address within VLAN 1 for management purposes. The 2524 switches use VLAN 1 for management by default, called the primary VLAN. A-Series devices have routing enabled by default.
4 4
Rev 10.41

Layer 3 (Network Layer) Troubleshooting and Problem Resolution

VRRP Basics

Figure 4.5: VRRP basics

Basic default gateway redundancy operation

Common goals for default gateway redundancy methods:

Enable continuity for off-network communication despite the failure of the primary default gateway Provide for automatic failover from primary to backup default gateway within typical session timeout intervals Routers use shared IP address (virtual address or interface on one router) that is the default gateway address for hosts Backup router takes over forwarding if Master router fails or is unavailable

Common technologies and implementation methods:

VRRP: automatic failover for default gateway

Virtual Router Redundancy Protocol (VRRP) provides automatic failover for default gateways

Specified in IETF RFC 3768 Enables load sharing in designs that coordinate VRRP and MSTP Provides industry standard for default gateway provisioning Implemented on all HP Networking E-Series ProVision ASIC switches

VRRP terminology review

A virtual router consists of a set of router interfaces on the same network that share:

A virtual router identifier (VRID) A virtual IP address

One router in the group becomes the VRRP Master; other routers are VRRP Backup(s)
4 5

Rev 10.41

Troubleshooting HP Networks

The VRRP Master router periodically sends advertisements to a reserved multicast group address VRRP Backup routers listen for advertisements and assumes Master role if necessary

A VRRP router can support many virtual router instances, each with a unique VRID/IP address combination

Client interacts with virtual router

Figure 4.6: Client interacts with virtual router

Hosts on VRRP-protected networks learn the default gateways virtual MAC address from the Masters via ARP request Hosts send all off-network traffic to the local virtual MAC address without knowing it is not a physical address

Automatic failover

Figure 4.7: Automatic fallover

If the Owner fails, the non-Owner (backup) begins forwarding traffic addressed to the VRID 2 virtual MAC address (same as the Router 1 virtual MAC address) Host does not require any configuration changes or session restarts

4 6

Host is unaware that a different router is forwarding its off-network traffic


Rev 10.41

Layer 3 (Network Layer) Troubleshooting and Problem Resolution

OSPF Basics

Benefits

Offers faster convergence than RIP Scales to meet the needs of very large intranets OSPF routers advertise the state of connected links

Characteristics

Flood advertisements to neighbors, who flood to other neighbors

Depends on router adjacency, formal relationship used to share routing information Intelligent path selection based on bandwidth-sensitive link costs Divide large domain into smaller areas to enhance efficiency Careful design can avoid router overload

As described in IP Routing Foundations, OSPF is a sophisticated routing protocol designed to scale to meet the needs of very large enterprise networks. OSPF offers several important advantages over the older Routing Information Protocol (RIP), including faster convergence times as well as scalability. OSPF uses hierarchical areas to enhance efficiency. By making sound decisions when defining area borders, network designers can develop routing hierarchies that scale readily without placing undue load on the routers. This module will describe the design, deployment, and configuration of OSPF networking using the E-Series ProVision ASIC switches.

Figure 4.8: OSPF basics

OSPF provides a hierarchical routing structure based on multiple areas

Rev 10.41

Backbone area (Area 0) required


4 7

Troubleshooting HP Networks

Other area types include stub and NSSA

Router roles:

Area Border Router (ABR) Autonomous System Boundary Router (ASBR)

As described in IP Routing Foundations, OSPF provides a hierarchical routing structure that can scale to meet enterprise needs. The graphic, adapted from IRF, illustrates some basic elements of the OSPF topology. For more detail, consult IRF.

Enabling OSPF Enabling OSPF


10.1.65.0/30 10.1.67.3024

Server VLAN 10
E5406_A

Student VLAN 30

5406zl_A(config)# ip router-id 10.1.0.3 5406zl_A(config)# router ospf 5406zl_A(ospf)# area 0 5406zl_A(ospf)# vlan 10 5406zl_A(vlan-10)# ip ospf [area 0] 5406zl_A(vlan-10)# ip ospf passive 5406zl_A(vlan-10)# vlan 30 5406zl_A(vlan-30)# ip ospf 5406zl_A(vlan-30)# ip ospf passive 5406zl_A(vlan-30)# vlan 65 5406zl_A(vlan-65)# ip ospf 5406zl_A(vlan-65)# vlan 67 5406zl_A(vlan-67)# ip ospf 5406zl_A(vlan-67)# interface loopback 0 5406zl_A(lo-0)# ip ospf all
7 Rev. 10.41

Define Router ID Enable OSPF and create Area 0 Enable OSPF on each VLAN and the loopback interface; area ID defaults to Area 0

Optionally, define stub networks as passive

Figure 4.9: Enabling OSPF

Before enabling OSPF on an IP router, it is advisable to statically define a Router ID. If no Router ID is configured, the switch will assign one automatically. On the ESeries ProVision ASIC switches, the choice of ID will depend on other configuration items. Five possible cases are: 1. 2. 3. A single loopback interface and multiple VLANs with addresses The loopback interface will be used as ROUTER ID. A single loopback interface with multiple IP addresses The lowest loopback IP address will be used as Router ID. Multiple loopback interfaces with multiple IP addresses The lowest loopback number and lowest loopback IP address will be used as Router ID. Multiple VLANs with a single IP Address in each VLAN The IP address of the VLAN that becomes active first will be used as a Router ID. Typically, on E-Series switches, the lowest number VLAN becomes active first. Consequently, if an address is defined in VLAN 1, it will become the Router ID.
Rev 10.41

4.

4 8

Layer 3 (Network Layer) Troubleshooting and Problem Resolution

If VLAN 1 is down, the switch will use the next lowest number VLAN IP address as the Router ID. 5. Multiple VLANs with multiple IP addresses in each VLAN The lowest IP address of the first active VLAN will be used as a Router ID. In most cases, this will be a default VLAN IP address.

After the ID is defined, two separate commands are required to enable OSPF globally on the E-Series ProVision ASIC switches. In the first, you simply enable OSPF by issuing the router ospf command. In the second, you define at least one area. To form adjacencies, which are fundamental to OSPF operation, two OSPF routers must agree on an area ID, among other items. Note that the configuration for the loopback interface must include an argument specifying which IP addresses will be included in OSPF advertisements. In the example on the previous page, all indicates that all addresses will be included. Alternatively, the administrator could specify any address configured on the interface as this argument. On the E-Series ProVision ASIC switches, configuration of OSPF at the global and interface level is dynamic. Enabling OSPF on an interface may cause the router to:

1. 2.

Begin sending Hello packets through this interface in an effort to establish adjacencies. Include the network address range associated with this interface in its Router LSA.

To minimize OSPF processing overhead, interfaces with no neighboring routers, such as VLANs 10 and 30 in the example on the previous page, may be defined as passive. The router does not send Hello messages over a passive interface, which means it can never form an adjacency and will never send Link State Updates over this type of interface.

Rev 10.41

4 9

Troubleshooting HP Networks

Verifying Verifying OSPF status OSPF Status


View status of OSPF interfaces
5406zl_A(config)# show ip ospf interface OSPF Interface Status IP Address ----------10.1.0.3 10.1.10.1 10.1.30.1 10.1.65.2 10.1.67.2 Status --------enabled enabled enabled enabled enabled Area ID --------backbone backbone backbone backbone backbone State -----LOOP DR DR DR DR Auth-type --------none none none none none Cost ---1 1 1 1 1 Pri --1 1 1 1 1 Passive ------no yes yes no no

View state of the routers neighbors


5406zl_A(config)# show ip ospf neighbor

OSPF Neighbor Information


Router ID ----------10.1.0.1 Pri ---1 IP Address ---------10.1.65.1 NbIfState --------BDR State -----FULL Rxmt QLen ---------0 Events -----6

Rev. 10.41

Figure 4.10

After assigning each IP interface to an OSPF area, you can verify the status of configured OSPF interfaces by issuing the show ip ospf interface command. In the example shown in the figure above, only the backbone area has been defined, and all interfaces are associated with the backbone area. All of these interfaces were configured with default settings for authentication type, cost, and priority. OSPF interfaces 10.1.10.1/24 and 10.1.30.1/24 were defined as passive. The State column indicates the relationship each OSPF interface has with neighboring routers. Note that the passive interfaces have the Designated Router state. The interfaces assume this role even though the router does not expect to find neighbors on these networks. This router has a neighbor on the network 10.1.65.0/30, which is indicated in the output from the OSPF neighbor table. The entry in this table shows the neighbors Router ID, its IP address on the network it shares with E5406_A, and the state of the neighbor relationship. In this case, the neighbor is the Backup DR of the network 10.1.65.0/30. The next slide will provide more detail on the OSPF neighbors table.

4 10

Rev 10.41

Layer 3 (Network Layer) Troubleshooting and Problem Resolution

Viewing OSPF neighbor states Viewing OSPF neighbor states


E8212_A has full adjacency with one neighbor on each of the following networks:

10.1.64.0/30

10.1.64.0/30 10.1.65.0/30 10.1.68.0/30

E8212_A 10.1.0.1

E8212_B 10.1.0.2

10.1.65.0/30 10.1.67.0/30
E5406_A 10.1.0.3

10.1.66.0/30 10.1.68.0/30
E5406_B 10.1.0.4

With equal interface priorities, the OSPF router with the highest router ID becomes the Designated Router
E8212_A(config)# show ip ospf neighbor OSPF Neighbor Information Router ID --------------10.1.0.2 10.1.0.3 10.1.0.4
9 Rev. 10.41

Pri --1 1 1

IP Address --------------10.1.64.2 10.1.65.2 10.1.68.2

NbIfState --------DR DR DR

State -------FULL FULL FULL

Rxmt QLen --------0 0 0

Events ---------6 6 7

Figure 4.11: OSPF neighbor states

The figure on the previous page showed how information from the OSPF interface and neighbor tables can be combined to learn the state the router interfaces on a given network. In the figure above, the neighbor table from a different router, E8212_A, which has three neighbors. Because all of E8212_As neighbors have Router IDs that are higher than E8212_As Router ID, which is 10.1.0.1, all three neighbors have assumed the role of Designated Router on their respective networks. If you were to view the OSPF interface table, you would see that E8212_A has the Backup DR state for the three networks that support its full adjacencies. As shown, the neighbor table identifies each adjacent router by its Router ID and the IP address on the interface where the adjacency has formed. The table also indicates each neighbors priority and state. Use the OSPF neighbor table to troubleshoot routing problems that may arise from the failure to form an adjacency.

Rev 10.41

4 11

Troubleshooting HP Networks

External and internal Border Gateway Protocol (BGP)


BGP uses the Transmission Control Protocol (TCP) as its transport protocol, using port 179 for establishing connections. Running over a reliable transport protocol eliminates the need for BGP to implement update fragmentation, retransmission, acknowledgment, and sequencing. The Internet is organized in a multitude of administratively independent networks called domains or Autonomous Systems (AS). For example, an AS can be an Internet Service Provider (ISP), a University campus or a corporate network. The Border Gateway Protocol is an inter-Autonomous System routing protocol. The primary function of a BGP speaking system is to exchange network reachability information with other BGP systems. This network reachability information includes information on the list of Autonomous Systems (ASs) that reachability information traverses. This information is sufficient to construct a graph of AS connectivity from which routing loops may be pruned and some policy decisions at the AS level may be enforced The route to each destination is called the AS path, and the additional route information is included in path attributes. BGP uses the AS path and the path attributes to completely determine the network topology, detect and eliminate routing loops and it can enforce administrative preferences and routing policy decisions.

Contrasting eBGP and iBGP


BGP-4 provides a new set of mechanisms for supporting CIDR. These mechanisms include support for advertising an IP prefix and they eliminate the concept of network "class" within BGP. BGP-4 also introduces mechanisms which allow aggregation of routes, including aggregation of AS paths. Once BGP speakers are connected they exchange messages to start a BGP session with a neighbor. This initial message identifies the senders AS number and BGP identifier. Dependent upon whether the two speakers are in the same AS or different will govern the session type. There are two basic session types for BGP, interior and exterior. While there are many similarities between exterior and interior BGP, the most important difference is that the BGP speakers in an interior BGP peer session are in the same AS. Interior BGP is used within a transit AS, as is shown in the diagram below.

4 12

Rev 10.41

Layer 3 (Network Layer) Troubleshooting and Problem Resolution

Figure 4-12: Contrasting eBGP and iBGP

Note that BGP routers at the "edge" of a domain will support both interior BGP peers and exterior BGP peers.

BGP messages and route selection


Routers send open messages to each other to open or establish a BGP connection. The two routers must first establish a TCP connection between them. After which the sending of the Open Message is bidirectional. Routers send Open Messages out and wait until they receive an Open Message from their peer before continuing. Once the BGP peer is established, routers can exchange routing information. This routing information is contained in Update Messages Once the connection has been established, the routers send incremental updates that include summarized address ranges and AS numbers. (Messages vary somewhat dependent upon whether they are between interior or exterior BGP speakers.) They also send keepalives to maintain the session. The router builds a graph or table of the destinations and the attributes. BGP uses the AS or Autonomous System number to select the shortest path to route data and avoid routing loops. The two routers use UPDATE messages to add new routes, replace existing routes, withdraw invalid routes, and communicate attributes. BGP Notification Messages are an error message. The router selects the error type, and puts it into the Notification Message and sends it to the peer. It then tears down the peer connection Notification Messages consist of multiple pieces, including the BGP header, error code, error sub-code, and data that describes the error. This is important as it helps the Notification Message recipient router to troubleshoot BGP peering problems
Rev 10.41

4 13

Troubleshooting HP Networks

Network Address Translation (NAT)

Figure 4.13: Network address translation (NAT)

Network Address Translation (NAT) was originally created as a solution to the limited number of public IP addresses. Internet Protocol version 4 (IPv4) uses four octets (32 bits) of address spacewhich does not provide enough IP addresses for the current demandand IPv6 is not yet widely implemented. NAT can provide an alternative to obtaining a large block of registered addresses. With NAT implemented on the network, a company does not need a public IP address for each of its computers. NAT uses a device (a router, firewall, or computer) as an agent between the trusted network and the untrusted network. When a packet destined for the untrusted network reaches this device, the senders private IP address is translated into either the companys one public IP address or one of a limited range of such addresses assigned to that company. NAT also provides security: you give away nothing about your companys internal network if you use NAT when communicating with untrusted networks. The NATenabled device adds an entry to its address translation table that maps the internal address it replaced with the new public IP address. When the destination computer sends a reply packet back through the router, the router uses the table to identify the original internal IP address and sends the reply back to the appropriate computer on the trusted network. The following sections discuss the various types of NAT technology available. These include single IP address translation, static NAT, dynamic NAT, Port Address Translation (PAT), and NAT Traversal (NAT T).

4 14

Rev 10.41

Layer 3 (Network Layer) Troubleshooting and Problem Resolution

Single IP Address Translation

Figure 4.12: Single address translation

Single IP address translation allows one public IP address to be used by a full IP network. In this version of NAT, the available port numbers of the NAT-enabled gateway (router) are assigned to different private IP addresses. This allows multiple simultaneous TCP/IP sessions to occur using only the routers public IP address.

How It Works
When an internal computer sends a packet (containing the source IP address, source port, destination IP address, and destination port), the packet must travel through the NAT-enabled router. At this point, the router rewrites the packet header so that it contains the routers public IP address instead of the source IP address. The router then encapsulates the package to send to its destination. When the router rewrites the packet, it adds an entry into the address translation table that maps the internal address it replaced with its own public IP address. When the destination computer sends a reply packet back through the router, the router identifies its original internal IP address from the address translation table and sends the reply back to the appropriate computer. The above figure illustrates this process.

Rev 10.41

4 15

Troubleshooting HP Networks

Static and Dynamic NAT

Figure 4.13: Static and dynamic NAT

Static NAT maps an internal IP address to a public IP address on a one-to-one basis. That is, static NAT will always assign a particular computer the same public IP address. For example, it will always assign the computer with IP address 192.168.45.10 the public IP address 213.18.121.1 10. Dynamic NAT maps an internal IP address to a public IP address from a range of public addresses assigned to that company. A computer on the trusted network is dynamically assigned a random IP address depending on which addresses are available at a given time. For example, NAT can assign a computer public IP address 213.18.121.1 one time and then assign that same computer IP address 10 213.18.121.1 the next time that computer tries to send a packet to the untrusted 16 network. Static NAT is particularly useful when a device needs to be accessible from outside the network. Conversely, implementing dynamic NAT automatically creates a firewall of sorts between a companys internal network and untrusted networks: NAT only allows connections that originate from the trusted network. Essentially, this means that a computer in an untrusted network cannot connect to a computer in the trusted network unless the trusted host initiates contact first.

4 16

Rev 10.41

Layer 3 (Network Layer) Troubleshooting and Problem Resolution

Lab 5: Layer 3 Practice and Tools


Lab 5 is design to ensure you can use the troubleshooting methodology and troubleshooting tools. There are three trouble tickets in this lab. Refer to your lab guide for instructions on how to do these labs.

Rev 10.41

4 17

Troubleshooting HP Networks

Lab 6: OSPF Routing Issues


Lab 6 is designed to ensure you can use a structured troubleshooting methodology to resolve OSPF routing problems. There is one trouble ticket in this lab. Refer to your lab guide for instructions on how to do this lab.

4 18

Rev 10.41

Layer 3 (Network Layer) Troubleshooting and Problem Resolution

Lab 7: Addressing Issues


Lab 7 is design to ensure you can use a structured troubleshooting methodology to resolve IP addressing problems. There is one trouble ticket in this lab. Refer to your lab guide for instructions on how to do this lab.

Rev 10.41

4 19

Troubleshooting HP Networks

Lab 8: Inter-VLAN and Routing


Lab 8 is design to ensure you can use a structured troubleshooting methodology to resolve inter-VLAN routing issues. There is one trouble ticket in this lab. Refer to your lab guide for instructions on how to do this lab.

4 20

Rev 10.41

Layer 4 (Transport Layer) Troubleshooting and Problem Resolution


Module 5

This module focuses on troubleshooting at the transport layer 4. Upper layer protocols such as TCP, UDP, HTTP, FTP and Telnet run on top of the IP layer 3.

Figure 5.1: The 5 layer IETF model In the course the five Layer IETF model is used to describe a layered approach to networking. The TCP/IP model consists of four Layers. Even though there are some architectural differences, both models have interchangeable transport and network layers and their operation is based upon packet-switched technology.

Rev 10.41

5 1

Troubleshooting HP Networks

Troubleshooting TCP/UDP
The Host-to-Host (Transport) Layer contains two protocols; Transmission Control Protocol (TCP) and User Datagram Protocol (UDP). TCP and UDP are used to transmit datagrams.

Figure 5.2: Contrasting TCP and UDP Below is a description of major differences between TCP and UDP.

Reliable/Connection-Oriented TCP is a connection-oriented protocol. When a file or message send it will get delivered unless connections fails. If connection lost, the server will request the lost part. There is no corruption while transferring a message.

Unreliable/connectionless UDP is connectionless protocol. When you a send a datagram or message, you don't know if it'll get there, it could get lost on the way. There may be corruption while transferring a message

Ordered Each message is sent with a sequence number, so that even if they arrive out of order, they can be reassembled in the correct order.

Not Ordered If you send two messages out, and they arrive out of order, the application itself would be responsible for reassembly in the proper order.

Heavyweight When the low level parts of the TCP "stream" are lost, resend requests have to be sent, and all the out of sequence parts have to be put back together, so requires a bit of work to piece together.

Lightweight No ordering of messages, no tracking connections, etc. This means it's a lot quicker, and the network card / OS have to do very little work to translate the data back from the packets.

Streaming
Rev 10.41

5 2

Layer 4 (Transport Layer) Troubleshooting and Problem Resolution

Data is read as a "stream," with nothing distinguishing where one packet ends and another begins.

Datagram Packets are sent individually and are guaranteed to be whole if they arrive.

The TCP header occupies quite a large space in the Ethernet frame.

Figure 5.3: TCP message segment format


Source Port: 16 bits - The source port number. Destination Port: 16 bits - The destination port number. Sequence Number: The sequence number of first data octet in the segment (except when SYN is present). If SYN is present the sequence number is the initial sequence number (ISN) and the first data octet is ISN+1. Acknowledgment Number: If the ACK control bit is set this field contains the value of the next sequence number the sender of the segment is expecting to receive. Once a connection is established this is always sent. Data Offset: The number of 32 bit words in the TCP Header. This indicates where the data begins. The TCP header (even one including options) is an integral number of 32 bits long. Reserved: 6 bits - Reserved for future use. Must be zero. Flags: 6 bits and contains:

URG: Urgent Pointer field significant ACK: Acknowledgment field significant PSH: Push Function RST: Reset the connection SYN: Synchronize sequence numbers FIN: No more data from sender

Rev 10.41

5 3

Troubleshooting HP Networks

Window: 16 bits - The number of data octets beginning with the one indicated in the acknowledgment field which the sender of this segment is willing to accept. Checksum: 16 bits The TCP Length: The TCP header length plus the data length in octets (this is not an explicitly transmitted quantity, but is computed). Urgent Pointer: This field communicates the current value of the urgent pointer as a positive offset from the sequence number in this segment. The urgent pointer points to the sequence number of the octet following the urgent data. This field is only be interpreted in segments with the URG control bit set. Options: Options may occupy space at the end of the TCP header and are a multiple of 8 bits in length. All options are included in the checksum. An option may begin on any octet boundary. There are two cases for the format of an option:

A single octet of option-kind. An octet of option-kind, an octet of option-length, and the actual optiondata octets.

The option-length counts the two octets of option-kind and option-length as well as the option-data octets. Note that the list of options may be shorter than the data offset field might imply.

Data: variable - The actual user data is included after the end of the header

To troubleshoot TCP and UDP it is often necessary to analyze TCP segments using a network analyzer tool such as Wireshark. The TCP Packet capture shown in the figure below is a request-response message sequence carried over TCP. Notice the fields discussed above: Source Port, Destination Port, Sequence number, Window size, Flags, Checksum and options.

5 4

Rev 10.41

Layer 4 (Transport Layer) Troubleshooting and Problem Resolution

Figure 5.4: TCP packet capture UDP does not ensure that the data bytes sent will arrive at the other site. Thus, UDP imposes less network overhead than TCP.

Source Port: The 16-bit port number of the process that originated the UDP message on the source device. This will normally be an ephemeral (client) port number for a request sent by a client to a server, or a well-known/registered (server) port number for a reply sent by a server to a client. Destination Port: The port number of the process that is the ultimate intended recipient of the message on the destination device. This will usually be a wellknown/registered (server) port number for a client request, or an ephemeral (client) port number for a server reply. Length: The length of the entire UDP datagram, including both header and Data fields. Checksum: An optional checksum computed over the entire UDP datagram plus a special pseudo header of fields. See below for more information. Data: The encapsulated higher-layer message to be sent.

Rev 10.41

5 5

Troubleshooting HP Networks

Figure 5.5: UDP message segment format Below is a picture of a packet capture of the UDP section of the Ethernet frame. Note that the UDP packet capture shows the Source port, Destination port, Length and Checksum

Figure 5.6: UDP packet capture

5 6

Rev 10.41

Layer 4 (Transport Layer) Troubleshooting and Problem Resolution

Firewalls
Layer 4 protocols are subject to packet filters and firewalls. It is possible to have IP connectivity between the network components but certain packets are unable to traverse between a source and destination address. These types of connectivity issues may cause by problems with:

Firewalls Packet filters Servers Authentication and authorization Application software interoperability Operating system interoperability

In this section we are going to look at troubleshooting firewall and packet filter issues.

Firewall configurations
You have many options when deciding where or how to implement your firewall. The configuration typically includes a combination of routers, gateways, and servers on the edge of a trusted network. Firewalls can be configured in (but are not limited to) the following architectures shown in the picture below.

Figure 5.7: Firewall configurations

Rev 10.41

5 7

Troubleshooting HP Networks

Denying or permitting packets


A firewall is a collection of components configured to enforce a specific access control policy between your internal (trusted) network and any other (untrusted) network. As the above figure shows, a firewall protects your companys internal network from the Internet. A firewall filters incoming and outgoing packets to ensure only authorized packets pass. You must set up a clearly defined security policy that delineates authorized traffic. For example, you can configure rules in which the firewall drops packets from specific untrusted servers that you identify by IP address. Essentially, you can use one of two principles when implementing rules for your companys firewall:

Deny everything except that which is explicitly permitted Permit everything except that which is explicitly denied

5 8

Rev 10.41

Layer 4 (Transport Layer) Troubleshooting and Problem Resolution

Firewall types
Firewalls fall into one or more of the following categories: Packet-filtering firewall:

Must establish a predefined table of rules against which a packet-filtering firewall compares the full association of the packets. Must specify which packets should be accepted and which denied. Can create rules that will drop packets from specific untrusted servers, which you identify by IP address. Can also create rules that permit particular types of connections (such as FTP connections) only if they are using the appropriate trusted servers (such as the FTP server). Acts as a proxy server to establish a circuit with the internal computers. All outgoing packets from the trusted clients appear to have the proxy servers source IP. After a connection is established, the circuit-level gateway simply copies and forwards packets back and forth without filtering them further. Acts as a proxy server between a trusted client and an untrusted host. Only accept packets generated by services they are designed to copy, forward, and filter. For example, only a telnet proxy can copy, forward, and filter telnet traffic. Combines all of the above. Filtering all incoming and outgoing packets based on source and destination IP addresses and port numbers.

Circuit-level gateway

Application-level gateway

Stateful-inspection firewall

Ensuring packets in a session are appropriateEvaluates the contents of each packet up through the application layer and ensures that these contents match the rules in your companys network security policy.

Algorithms compare packets against known bit patterns of authorized packets.

Rev 10.41

5 9

Troubleshooting HP Networks

Table 5.1: Contrasting firewall types Few firewalls belong in only one of these categories, and fewer still exactly match the definition for any one category. These categories, however, do reflect the key capabilities that differentiate one firewall from another.

Figure 5.8: Stateful-inspection firewalls In a specific firewall implementation, the various types can be combined to create complex, sophisticated solutions. For example, a dual-homed host can be either a circuit-level gateway or an application-level gateway. A screened subnet includes at least two packet-filtering firewalls.

5 10

Rev 10.41

Layer 4 (Transport Layer) Troubleshooting and Problem Resolution

Network address translator (NAT)


There are various types of NAT technology available. These include

Single IP address translation Static NAT and dynamic NAT Port Address Translation (PAT) NAT Traversal (NAT T)

Network address translation (NAT) was discussed in an earlier module. This module extends this discussion to include Port Address Translation (PAT). Often, a companys global address pool does not contain enough public IP addresses to ensure all hosts in the trusted network can be mapped to an Internet address when they need to be. In this situation, the company should implement Port Address Translation (PAT). PAT maps each host in the trusted network to a global IP address and also to a unique TCP or UDP port number on the NAT-enabled router. In this way, PAT can map the same global IP address to a number of private IP addresses; it uses the unique port number to distinguish between them.

Figure 5.9: Port address translation (PAT) The router stores the original IP address and port against the new IP address and port in the address translation table. When the destination computer on the untrusted network sends a reply packet back through the router, the router identifies the recipient on the trusted network using the address translation table and routes the packet appropriately.

Enabling PAT NAT


PAT must be enables when you configure NAT, that is:

Configure a basic or advanced ACL for each range of private addresses for which you want to provide NAT. Configure a pool for each consecutive range of Internet addresses to which you want NAT to be able to map the private addresses specified in the ACLs. Each pool must contain a range with no gaps. If your Internet address space has
5 11

Rev 10.41

Troubleshooting HP Networks

gaps, configure separate pools for each consecutive range within the address space.

Associate a range of private addresses (specified in a basic or advanced ACL) with a pool. Enable the Port Address Translation feature if you have more private addresses that might need NAT than the Internet address pools contain.

Enable outbound NAT on the interface connected to global addresses. The following commands configure a basic ACL for the private subnet 10.10.10.x/24, then enable inside NAT for the subnet. This example has Port Address Translation Enabled. # acl number 2001 rule permit source 10.10.10.0 0.0.0.255 # nat address-group 1 209.157.1.2 209.157.1.254 )# interface Serial 5/0 nat outbound 2001 address-group 1

5 12

Rev 10.41

Layer 5 (Application Layer) Troubleshooting and Problem Resolution


Module 6

In this module, the common issues around troubleshooting application layer 5 issues will be reviewed and common problems will be discussed. The most common application layer problems evolve around QoS. The focus of this module is therefore around QoS.

Rev 10.41

6 1

Troubleshooting HP Networks

QoS process flow


QoS attributes include:

QoS parameters Congestion QoS mechanisms Switch QoS configurations Traffic handling techniques QoS policies

QoS parameters
The reason that networks exist is to enable users to access and run their applications. Applications include web browsing, file transfers, video streaming, email exchange, and voice conversations. These applications have different Quality of Service requirements, where Quality of Service defines the level of service that the application requires from the network. Quality of Service (QoS) parameters may include minimum data rates, packet error rates, jitter and latency. When making a QoS scheme, a network administrator must consider the characteristics of various applications to balance the interests of diversified users and fully utilize network resources. In addition, enterprises today are experiencing increased voice and video traffic over their networks, and many have fully migrated their voice traffic from a separate PBX network to run over their IP networks. Voice and video have different network requirements such that the voice and video quality will be perceived acceptable by the users:

Video and voice are both sensitive to jitter.


The variation in intervals between the arrival of packets. Can cause dead spots in real-time transmission. Relates to the amount of time that passes between the sending of a transmission and its arrival at the receiving station.

Voice is sensitive to delay, sometimes called latency.

Switches and routers can be configured to support these QoS needs.

Congestion
When the rate at which traffic arrives at a device exceeds the rate that the devicecan forward the traffic on a specific interface then congestion occurs. As such the interface that forwards packets is a basic network resource.

TCP applications such as web browsing can tolerate congestion


Rev 10.41

6 2

Layer 5 (Application Layer) Troubleshooting and Problem Resolution

Acknowledgement and flow-control mechanisms Lost packets retransmitted Back off procedure when congestion is detected No acknowledgement or flow control at the transport layer

UDP applications such as voice and video are more susceptible

Applications may provide acknowledgement and flow control Single application might monopolize link

No back off when congestion is detected

Queuing processes
Congestion management uses queuing and scheduling algorithms to classify and sort traffic leaving a port. Each queuing algorithm addresses a particular network traffic problem, and has a different impact on bandwidth resource assignment, delay, and jitter. Queue scheduling processes packets by priority, and preferentially forwards high-priority packets. Queuing processes include:

Strict Priority (SP) queuing. SP queuing is specially designed for mission-critical applications, which must be served first to reduce response delays when congestion occurs. SP queuing classifies eight queues on an A-Series switch port into eight classes, numbered 7 to 0 in descending priority order. SP queuing schedules the eight queues strictly according to the descending order of priority. It sends packets in the queue with the highest priority first. When the queue with the highest priority is empty, it sends packets in the queue with the second highest priority, and so on. Thus, you can assign mission-critical packets to the high priority queue to ensure that they are always served first and common service packets to the low priority queues and transmitted when the high priority queues are empty. The disadvantage of SP queuing is that packets in the lower priority queues cannot be transmitted if there are packets in the higher priority queues. This may cause lower priority traffic to be starved and never be transmitedt.

Weighted Round Robin (WRR) queuing WRR queuing schedules all the queues in turn to ensure that each can be served for a certain time. Assume there are eight output queues on a port. WRR assigns each queue a weight value (represented by w7, w6, w5, w4, w3, w2, w1, or w0) to decide the proportion of resources assigned to the queue. On a 100 Mbps port, you can configure the weight values of WRR queuing to 5, 5, 3, 3, 1, 1 1, and 1 (corresponding to w7, w6, w5, w4, w3, w2, w1, and w0 respectively). In this way, the queue with the lowest priority is assured, thus avoiding the disadvantage of SP queuing that packets in low-priority queues may fail to be served for a long time.

Rev 10.41

6 3

Troubleshooting HP Networks

Another advantage of WRR queuing is that while the queues are scheduled in turn, the service time for each queue is not fixed, that is, if a queue is empty, the next queue will be scheduled immediately. This improves bandwidth resource use efficiency.

Weighted Fair Queuing (WFQ) The only difference between WFQ and WRR is that: WRR schedules certain number of packets from a queue in each cycle of scheduling, while WFQ schedules certain number of bytes from a queue in each cycle of scheduling. Additionally, WFQ can work with the minimum guaranteed bandwidth mechanism. You can configure a minimum guaranteed bandwidth for each WFQ queue, so that each WFQ queue is guaranteed of the bandwidth when congestion occurs. The assignable bandwidth (total bandwidth the sum of the minimum guaranteed bandwidth for each queue) is allocated to queues based on queue priority. Because WFQ can balance delay and jitter among congested flows, it can be applied in certain special scenarios. For example, WFQ is used for the assured forwarding (AF) services of the Resource Reservation Protocol (RSVP). In Generic Traffic Shaping (GTS), WFQ schedules buffered packets.

SP+WRR queuing. By assigning some queues on the port to the SP scheduling group and the others to the WRR scheduling group (group 1), you implement SP + WRR queue scheduling on the port. Packets in the SP scheduling group are scheduled preferentially. When the SP scheduling group is empty, packets in the WRR scheduling group are scheduled. Queues in the SP scheduling group are scheduled with the SP queue scheduling algorithm. Queues in the WRR scheduling group are scheduled with WRR.

QoS mechanisms
QoS mechanisms enable network administrators to manage the use of network resources, enabling mission critical applications to receive priority access to network resources over lower priority traffic. Traffic arriving at the device is separated into flows via a process referred to as Classification.

Classification

Recognize traffic that should be prioritized Assign an internal traffic class (internal forwarding priority)

The device maps priority values to its internal queues and forwards appropriately. If transmitting host does not mark its own traffic, devices can apply policies to inbound traffic

Marking
Rev 10.41

6 4

Layer 5 (Application Layer) Troubleshooting and Problem Resolution

Indicates within the header how traffic should be handled for the benefit of other devices

Layer 2 marking: IEEE 802.1p Layer 3 marking: IP Precedence or Differentiated Services Code Point (DSCP)

Scheduling algorithms determine the packets and the rate of the packets that will be forwarded on the interface.

Scheduling / traffic shaping


Place traffic in queues based on traffic class Allocate sufficient percentage of outbound bandwidth for high priority traffic

Figure 6.1: QoS mechanism

Switch QoS configuration


As such when configuring the switch to provide QoS to application traffic, you must configure the QoS interface parameters including:

Classifications Behavior Number of queues

Traffic filtering
You can filter in or filter out a class of traffic by associating the class with a traffic filtering action. For example, you can filter packets sourced from a specific IP
Rev 10.41

6 5

Troubleshooting HP Networks

address according to network status. By using ACL rules configured with a time range for traffic classification, you can implement time-based traffic filtering. Class of Service (CoS) is:

The process of classifying traffic based on:


Layer 2: IEEE 802.1p Layer 3: IP Precedence or DSCP

A classification method only A tool used by scheduling (queuing) mechanisms to limit delay

To illustrate traffic filtering, below is an example configuration for a host connected to interface GigabitEthernet 1/0/1 of the switch. The requirement is to configure traffic filtering to filter the packets whose TCP source port number is 21 received on the interface # Create advanced ACL 3000, and configure a rule to match packets whose source TCP port number is 21. [DeviceA-acl-basic-3000] rule 0 permit tcp source-port eq 21 [DeviceA-acl-basic-3000] quit # Create a class named classifier_1, and use ACL 3000 as the match criterion in the class. [DeviceA] traffic classifier classifier_1 [DeviceA-classifier-classifier_1] if-match acl 3000 [DeviceA-classifier-classifier_1] quit # Create a behavior named behavior_1, and configure the traffic filtering action for the behavior to drop packets. [DeviceA] traffic behavior behavior_1 [DeviceA-behavior-behavior_1] filter deny [DeviceA-behavior-behavior_1] quit # Create a policy named policy, and associate class classifier_1 with behavior behavior_1 in the policy [DeviceA] qos policy policy [DeviceA-qospolicy-policy] classifier classifier_1 behavior behavior_1 [DeviceA-qospolicy-policy] quit # Apply the policy named policy to the incoming traffic on interface GigabitEthernet 1/0/1. [DeviceA] interface gigabitethernet 1/0/1
6 6
Rev 10.41

Layer 5 (Application Layer) Troubleshooting and Problem Resolution

[DeviceA-GigabitEthernet1/0/1] qos apply policy policy inbound

Rev 10.41

6 7

Troubleshooting HP Networks

802.1p traffic prioritization


Traffic handling techniques generally involve the host sending time-sensitive traffic modifies bits in packet header. Examples of protocols that do this include:

DSCP or IP Precedence: Priority field within IP datagram header IEEE 802.1p: Priority field within 802.1Q tag

802.1p is a layer 2 marking that is used in many LANs. 802.1p defines a field in the MAC Ethernet header that carries one of eight priority values as shown in the picture below.

IEEE 802.1p reserves a three-bit field in the 802.1Q tag Some end stations set priorities for their traffic

Figure 6.2: 802.1p priority tag Switches can retain or modify markers for prioritized traffic forwarded over tagged links. The table below provides an example of parameters that can be configured in a ESeries switch.

Minimum percentages shown below are configurable per port If all waiting traffic has the same priority level (e.g. normal) in a given time period, 100% of the bandwidth is given to that traffic.

6 8

Rev 10.41

Layer 5 (Application Layer) Troubleshooting and Problem Resolution

Table 6.1: Illustration of 802.1p switch settings

Configuring QoS policy


Switches can act as QoS policies enforcement points (PEP) to control access. PEPs determine whether traffic can be admitted.

Figure 6.3: Configuring QoS A QoS policy can be applied to:

An interface, the policy takes effect on the traffic sent or received on the interface.
6 9

Rev 10.41

Troubleshooting HP Networks

A user profile, the policy takes effect on the traffic sent or received by the online users of the user profile. A VLAN, the policy takes effect on the traffic sent or received on all ports in the VLAN. Globally, the policy takes effect on the traffic sent or received on all ports.

Default QoS example


The picture below shows an example of the E-Series switch supporting both a data VLAN and a voice VLAN.

Figure 6.4: Default QoS on A-Series 3500 switches

6 10

Rev 10.41

Layer 5 (Application Layer) Troubleshooting and Problem Resolution

Traffic marking by an end station

Many IP phones mark their traffic for high-priority handling. In this illustration: 1. Phone marks priority level in IEEE 802.1Q tag 2. The edge switch a. Classifies traffic based on priority marker in tag b. Schedules packet for delivery by placing it in queue associated with traffic class

Figure 6.5: IP phone illustration

Display the QoS policy applied to VLAN


Below is an example of how to display the parameters of VLAN 2. # show qos vlan-policy vlan 2 VLAN priorities VLAN ID Apply rule | DSCP Priority ------- ----------- + ------ ----------1 500 No-override | No-override | No-override No-override

Rev 10.41

6 11

Troubleshooting HP Networks

Retaining priority between VLANs


Continuing the previous example: 3. The core switch classifies traffic based on priority marker in tag 4. The core switch a. Marks priority in 802.1p field of outbound packets 802.1Q tag b. Schedules packet for delivery by placing it in appropriate queue c. Classifies and schedule delivery

Figure 6.6: Continuation of IP phone illustration

Configuring port priority


Below is an example of how port priorities can be set per VLAN or per interface. # vlan 500 (vlan-500)# qos priority (vlan-500)# qos dscp priority 0 1 2 3
6 12
Rev 10.41

Specify DSCP policy to use. Specify priority to use.

(vlan-500)# qos priority

Layer 5 (Application Layer) Troubleshooting and Problem Resolution

4 5 6 7 # interface A1 (eth-A1)# qos priority (eth-A1)# qos dscp priority 0 1 2 3 4 5 6 7 Specify DSCP policy to use. Specify priority to use.

(eth-A1)# qos priority

Rev 10.41

6 13

Troubleshooting HP Networks

Normal priority data traffic

This last illustration show the normal priority data traffic. In this example the edge switch uplink (port 50) is a tagged member of VLAN 10; 802.1p field in tag contains 0 value. The steps are: 1. The users data traffic is sent untagged, with no priority marker 2. The edge switch a. Classifies the traffic as normal b. Then marks 0 value in 802.1p field of outbound packets 802.1Q tag c. Schedules packet for delivery, assigning it to the queue associated with normal traffic

Figure 6.7: Normal priority data traffic

6 14

Rev 10.41

Layer 5 (Application Layer) Troubleshooting and Problem Resolution

Lab 10: Quality of Service


Lab 10 is designed to ensure you can use a structured troubleshooting methodology to resolve Quality of Service problems. There is one trouble ticket in this lab. Refer to your lab guide for instructions on how to do this lab.

Rev 10.41

6 15

Troubleshooting HP Networks

This page left blank intentionally.

6 16

Rev 10.41

Troubleshooting an End-to-End Complex, Integrated Multi-Protocol Network


Module 7

This module brings all the lessons from the previous modules and challenges you to resolve a complex multi-protocol problem. Stable network operations are critical to most enterprises. Failure of the network results in productivity and revenue losses. Troubleshooting multiprotocol networks can be complex and formidable, however following a structured approach diagnosis and resolution can help resolve problems quickly and effectively. In this lab you will solve a trouble ticket that has several problems. To do this lab, you should use a structured approach to troubleshooting and document your steps.

Rev 10.41

7 1

Troubleshooting HP Networks

Lab 1 Final lab 1:


Lab 1 is designed to ensure you can use a structured troubleshooting methodology 1 to resolve problems at multiple protocol layers. There is one trouble ticket in this lab that contains several problems. Refer to your lab guide for instructions on how to do this lab.

7 2

Rev 10.41

To learn more about HP Networking, visit www.hp.com/networking


2010 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. The only warranties for HP products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.

You might also like