http://www.zog.net/Docs/serverdown.

html

HELP, I CAN'T REACH MY SERVER!

A Quick & Dirty Guide to Connection Troubleshooting for Beginners

This is not intended to be a technically intricate document to

allow a systems administrator to understand why an application or
server is not working correctly. Rather, it is meant as a step-by-step
guide for a relative technical novice to narrow down possible causes
for an unreachable server, before having to call a helpdesk.
Some of this stuff is pretty basic, so just skip around until you
find something you can use.
N.b. all output is from entering these commands on a laptop running
FreeBSD (Unix); syntax may vary depending on your operating system.

I.

Protocols & Ports

Out of academic interest, it might be useful for you to take a look at the
OSI (Open Systems Interconnect) model. This is a theoretical, schematic
model for looking at network connections--rather than being a written-in-stone
classification for different levels of network traffic, it's meant to allow
you to think about parts of network connections a bit more abstractly, and
to help you figure out what does what.
Here the main concepts you should consider:
A.

Internet Protocol (IP)

Most of the Internet, and most local area networks (LANs) function based
on the IP protocol, also referred to as IP version 4 (IPv4.) IP addresses
are unique on the internet, and usually look like this:
205.171.14.93
Each of these four 'octals' is an 8-bit number, and cannot be higher than
255.
IP addresses are parts of 'subnets', or 'ranges', which are assigned by
organizations such as IANA and RIPE. In order to be properly visible,
all IP addresses you want a single machine to reach on the internet must be
unique. Likewise, it is a bad idea to simply grab an IP on your local
network
and assume it will work--you may run into an 'IP conflict' which will make
your network administrator, and possibly another user, very unhappy.
B.

Transmission Control Protocol (TCP)

http://www.zog.net/Docs/serverdown.html (1 of 14) [4/11/2008 3:23:29 PM]

http://www.zog.net/Docs/serverdown.html

TCP is the backbone of internet traffic. Simply put, it is a means to

transport
'packets' between applications in a way that assures delivery. Every TCP
packet
has a unique identifier ("sequence number") which is incremented with each
packet in a given connection. TCP connections look as follows:
Client
Client
Client
Client

-->
<-<-<->

Server
Server
Server
Server

"SYN" (Synchronize)
"SYN/ACK" (Synchronize Acknowledged)
"ACK" (Acknowledge)
"ACK" (at this point the connection is

"established"
Client --> Server
Client <-- Server
Client <-- Server

"FIN" (Finish)
"FIN/ACK" (Finish Acknowledged)
"RST" (Reset--the connection is now dead.)

Depending on the type of connection, not all of these are necessary--for

example,
the server may just "RST" the connection without prompting, effectively
cutting it off.
C.

Unreliable Datagram Protocol (UDP)

UDP is a lot faster than TCP, as it does not do any error checking.
Concordantly,
it is used for applications where you don't care so much if the occasional
chunk
does not arrive. Internet video streaming is a typical application of UDP.
D.

Internet Control Message Protocol (ICMP)

ICMP is a way for traffic on the internet to decide where to go,

whether hosts are reachable, etc. It advises hosts that a next hop may not
be
available for traffic.
E.

Stacks

An IP stack is an area of virtual memory allotted in a machine to dealing

with
network connections.
with in this stack.
F.

All incoming and outgoing network connections are dealt

Ports

A "port" is a virtual address configured on a server to listen to

connections. It
is analogous to a mailbox in a highrise apartment building. Each
application
has a unique port, although many individual connections of the same type can
http://www.zog.net/Docs/serverdown.html (2 of 14) [4/11/2008 3:23:29 PM]

http://www.zog.net/Docs/serverdown.html

be
handled by a single port. Ports 1-512 by convention are "privileged",
meaning they
are supposed to only be handled by applications approved by a server
administrator.
Ports 512-1024 are "privileged", which implies that they are not to be used
by applications
run by a mere mortal user. However, whether this convention is adhered to
or not is
up to a server's administrator.
For a list of commonplace ports, look at
http://www.iana.org/assignments/port-numbers

II.

What's happening?

You may be seeing that a server "just isn't reachable". This be due
to a number of reasons--there's no way to predict all of them, but
this might give you a few hints as to what the cause could be.
Let's do some preliminary testing.

A.

Machine Settings (Very Basic)

First, check to see that you are on the net. Can you reach anything?
any of your networked applications work? Are Windows network drives
accessible?
Can you access any web pages at all? If not, check the following:
-Is the cable plugged in?
is it lit?

Do

If you have a link light on your network card,

-Do you have a default route? Is your interface configured?

Under Windows, run the command
ipconfig
in a DOS Window.

If you see an entry for 'Default Gateway', you're

set.
Under Unix, run
ifconfig -a
and

http://www.zog.net/Docs/serverdown.html (3 of 14) [4/11/2008 3:23:29 PM]

http://www.zog.net/Docs/serverdown.html

netstat -rn|grep default

Ifconfig (InterFace Config) allows you to see the configuration of
all your
active network interfaces.

Netstat is a general network utility

command;
If netstat -an shows you a default gateway, you should be okay.
-Are your general settings correct? Compare your station with others on
your local
network. If you have a network administrator handy, check your settings
against
what he gave you. Can you ping them? Can they ping you? Can you ping
your default
gateway? See below for details on Ping.

B.

ICMP/Ping

First, let's find out whether a host responds to basic pings.

"Ping" (Packet InterNet Groper).
is a common implementation of the ICMP protocol, and used to probe hosts. On
Unix servers, it is usually found in /usr/sbin or /usr/bin, and on Windows
machines it sits in c:\windows\system32\ or c:\winnt\system32.
The Syntax of this command is
ping
Successful pings generally mean that a server's address is reachable,
and look like this:
bolo:[15:23]~> ping www.berkeley.edu
PING arachne.berkeley.edu (169.229.131.109): 56 data bytes
64 bytes from 169.229.131.109: icmp_seq=0 ttl=237 time=180.286 ms
64 bytes from 169.229.131.109: icmp_seq=1 ttl=237 time=173.291 ms
^C
--- arachne.berkeley.edu ping statistics --3 packets transmitted, 2 packets received, 33% packet loss
round-trip min/avg/max/stddev = 173.291/176.788/180.286/3.497 ms
Under Solaris, 'ping -s ' will return this same result.
just will return ' is alive.'

'Ping' by itself

If your client station does not know how to get to a net (it does not have a
'route'),
or does not have a default gateway, you will see something like
bolo:[15:25]~> ping www.cnn.comPING cnn.com (207.25.71.5): 56 data
bytes
http://www.zog.net/Docs/serverdown.html (4 of 14) [4/11/2008 3:23:29 PM]

http://www.zog.net/Docs/serverdown.html

ping: sendto: No route to host

ping: sendto: No route to host
ping: sendto: No route to host
^C
--- cnn.com ping statistics --13 packets transmitted, 0 packets received, 100% packet loss
When you try to reach a remote address, you may pass many network devices
("routers")
on the way. If you are trying to reach an address that they don't know how
to get to,
you could see
bolo:[15:28]~> ping 10.0.0.1
PING 10.0.0.1 (10.0.0.1): 56 data bytes
36 bytes from Serial10-1-0.GW3.ZUR4.ALTER.NET (146.188.39.89):
Destination Host Unreachable
Vr HL TOS Len
ID Flg off TTL Pro cks
Src
Dst
4 5 00 5400 ae7e
0 0000 3c 01 0384 192.168.1.254 10.0.0.1
Lastly, if your output looks as follows, it is possible that there is a
firewall or
similar security device in the way, which is not permitting ICMP traffic.
This does not
mean that the host is down, just that it cannot be pinged.
bolo:[15:32]~> ping 165.222.187.100
PING 165.222.187.100 (165.222.187.100): 56 data bytes
^C
--- 165.222.187.100 ping statistics --32 packets transmitted, 0 packets received, 100% packet loss

C.

Traceroute

Traceroute is a program which allows you to find out which hops your traffic
passes on the way to a target server. It is usually found in the same
location
as ping, both under Unix and Windows. Traceroute's main function is to
allow you
a look at whether the path a transmission takes from client to server looks
sane. A successful traceroute means that the host is up and running, and
that
there is a proper route to it. Traceroute output looks like this:
soda:[6:38]~> traceroute www.berkeley.edu
traceroute to arachne.berkeley.edu (169.229.131.109), 64 hops max,
40 byte packets
ms

gig6-1v247.snr1.CS.Berkeley.EDU (128.32.247.1)

0.744 ms

vlan241.inr-201-eva.Berkeley.EDU (128.32.255.161)

0.655

0.610 ms

http://www.zog.net/Docs/serverdown.html (5 of 14) [4/11/2008 3:23:29 PM]

0.858 ms

http://www.zog.net/Docs/serverdown.html

0.761 ms

0.818 ms
3
ms 0.774 ms
4
0.732 ms

vlan209.inr-203-eva.Berkeley.EDU (128.32.255.2)
arachne.Berkeley.EDU (169.229.131.109)

0.861 ms

0.781 ms

0.986

0.717 ms

It can also look as follows. This does not mean that the server is
necessarily
down or unreachable, but could imply that there is a firewall/security device
in the middle:
soda:[6:39]~> traceroute www.google.com
traceroute to www.google.com (216.239.33.101), 64 hops max, 40 byte
packets
1
ms

gig6-1v247.snr1.CS.Berkeley.EDU (128.32.247.1)

0.777 ms

0.663

0.612 ms

2
0.785 ms
3
ms 0.515 ms
4
ms 0.559 ms
5
1.992 ms
6
2.236 ms
7
2.836 ms 2.906 ms
8
9
2.848 ms
10
ms 3.366 ms
11
ms 9.461 ms
12
ms 4.251 ms
13
4.610 ms
14
ms 5.315 ms
15
16
17

vlan241.inr-201-eva.Berkeley.EDU (128.32.255.161)

0.870 ms

0.736 ms

gigE2-0.inr-000-eva.Berkeley.EDU (128.32.0.193)
pos3-0.c2-berk-gsr.Berkeley.EDU (128.32.0.90)

0.599 ms
0.692 ms

0.537
0.579

SUNV--BERK.POS.calren2.net (198.32.249.14)

1.810 ms

1.852 ms

STAN--SUNV.POS.calren2.net (198.32.249.74)

2.230 ms

2.167 ms

PAIX-7206--STAN-3.ATM.calren2.net (198.32.249.186)

3.346 ms

paix.exodus.net (198.32.176.15) 3.071 ms 3.164 ms 3.503 ms

ibr02-g1-0.paix01.exodus.net (206.79.9.242) 3.875 ms 2.931 ms
bbr01-p6-0.sntc03.exodus.net (209.185.9.241)

3.404 ms

3.320

dcr04-g4-0.sntc03.exodus.net (216.33.153.68)

5.334 ms

3.382

csr01-ve242.sntc03.exodus.net (216.33.153.181)
google-exodus.exodus.net (64.68.64.210)

4.482 ms

4.574 ms

exbi1-gige-1-3.net.google.com (216.239.47.2)

3.876

3.610 ms

5.256 ms

5.382

* * *
* * *
* *^C

Lastly, you may see the next block of output--this implies that there is a
firewall
in the middle which allows traces to the final destination, but not to
itself (thus
nobody can identify it as a hop in the middle):

http://www.zog.net/Docs/serverdown.html (6 of 14) [4/11/2008 3:23:29 PM]

http://www.zog.net/Docs/serverdown.html

soda:[6:43]~> traceroute altavista.digital.com

traceroute to altavista.digital.com (209.73.180.1), 64 hops max, 40
byte packets
1
ms

gig6-1v247.snr1.CS.Berkeley.EDU (128.32.247.1)

0.837 ms

0.668

0.685 ms

2
0.785 ms 0.754 ms
3
ms 2.628 ms 2.364
4
ms 3.259 ms
5
3.361 ms
6
2.802 ms
7
3.094 ms
8
5.857 ms
9
ms 67.408 ms
10
ms 67.102 ms
11
ms 66.156 ms
12
13
14

vlan241.inr-201-eva.Berkeley.EDU (128.32.255.161)

1.465 ms

fast4-1-0.inr-new-666-doecev.Berkeley.EDU (128.32.0.73)
ms
qsv-juniper--ucb-gw.calren2.net (128.32.0.70) 3.717 ms

1.779
3.283

svl-edge-09.inet.qwest.net (65.113.32.209)

3.247 ms

2.917 ms

svl-core-01.inet.qwest.net (205.171.14.93)

3.315 ms

3.632 ms

sjo-core-01.inet.qwest.net (205.171.5.99)

3.831 ms

2.907 ms

sfo-core-02.inet.qwest.net (205.171.5.123)

4.591 ms

4.839 ms

jfk-core-01.inet.qwest.net (205.171.5.113)

66.522 ms

67.079

jfk-core-03.inet.qwest.net (205.171.230.6)

66.856 ms

66.065

jfk-edge-04.inet.qwest.net (205.171.30.114)

66.596 ms

66.322

63.148.0.22 (63.148.0.22) 74.506 ms 74.240 ms 73.975 ms

* * *
altavista.com (209.73.180.1) 74.845 ms 73.763 ms 74.196 ms

Traceroute and ping are two different applications. Furthermore, traceroute

on Windows and NT are not the same type of traffic. Windows traceroute uses
"ICMP ECHO REQUEST"
to see whether routes to a host are alive. Unix and Cisco traceroute is
generally based on UDP.
Some firewalls allow one, but not the other; if you suspect a firewall
between your client
and the target server, try traceroute from both types of platforms.
D.

Telnet

Telnet is a very basic login application designed to emulate a mainframe

terminal.
However, telnet has the ability to connect to any TCP ports to show you
whether a service
is even listening or not. On unix, telnet is normally located in /usr/bin/,
in Windows
it should be in C:\{Windows,Win98,WINNT}\System32.
On Unix, you can telnet to a given port for an application using the syntax

http://www.zog.net/Docs/serverdown.html (7 of 14) [4/11/2008 3:23:29 PM]

http://www.zog.net/Docs/serverdown.html

telnet hostname portnumber

Based on the documentation in section I.F., to see if my webserver on
www.switch.ch is listening on the default http port, I would try the
following:
bolo:[17:07]~> telnet www.switch.ch 80
Trying 130.59.10.30...
Connected to etna.switch.ch.
Escape character is '^]'.
This means that I have a connection established to the server.

In effect,

what
I have just done is "faked" the same sort of connection that a web browser
(Netscape,
Opera, Internet Explorer) makes to a webserver. Under Windows NT/9x telnet,
the equivalent
of this is when the server address is displayed in the telnet window's title
bar. Don't
forget to change the port number from the default 'telnet' (port 23) before
connecting.
Windows 2000 telnet works like Unix telnet.
If my service uses TCP and I get this prompt, and I am still having
connection difficulties,
I can rule out that there is a firewall or routing problem; the issue is
most likely
with the server's configuration or my client application.
Another possible response to a telnet attempt, this time to the SMTP (Simple
Mail Transport
Protocol, the most-used Internet email standard) port on an address not
configured to
receive SMTP mail, may yield this:
bolo:[17:15]~> telnet cfpa11.berkeley.edu 25
Trying 128.32.124.189...
telnet: connect to address 128.32.124.189: Connection refused
telnet: Unable to connect to remote host
This means that the target has sent me a "RST", or "RESET" TCP packet in
response to my
Attempt to "SYN". It's effectively telling me, no, I'm not paying attention
to this
port. The server is not configured correctly, or the program on the server
which is
supposed to be listening on the port isn't even running.
Telnet can also yield this:
bolo:[17:15]~> telnet www.google.com 12345
http://www.zog.net/Docs/serverdown.html (8 of 14) [4/11/2008 3:23:29 PM]

http://www.zog.net/Docs/serverdown.html

Trying 216.239.37.101...
telnet: connect to address 216.239.37.101: Operation timed out
telnet: Unable to connect to remote host
This usually indicates the presence of a firewall or other similar security
filter
in front of or on the server which explicitly disallows connections to port
12345.
Under Windows NT/9x telnet, you will see 'Telnet (none)' in the title bar,
and the cursor will be an hourglass.

III.

What's Happening on the Server?

Now, let's see if you can log into your server. If you are able to telnet/ssh/rlogin
to your Unix server, or have console/remote desktop access to a Windows NT server
on which your network server

A.

What's listening to what?

On your unix server, you can have a look at some locally defined ports in
the file
/etc/services. Furthermore, the file /etc/inetd.conf specifies which of
these services
your inetd, which is the program run by Unix at startup to listen to network
connections,
has an actual program (server) associated with it. This means that when
Inetd "sees" a
connection arriving on a given port, it starts a service to deal with it.
However, this is not the only way to run servers.

Different types of Unix

(Linux,
Solaris, AIX, etc.) have various methods for starting servers via scripts at
boot.
The easiest way to look at this is (on the server!) to run the commands
netstat -an
or
lsof -i (if installed)
To see which services are (supposedly) listening on which ports. If you
don't understand
these commands, see whether there is a manual page by running the command
man netstat (or lsof)
This should explain the syntax to you in the usual clear, concise Unix

http://www.zog.net/Docs/serverdown.html (9 of 14) [4/11/2008 3:23:29 PM]

http://www.zog.net/Docs/serverdown.html

manual page
language.

In short, for netstat -an, you may see these keywords:

CLOSED, LISTEN, SYN_SENT, SYN_RECEIVED, ESTABLISHED, CLOSE_WAIT,

FIN_WAIT_1,
CLOSING, LAST_ACK, FIN_WAIT_2, TIME_WAIT
Consult the netstat man page for the meaning of these. However, if you see
your port
listed with one of these comments, it means something's happening with it.
Windows NT
also has a netstat command, which should be located in your System32
directory.
Lsof (LiSt Open Files) with the -i argument should tell you which actual
application
is using which port. Have a look at its output, and see if it looks
reasonable.
Under Windows, there isn't much I can tell you except to look at your Task
Manager
(taskmgr.exe), and your services (under 'Administrative Tools'), and to not
run
servers on Windows :-)
B.

Sniffing

Lastly, if you have administrator rights on your server, you can run a
sniffer for
traffic reaching the server. Under Solaris, 'snoop' is an example, while
BSD-based
systems may have 'tcpdump' installed. Windows systems generally require an
additional
piece of software to be installed. RTFM is left to the user. The point
behind this
is to see whether traffic from your client address actually reaches the
server. Generally,
sniffers let you sort traffic by destination port or source IP address,
allowing you to
see whether traffic from the client even reaches your server system.
Please please be aware of the legal/administrative consequences of sniffing
network
traffic on a system--you may see things people don't want you to see, such
as passwords
or confidential data. This is why most sniffer software is restricted to use
by people with administrator rights.

IV.

Common Services

http://www.zog.net/Docs/serverdown.html (10 of 14) [4/11/2008 3:23:29 PM]

http://www.zog.net/Docs/serverdown.html

A server can be down for a number of reasons.

and what could cause them to not respond.

A.

Let's take a few common applications

Hyper Text Transfer Protocol (HTTP), Web Pages

Some common errors found on webservers are listed at

http://offline.home.cern.ch/offline/web/http_error_codes.html
CERN came up with the World Wide Web, so they should know.

By the way, the

above
is an example of a "Uniform Resource Locator", or "URL", which is nothing
more than a
means of specifying a protocol/application (in this case, HTTP, but can be
FTP, Gopher,
telnet, etc etc) an address, and a path (the stuff after the first single
"/").
These mean that, while the webserver you are trying to access is listening,
there
is some problem either with your client (browser) or the web server.
B.

Web Applications

If you are behind a firewall, or a proxy server, you may have troubles with
certain
applications (java applets for example) which your browser attempts to
download.
Specifically, this can be caused by an application attempting to access a
server
which your firewall does not allow, even though the application was actually
installed
via regular HTTP.
Certain multimedia applications, such as RealPlayer and other streaming
media clients
require various other ports to be opened--this means that your browser
understands
that a certain kind of file a webserver is giving it should be opened with a
specific
application, and the application then handles its own traffic.
C.

File Transfer Protocol (FTP)

FTP can either be via anonymous FTP (what you usually see when you have a
URL in
your browser window starting with 'ftp://') or interactively, either via
command-line
ftp from Unix or Windows, or a graphical client such as 'CuteFTP' or
http://www.zog.net/Docs/serverdown.html (11 of 14) [4/11/2008 3:23:29 PM]

http://www.zog.net/Docs/serverdown.html

'WS_FTP'.
There are two types of FTP, active and passive.
Active FTP opens a connection from your client to the server, which is used
for passing
'administrative' information--this includes your login and password.
However, the FTP server
then opens a connection back to you for the actual file transfer, regardless
of what
direction the file actually flows in (whether you 'PUT' or 'GET' the file
from the server.)
Certain firewalls can have trouble with this.
Passive FTP is an answer to this; it only opens the first connection from
client to server,
and all data also flows over this 'channel'. However, both the client and
server must understand
passive mode.

V.

My Server is Slow!

As so often happens, servers slow down.

server
access.
A.

There are a few things you can do, without

Ping the server

If the 'Round-trip' times you see in your ping are above a few hundred
milliseconds, there
is possibly a problem with the network between your client and server. This
can be
due to any number of factors; Use traceroute to see where the bottleneck is.
B.

Check client load

Self-Explanatory.
C.

However, everything else shouuld be slow as well.

Check server load

On the server, if it's an NT server you can run taskmgr to see what the
server load is.
On Unix, there are a number of commands you can look at;
(insofar as they
are installed)
top
uptime
http://www.zog.net/Docs/serverdown.html (12 of 14) [4/11/2008 3:23:29 PM]

these include

http://www.zog.net/Docs/serverdown.html

vmstat
ps
dmesg
You can also look at syslog and messages (in /var/log/ or /var/adm/) for
some further information.
I can't tell you what proper output of these will look like, but they will
give you information
on what could be wrong with a given application
D.

Check network load

Are you connected to a switch or a hub? Ethernet, the underlying protocol

for many IP
connections, is essentially a 'shared medium'. This means, that if two
machines on a local
physical segment simultaneously try to transmit data, you have what is
called a 'collision'.
The more collisions you get, the slower things are. A hub creates a larlge
physical segment,
leading to the possibility of collisions, while a switch is designed to
separate so-called
'collision domains'.
Make sure that your network card speed is correct. Some network cards try
to 'autonegotiate'
their speed with whatever switch/hub port they are connected to; sometimes
this breaks. Make
sure with the network administrator that both your station and the network
device are
trying to communicate at the same speed.
While you're at it, have them check network traffic load on that port, as
Most network
devices allow 'polling' of their load, via a protocol called SNMP (Simple
Network Management
Protocol). It could simply be a case of putting too much data on a slow
line, such as
downloading enormous files over a modem link.
well.

E.

Check the cable, network hardware

Ethernet/Cat5/UTP cabling can be sensitive to various factors, such as

errors introduced
by radiation from nearby fluorescent lighting, untwisting of the wires (UTP
= "unshielded
twisted pair"), etc.
Your network staff can also usually check logfiles on a router/switch/hub
http://www.zog.net/Docs/serverdown.html (13 of 14) [4/11/2008 3:23:29 PM]

http://www.zog.net/Docs/serverdown.html

for errors, or
run a debug command on the device. Network hardware does occasionally
break, as do network
interface cards (NIC) on servers and clients.

VI.

Compendium

If you can't figure out what's wrong at this point, you'll probably need help
(helpdesk, USETNET
newsgroup, etc.) Remember that the more information you supply from system debug
commands, the
easier it'll be to help you find out where your problem lies.
(c) 2001 John Morgan Salomon

http://www.zog.net/Docs/serverdown.html (14 of 14) [4/11/2008 3:23:29 PM]