Website Footprinting against DIGG.COM

Andrew Robinson -1103429
CE0973A
Issues in Network Security
Website Footprinting against DIGG.COM

What is Footprinting?
Web footprinting is the process of gathering as much information as possible about a target (website), this includes information such as all sub domains, active machines, applications and operating systems in use. This information is used to prepare for an attack on the target, this document will cover the tools and methodologies used to gather this information based on a live target.
Target Scoping
Scoping a target is the process of gathering the requirements and limitations to build a test plan. Ultimately ethical hacking vulnerability and penetration testing is carried out with the acceptance and permissions of the target. However a malicious attacker will also use this process depending on what they want to achieve, for example defacing a website, they may need certain information such as versions of web server software. This stage will also outline what types of methods will be used, for example will social engineering be involved or not. In this example limitations exist by law, university policies and there is no contact with the client (digg.com). Information will be collected within these restrictions not aimed at a particular attack.
Passive Open Source Information Gathering

Passive information gathering is the process of using publically accessible resources without direct interaction with the target itself this is also known as open source footprinting. WHOIS with Online Domain Tools Typically the first step is using a domain name tool, popular ones include http://www.domaintools.com, http://www.netcraft.com, http://robtex.com and http://www.serversniff.net. These all generally offer many tools and options for information, serversniff.net is referred to as the Swiss Army knife of networking tools. The table below contains the general information obtained from a basic domain scan or a WHOIS against digg.com. Digg.com Registered through Registrant Registrar history Name server Name server organisation Creation date IP history Current IP
Figure 1
GoDaddy.com, LLC Domains By Proxy, LLC 5 Dns11(14).cotdns.net Cotendo, Inc. 20-feb-2000 10 changes, 7 unique IP address over 8 years 64.191.203.30
CE0973A
I can also tell that theyre using a service provided by Citrix Systems called NetScaler. This systems aims to offer multiple benefits to the customer and user. A couple interesting points are, NetScaler offers both an integrated application firewall and L4-7 load balancing. These are partly designed to help combat Denial of Service attacks by filtering traffic destined for the web application. Domains by proxy essentially registers the proxy on your behalf, so this is why the registrants name shows this, opting in for a private domain usually just hides addresses and phone numbers not your name. The original registrars name isnt particularly relevant to this situation, however I was able to find it was Robert Kevin Rose. The original registered date was 1998, although that was for a different company (Digg Records), 2004 is when R K Rose took the name. I used websites such as www.archive.org, www.wikipedia.org, Google and even YouTube to find this information. Sub Domains Using Name Servers Knowing the sub domains for a website can shine a light on more vulnerable areas that the target doesnt normally wish to be completely public. With DNS its possible to connect to the name server and request a zone transfer. Its also possible to use a brute-force technique using applications such as DNSMAP to retrieve sub domains. Although it is also possible to obtain this information using Google, plus the Google method is the only passive method. To automate this process there is a Python script gxfr.py available at http://ptscripts.googlecode.com/svn/trunk/gxfr.py, however Digg.com is not a great place to run this, as the website contains usernames, these are accessible as www.digg.com/username which means this will attempt to return tens of thousands of addresses. Manually the Google command would be site:digg.com, from this I gained jobs, developers and about. Documents with Metagoofil Documents contained within a website can contain useful information, the Python application metagoofil uses Google to obtain this information and displays the meta data related to the documents. As it uses Google it is passive, youre not actually the one connecting to the website. Again manually this is using Google to do site:digg.com filetype:pdf,doc etc, it downloads the results locally and extracts the meta data of interesting information. It can also take the MAC addresses from .doc files. However I was unsuccessful using the tool or manually, results merely linked to news stories. However I did try this technique against the University of Abertay and found documents aimed at the technical support team outlining all the names of used servers, so it can be very useful. Passive Operating System Fingerprinting This is a method of obtaining information about the target operating system, if a passive method works it is a better option. Organisations can detect active methods and will generally follow up as to why they have happened, if they arent happy legal action can be taken. Active fingerprinting works by examining the TCP/IP responses, each operating system handles TCP/IP communications differently, thus information can be gained from this without raising
CE0973A
suspicions. The basic principle is listen on local port, access resource such as the website and view the information. A popular tool for many Man In The Middle (MITM) attacks is ettercap, this also have the function of passive fingerprinting, so this is what I shall use. Its as simple as stating the local network adapter you want to listen on, start sniffing and view profiles. Whenever the system connects to a server, such as a web server it will create a new profile where the information can be viewed. I started off by opening a browser and viewing digg.com, here are my results in figure 2:

Figure 2
As this shows, the web server is running Apache which we knew already and it reports FreeBSD 4.5. However I believe this is likely incorrect as most of the places I tested show this, if its unable to confirm the OS it generally spits out FreeBSD 4.5 although I am not 100% sure. So overall this technique is nice and quiet, I havent done anything but connect to the web server and request the home page, now this may seem like its not passive however it generally is considered passive as the server believes its merely responding to a web page request. Although I dont believe this to be overly successful. Database As this website contains users its going to need a database. From using various sources such as Google and Wikipedia I was able to find Digg.com has moved away from MySQL to a distributed database system called Cassandra powered by Apache initially created for Facebook. I found this by simply searching for Kevin Rose digg.com database update. Many articles came back explaining why they made the move. Further non passive methods can be used to find exact version details.
CE0973A
An add on for Firefox can be used called Hackbar to try and get the website to spit out valuable information via errors.
Active Information Gathering

Active information gathering involves connecting to the servers and systems directly. This potentially leaves a trace and can break the laws of the Computer Misuse Act. DNS As mentioned using Google is the only true passive method, active methods do include brute-force. However a more common way is to attempt a transfer of zones. Within a Unix like operating system most have the Domain Information Groper (DIG) application, this can achieve the same as the WHOIS as far as returning the name server, this information can then be used with dig and the opcode mnemonic AXFR to attempt a transfer of zones from the name server, this would then allow you to see all subdomains associated with the main domain; Linux Security Tools Distributions (STD) such as Blackbuntu and BackTrack also contain DNSWalk which automates this process. However many administrators will disable transfer of zones to non- authenticated servers as this as it can be a security threat. My attempt was unsuccessful due to security settings as shown in the below output:

Figure 3
DNSenum is another application that can combine the two approaches using the above method as well as passive. I was able to obtain the Google based mail server records using dnsenum.
CE0973A

Figure4
As both the passive method and the active methods for obtaining DNS information failed, the next step would be to brute force the records. This can be achieved using dnsenum usinf the f switch and as mentioned earlier dnsmap is another option for brute forcing. However I wont be doing this due to ethical and legal reasons. Dmitry is a tool thats able to do all of the above, that includes: Whois based on IP Host information from http://www.netcraft.com Subdomains E-mail address of the target Port scan
As port scans are not permitted through the university this tool wasnt an option. LEGAL Currently the act of requesting a public DNS zone is not illegal, if the server is not configured correctly and they intend for this to be blocked its their negligence. Nonetheless in North Dakota a judge ruled it as an illegal activity, he also claimed that using WHOIS is illegal. So generally this isnt something that is or will be considered illegal but there are cases out there. (http://www.circleid.com/posts/811611_david_ritz_court_spam/ - ND Judge gets it wrong) Port Scanning Port scanning allows me to target the server and find what services are active by which ports are open. This is useful information as it shows standard ports, for example port 80 is the standard for HTTP, depending on the server type there might be vulnerabilities known that are exploitable. Port scanning in the UK is not illegal, however it may be against the terms and conditions of an ISP and the target may raise concerns and take legal action if detected and is seen as an aggressive action. Regardless of which method is used it isnt allowed within the university, or at least external scans arent. However if I wasnt restricted by these rules I could carry out a ping sweep to find which IPs are active from the ones I found. From this I can scan the individual systems for all active ports or a port sweep for a specific port, such as 23 for Telnet if I wanted to exploit a Telnet vulnerability. So this is not an option for me however I have included an example scan output of my own remote server as
CE0973A
an example of the output. The tool used is nmap, nmap is also a useful tool for active operating system fingerprinting. A basic scan returned this:

Figure 5.1
This indicates that my security measures on the server prevented the scan from working and are dropping the traffic. I know there are no restrictions on IP access, however this can be a tactic and nmap will allow you to spoof an IP. Also a very common method for Intrusion Detection Systems and firewalls are to detect this behaviour as a signature so another tactic is to changed how the scan happens. One of the more effective but slow methods it to drip out a scan, this may be carried out over a period of 12 hours or more. Rather than scan the first 1000 ports as quick as possible it will happen slowly over time, often this will return some results as it wont match the signature behaviour. Its also important to note that filtered doesnt mean closed or open. However as previously mentioned nmap is also a tool that can be used for Operating System fingerprinting, using this option I was able to gain some results on the port, although the target OS was incorrect it did report the switch and router IOS versions.
CE0973A

Figure 5.2
However this method is very noisy and leaves traces. Ping and Trace Route Tracing the route to the web server can give information on the topology and potentially lead the attacker to weaker more vulnerable areas allowing them to penetrate their target from another angle. In Windows the command is tracert, in Unix like and Unix systems it is traceroute. This will display the route a packet takes to reach the destination, the later stages can be investigated to show whom they belong to. Below is the output for www.digg.com:
CE0973A

Figure 6
From my previous information gathering, I do know Digg Inc. is based out of San Francisco, California. As can be seen from the output it does eventually land in San Jose in California. Not only does this give me IP addresses but also naming conventions and information for the Fully Qualified Domain Name (FQDN). BANDCON happens to be an ISP based in San Jose, alpha appears to be a server that my IP is possibly directed to, there may be others such as bravo. Serversniff also has these tools so theres no need to actually run them from your local machine providing a more passive method.
The Website
The website itself if obviously a place that can contain information and it is the next step. Banner Grabbing Again using serversniff.net Im able to view the headers, this returns information such as PHP version which can be vulnerable to attacks. Digg.com is running PHP 5.2.9. You are also able to use tools such as Netcat or Telnet to connect to port 80 and send a bad request, in a Telnet session this can be as simple as typing random characters and looking at the information sent back in the header. Most web browsers allow you to view the page source, this can show directories the original developer didnt intend for the public to see, it can also reveal JavaScript code which may contain validation logic and even comments that were not intended to be left in. For example it may be a very simple way of authenticating a user and can contain the password. However Digg.com uses cdn1.diggstatic.com to hold all images and scripts. The permissions have been correctly set meaning I can view the content but I cannot get a directory listing.
CE0973A
Robots.txt This check calls a webservers robots.txt. Site-Owners use this file for telling search engine-spiders where to search. - www.serversniff.net /httprobots.php The robots.txt file for Digg.com is included within the appendix. This essentially blocks the search engine from certain directories, however it does mean usually the user can manually visit these locations. It also provides the user with a true sitemap.
Other Applications
There are many applications associated with information gathering and these usually come pre- loaded with STDs such as Knoppix, Blackbuntu and Back|Track. One of these includes the open source (passive) tool Maltego, however Maltego is essentially an application based version of Domaintools.com. This is because the free version is limited and since version three you are required to register before even using it. WebFerret is also a tools often used to return specific meta data and content from multiple search engines. Dradis can be used to consolidate all of the information found during each stage and method.

Figure 7
CE0973A
[PAGE INTENTIONALLY LEFT BLANK]
10
CE0973A
SQL Injection
Disclaimer, any examples used within this document were carried out on a local website I am creating for another module, the website therefore does share images and its name with a live website, however it is not. [viga.co.uk]
Structured Query Language

SQL sometimes referred to as Structured Query Language is a programming language designed for managing data in relational database management systems (RDBMS). Wikipedia Its important to have a basic grasp of why SQL exists and what it does to truly understand SQL injection and why an attacker might want to do this. As the full name suggests SQL is about creating queries, usually to query a database to return data. This could be used in an online store to display all items of a particular category, or as an authentication system. A user enters their username and password, the database is then queried to see if these match, if they do the user is authenticated and granted access. A basic select query may look something like: SELECT password FROM customers WHERE email='$emailaddress' In the above example a login form has asked for an email address and password. This query tries to find a match for the email address within the customers table, if it matches it will retrieve the password from the table. These will then be compared to what the user entered, if they match the user is authenticated, if they dont the user is returned some form of error message.
Advanced SQL
Advanced SQL techniques exist to allow developers to develop more efficient code, this includes the likes of intersect, except and union. I will briefly cover union here as its part of a very common attack method. Union essentially allows two queries to be put together and works much like or and combines the output. If we extend the above example to: SELECT password FROM customers WHERE email='$emailaddress' UNION SELECT password FROM oldCustomers WHERE email='$emailaddress' Were now looking in two different tables, this can be any two queries and providing one returns a true value, the application will continue. A More Simple Approach Of course, its also possible to have multiple conditions within a query, such as: SELECT password FROM customers WHERE email='$emailaddress' ' or 'a' = 'a'
11
CE0973A
Its also possible to stack queries separating them with a semicolon. Often this is disabled for security reasons and isnt supported by all application types. For example MySQL with PHP is disabled, however MySQL with ASP.NET is enabled. Connecting to a Database as a User Depending on the scripting language being used there are slightly different methods. In PHP the user will use a username and password that has permissions on the SQL server. For example a user must access the database to execute the queries on the server; these credentials are often in a separate PHP file thats included on each page that requires connecting to the database. Best practice security states multiple users should be created with varying permissions known as roles, if SELECT is needed and nothing more, a user should be limited to this and that user should be used to execute the query. However many applications use one user who is often the owner of the database, meaning they have full control.
What is SQL Injection?

SQL injection is a form of interpreter injection that involves the process of manipulating queries on a website that constructs SQL statements from user input. Typically this is used to read, modify and delete database data. Often an attacker will look to dump an entire database. Its also possible to launch system-level commands to perform a Denial of Service (DoS) attacks on the server. Remote code execution is possible using tools such as Meterpreter part of the Metasploit framework to provide a shell. The Open Web Application Security Project (OWASP) states injection as the top security risk for web applications in 2010. - https://www.owasp.org/index.php/Category:OWASP_Top_Ten_Project
Types of SQL Injection

There is a slight grey area regarding the types and name thereof, however the following classes best contain the known types of attacks. Classes In-band data is extracted using the same channel that is used to inject the SQL code. This is the most basic and straightforward kind of attack, retrieved data is presented directly within the application web page. This usually takes the form of error messages displayed within the website and includes error and union based attacks. Out-of-band data is retrieved using a different channel, for example an email with the results of the query are generated and sent to the attacker. This can also be delivered via HTTP or even DNS, the main aspect though is it out of the website. Inferential there is no actual transfer of data but the attacker is able to reconstruct the information by sending particular requests and observing the behaviour of the website and or database server.
12
CE0973A
This means you dont actually receive an error message, some servers are configured not to display database errors and even use their own custom 404 error page. This class contains the blind form of attack. Methods Again descriptors of these types differ from person to person, for example error and union can be combined into verbose mode, meaning you receive an output. Error based o Asking the database a question that will cause an error and gleaning information from the error. UNION o The SQL UNION is used to combine results of two or more SELECT SQL statements into a single result. Blind o Asking the database a true or false question and using the response as an indication, for example did it return a valid page? A time delay can also be used.
Tools for Identifying Vulnerable Sites and Exploitation

Typically these tools identify that a vulnerability may exist but arent used to perform the attack. Tool / Script Mieliekoek.pl Wpoison Sqlmap Wapiti W3af paros sqid Attack Method Identified Error based Error based Default -> blind, can use UNION Error based Error and blind Error and blind Error
Therefore if these tools are utilised none of them identify all types, so a mixture of tools should be used. There are closed source applications that cost money and require registration. Many of these tools can also automate the injection process. Mozilla Firefox also supports an add on called Hackbar making it a lot easier for SQL injection. Here is an example of me creating a UNION select statement for 10 columns this shows some of the options Hackbar supports, the SQL option supports MSSQL, MySQL and Oracle.
13
CE0973A

Figure 1
Manually Testing for Injection

As previously mentioned the goal of SQL injection is to modify an SQL statement to force a true response. The first method used to check if a site is vulnerable is to find an input such as the image below in figure 2:

Figure 2
The process of this form sends to login_action.php, where the SQL statement finds a match and compares the inputs entered. Here an attacker will use a variety of SQL snippets to eliminate the need for a password. Rather than the query saying match the email and password, it will become match the email and password or CONDITON. Popular methods include: ' or 1=1-- " or 1=1-- or 1=1-- ' or 'a'='a " or "a"="a
14
CE0973A
') or ('a'='a
Depending on the syntax of the SQL statement will depend on what is required, however if none of these work the site could still be vulnerable, but often attackers will move on unless theyre specifically targeting the website. POST and GET A typical HTML form will post the information to an action page, for example figure 2 process described above. Get is used on the action page to retrieve variables from the URL as opposed to the POST method. Injection attacks work differently depending on whether the site is using POST or GET. With GET attackers will use manual tools such as Hackbar and modify the URL directly to achieve the same results as displayed earlier in figure 1.
Common Tactics
Nature of Vulnerabilities If an application or software version, such as forum software is identified as vulnerable this doesnt change. These will generally be patched but it is then up to the system administrator to apply this update and ensure they are no longer susceptible to this vulnerability, if the patch isnt applied, its still exploitable. Website such as milw0rm, shell storm and the exploit database from Offensive Security allow attackers to search for vulnerabilities based on the application. For example I may search for Bulletin Board System forums and find version 2.3 is vulnerable to a particular attack due to a code error. However while information such as server type and application revisions are useful and are used to find further information, SQL injection doesnt rely on these vulnerabilities existing, this is often an attack when there are a lack of vulnerabilities. The server administrator may have active Intrusion Detection Systems (IDS), firewalls and all ports locked down apart from 80 and 443. This is when SQL injection potentially comes into its own. SQL injection can take advantage of a weak application or as stated blind attacks. Patching a server is not a solution to this potential attack as its an inherent weakness within the application, unless the error is found and recoded it can be difficult to close this particular hole. Google Dorks Google dorks are search terms used to find particular pages, sites and files that may be vulnerable to attack. Allintext:D.N.I filetype:xls The above is an example of a Google dork that may find files that contain passwords and other sensitive information. This again can be searched randomly and the fly by attacks may choose the site. However they can be used within SQL injection for example the previously mentioned tool
15
CE0973A
sqlmap.py allows the attacker to use Google dorks to find target pages linked to a domain rather than one specific page. SQL attacks may use Google dorks to find pages that are likely to allow SQL injection. For example: http://studentplus.abertay.ac.uk/1103429/index.asp?id=1
Putting This Altogether

This is based on the assumption the penetration tester has a specific website or the malicious attacker has chosen a target not randomly searching Google for potentially vulnerable targets. The attacker must first identify the injection, whether this be with a tool or manually. The attacker then needs to know if theyre dealing with a string or integer, if an integer the single tick ` is required, for example, the URL may indicate .php?id=1 or .php?id=a, 1 is obviously an integer and a represents a string. Now the attacker must find what methods they can use, error based is the easiest so its sensible to try and force an error. If unsuccessful, UNION based statements are the next choice, if all fails blind injection is the last resort. Blind injection is usually a last resort and can take days to dump a database. Information such as username, database name, server name and version will be collected with the ultimate aim of gaining ownership rights and dumping the database.
Countermeasures
Client Side JavaScript This involves checking inputs on the client side, most average users do not disable JS and it does provide validation. However its so easy to block JS on your web browser its pointless spending any time developing a system using JS. Most attackers will disable scripts, so dont rely on JS.
Disable Errors
Database errors should be disabled as they provide the easiest method and too much information. Custom 404 error pages can also be implemented to prevent server details from being leaked as easily. However both of these methods arent enough on their own. Validating Input and Values With a server youre often told, patches, patches and patches. Always keep applications up-to-date to prevent legacy vulnerabilities. With web application utilising SQL, validation, validation and validation applies. This isnt just a catchy way to remember to validate, but the three iterations represent input from users, parameters from URLs and values from cookies. Depending on the scripting language in use, they can support their own methods. For example a common solution within ColdFusion is the use of URLEncodedFormat(), this will encode the URL. The problem with this however is it isnt hidden, attackers are able to simple use the encoded format within their attacks, for example %20 is a space character.
16
CE0973A
PHP includes an escape function, normally when coding in PHP if you want something to be ignored by PHP and displayed as is, you escape the characters using a backslash /. The addslashes() function does exactly that, it will escape special characters, a custom version is also available, addcslashes() allowing the user to specify which characters are to be escaped. However again this does have limitations, for example or 1=1 can be displayed as or 1 like 1, this is saying is 1 like 1, well it is so the injection can still work. Do you display allow the string like? Its not a great solution. With that being said its important to validate all inputs, this can enforce security policies such as password length, but also helps prevent injection. Regular expressions and pattern matching are a common method used to detect unanticipated input. SQL Statement Construction If you construct your SQL statements using the variable directly, youre open to injection. For example: "SELECT email, password FROM customers WHERE email=' $_POST['emailaddress']'" This is taking the raw variable, POST indicates the web application has used the POST method to take an input from a user and send to this page, where it is then used in an SQL statement. Obviously this means anything that is the variable i.e. anything the user entered for their email address will now be part of the SQL statement. A much better approach is to take the input in, assign it to a new variable: $emailaddress = $_POST['emailaddress']; Validate $emailaddress, and then use $emailaddress within the query. So previously mentioned methods such as addslashes() can work on the variable before it becomes part of the SQL statement.
Intrusion Detection / Prevention Systems

IDPSs can be used to detect and kill attacks. The do require direct access to the traffic as HTTPS can prevent them from functioning. The most common type is signature based, meaning the system will recognise attempts as being malicious. A common attack is to use xp_cmdshell, if this is detected it will be prevented. General SQL injection prevention works using keywords in HTTP requests, if they pop up the user will be blocked as suspected SQL injection. Web application firewalls can also filter traffic destined for the particular site.
Conclusion
If an attacker is determined and knows what theyre doing, SQL injection is a very real threat and attackers are learning more and more as other security areas are increased. Penetration testers and malicious hackers need to find other ways into systems, and SQL injection is only becoming more common.
17
CE0973A
Each of the mentioned steps should be considered when deploying a new system and security should be a concern from the beginning, especially within the application design.
References
Footprinting
http://www.pauldotcom.com http://www.irongeek.com http://www.offensive-security.com http://www.paterva.com/web5/ http://training.thehackernews.com/Certified-Cyber-Security-Ninja.html http://searchsecuritychannel.techtarget.com/feature/Ethical-hacking-tools-and-techniques- Information-gathering http://www.ehacking.net/2011/03/footprinting-information-gathering.html http://www.informit.com/articles/article.aspx?p=472323&seqNum=5 http://www.digg.com http://wikipedia.org http://www.circleid.com/posts/811611_david_ritz_court_spam/
SQL injection
https://www.owasp.org/index.php/Category:OWASP_Top_Ten_Project http://www.cisco.com/web/about/security/intelligence/sql_injection.html http://www.securiteam.com/securityreviews/5DP0N1P76E.html http://www.defcon.org http://en.wikipedia.org/wiki/Sql http://en.wikipedia.org/wiki/Sql_injection http://www-01.ibm.com/support/docview.wss?uid=swg21283303 http://st-curriculum.oracle.com/tutorial/SQLInjection/index.htm
18
CE0973A
Appendix
Robots.txt file for www.digg.com: User-agent: Mediapartners-Google Disallow: User-agent: AdsBot-Google Disallow: User-agent: * Disallow: /ad/* Disallow: /ajax/* Disallow: /error/* Disallow: /onboard/* Disallow: /saved Disallow: /settings Disallow: /settings/* Disallow: /news/*/v/* Disallow: /verification/* Disallow: /newsbar Disallow: /newsbar/* Disallow: /adcentric/* Disallow: /adinterax/* Disallow: /atlas/* Disallow: /doubleclick/* Disallow: /eyeblaster/* Disallow: /eyereturn/* Disallow: /eyewonder/* Disallow: /klipmart/* Disallow: /pointroll/* Disallow: /smartadserver/* Disallow: /unicast/* Disallow: /viewpoint/* Disallow: /widgetbox/* Sitemap: http://digg.com/sitemap_index.xml Sitemap: http://digg.com/sitemap_v3_index.xml
19

Website Footprinting against DIGG.COM

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Website Footprinting against DIGG.COM

Uploaded by

Copyright:

Available Formats

Andrew Robinson -1103429

Issues in Network Security