The User Agent on my Header

May 9, 2011, by | Start Discussion

This article will introduce you to User Agent, what is it used for and from the aspect of Security, to know what are the possible attacks.

What is a Header?

HTTP header fields are components of the message header of requests and responses in the Hypertext Transfer Protocol (HTTP). They define the operating parameters of an HTTP transaction..

Example: Request HTTP

GET / HTTP/1.1
Connection: Keep-Alive
Keep-Alive: 300
Accept:*/*
Host: www.google.com
Accept-Language: en-us
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;
rv:1.9.2.16) Gecko/20110319 Firefox/3.6.16 ( .NET CLR 3.5.30729; .NET4.0E)

Now, a user agent is a client application implementing a network protocol used in communications within a client–server distributed computing system. (Wikipedia)

For example the user agent is used by different Web browsers, to show the Web pages to the users according to the different proposed scenarios. The idea is provide content and operating parameters if it is a desktop computer, smartphone or whatever.

Sometimes the User Agent of a spider or crawlers includes the URL or e-mail address of the organization to contact them.
If you want to limit the access from some User Agent, you should use the Basic Exclusion with the robots.txt (http://www.robotstxt.org)

Example:

User-agent: *
Disallow: /cgi-bin/
Disallow: /tmp/

Remember this

  • Robots can ignore your /robots.txt. Especially malware robots that scan the web for security vulnerabilities, and email address harvesters used by spammers will not pay attention.
  • The /robots.txt file is a publicly available file. Anyone can see what sections of your server you do not want robots to use.

Basically, you should not try to use/robots.txt to hide information.

User Agent explained

Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.16)
Gecko/20110319 Firefox/3.6.16 (.NET CLR 3.5.30729; .NET4.0E)
Firefox 3.6.16
Mozilla MozillaProductSlice. Claims to be a Mozilla based user agent, which is only true for Gecko browsers like Firefox and Netscape. For all other user agents it means 'Mozilla-compatible'. In modern browsers, this is only used for historical reasons. It has no real meaning anymore
5.0 Mozilla Version
Windows Platform
U

Security Values:

  • N for no security
  • U for strong security
  • I for weak security
Windows NT 5.1 Operating System: Windows XP
en-US

Language Tag, indicates the language for which the client had been localized (eg menus and buttons in the user interface)

en-US= English – United States

rv:1.9.2.16

CVS Branch Tag

The version of Gecko being used in the browser

Gecko Gecko engine inside
20110319 Build Date: the date the browser was built
3.6.16 Firefox Version
.NET CLR 3.5.30729

.NET framework

Version: 3.5.30729

.NET4.0E .NET framework Version 4.0 Extended

Vulnerabilities

This section is the purpose of the article, describe the possible vulnerabilities known and in some cases still have no solution.

Security bypass: Is the lack or poor security validation. It often depends on the implementation used and how this can be avoided.

SQL Injection: Is a technique used to exploit applications that construct SQL statements from user-supplied input. When successful, the attacker is able to change the logic of SQL statements executed against the database.

Denial of Service: Is a technique with the intent of preventing a web site from serving normal user activity. DoS attacks, which are easily normally applied to the network layer, are also possible at the application layer. These malicious attacks can succeed by starving a system of critical resources, vulnerability exploit, or abuse of functionality.

Script Injection: Is the exploitation of a computer bug that is caused by processing invalid data. Code injection can be used by an attacker to introduce (or "inject") code into a computer program to change the course of execution.

Cross-site Scripting (XSS): Cross-site Scripting (XSS) is an attack technique that involves echoing attacker-supplied code into a user's browser instance. A browser instance can be a standard web browser client, or a browser object embedded in a software product such as the browser within WinAmp, an RSS reader, or an email client.

Session Hijacking: The attack consists of the exploitation of the web session control mechanism, which is normally managed for a session token. The Session Hijacking attack compromises the session token by stealing or predicting a valid session token to gain unauthorized access to the Web Server.

How to prevent these vulnerabilities?

You should read guidelines, best practices related to security, code programming and application testing.

An excellent guide that every person should read is OWASP Top Ten, “The OWASP Top Ten provides a powerful awareness document for web application security.”

As per OWASP the Top 10 Web Application Vulnerabilities are:-

•    A1: Injection
•    A2: Cross-Site Scripting (XSS)
•    A3:Broken Authentication and Session Management
•    A4: Insecure Direct Object References
•    A5: Cross-Site Request Forgery (CSRF)
•    A6: Security Misconfiguration
•    A7: Insecure Cryptographic Storage
•    A8: Failure to Restrict URL Access
•    A9: Insufficient Transport Layer Protection
•    A10: Unvalidated Redirects and Forwards
 

https://www.owasp.org/index.php/Category:OWASP_Top_Ten_Project
 

Considerations

The W3C (World Wide Web Consortium) have proposed a good practice to implement web sites that are easy and simple for any user. The User Agent Accessibility Guidelines (UAAG) documents explain how to make user agents accessible to people with disabilities, particularly to increase accessibility to Web content. UAAG is primarily for developers of Web browsers, media players, assistive technologies, and other user agents.

More information: http://www.w3.org/WAI/intro/uaag.php

Tools

There are number of tools available to perform the attacks, this is only a recommendation:

Products    Vendor
Fiddler    Fiddler is a Web Debugging Proxy which logs all HTTP(S) traffic between your computer and the Internet. Fiddler allows you to inspect all HTTP(S) traffic, set breakpoints, and "fiddle" with incoming or outgoing data.
http://www.fiddler2.com/fiddler2

Havij    Havij is an automated SQL Injection tool that helps penetration testers to find and exploit SQL Injection vulnerabilities on a web page.
http://www.itsecteam.com

SQL Power Inyector    SQL Power Injector is an application created in .Net 1.1 that helps the penetration tester to find and exploit SQL injections on a web page.
http://www.sqlpowerinjector.com

SQID    SQL injection digger is a command line program that looks for SQL injections and common errors in web sites. You can specify the user agent, the referer, supports HTTPS, Proxy with authentication and more.
http://sqid.rubyforge.org

Tamper Data    Firefox addon: Use tamperdata to view and modify HTTP/HTTPS headers and post parameters.
https://addons.mozilla.org/en-US/firefox/addon/tamper-data

User Agent Switcher    Firefox addon: The User Agent Switcher extension adds a menu and a toolbar button to switch the user agent of a browser.
https://addons.mozilla.org/en-US/firefox/addon/user-agent-switcher

Online Resources
Hypertext Transfer Protocol — HTTP/1.1 (RFC 2616)
http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14

What's My User Agent?
http://whatsmyuseragent.com

List of User-Agents
http://www.user-agents.org

UAProf
http://www.uaprof.com

Bots vs Browsers
http://www.botsvsbrowsers.com

Security Analyst working in an International Bank and participating in some Projects like Vulnerability Database, Zero Science Lab, OWASP. Fanatic of open standards.

Leave a Reply