This article will introduce you to User Agent, what is it used for and from the aspect of Security, to know what are the possible attacks.
What is a Header?
HTTP header fields are components of the message header of requests and responses in the Hypertext Transfer Protocol (HTTP). They define the operating parameters of an HTTP transaction..
Example: Request HTTP
GET / HTTP/1.1 Connection: Keep-Alive Keep-Alive: 300 Accept:*/* Host: www.google.com Accept-Language: en-us Accept-Encoding: gzip, deflate User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:126.96.36.199) Gecko/20110319 Firefox/3.6.16 ( .NET CLR 3.5.30729; .NET4.0E)
Now, a user agent is a client application implementing a network protocol used in communications within a client–server distributed computing system. (Wikipedia)
For example the user agent is used by different Web browsers, to show the Web pages to the users according to the different proposed scenarios. The idea is provide content and operating parameters if it is a desktop computer, smartphone or whatever.
Sometimes the User Agent of a spider or crawlers includes the URL or e-mail address of the organization to contact them.
If you want to limit the access from some User Agent, you should use the Basic Exclusion with the robots.txt (http://www.robotstxt.org)
Example: User-agent: * Disallow: /cgi-bin/ Disallow: /tmp/
- Robots can ignore your /robots.txt. Especially malware robots that scan the web for security vulnerabilities, and email address harvesters used by spammers will not pay attention.
- The /robots.txt file is a publicly available file. Anyone can see what sections of your server you do not want robots to use.
Basically, you should not try to use/robots.txt to hide information.
User Agent explained
Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:188.8.131.52) Gecko/20110319 Firefox/3.6.16 (.NET CLR 3.5.30729; .NET4.0E)
|Mozilla||MozillaProductSlice. Claims to be a Mozilla based user agent, which is only true for Gecko browsers like Firefox and Netscape. For all other user agents it means 'Mozilla-compatible'. In modern browsers, this is only used for historical reasons. It has no real meaning anymore|
|Windows NT 5.1||Operating System: Windows XP|
Language Tag, indicates the language for which the client had been localized (eg menus and buttons in the user interface)
en-US= English – United States
CVS Branch Tag
The version of Gecko being used in the browser
|Gecko||Gecko engine inside|
|20110319||Build Date: the date the browser was built|
|.NET CLR 3.5.30729||
|.NET4.0E||.NET framework Version 4.0 Extended|
This section is the purpose of the article, describe the possible vulnerabilities known and in some cases still have no solution.
Security bypass: Is the lack or poor security validation. It often depends on the implementation used and how this can be avoided.
SQL Injection: Is a technique used to exploit applications that construct SQL statements from user-supplied input. When successful, the attacker is able to change the logic of SQL statements executed against the database.
Denial of Service: Is a technique with the intent of preventing a web site from serving normal user activity. DoS attacks, which are easily normally applied to the network layer, are also possible at the application layer. These malicious attacks can succeed by starving a system of critical resources, vulnerability exploit, or abuse of functionality.
Script Injection: Is the exploitation of a computer bug that is caused by processing invalid data. Code injection can be used by an attacker to introduce (or "inject") code into a computer program to change the course of execution.
Cross-site Scripting (XSS): Cross-site Scripting (XSS) is an attack technique that involves echoing attacker-supplied code into a user's browser instance. A browser instance can be a standard web browser client, or a browser object embedded in a software product such as the browser within WinAmp, an RSS reader, or an email client.
Session Hijacking: The attack consists of the exploitation of the web session control mechanism, which is normally managed for a session token. The Session Hijacking attack compromises the session token by stealing or predicting a valid session token to gain unauthorized access to the Web Server.
How to prevent these vulnerabilities?
You should read guidelines, best practices related to security, code programming and application testing.
An excellent guide that every person should read is OWASP Top Ten, “The OWASP Top Ten provides a powerful awareness document for web application security.”
As per OWASP the Top 10 Web Application Vulnerabilities are:-
• A1: Injection
• A2: Cross-Site Scripting (XSS)
• A3:Broken Authentication and Session Management
• A4: Insecure Direct Object References
• A5: Cross-Site Request Forgery (CSRF)
• A6: Security Misconfiguration
• A7: Insecure Cryptographic Storage
• A8: Failure to Restrict URL Access
• A9: Insufficient Transport Layer Protection
• A10: Unvalidated Redirects and Forwards
The W3C (World Wide Web Consortium) have proposed a good practice to implement web sites that are easy and simple for any user. The User Agent Accessibility Guidelines (UAAG) documents explain how to make user agents accessible to people with disabilities, particularly to increase accessibility to Web content. UAAG is primarily for developers of Web browsers, media players, assistive technologies, and other user agents.
More information: http://www.w3.org/WAI/intro/uaag.php
There are number of tools available to perform the attacks, this is only a recommendation:
Fiddler Fiddler is a Web Debugging Proxy which logs all HTTP(S) traffic between your computer and the Internet. Fiddler allows you to inspect all HTTP(S) traffic, set breakpoints, and "fiddle" with incoming or outgoing data.
Havij Havij is an automated SQL Injection tool that helps penetration testers to find and exploit SQL Injection vulnerabilities on a web page.
SQL Power Inyector SQL Power Injector is an application created in .Net 1.1 that helps the penetration tester to find and exploit SQL injections on a web page.
SQID SQL injection digger is a command line program that looks for SQL injections and common errors in web sites. You can specify the user agent, the referer, supports HTTPS, Proxy with authentication and more.
Tamper Data Firefox addon: Use tamperdata to view and modify HTTP/HTTPS headers and post parameters.
User Agent Switcher Firefox addon: The User Agent Switcher extension adds a menu and a toolbar button to switch the user agent of a browser.
Hypertext Transfer Protocol — HTTP/1.1 (RFC 2616)
What's My User Agent?
List of User-Agents
Bots vs Browsers