Webscarab – An Overview
Nowadays, because of increasing internet usage, most companies around the world use web applications which have all the information about the company and its activities. Usually, web applications have access to a database which contains all this sensitive information. If an attacker exploits the web application, they can get to all the data, so securing web applications has become a critical issue that all companies have to handle.
There are different approaches to secure web applications. One of them is manual penetration testing. During these tests, the pen-tester needs to intercept web browser web requests and web server replies. Our article today is about a web security application testing tool which can aid a pen-tester in intercepting HTTP traffic between a browser and a web server and also provide other functionalities which will semi-automate the pen-tester’s tasks.
WebScarab is a framework for analyzing applications that communicate using the HTTP and HTTPS protocols. It is written in Java, and is thus portable to many platforms. It has several modes of operation, implemented by a number of plugins. In its most common usage, WebScarab operates as an intercepting proxy, allowing the operator to review and modify requests created by the browser before they are sent to the server, and to review and modify responses returned from the server before they are received by the browser. WebScarab is able to intercept both HTTP and HTTPS communications. The operator can also review the conversations (requests and responses) that have passed through WebScarab.
WebScarab provides some great features and plugins that will aid the pen-tester to get a solid understanding about a web application’s behavior. Here are the plugins along with their short descriptions according to WebScarab’s official website:
- Fragments – extracts Scripts and HTML comments from HTML pages as they are seen via the proxy or other plugins.
- Proxy – observes traffic between the browser and the web server. The WebScarab proxy is able to observe both HTTP and encrypted HTTPS traffic by negotiating an SSL connection between WebScarab and the browser, instead of simply connecting the browser to the server and allowing an encrypted stream to pass through it. Various proxy plugins have also been developed to allow the operator to control the requests and responses that pass through the proxy.
- Manual intercept – allows the user to modify HTTP and HTTPS requests and responses on the fly, before they reach the server or browser.
- Beanshell – allows for the execution of arbitrarily complex operations on requests and responses. Anything that can be expressed in Java can be executed.
- Reveal hidden fields – changes all hidden fields found in HTML pages to text fields, making them visible and editable. Sometimes it is easier to modify a hidden field in the page itself, rather than intercepting the request after it has been sent.
- Bandwidth simulator – allows the user to emulate a slower network in order to observe how their website would perform when accessed over, say, a modem.
- Spider – identifies new URLs on the target site, and fetches them on command.
- Manual request – allows editing and replay of previous requests, or creation of entirely new requests.
- SessionID analysis – collects and analyzes a number of cookies to visually determine the degree of randomness and unpredictability. Note that this analysis is rather trivial, and does not do any serious checks, such as FIPS, etc.
- Scripted – operators can use BeanShell (or any other BSF supported language found on the classpath) to write a script to create requests and fetch them from the server. The script can then perform some analysis on the responses, with all the power of the WebScarab Request and Response object model to simplify things.
- Parameter fuzzer – performs automated substitution of parameter values that are likely to expose incomplete parameter validation, leading to vulnerabilities like Cross Site Scripting (XSS) and SQL Injection.
- Search – allows the user to craft arbitrary BeanShell expressions to identify conversations that should be shown in the list.
- Compare – calculates the edit distance between the response bodies of the conversations observed and a selected baseline conversation. The edit distance is “the number of edits required to transform one document into another.” For performance reasons, edits are calculated using word tokens, rather than byte by byte.
- SOAP – parses WSDL and presents the various functions and the required parameters, allowing them to be edited before being sent to the server. NOTE: This plugin is deprecated, and may be removed in the future. SOAPUI is way beyond anything that Webscarab can do, or will ever do, and is also a free tool.
- Extensions – automates checks for files that were mistakenly left in the web server’s root directory (e.g. .bak, ~, etc). Checks are performed for both files and directories (e.g. /app/login.jsp will be checked for /app/login.jsp.bak, /app/login.jsp~, /app.zip, /app.tar.gz, etc). Extensions for files and directories can be edited by the user.
- XSS/CRLF – a passive analysis plugin that searches for user-controlled data in HTTP response headers and body to identify potential CRLF injection (HTTP response splitting) and reflected cross-site scripting (XSS) vulnerabilities.
The first thing we’ll need to do is donwload Webscarab. You can do so here, and its github repository here. After downloading WebScarab, it’s time to start it up and go through some of its plugins to see how we can use them.
An HTTP proxy is very important for the pen-tester when testing web applications. A proxy will record all HTTP transactions and will produce a history of pages you have visited.
As you can see from the previous screenshot, inside the Proxy tab are four sub-tabs: Listeners, Manual Edit, Bean Shell and Miscellaneous.
Listeners Tab is used to set which TCP port Webscarab will listen to. Webscarab listens by default on port 8008 but we can change it to any port we want.
Manual Edit Tab is used for intercepting HTTP requests and responses. The main reason why pen-testers intercept HTTP requests and responses is to learn more about what the web application is doing certain inputs. Intercepting an HTTP request and response will allow us to see and modify them while they’re being sent back and forth. As you can see, we can include or exclude any type of extensions and can determine which type of response we can intercept.
Bean Shell Tab allows for the execution of arbitrarily complex operations on requests and responses. Anything that can be expressed in Java can be executed.
Miscellaneous Tab lets you select some options like show hidden fields in HTML pages, etc.
As you can see from the following screenshot, this is a POST request where you can edit any HTTP request headers, add any custom header, or change and edit any parameters of the request, like userid or pwd, then send the request by clicking on the Accept changes button.
This tab is a summary of your history while surfing with the web application.
Here you can see the tree of URL’s and a summary of your history while surfing the web application. This tree represents the site layout.
To see more details about any conversation, double click on that particular row in the table, and a new window will show up which has all the request and response details for this conversation.
You can see the request and response in two forms according to content type: Parsed view and Raw view. You can parse it in many ways in parsed view, like HTML, XML, Text and Hex.
SessionID Analysis Tab
For people who don’t know what session ID is, here is the Wikipedia definition:
“A session ID is typically granted to a visitor on his first visit to a site. It is different from a user ID in that sessions are typically short-lived (they expire after a preset time of inactivity which may be minutes or hours) and may become invalid after a certain goal has been met (for example, once the buyer has finalized his order, he cannot use the same session ID to add more items).
As session IDs are often used to identify a user that has logged into a website, they can be used by an attacker to hijack the session and obtain potential privileges. A session ID is often a long, randomly generated string to decrease the probability of obtaining a valid one by means of a brute-force search. Many servers perform additional verification of the client, in case the attacker has obtained the session ID. Locking a session ID to the client’s IP address is a simple and effective measure as long as the attacker cannot connect to the server from the same address.”
Webscarab analyzes the session ID to these if it’s easy or hard to guess.
The SessionID Analysis tab has three sub-tabs :Collection, Analysis and Visualisation.
Collection Tab: As you can see, there is a drop down list labeled Previous Requests. Here you can pick a conversation from your history of transactions within the web application to analyze.
We should choose the conversation in which the web application saved cookies in your browser. Usually this is is a login page but to make sure, you can choose the conversation and click test. A window should pop up showing the cookie that the web application set to your browser.
After you’ve selected your conversation, click on the Fetch button to start collecting session IDs from the web server. The default number of session IDs collected is set at 100, but the more samples you select the more accurate the analysis will be.
Analysis Tab: After clicking the Analysis tab, you will find a drop down list called Session Identifier which lets you choose which cookie you want to analyze.
Visualisation Tab: This tab creates a graph using the result of the Analysis tab. This will give you a quick visual representation of how weak or strong each session IDs is. As we can see, the session IDs created for phpmyadmin are random or at the very least hard to guess.
This concludes Part 2 of the Webscarab tutorial. If you have something to add or you see an error, please leave a comment or contact me. Part 3 is going to focus on WebScarab’s fuzzing functionality. Until then, try the session ID functionality on a web application that you use often and see how weak or strong their session ID management is.
For people who don’t know what fuzzing means, here is the Wikipedia definition:
“Fuzz testing or fuzzing is a software testing technique, often automated or semi-automated, that involves providing invalid, unexpected, or random data to the inputs of a computer program. The program is then monitored for exceptions such as crashes, or failing built-in code assertions or for finding potential memory leaks. Fuzzing is commonly used to test for security problems in software or computer systems.”
To fuzz a parameter in a specific conversation, we should use this particular conversation as a fuzz template by right clicking on it then choosing “Use As Fuzz Template” before going to the Fuzzer tab.
You can see from the following screenshot that your request is now used as a template.
Now we create a category to use as a fuzzer string. Click on Sources, write its description, and select the file that contains the fuzzer string. Then click Add.
After that, select the fuzz source and click Start.
WebScarab is a great tool which will help pen-testers during web application penetration testing and assessment.