Thursday, June 21, 2012

Enhancing AJAX Crawling capabilities of ZAProxy

Zed Attack Proxy
Zed Attack Proxy (ZAP) is a tool for penetration testing of web applications that can be easily used by both security aware  an unaware engineers .

It has several features such as a spider to discover the pages of a given site, and a set of scanners to automatically and semi-automatically discover security flaws. You can find more information about its specifications it the ZAP wiki.


I recently started working on improving its AJAX crawling capabilities by developing a plugin to integrate the OWASP Ajax Crawling Tool and Crawljax.

I am focusing my work in two different lines.
  • Building a plugin to facilitate the ajax spider integration in zaproxy.
  • Improving the results of the spiders when targeting AJAX-based sites.

To accomplish the second one, I am testing ACT and Crawljax in a security testing framework called Wivet that builds a set of links in a tricky way to difficult spiders crawling those.

When I started testing crawljax, it was not able to crawl any link of wivet due to unsupported tags issues and the spider of ZAP only a 10% of the pages.

 Since then, I improved its results up to 72% of the links so far. However, I am still working on reducing that 28% of the most hard to crawl ones.

For instance, I modified the clickTag() method of the Crawler class to support meta refresh tags as follows:
if (eventable.getElement().getTag().toLowerCase().equals("meta")) {
    Pattern p = Pattern.compile("(\\d+);\\s+URL=(.*)");
    for (Attribute e : eventable.getElement().getAttributes()) {
        Matcher m = p.matcher(e.getValue());
        if (m.find()) {
            try {
                //time*1000 to convert seconds to ms
                Thread.sleep(Integer.parseInt(m.group(1))*1000);
            } catch (Exception e) {
                LOGGER.error(e);
            }
        }
    }
}
LOGGER.info("Executing " + eventable.getEventType() + " on element: " + eventable+ "; State: "+this.getStateMachine().getCurrentState().getName());
    if (this.fireEvent(eventable)) {
        ...

I also improved in crawljax appendFrameContent() of WebDriverBackedEmbeddedBrowser and getFramesCandidates() of CandidateElementExtractor to support frame tags among other enhancements.

Below, I show a video of a test I did with the different spiders targeting  my wivet test environment.


I used a tool called Istanbul to record the video, but it looks like the time between frames is kind of high and it is not possible to change it.

6 comments:

  1. Your Post Updated and tested, it's working perfectly fine. I love this blog. Online Florist Services in Gurgaon

    ReplyDelete
  2. I advise you to lead your youtube channel. You can use this site https://soclikes.com to get more likes and views for your video

    ReplyDelete
  3. And I recommend you to create and lead instagram blog. Now is very easy to buy instagram followers for your page

    ReplyDelete
  4. Nicely written article. Thank you for this. The international tourist who wish to travel to Kenya should apply for Kenya visa application online. Applying online saves time and provide you the best services where you can check the Kenya visa documents requirement and other information very easily.

    ReplyDelete
  5. The participant can then use this credit score in the on-line on line casino. This may be very straightforward to do with the help of a short research on the Internet. Even if the PayPal logo is displayed on the provider’s web page, this doesn't imply 퍼스트카지노 that could be a|it is a} safe provider. Be suspicious and don't be lured by very high bonus offers. If these are unusually high, this is an indication of a dubious on-line gambling home. So, ought to not|you shouldn't} blindly trust every website and supply your PayPal account data there.

    ReplyDelete