Tuesday, 22 August 2017

ZAP Browser Launch

We have just released a new feature for ZAP that allows you to launch browsers from within ZAP. The browsers are automatically configured to proxy via ZAP and ignore certificate warnings, making it much easier for people to get started with ZAP as well as for more experienced users who want to use ZAP with a variety of browsers. You can install and use Browser Launch right now via the ZAP Marketplace, which can be accessed via the 'Manage Add-ons' button in ZAP:



just 'Check for Updates':

and 'Update All':

You will now get a new 'Launch Browser' option in the Quick Start tab:


To see a demo of this feature see the following video:



Note that you must be using the latest version of ZAP (currently 2.6.0) and at least Java 8.
Version 2.6.0 does support Java 7, but this functionality requires Java 8. It’s worth noting that the next full release of ZAP will also require Java 8 as a minimum.

If you have any problems or questions about this new feature then please head over the the ZAP User Group.

Monday, 19 June 2017

Scanning APIs with ZAP

The previous ZAP blog post explained how you could Explore APIs with ZAP.
This blog post goes one step further, and explains how you can both explore and perform security scanning of APIs using ZAP from the command line.
This allows you to easily automate the scanning of your APIs.

Following the approach taken by the Baseline Scan we have introduced a new API scanning script which has only one dependency – Docker. You don’t need to install either ZAP or Java.
The script, zap-api-scan.py is included in the Weekly and Live ZAP Docker images, it will also be included in the next Stable image.

To use the API scanning script you just need to use the commands:
    docker pull owasp/zap2docker-weekly
    docker run -t owasp/zap2docker-weekly zap-api-scan.py -t \
        https://www.example.com/openapi.json -f openapi

By default the script:
  1. Imports the API definition supplied
  2. Actively scans the API using a custom scan profile tuned for APIs
  3. Reports any issues found to the command line
If no issues are reported then that does not mean that your API is safe.
If your API is particularly important or sensitive then it would be sensible to follow the scan up with a manual penetration test. You should also test the applications that use the API as data returned via the API could still be used to attack the application if it does not suitably escape data that has been originally entered via a user.

Command Line Options


The script has a number of command line options that allow it to be tuned to your requirements:

Usage: zap-api-scan.py -t <target> -f <format> [options]
    -t target         target API definition, OpenAPI or SOAP, local file or URL, eg https://www.example.com/openapi.json
    -f format         either openapi or soap
Options:
    -c config_file    config file to use to INFO, IGNORE or FAIL warnings
    -u config_url     URL of config file to use to INFO, IGNORE or FAIL warnings
    -g gen_file       generate default config file(all rules set to WARN)
    -r report_html    file to write the full ZAP HTML report
    -w report_md      file to write the full ZAP Wiki(Markdown) report
    -x report_xml     file to write the full ZAP XML report
    -a                include the alpha passive scan rules as well
    -d                show debug messages
    -P                specify listen port
    -D                delay in seconds to wait for passive scanning
    -i                default rules not in the config file to INFO
    -l level          minimum level to show: PASS, IGNORE, INFO, WARN or FAIL, use with -s to hide example URLs
    -n context_file   context file which will be loaded prior to scanning the target
    -p progress_file  progress file which specifies issues that are being addressed
    -s                short output format - dont show PASSes or example URLs
    -z zap_options ZAP command line options e.g. -z "-config aaa=bbb -config ccc=ddd"



Scan Rules


By default the script will use a Scan Policy tuned for APIs.
This disables rules that are focused on client side (e.g. browser) issues, such as the ones for detecting Cross Site Scripting, and also adds 2 additional rules that are implemented as scripts:
You can change which rules are run and how failures are reported using a configuration file. This allows you to tune the scanning script to meet your requirements for each of your APIs.
To generate a configuration file use the ‘-g’ option. This will create a file which includes all of the active and passive scan rules available: https://github.com/zaproxy/zaproxy/wiki/ZAP-API-Scan#configuration-file. You can edit this file using a text editor.
Changing a passive rule will only affect how failures are reported, but changing an active rule to IGNORE will prevent the rule from running. This is to reduce the overall scan time – passive rules are very quick while active rules can take a significant amount of time.

Specifying Values


ZAP will use a set of default values when importing APIs. In some cases these will not be suitable values for a specific application, and therefore will not exercise enough of the code. For example a username of “test” might not cause a new user to be created as it is not a valid email address.
For APIs defined using OpenAPI/Swagger you can specify the values you want ZAP to use via ZAP command line options.
For example the options:
  -config formhandler.fields.field\(0\).fieldId=username \
  -config formhandler.fields.field\(0\).value=test@example.com \
  -config formhandler.fields.field\(0\).enabled=true \
  -config formhandler.fields.field\(1\).fieldId=phone \
  -config formhandler.fields.field\(1\).value=012345678 \
  -config formhandler.fields.field\(1\).enabled=true

Will supply the following values to the named fields:
  username -> test@example.com
  phone -> 012345678

Support for specifying values for APIs defined using SOAP is also planned – if you need this then get in touch with the ZAP development team and we will do our best to prioritize this.

Note that as these are ZAP command line options you will need to specify them to the script using the -z script option.
If you need to specify lots of options then you can put them all in a property file, eg called options.prop
You can then run the API scan using a command like:

docker run -v $(pwd):/zap/wrk/:rw -t owasp/zap2docker-weekly zap-api-scan.py \
    -t https://www.example.com/openapi.json -f openapi \
    -z "-configfile /zap/wrk/options.prop"


The "-v $(pwd):/zap/wrk/:rw" is a Docker option which maps the current working directory to a folder called /zap/wrk in the Docker instance.

Authentication


Some of your APIs may be protected using authentication mechanisms.
For mechanisms that use header values we recommend that you obtain suitable tokens for your application using whatever means are appropriate and then tell ZAP to use them via another set of command line options.
For example the options:
  -config replacer.full_list\(0\).description=auth1 \
  -config replacer.full_list\(0\).enabled=true \
  -config replacer.full_list\(0\).matchtype=REQ_HEADER \
  -config replacer.full_list\(0\).matchstr=Authorization \
  -config replacer.full_list\(0\).regex=false \
  -config replacer.full_list\(0\).replacement=123456789 \
  -config replacer.full_list\(1\).description=auth2 \
  -config replacer.full_list\(1\).enabled=true \
  -config replacer.full_list\(1\).matchtype=REQ_HEADER \
  -config replacer.full_list\(1\).matchstr=AnotherHeader \
  -config replacer.full_list\(1\).regex=false \
  -config replacer.full_list\(1\).replacement=abcdefghi

will cause the following headers to be added to every request ZAP makes:
  Authorization: 123456789
  AnotherHeader: abcdefghi

You can specify as many headers as you need to by using incrementing indexes.

This functionality is provided by the Replacer add-on included by default with ZAP. It is very powerful and can do much more than just inject new header values, so if you need to manipulate the requests ZAP makes in other ways then this could be a very good option for you.

Conclusion


The API scanning script is an easy way for you to automate security scanning of APIs defined using OpenAPI/Swagger or SOAP.
It can be used ‘out of the box’ or quickly tuned to meet your requirements via simple command line and configuration file options.

A future blog post will explain how you can automate the security scanning of HTML based web applications in a very similar way.

Monday, 3 April 2017

Exploring APIs with ZAP

APIs can be challenging for security testing for a variety of reasons.
The first problem you will encounter is how to effectively explore an API - most APIs cannot be explored using browsing or standard spidering techniques.
However many APIs are described using technologies such as:

These standards define the API endpoints and can be imported into ZAP using 2 optional add-ons.

Installing the add-ons

In order to import the API definitions you will need to add the relevant add-ons from the ZAP Marketplace.

To do this via the UI:


  1. Click on the ‘Manage Add-ons’ button
  2. Select the ‘Marketplace’ tab
  3. Click on the ‘Check for Updates’ button if no add-ons are shown
  4. Select and install the add-ons:
    • OpenAPI Support
    • SOAP Scanner



To install them via the command line is even easier - just start ZAP with the command line flags: “-addoninstall soap -addoninstall openapi”
Eg:
./zap.sh -addoninstall soap -addoninstall openapi

Importing via the UI

These add-ons add the following items under the ‘Tools’ menu:
  • Import a WSDL file from local file system
  • Import a WSDL file from a URL
  • Import an OpenAPI definition from the local file system
  • Import an OpenAPI definition from a URL

These menu items will open new dialogs which will allow you to specify the relevant file or URL to import.

When the definitions have been imported they will be shown in the Sites tree:



Importing via the API

Both add-ons support importing API definitions from URLs and local files.
You can try these out via the API Web UI:


Using the Java API client this can be done with calls like:

Map<String, String> map1 = new HashMap<>();
map1.put("file", "/home/user/openapi.json");
ApiResponse resp =
  api.callApi("openapi", "action", "importFile", map1);

Map<String, String> map2 = new HashMap<>();
map2.put("url", "https://localhost/openapi.json");
ApiResponse resp =
  api.callApi("openapi", "action", "importUrl", map2);

Map<String, String> map3 = new HashMap<>();
map3.put("file", "/home/user/soap.xml");
ApiResponse resp =
  api.callApi("soap", "action", "importFile", map3);

Map<String, String> map4 = new HashMap<>();
map4.put("url", "https://localhost/soap.xml");
ApiResponse resp =
  api.callApi("soap", "action", "importUrl", map4);

Using the Python API client this can be done with calls like:

print zap._request(
  zap.base + 'openapi/action/importFile/',
  {'file':'/home/user/openapi.json'})

print zap._request(
  zap.base + 'openapi/action/importUrl/',
  {'url':'https://localhost/openapi.json'})

print zap._request(
  zap.base + 'soap/action/importFile/',
  {'file':'/home/user/soap.xml'})

print zap._request(
  zap.base + 'soap/action/importUrl/',
  {'url':'https://localhost/soap.xml'})

Spidering

Both of the add-ons automatically detect the relevant API definition file while spidering and will explore the definitions as long as they are in scope.
This means that you can explore an API by spidering from the URL of the definition as long as it is on the same domain as the API.

Next steps

Once you have added your API to the Sites tree using any of the above options you can then use any of the other ZAP components on the API, including the active scanner and fuzzer.

Note that the SOAP Scanner add-on also adds 2 additional scan rules that specifically target SOAP and XML URLs:


Monday, 6 February 2017

Introducing the JxBrowser add-on for ZAP

As modern web applications are increasing their reliance on JavaScript, security tools that do not understand JavaScript will not be able to work effectively with them.  ZAP already has components like the Ajax Spider and DOM XSS scanner that work by launching browsers and controlling them via Selenium, and we are planning to make much more use of browsers in the future.

To that end we are planning on releasing a new ZAP add-on which will contain JxBrowser, a wrapper around Chromium. We want ZAP to work as effectively as possible out of the box, and to be as easy to automate as possible. Being able to package JxBrowser in an add-on gives us an up to date browser that we know will work without any other user actions. While the add-on does contain a bundled version of Chromium it will not install Chrome on your computer. Your existing browsers and browser preferences will not be affected.

Please note: While JxBrowser is a commercial closed source wrapper around Chromium, TeamDev has generously given us a permanent free license to allow us to redistribute JxBrowser with ZAP. Most importantly ZAP will stay completely free. Additionally we will not be including the JxBrowser with the ZAP ‘core’ release which we maintain. It will also be very easy to remove the JxBrowser add-on from ZAP via the command line, in the same way that any other add-ons can be removed:
./zap.sh -addonuninstall jxbrowser

Why are we doing this?
As this is the first time we will have done anything like this I wanted to explain why we are packaging a closed source product with ZAP, what the implications are and how you can contact us to discuss any concerns you may have.

Unfortunately it is difficult for us to know which browsers will work on any specific system. Both Firefox and Chrome could be present, but we can’t tell until we try to launch them. IE and/or Edge are only going to be available on Windows systems and although Safari is always likely to be available on Mac OS it will not work with Selenium until an extra plugin is installed. As part of this on going effort we have recently decided to package the WebDrivers for Firefox, Chrome and IE in ZAP add-ons so that you will not have to download them manually. We want to be able to default to JxBrowser while allowing you to choose to use any of the browsers as you see fit. We will also be able to do things like launch JxBrowser from within ZAP pre-configured to proxy via ZAP.

Are there any licensing restrictions for JxBrowser?
As a ZAP user you will be able to use the ZAP JxBrowser add-on for any purposes.
However if you change the ZAP source code and call the JxBrowser API from one of your own products then you will need to arrange a suitable licence with TeamDev.
It is also worth noting that while JxBrowser does not ‘phone home’ the internal Chromium functionality may call 3rd party services (spell checking, geolocation, etc) in a similar way to other browsers.

When will the add-on be available?
It's available now :) You can install it from within ZAP using the 'Manage Add-Ons' button on the ZAP toolbar. Make sure that you update any add-ons that are flagged as needing updates, in particular the Selenium and Ajax Spider ones.

If you do have any concerns about our bundling of JxBrowser then please join in the discussion on the ZAP User Group or contact me directly.

Tuesday, 22 November 2016

Announcing the Official ZAP Jenkins Plugin

Using ZAP during the development process is now easier than ever. We are proud to present the Jenkins plugin, it extends the functionality of the ZAP security tool into a CI Environment.

The process explained
  1. A Jenkins CI Build step initializes ZAP
  2. Traffic flows (Regression Pack) through ZAP (Web Proxy)
  3. ZAP modifies requests to include Vulnerability Tests
  4. Target Application/Server sends Response back through ZAP
  5. ZAP sends reporting data back to Jenkins
  6. Jenkins publishes and archives the report(s)
  7. Jenkins creates JIRA tickets for the alerts

The ZAP Jenkins plugin makes use of the readily available and diverse ZAP API, allowing you to use the same session files and scan policy profiles between ZAP and the Jenkins plugin, so they can be interchangeably loaded.

So what can you do?
Automate the site mapping process with a selenium script, have ZAP act as an intercepting proxy to map the structure of your site and record passive alerts. Fire off an active scan and finish it off by generating a report in one of three available formats (xhtml, xml or json). These can be sent off to management or you can load the session later and inspect each raised alert at your convenience.

Providing a seamless workflow and the same functionality as the GUI. You can
  • Manage Sessions (Load or Persist)
  • Define Context (Name, Include URLs and Exclude URLs)
  • Attack Contexts (Spider Scan, AJAX Spider, Active Scan)
You can also:
  • Setup Authentication (Form Based or Script Based)
  • Run as Pre-Build as part of a Selenium Build
  • Generate Reports (.xhtml, .xml, .json)
All while giving you all the benefits of Jenkins to automate the process. Scan between build and deployment all from taking advantage of the automation server.


Where we go from here...
We plan to extend the authentication method to allow authenticated AJAX Spider Scans and support HTTP/NTLM Authentication. To further the continuous integration process, we will be adding Build Management tools in the near future which will allow you to set the thresholds that will determine a builds pass or failure. But we’re not stopping here, we will be continuously advancing our API to meet the needs of community requests for the Jenkins Plugin.

We will work with our community, taking advice and feedback to improve and support this plugin in the short and long term.

ZAP is, as it always has been; one of the largest open source security software. As such this plugin follows in its footsteps. We hope it will become one of your favorite additions into your development lifecycle.

We hope to expand this project and open it to community involvement, we welcome you; the community members to help us maintain and improve the plugin and we thank you for your continued support.

Documentation: See the Wiki for more details.

Questions: Ask on our Google Group.

Issue Tracking: Report on the Jenkins JIRA for the project, please read the JIRA guidelines before reporting an issue.

Monday, 22 August 2016

Announcing ZAP Unit Test Bounties

Unit tests are wonderful things, but they are painful to add to a mature project that doesn’t have enough of them. We would love to have more ZAP unit tests, and we are therefore launching a Unit Test Bounty program, where we pay for unit tests for specific areas of the ZAP codebase.

We are going to start with the passive scan rules (release and beta quality).
These are all defined in the zap-extensions src packages:
We do already have some release quality unit tests in the corresponding test package:
These can be used as a basis of the new tests. Unit tests should be included for real issues (true positives) as well as false positives.
For details of how passive scan rules work see the blog post: Hacking ZAP #3 Passive Scan Rules

We have raised issues for all of the release and beta quality passive scan rules that do not have unit tests, and assigned a bounty on them in Bounty Source. The bounties vary based on the amount of work we expect will be required. We will increase the bounties paid if the pull request (PR) include fixes for false positives or false negatives.

The open bounties can all be seen on the Bounty Source page

We plan to extend the bounties to cover active scan rules in the future. We expect the bounties to typically be higher than for the passive scan rules due to the added complexity.

We will pay out these bounties only when we have merged a PR that has an associated bounty. The PRs must test all of the associated rule functionality and must conform to our Development Rules and Guidelines. Also see CONTRIBUTING.md
We will work with submitters to help them improve any non-conforming PRs so that they meet our standards. We will also give the first person who submits a nearly conforming PR a reasonable time (eg 2 weeks) to reach the required standard rather than immediately accepting another later PR from someone else that does conform.
We will not reserve issues in advance, nor will we reserve an issue based on an inadequate PR.

You will see that the bounties are not huge. In our defence there are quite a few of them, implementing them should not take too much work and in any case we are an open source project with no revenue stream other than donations!

We also have plans to expand this program to other aspects of ZAP. If you would be interested in sponsoring such plans then please get in touch, or you can just donate money directly to the ZAP project via the Bounty Source page or via the ‘Donate’ Paypal button on https://www.owasp.org/index.php/ZAP. You can also donate money to ZAP when you join or renew your OWASP membership.
Do let us know if you are explicitly sponsoring this initiative so that we can give you the public credit you deserve :)

Friday, 3 June 2016

ZAP 2.5.0

ZAP 2.5.0 is now available: https://github.com/zaproxy/zaproxy/wiki/Downloads

This release contains a large number of enhancements and fixes which are detailed in the release notes: https://github.com/zaproxy/zap-core-help/wiki/HelpReleases2_5_0

API changes

There have been some API changes which are not backwards compatible, and the reason for the version change to 2.5. These are detailed in the release notes.
The API has also been extended to cover even more of the functionality in ZAP, including full access to the statistics.

The Java API is no longer packaged with this release.
You can download the latest version from: https://github.com/zaproxy/zap-api-java/releases

It will also be available on Maven Central:
  • GroupId: ‘org.zaproxy’
  • ArtifactId: ‘zap-clientapi’
  • Version: ‘1.0.0’
Note that this may not be available immediately, depending on how long the publishing process takes.

Daemon obeys Mode

ZAP now obeys the mode setting when running as a headless daemon.

Spider subtree option

The spider now has an option to constrain it to a specific subtree. This allows you to explore one part of an application without having to cover all of it.

Statistics

ZAP now maintains a wide range of statistics which can be invaluable for understanding what is really happening when interacting with large applications.
These are available via the API and can also be sent to a Statsd server.
For more information see https://github.com/zaproxy/zap-core-help/wiki/HelpStartConceptsStats

Docker

The stable and weekly docker images now allow you to run the ZAP Desktop UI in a browser.
This means that you can run ZAP without having to install Java.
For more details see https://github.com/zaproxy/zaproxy/wiki/WebSwing

The Docker images also include the ‘ZAP Baseline’ script.
This runs the ZAP spider against the specified target for (by default) 1 minute and then waits for the passive scanning to complete before reporting the results.
This means that the script does not perform any actual 'attacks' and will run for a relatively short period of time (a few minutes at most).
This script is intended to be ideal to run in a CI/CD environment, even against production sites.
For more details see https://github.com/zaproxy/zaproxy/wiki/ZAP-Baseline-Scan

Thank you to everyone who contributed to this release.
To keep up to date with ZAP related news follow @zaproxy on twitter.