Core Features

ScalaWebTest has two main features.

  • Handling of web requests and state
  • Gauges for simple tests

The short description of the two main features is followed by a getting started section suitable for beginners and a detailed description of all features.

Handling of web requests and state

ScalaWebTest's IntegrationSpec trait helps you to structure your tests and keep them boilerplate free. It extends the most important traits, to write integration tests for web applications, from ScalaTest. It provides an easy way to configure the HtmlUnit webdriver and it takes care of login, web requests and cookies. Before executing your tests, it takes care of login, if needed. Next it requests the resource, which your test wants to verify. To keep your tests boilerplate free, the URL to the tested resource is split in 3 segments. The host and projectRoot are part of the IntegrationSettings. The last part is the path itself. In most cases a test should only work with a single resource, in this case the path doesn't change throughout the test. Per default the IntegrationSpec will, before each test, take care that the current resource in the WebDriver is the one belonging to the defined path. This behavior can be changed. Therefore your test does not have to interact with the WebDriver. Instead it can directly verify the content of the current resource via webdriver.

Gauges for simple tests

The HtmlUnit webdriver and selenium are very powerful. But writing integration tests for web application is still very cumbersome and the resulting tests are often hard to read. We think the easiest way to define the expected result of a web request is using the returned data format itself. Therefore we use pseudo HTML (let's focus on HTML for the moment, but we also support JSON) to describe the expected result. Of course a simple string comparison would be to naive. Therefore we treat the HTML (technically we require XML, because it has native support in Scala), which is used to describe the result, as gauge definition (other would call it a template). If we can find all elements from the gauge definition in the verified HTML page, it is considered valid.

Getting started

Add ScalaWebTest to your project

Before you get to use all the nice features, you have to introduce ScalaWebTest to your project. All you have to do is adding the core module to your test or integration test dependencies.

Add ScalaWebTest to your SBT project

You can add ScalaWebTest to your testing dependencies in the build.sbt/scala of your sbt project as follows.

libraryDependencies += "org.scalawebtest" %% "scalawebtest-core" % "2.0.1" % "test"

We recommend to bind the ScalaWebTest dependencies to the IntegrationTest configuration, which can be referenced with "it". The IntegrationTest configuration is not active by default. Follow the guide for SBT 1.0 or SBT 0.13 to make it part of your build. Then you can add ScalaWebTest to your project as follows.

libraryDependencies += "org.scalawebtest" %% "scalawebtest-core" % "2.0.1" % "it"

If you want to manage the most important transitive dependencies of ScalaWebTest, you might configure it as follows (replace "it" with "test", if you do not make use of the IntegrationTest configuration. Especially selenium-java and htmlunit-driver have to align with each other.

libraryDependencies ++= Seq(
    "org.scalawebtest" %% "scalawebtest-core" % "2.0.1" % "it",
    "org.scalatest" %% "scalatest" % "3.0.4" % "it",
    "org.seleniumhq.selenium" % "selenium-java" % "3.6.0" % "it",
    "org.seleniumhq.selenium" % "htmlunit-driver" % "2.27" % "it"
)

Add ScalaWebTest to your maven project

You have to add the following dependency to your maven project to use ScalaWebTest.

<dependency>
    <groupId>org.scalawebtest</groupId>
    <artifactId>scalawebtest-core_2.12</artifactId>
    <version>2.0.1</version>
    <scope>test</scope>
</dependency>

ScalaWebTest is also available for Scala 2.10 and 2.11, change the version after the underscore, to get a ScalaWebTest version which is binary compatible with the Scala version which you use.

<dependency>
    <groupId>org.scalawebtest</groupId>
    <artifactId>scalawebtest-core_2.11</artifactId>
    <version>2.0.1</version>
    <scope>test</scope>
</dependency>

In case you want to manage the most important transitive dependencies of ScalaWebTest, you can add our bill-of-materials to your project. Just add the following to your dependencyManagement.

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.scalawebtest</groupId>
            <artifactId>bom_2.12</artifactId>
            <version>2.0.1</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

Write your first test

To write your first test, you have to create a scala file in src/it/scala or src/test/scala, depending on your test setup. Lets assume we want to verify our homepage. Create a class named HomepageSpec, and let it extend IntegrationFlatSpec. This uses FlatSpec style from ScalaTest, other styles are available.

package org.scalawebtest.documentation

import org.scalawebtest.core.IntegrationFlatSpec

class HomepageSpec extends IntegrationFlatSpec {
    override val host = "http://www.scalawebtest.org"
    path = "/index.html"

    "Our homepage" should "contain a succinct claim" in {
        webDriver.findElementByTagName("h2").getText shouldEqual "Reduce the effort needed to write integration tests"
    }
}
Try it in Scastie

That's all you need. The IntegrationSpec, inherited from IntegrationFlatSpec will automatically call http://localhost:8080/index.html before executing a test. When using FlatSpec style the block following the in keyword is the body of the test. Within the body of a test you can access the webDriver, to test the content of the current page. The webDriver is backed by the HtmlUnit driver from selenium. It emulates a browser and supports among other things JavaScript execution, clicking elements and submitting forms.

Write your first gauge

This test can be rewritten making use of the HtmlGauge. Writing gauges provides a detailed introduction to this feature.

package org.scalawebtest.documentation

import org.scalawebtest.core.IntegrationFlatSpec
import org.scalawebtest.core.gauge.HtmlGauge

class HomepageSpec extends IntegrationFlatSpec with HtmlGauge {
    override val host = "http://www.scalawebtest.org"
    path = "/index.html"

    "Our homepage" should "contain a succinct claim" in {
    fits(<h2>Reduce the effort needed to write integration tests</h2>)
    }
}
Try it in Scastie

Structuring your project

Build your base trait

Configurations, style choice and possible custom extension should be shared across your project. To do so, we recommend to create a base trait, which is extended by all your tests. ScalaWebTest uses this concept for it's own integration tests as well.
package org.scalawebtest.documentation

import org.scalatest.AppendedClues
import org.scalatest.concurrent.PatienceConfiguration.Timeout
import org.scalatest.time.SpanSugar._
import org.scalawebtest.core.gauge.HtmlGauge
import org.scalawebtest.core.{FormBasedLogin, IntegrationFlatSpec}

import scala.language.postfixOps

trait MyProjectBaseSpec extends IntegrationFlatSpec with FormBasedLogin with AppendedClues with HtmlGauge {
    override val host = "http://localhost:9090"
    override val loginPath = "/login.php"

    override val projectRoot = ""

    override def loginTimeout = Timeout(5 seconds)
}
This base trait uses FlatSpec style, FormBasedLogin and HtmlGauge. It configures host, loginPath and projectRoot and sets a suitable loginTimeout. Our integration tests have two purposes. Not only do we use them to test our framework, but we also use it as documentation. They are a good starting point when looking for best practice on how to make the most out of ScalaWebTest.

Selecting a style

ScalaTest provides a wide variety of testing styles. ScalaWebTest supports all available styles. As we believe the FlatSpec style is easy to read for most people, we choose it for our own documentation and most of our integration tests. There is no technical reason behind it, it is simply a matter of taste. We recommend that you extend one of the Integration*Spec/Suite classes from Styles.scala in your base trait.

Configure your tests

A reasonable default configuration is provided, which can easily be adapted. If you want to change a configuration for all your tests, we recommend to change it in your base trait, otherwise best practice is to change it in the constructor of your TestSpec. Two config objects exist, one which is used during login (if login is needed) and one which is used during test execution.

In the following TestSpec JavaScript errors are swallowed during Login phase, no matter if JavaScript execution is enabled or disable. During test execution JavaScript is enabled and JavaScript errors let the test fail.

package org.scalawebtest.documentation

import org.scalawebtest.core.IntegrationFlatSpec

class HomepageSpec extends IntegrationFlatSpec {
    override val host = "http://www.scalawebtest.org"
    path = "/index.html"

    loginConfig.swallowJavaScriptErrors()
    config.enableJavaScript(throwOnError = true)

    "Our homepage" should "contain a succinct claim" in {
        webDriver.findElementByTagName("h2").getText shouldEqual "Reduce the effort needed to write integration tests"
    }
}
Try it in Scastie

By default JavaScript is not executed and JavaScript errors don't throw. Also CSS is not interpreted. This applies to both loginConfig and config.

Writing gauges

Using gauges

Using gauges to write your integration tests, is the core idea of ScalaWebTest. We think using org.scalatest.selenium.WebBrowser.Query, i.e CssSelectorQuery, to select elements in a HTML document and then verifying it is cumbersome and resulting tests are hard to read. Instead we copy a concept from the manufacturing industry. They build gauges (or templates), which they then lay their workpiece into. If the workpiece fits in the gauge, it satisfies the requirements. Our gauges are defined in HTML (XML to be more precise, or scala.xml.NodeSeq to be exact) instead of being forked from steel. Let's have a look at a simple example to get an idea how those gauges work.

package org.scalawebtest.documentation

import org.scalawebtest.core.IntegrationFlatSpec
import org.scalawebtest.core.gauge.HtmlGauge

class SimpleGaugeSpec extends IntegrationFlatSpec with HtmlGauge {

    path = "/navigation.jsp"

    "The navigation" should "contain our navigation links in correct structure, order and with the expected text" in {
        fit(
            <nav id="mainNav">
                <ul class="blue_theme">
                     <li>
                          <a href="/path/to/first/element">first navigation element</a>
                     </li>
                     <li class="active">
                          <a href="/path/to/second/element">second navigation element</a>
                     </li>
                     <li>
                          <a href="/path/to/third/element">third navigation element</a>
                     </li>
                </ul>
             </nav>
        )
    }
}

This test would be successful, when run against, when navigation.jsp contains the following HTML

<!DOCTYPE html>
<html>

<head>
</head>

<body>
<nav id="mainNav">
    <ul class="blue_theme">
        <li>
            <a href="/path/to/first/element">first navigation element</a>
        </li>
        <li class="active">
            <a href="/path/to/second/element">second navigation element</a>
        </li>
        <li>
            <a href="/path/to/third/element">third navigation element</a>
        </li>
    </ul>
</nav>
</body>
</html>

We can already, see that fits() ignored html, head and body, elements before the navigation. But following the idea from manufacturing, we should strip our gauge, from everything which we don't need. Let's assume we only want to make sure our navigation contains the correct links. We don't care about classes added for design or text contained in the links. Therefore we reduce our test to.

"The navigation" should "contain our navigation links in correct structure and order" in {
    fit(
        <nav id="mainNav">
            <ul>
                <li>
                    <a href="/path/to/first/element"></a>
                </li>
                <li>
                    <a href="/path/to/second/element"></a>
                </li>
                <li>
                    <a href="/path/to/third/element"></a>
                </li>
            </ul>
        </nav>
    )
}

If we don't care about the unnumbered list, we can reduce our test even further.

"The navigation" should "contain our navigation links in correct order" in {
    fit(
        <nav id="mainNav">
            <a href="/path/to/first/element"></a>
            <a href="/path/to/second/element"></a>
            <a href="/path/to/third/element"></a>
        </nav>
    )
}

The document then fits our gauge as follows. Matching parts are green, parts in grey have no influence whether the document fits or not.

<!DOCTYPE html>
<html>
    <head>
    </head>
    <body>
        <nav id="mainNav">
            <ul class="blue_theme">
                <li>
                    <a href="/path/to/first/element">first navigation element</a>
                </li>
                <li class="active">
                    <a href="/path/to/second/element">second navigation element</a>
                </li>
                </li>
                    <li>
                <a href="/path/to/third/element">third navigation element</a>
            </li>
        </ul>
        </nav>
    </body>
</html>

If the gauge doesn't fit, we get an error. Lets change the test to the following.

"The navigation" should "contain our navigation links in correct order" in {
    fit(
        <nav id="mainNav">
            <a href="/path/to/first/element">second navigation element</a>
            <a href="/path/to/second/element"></a>
            <a href="/path/to/third/element"></a>
        </nav>
    )
}

This will of course not match, because the text within the first link is wrong and belongs to the second link. See the difference highlighted in the following html block.

<!DOCTYPE html>
<html>
    <head>
    </head>
    <body>
        <nav id="mainNav">
            <ul class="blue_theme">
                <li>
                    <a href="/path/to/first/element">second navigation element first navigation element</a>
                </li>
                <li class="active">
                    <a href="/path/to/second/element">second navigation element</a>
                </li>
                </li>
                    <li>
                <a href="/path/to/third/element">third navigation element</a>
            </li>
        </ul>
        </nav>
    </body>
</html>

This results in the following error message from ScalaWebTest

23:06:37.482 [ScalaTest-run-running-SimpleGaugeSpec] INFO IntegrationSpec - Going to
http://localhost:8080/navigation.jsp

Misfitting Text: The text [first navigation element] within [HtmlAnchor[<h4
    href="/path/to/first/element"></h4>]] didn't equal [second navigation element]
Current document does not match provided gauge:
<nav id="mainNav">
    <ul>
        <li>
            <a href="/path/to/first/element">second navigation element</a>
        </li>
        <li>
            <a href="/path/to/second/element"></a>
        </li>
        <li>
            <a href="/path/to/third/element"></a>
        </li>
    </ul>
</nav>
ScalaTestFailure

Test for classes on elements

The class attribute is special, because it is basically an unsorted set of class attributes. Usually we don't bother, if additional classes are present in our HTML and for sure we never care about the order. ScalaWebTest therefore handles the class attribute different from the rest. Per default the attribute content, has to match exactly the one you provided in your gauge, but for classes, it only asserts, that the classes which an element contains within your gauge definition, are all present on that element in the HTML document. The following gauge therefore matches all the elements shown below.

fit(
    <div class="container red"></div>
)
<div class="container red"></div>
<div class="red container"></div>
<div class="red important container main"></div>

Test single elements

Often trying to fit the complete document into the defined gauge isn't the most natural and efficient thing to do. Especially when a website contains multiple elements of the same kind, such as content cards, gallery images or items of a product list, finding all those elements first, and then trying to fit each element with the gauge, is better.

package org.scalawebtest.documentation

import org.scalatest.AppendedClues
import org.scalawebtest.core.IntegrationFlatSpec
import org.scalawebtest.core.gauge.HtmlElementGauge

class ElementGaugeSpec extends IntegrationFlatSpec with HtmlElementGauge with AppendedClues {
    path = "/galleryOverview.jsp"

    def images = findAll(CssSelectorQuery("ul div.image_columns"))

    val imageGauge =
    <div class="columns image_columns">
        <a>
            <figure class="obj_aspect_ratio">
                <noscript>
                    <img class="obj_full"></img>
                </noscript>
                <img class="obj_full lazyload"
                     srcset="data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7"
                     data-sizes="auto"></img>
            </figure>
        </a>
    </div>

    "The gallery" should "contain the expected HTML for every image" in {
        images.size should be > 5 withClue " - gallery didn't contain the expected amount of images"

        for (image <- images) {
            image fits imageGauge
        }
    }
}

When using findAll with a for-comprehension, one should always verify, that the expected amount of elements was found.

When using fits or doesntFit, the same features are available, no matter if the whole document, or a single element is checked. Therefore the following chapters apply to both usecases. There are two restrictions when using gauges for single elements (built using ElementGaugeBuilder.GaugeFromElement)

  • the gauge may only contain one top level element
  • the top level element of the gauge has to be the same, as the one which is checked (not one of its children)

Test for containment

It is common that you don't want to verify the complete attribute value, or text of an element. You may use @contains to trigger the ContainsMatcher instead of the DefaultMatcher. Thanks to this matcher, this gauge matches the following HTML element

fit(<a href="@contains ScalaWebTest"></a>)
<a href="https://github.com/unic/ScalaWebTest"></a>

The containsMatcher is also available, when matching text.

fit(<a>@contains available</a>)
<a>ScalaWebTest is available on github</a>

Hint: always add a space after the matcher annotation. We enforce this space to improve readability of tests.

Test for regex matches

When we want to enforce textual rules on our content, we need a more powerful Matcher. The RegexMatcher is our friend. It is triggered using @regex

fit(<a href="@regex http:\\/\\/[a-zA-Z]+\.scalawebtest.org.*,"></a>)

This gauge will fit

<a href="http://www.scalawebtest.org/documentation.html"></a>

but it won't fit

<a href="http://scalawebtest.org/documentation.html"></a>

Negating your tests

Sometimes you want to make sure that a certain content is not shown. For example you want to make sure, that no login form is shown, when a user is logged in. Or you want to make sure that a post doesn't appear before it is made public. To do so you may use doesnt fit() or not fit(). They are synonyms. Choose whatever reads better in your current context.

package org.scalawebtest.documentation

import org.scalawebtest.core.IntegrationFlatSpec
import org.scalawebtest.core.gauge.HtmlGauge

class LoggedInSpec extends IntegrationFlatSpec with HtmlGauge {
    path = "/protectedContent.jsp?username=admin&password=secret"

    "When logged in the protectedContent page" should "not show the login form" in {
        not fit <form name="login_form"></form>
    }
}

In case the login form is mistakenly rendered, the following error will appear.

Current document matches the provided gauge, although expected not to!
Fitting node
<html>
<head>
    <title>Mock of a protected content page
    </title>
</head>
<body>
<form name="login_form" action="protectedContent.jsp" method="get">
    <label for="username">username
    </label>
    <input type="text" name="username" id="username">
    </input>
    <label for="password">password
    </label>
    <input type="password" name="password" id="password">
    </input>
    <button type="submit">login
    </button>
</form>
</body>
</html>
found

Testing a complete process

Today's web applications are of course not that static. Usually it is more interesting to test a complete process. This is easily possible with Selenium. You can use it to press buttons, fill and submit forms and even execute JavaScript. As the gauges are based on Selenium, they can be combined with actions as well. As soon as an action is executed, the currentPage is updated and our next call to fit() or doesnt fit() will be executed against the updated page.

Lets try this out using a page with protected content. On first load a form will be shown, after logging in the form disappears, instead the text sensitive information appears.

package org.scalawebtest.documentation

import org.scalawebtest.core.IntegrationFlatSpec

import org.scalawebtest.core.gauge.HtmlGauge

class LoginSpec extends IntegrationFlatSpec with HtmlGauge {
    path = "/protectedContent.jsp"

    "When accessing protectedContent it" should "show the login form" in {
        fits(
            <form name="login_form">
                <input name="username"></input>
                <input name="password"></input>
            </form>
        )
    }

    it should "hide the protected content, when not logged in" in {
        not fit <p>sensitive information</p>
    }

    it should "show the protected content, after logging in" in {
        textField("username").value = "admin"
        pwdField("password").value = "secret"

        submit()

        fits(<p>sensitive information</p>)
    }
}

This example test was split into three smaller tests. This is technically not necessary, but has the following advantage. When only part of the tests fails, you have a better idea what went wrong. For example, if the login fails, but the login form is correctly shown, the issue has to be your login process. But if the login form isn't shown, the problem has a very different cause. In addition the verbose output from the gauge should help you finding the root cause.

Using advanced Features

Executing JavaScript

Thanks to Selenium executing JavaScript is not an issue. Per default JavaScript is not executed with ScalaWebTest, because tests run faster without. To execute JavaScript, you have to enable it.

config.enableJavaScript(throwOnError = true)

In addition, you have to decide whether a JavaScript error should cause your test to fail or not. Usually failing on JavaScript errors is good, but sometimes you have to write tests for a web application with major issues in it's JavaScript. This is one of the rare moments, where you have to disable throwOnError to be able to test the web application.

After receiving document, your browser needs some time to execute JavaScript. The same is true for the Selenium webdriver. We have to give it some time to execute JavaScript, before we can expect it to have transformed the HTML. ScalaTest provides eventually, which does exactly what we need. It repeats a given test until it succeeds or the given timeout has been surpassed.

eventually(timeout(3 seconds)) {
    fits(
        <div id="container">Text loaded with JavaScript</div>
    )
}

Additional modules

Modules

Every web application framework has its own specialities, which are relevant for testing. ScalaWebTest allows to share this framework specific logic via modules. We encourage others, to create additional modules for their most used framework.

What should be shared via modules?

We think even simple things such as default login method, ports and urls might be worth sharing. Functions to create content, update accounts, or requesting a page in debug mode, could be even better candidates for a module.

AEM module

As the creators of ScalaWebTest work a lot with Adobe Experience Manager, they decided to create the first module for this CMS

To use this module extend AemTweaks from the aem module in your BaseTrait. This will cause the following.

  • FormBasedLogin is activated
  • loginPath is set to the default AEM 6 login path
  • pages are requested with wcmmode cookie set to DISABLED

PageProperties

Disclaimer: This trait is currently only available in scalawebtest-aem_2.11, due to dependencies on play-json which wasn't yet released for 2.12

The PageProperties trait populates the pageProperties, and if applicable componentProperties and suffixProperties fields with a JsValue representing the properties of the currentPage, component and suffix respectively. It does so by retrieving the JSON representation of the currentPage. This works by default on all CQ/AEM author instances. In addition it provides convenience methods to access the pageProperties content.

  • pageProperties(name: String) - retrieve a page property by name
  • jcrContent(name: String) - retrieve a property from jcr:content by name
  • findByResourceType(value: String) - search through a parsys field in the pageProperties and find all component with given resourceType
  • findByProperty(name: String)(value: String) - search through a parsys field in the pageProperties and find all component with given property name and value

It populates the pageProperties field with a play.api.libs.json.JsValue, which represents the properties of the currentPage. In case the url/path points to something below jcr:content, the componentProperties will be populated with the properties of the component, and the pageProperties with those of the containing page. In case the url/path contains a suffix, the suffixProperties will be populated with the properties of the page referenced in the suffix. It does so by manipulating the url field, to request the JSON representation of the currentPage from CQ/AEM. This feature is available on CQ/AEM author instances by default. The enable.json property of the org.apache.sling.servlets.get.DefaultGetServlet of your CQ/AEM instance has to be set to true. Only extend this trait in tests which need the feature, as it otherwise unnecessarily slows down your tests, due to additional requests for page properties.

JSON module

JSON is currently the most important data exchange format in the web. We think writing integration tests for web-services, which return JSON, should be as easy as for websites. Therefore we created the JsonGauge, which provides functionality comparable to the HtmlGauge. All its magic is exposed via the according builder, which provides implicit conversions for JsValue, JsLookup and functions retrieving the currentPage from the webDriver. We choose to use play-json to parse the JSON response. To keep the core module clean from additional dependencies a module was created.

Only verifying a few values

To verify the value or type of a few values, we consider standard play-json sufficient. Just use Json.parse to parse the response and then use \ and \\ to navigate the json structure.

package org.scalawebtest.documentation

import org.scalawebtest.core.IntegrationFlatSpec
import play.api.libs.json.Json

class ReducedJsonSpec extends IntegrationFlatSpec {
    path = "/dijkstra.json"
    def json = Json.parse(webDriver.getPageSource)

    "Dijkstra" should "have the correct firstname" in {
        def firstName = (json \ "firstName").as[String]
        firstName should equal("Edsger")
    }
}

All JSON tests in the following examples use the following JSON response

{
    "name": "Dijkstra",
    "firstName": "Edsger",
    "yearOfBirth": 1930,
    "theories": [
    "shortest path",
    "graph theory"
],
    "isTuringAwardWinner": true,
    "universities": [
{
    "name": "Universit├Ąt Leiden",
    "begin": 1948,
    "end": 1956
},
{
    "name": "Mathematisch Centrum Amsterdam",
    "begin": 1951,
    "end": 1959
},
{
    "name": "Technische Universiteit Eindhoven",
    "begin": 1962,
    "end": 1984
},
{
    "name": "University of Texas at Austin",
    "begin": 1984,
    "end": 1999
}
]
}

Of course this way of verifying JSON responses reaches its limitations very soon. Therefore we created JsonGauge

JsonGauge - the ScalaWebTest way

Same as when verifying HTML, we think the easiest and most natural way to formulate your expectation is using the same language, as the tested response itself. With the JSON gauge you can use JSON to specify the gauge, into which the JSON response has to fit.

package org.scalawebtest.documentation
import org.scalawebtest.core.IntegrationFlatSpec
import org.scalawebtest.json.JsonGauge
import org.scalawebtest.json.JsonGaugeFromResponse.fitsValues

class DocumentFitsValuesSpec extends IntegrationFlatSpec with JsonGauge {
    path = "/jsonResponse.json.jsp"

    "FitsValues" should "report success, when the json gauge contains the same values as the response it is tested against" in {
        fitsValues(
            """{
            "name": "Dijkstra",
            "firstName": "Edsger",
            "yearOfBirth": 1930,
            "isTuringAwardWinner": true,
            "theories": [
                "shortest path",
                "graph theory"
            ]
            }"""
        )
    }
}

Often verifying the complete response isn't ideal. We recommend to parse and traverse the document using play-json and then verify if the JsLookup or JsValue fits the defined gauge.

package org.scalawebtest.documentation

import org.scalawebtest.core.IntegrationFlatSpec
import org.scalawebtest.json.JsonGauge
import play.api.libs.json.Json

class DijkstraJsonGaugeSpec extends IntegrationFlatSpec with JsonGauge {
    path = "/dijkstra.json"
    def dijkstra = Json.parse(webDriver.getPageSource)

    "The response for Dijkstra" should "contain the expected values" in {
        dijkstra fits values of
            """{
                "firstName": "Edsger",
                "name": "Dijkstra",
                "yearOfBirth": 1930,
                "theories": [
                    "shortest path",
                    "graph theory"
                    ]
                }
            """
        }
    it should "contain the correct universities" in {
        val universities = dijkstra \ "universities"
        universities fit values of
        """
        [
            { "name": "Universit├Ąt Leiden","begin": 1948, "end": 1956 },
            { "name": "Mathematisch Centrum Amsterdam", "begin": 1951, "end": 1959 },
            { "name": "Technische Universiteit Eindhoven", "begin": 1962, "end": 1984 },
            { "name": "University of Texas at Austin", "begin": 1984, "end": 1999 }
        ]
        """
    }
}

As with the HTML gauge, controlled variance is allowed. The original response contains additional key/value pairs (isTuringAwardWinner and universities). Also the order of the name and firstName is the other way around. Nevertheless this test would succeed, when run. It would report an error, if the response would:

  • contain a different value for one of the keys
  • miss a key, which is contained in the gauge
  • contain a different hierarchy of the key/value pairs

Behaviors are a good way to share tests. When sharing test via behaviors, verifying values is often too specific. Therefore we provide fits types of in addition to fits values of.

package org.scalawebtest.documentation

import org.scalawebtest.core.IntegrationFlatSpec
import org.scalawebtest.json.JsonGauge
import play.api.libs.json.Json

class DijkstraJsonGaugeFitsTypesSpec extends IntegrationFlatSpec with JsonGauge {
    path = "/dijkstra.json"
    def dijkstra = Json.parse(webDriver.getPageSource)

    "The response for Dijkstra" should "contain the expected types" in {
        dijkstra fits types of
            """{
                "firstName": "",
                "name": "",
                "yearOfBirth": 0,
                "theories": [ "" ]
            }
        """
    }
}

Now the values no longer matter. They are only used to determine the expected type. Therefore we recommend to use values, which indicate that only the type matters. Use empty strings and 0. This test would succeed. It would fail, when the response would:

  • contain a value with a type different from the one expected for the given key
  • miss a key, which is contained in the gauge
  • contain a different hierarchy of the key/value pairs
  • when an array value would contain an element with a mismatching type

Lets have a closer look at arrays. Usually we expect all array elements to match certain expectations. When using fits types of, you only have to define those expectations once. All array elements then have to fulfill them. Lets have a look at an example. Lets assume the test of the previous example would be replaced with the following

"The response for Dijkstra" should "contain the expected types" in {
    dijkstra fits types of
        """{
            "firstName": "",
            "name": "",
            "yearOfBirth": 0,
            "theories": [ "" ]
            "universities":
            [{
                "name": "",
                "begin": 0,
                "end": 0
            }]
        }
        """
}

This would enforce all elements of the universities array to contain a key name with a value of type string and the keys begin and end with a value of type number. The elements might contain additional keys, such as department, but no contradictions. Sometimes this is not the behavior one is looking for. Two more options exist to verify arrays. One common scenario is, that the size of the array matters, i.e. coordinates in a GeoJson document. To test this use fits typesAndArraySizes of. In this case every array element is considered unique, you have to provide specific expectations for every element, but those expectations are allowed to differ. For example, we expect Martin to have studied and worked at three universities. For the first two an end is known. This isn't the case for the last one.

"The response for Odersky" should "contain the expected types and array sizes" in {
    odersky fits typesAndArraySizes of
        """{
            "firstName": "",
            "name": "",
            "universities":
        [{
            "name": "",
            "begin": 0,
            "end": 0
        },
        {
            "name": "",
            "begin": 0,
            "end": 0
        },
        {
            "name": "",
            "begin": 0
        }]
        }
        """
}

One more common situation exists with arrays. Often the array elements are not sorted. Although we expect the array to contain a specific element, we don't know it's position. For this situation containsElementFitting values/types/typesAndArraySizes of exists.

package org.scalawebtest.documentation

import org.scalawebtest.core.IntegrationFlatSpec
import org.scalawebtest.json.JsonGauge
import play.api.libs.json.{JsLookupResult, JsValue, Json}

class ContainsElementFittingSpec extends IntegrationFlatSpec with JsonGauge {
    path = "/jsonResponse.json.jsp"
    def dijkstra: JsValue = Json.parse(webDriver.getPageSource)
    def universities: JsLookupResult = { dijkstra \ "universities" }

    "The universities array" should "contain an element with the expected types" in {
        universities containsElementFitting types of
            """{
                | "name": "",
                | "begin": 0,
                | "end": 0
                | } """.stripMargin
    }
    it should "contain an element with the expected values" in {
        universities containsElementFitting values of
            """{
                | "name": "Technische Universiteit Eindhoven",
                | "begin": 1962,
                | "end": 1984
                | }""".stripMargin
    }
}

Because of its simplicity we prefer play-json over Argonaut and Circe. A good alternative to our JSON module is cornichon. Cornichon requires your tests to be a bit more specific then the ScalaWebTest ones, but it provides nice ways to do so. As cornichon uses Akka Http instead of HtmlUnit to retrieve the JSON response, it is only partially compatible with ScalaWebTest. They might coexist in the same codebase and both are based on ScalaTest, but cornichon can't make much use of the boilerplate reduction provided by our IntegrationSpec.

Extending ScalaWebTest

One of the goals for ScalaWebTest is to make it as as easy as possible, to extend it. Therefore the foreseen extension points are a part of the documentation.

Create a custom matcher

One of the core extension points for gauge testing, are the matchers. ScalaWebTest uses two types of matchers, one for text and one for attributes. You may create your own TextMatcher or AttributeMatcher by extending those traits. You can then add the your own matcher, by prepending to the list of textMatchers or attributesMatchers in the Matchers object.

Here is a full example, of how to implement your own Matcher

Create a custom login

If the available login mechanisms don't work with your web application. No worries, simply create your own implementation of the Login interface and extend it in your base trait. Ideally you then contribute it back to the project via pull request.

The FormBasedLogin provides a good example on how to implement your custom login

Create your own module

If you implement multiple web applications with the same framework, you might want to create a framework specific module. This allows to share framework specific testing code among multiple projects. You can start using it within your company/project and share it later with the rest of the community by creating a pull request. If you provide your own module to ScalaWebTest, we ask you to help maintaining it.

Internal Structure

IntegrationSpec

The IntegrationSpec is the base trait, which all testing style specific traits extend. When writing a spec you should not extend IntegrationSpec, but one of the testing style specific traits, such as IntegrationFlatSpec

The base trait extends the following traits

  • Webbrowser - Selenium DSL for ScalaTest
  • Suite - encapsulates a conceptual suite of tests
  • BeforeAndAfterEach - provides beforeEach and afterEach test hooks
  • BeforeAndAfterAll - provides beforeAll and AfterAll tests hooks
  • WebClientExposingHtmlUnit - wrapper for HtmlUnitDriver, which exposes the WebClient
  • IntegrationSettings - a set of fields to configure IntegrationSpec
  • Eventually - provides eventually function

Integration_Spec

There is a complete set of test style specific base traits. Choose a testing style and then extend the according style specific Integration_Spec trait to create your test.

This traits extend the following traits

  • _Spec i.e. FlatSpec - provides the testing style specific DSL
  • IntegrationSpec - integrates ScalaTest and Selenium with additional features from ScalaWebTest
  • Matchers - provides DSL for assertions using the word should
  • Inspectors - provides nestable inspector methods that enable assertions to be made about collections

_SpecBehavior

One way to share assertions between testing suites, is to use behavior. A behavior contains a set of assertions. You can then use X behaves like SomeBehavior within the suite. Behavior and Suite have to use the same testing style. Therefore ScalaWebTest builds its style specific base traits on the style specific behavior traits. The following traits, which are part of the Integration_Spec, are in fact inherited from the behavior trait

  • _Spec i.e. FlatSpec
  • Matchers
  • Inspectors

Gauge

The Gauge provides the methods fit, fits, doesnt fit and not fit. All of them are used to verify if the current webpage matches the given gauge specification. To do so, the gauge, searches for elements, as defined by the specification, in the current webpage. Elements are found by element name and containing classes. All elements which fulfill the search criteria, are considered candidates. Candidates are then verified for correct attributes and textcontent, using Matchers, next they are verified by checking whether their children match. As soon as something doesn't match a Misfit is reported. If no candidate matches all criteria, the gauge doesn't fit. While verifying the candidates, a list of Misfits was acquired, the gauge will only report the most specific of all Misfits.

Misfit

A Misfit is a container for an error message, when something didn't fit a given Matcher. It contains an error message and a relevance. The deeper the current check in the gauge specification is, the higher its relevance. In the end only the Misfits with the highest relevance will be part of the error message.

Matchers

When working with gauges for testing, Matchers are used to test attributes and text content of elements. By default the following Matchers are available.

  • Default Matcher: tests whether an attribute or text exactly matches a given String
  • Contains Matcher: tests whether an attribute or text contains a given String
  • Regex Matcher: tests whether an attribute or text matches the given Regex

All Matchers, which are available by default, can be used for attributes and text. When creating a new Matcher, one is not forced to implement for both. For each matcher type a specific trait exists. Extend AttributeMatcher or TextMatcher or both, when creating your own Matcher

When creating a custom Matcher the most important thing is to provide a detailed Some(Misfit) in case the Matcher doesn't match. In case of a match, the Matcher simply returns None.

Login

The Login trait is very simple. It only defines what the username and password field should be called. It is used to mark specific implementations as Login and to assert that username and password are consistent over all implementations. Depending on the authentication process for which a Login trait was implemented, it might make sense to overwrite the login function from IntegrationSpec.

WebDriver

ScalaWebTest uses a custom web driver, more precisely a wrapper for the HtmlUnitDriver. Thanks to this wrapper, we can expose the WebClient to enable more control over the WebClient. We also rely on the HtmlUnitDriver, because it has the option to compare the order of html elements within the document. This is not available with all WebDriver implementations, but crucial for the implementation of Gauge.

WebDriverConfigFixtures

The WebDriverConfigFixtures provide the option to execute a closure with a specific configuration. After the closure is executed, the WebDriver configuration will be reverted to what it was before the call of the fixture.