Say TaTTa to your TuTTu

Anti-patterns in automation

About me...

- www.mwtestconsultancy.co.uk

- @2bittester

- linkedin.com/in/markwinteringham

Anti-patterns in our Automation

TuTTu

Automating a form check


  Webdriver driver = new Webdriver();
  driver.get("http://some.site.com/form");

  driver.findElement(By.id("firstname")).sendKeys("Mark");
  driver.findElement(By.id("lastname")).sendKeys("Winteringham");
  driver.findElement(By.id("submit")).click();

  String title = driver.getTitle();
  assertThat(title, "The second page of the form");

Where is the anti-pattern?

Checking that each browser renders the form correctly so it can be filled in by a human

Checking that each browser takes us to the same page after submission

Checking that each browser is using JavaScript correctly to create the form

Checking that each browser renders the form correctly so it can be filled in by a human

The anti-pattern here comes from the assumption around the perception of ‘rendering’ and WebDriver. WebDriver allows us to query a DOM that is being built in a browser so this means we CAN check to see if the DOM contains the HTML we require to submit our form but we CAN’T check if the HTML is rendered in a way that a human could have trouble submitting the form (due to CSS or accessibility issues). Which means that running the check across multiple browsers is redundant. It bears little influence on how the backend is serving the HTML since all browsers are capable of injecting HTML into a DOM (if it couldn’t no-one would use that browser!)

As a side note, WebDriver can inform us if an element can’t be reached, say for example if the HTML is missing but if the HTML is missing the issue lies with what is serving the HTML and not the browser (Unless it is created by JavaScript but we will come back to that). Therefore, it would be better to create an API level check with different User agents (if necessary) to determine if the correct HTML is returned by the backend, an approach which is cheaper and more stable.
Checking that each browser takes us to the same page after submission

The anti-pattern here is the assumption that each browser might handle a redirect from a backend differently (Which they generally don’t, again no-one would use them if they did). An API level check with different User agents (again, if necessary) to determine that the correct redirect path is returned would suffice.
Checking that each browser is using JavaScript correctly to create the form

So what if the form was created by JavaScript and not by the backend? There is a benefit to running the check across different browsers since they all have different JavaScript engines that behave in different ways, and we want to ensure that our form creation works with all of them.

However, the anti-pattern still appears in the submission of form and the assertion we’re being redirected to a new page since this is all being handled by the backend. We should consider breaking the check into two separate checks. A WebDriver based check for checking the JavaScript renders the HTML, and an API level check for checking the backend returns us a redirect.

Testing the UI or Testing Through the UI

Postman

Rest-assured


  given().
    parameters("firstName", "Mark", "lastName", "Winteringham").
  when().
    post("/some.site.com/form").
  then().
    statusCode(304)
    header(equalTo("http://some.site.com/form/2"));

'Checking validation is triggered on the form before submission'

Testing the UI or Testing Through the UI?

'Checking a record is created in the database on form submission'

Testing the UI or Testing Through the UI?

'Checking an edit version of the form contains previous values'

Testing the UI or Testing Through the UI?

'Checking a pop up appears when clicking a help icon next to a form field'

Testing the UI or Testing Through the UI?

'Checking the form uploads an attached PDF correctly'

Testing the UI or Testing Through the UI?

TaTTa

Rest-assured


  given().
    parameters("firstName", "Mark", "lastName", "Winteringham").
  when().
    post("/some.site.com/form").
  then().
    statusCode(200)

A Web Service example

http://adrianmejia.com/blog/2014/10/01/creating-a-restful-api-tutorial-with-nodejs-and-mongodb/

A '200' response

HTTP/1.1 200 OK
Content-Type: text/html
Connection: keep-alive

<body>
  <p>An error occurred</p>
</body>

Testing the API or Testing Through the API

Rest-assured


  given().
    parameters("firstName", "Mark", "lastName", "Winteringham").
  when().
    post("/some.site.com/form").
  then().
    statusCode(200)
    body("html.body.p", isNot(equalTo("An error occurred")))

Postman

Unit check


  public void testStorage() {
    Datastore store = new Datastore();
    Boolean result = store.createNewRecord("Mark", "Winteringham");

    assertTrue(result);
  }

'Checking a third party Web service can be accessed'

Testing the API or Testing Through the API?

'Checking a Web service evaluates data and returns a result'

Testing the API or Testing Through the API?

'Checking a Web service returns a record in the correct structure'

Testing the API or Testing Through the API?

'Checking a Web service handles overflow data such as an Integer'

Testing the API or Testing Through the API?

'Checking a redirect is returned after an unauthorised request is made'

Testing the API or Testing Through the API?

Questions?

Training

Testing Web Services
Friday 23rd September @ The Studio, Manchester

Taking Automated Checking Beyond WebDriver
Wednesday 30th September @ Manchester

Thank you

www.mwtestconsultancy.co.uk

@2bittester

linkedin.com/in/markwinteringham