Visual Testing With Cypress

Setting the Scene 🖼️

It’s Friday evening, the sun is shining 🌞, you’ve just deployed the latest UI changes to production. Just as you’re about to put those feet up and order that takeaway…

Those shrieking voices coming from all angles 😨;

" The website isn't loading! "

" The screen is completely blank!! "

" We can't see anything!! "

Clearly, something is amiss here. Let’s check the build…

Unit testing passed ✅
Component testing passed ✅
Even the E2E testing passed! ✅

It must be the backend guys right? Us frontend devs never make any mistakes. 😀

Time to open up those dev tools and take a deeper look.

No network errors, no worrying console output. Wait…

What’s this…

/* Todo: debug code - remove before deploy */
body {
  transform: scale(0.1);
  opacity: 0.1;
}

There it is! That debug code we added that we knew we would remove since we added a Todo comment! Now the entire document body has been shrunk 1/10th in size and its opacity has been set to 0.1 😧. Meaning all the content is there, just virtually invisible to our eyes!

But wait… Why did the tests pass? 🕵️

Most of the time the tests are just grabbing an element and verifying the visibility (opacity > 0) or asserting some text but never actually visually checking the page. We can’t automate our eyesight after all!

Or can we…

While this was an extreme example to demonstrate the problem, it is still a very plausible scenario. This kind of issue could occur in a small but extremely important section of your application today. Like a “Submit” button, for example.

Let’s take a look at how we can use visual testing to prevent these kinds of issues from happening.

Visual Testing 🧪

Overview 📓

We’re going to use Cypress along with a free community plugin cypress-plugin-snapshots.

Cypress is going to be used to control the browser while the additional plugin, cypress-plugin-snapshots, will be used to perform the visual testing.

There are a vast number of plugins available for performing visual testing with Cypress, ranging from free to paid, so be sure to check them all out before deciding.

The Test 📝

We’re going to create 2 very simple tests for our demo. They will both be testing the same thing, with one being a standard Cypress test and the other being a visual test.

The test will involve connecting to https://philip-griffin.com, clicking the first blog post and ensuring that the post title exists.

See the manual perspective of this below.

Let’s get coding! 💻

Prerequisite

If you don’t have Node.js installed head over to nodejs.org to get set up before continuing.

Laying the Foundation ⚙️

Let’s get a package.json, no questions asked by typing the below command into your chosen terminal.

npm init -y

We’ll install our dependencies next.

npm i cypress@7.6.0 cypress-plugin-snapshots@1.4.4 -S

Note the fixed versions here (this is purely to improve compatibility with this tutorial in the future).

We’ll also add a command to our package.json to make things a little easier. Drop the below into the script section.

"start": "cypress open"

Now we can fire up Cypress with an npm start.

npm start

We’re going to head into the newly created Cypress folder in our workspace and create a file under the integration folder named demo.spec.ts.

We can populate this file with our first test. This will be a regular Cypress test that verifies the blog post title.

// demo.spec.js
describe("Demo Testing", () => {
  it("can open a blog page", () => {
    cy.visit("localhost:1313/");

    cy.get(".post-entry:first").click();

    cy.get(".post-title")
      .should("be.visible")
      .contains("Web scraping to create an api in 3 minutes!");
  });
});

Now we will select the file from the Cypress file selector that popped up after we ran our npm start command. See below.

Great! We have a passing test! Let’s take a look at what happens if we set the .post-title to be:

.post-title {
  transform: scale(0.1);
  opacity: 0.1;
}

Even though the title seems invisible, the test still passes. This is not ideal.

The Visual Test 👁️

Let’s configure the plugin.

Jump over to the cypress/integration/plugins/index.js and replace the contents with the following.

const { initPlugin } = require("cypress-plugin-snapshots/plugin");

module.exports = (on, config) => {
  initPlugin(on, config);
  return config;
};

Add the following import to cypress/integration/support/index.js.

import "cypress-plugin-snapshots/commands";

For now we will revert the CSS changes we made.

Let’s add a second test to our demo.spec.js.

it("visually check a post", () => {
  cy.visit("localhost:1313/");

  cy.get(".post-entry:first").click();

  cy.get(".post-title").toMatchImageSnapshot({ imageConfig: { threshold: 0 } });
});

After running this test it will pass and an image will be saved in the workspace. Check it out below.

This image will now be our baseline image to be used to compare future test runs against. Let’s reintroduce our bad CSS and re-run the tests.

We now have a failing test and we can also get a look at the expected and actual image results. There is also an image diff available so you can see an overlay of both images with highlighted areas where potential differences lay.

Success! 👍

Conclusion

Visual testing can be used to test an entire page, individual elements, on a component by component basis and so on. Like any tool developers utilise, the usage will depend on your own use cases. Large screenshots can easily become brittle over time so consider your tests carefully.

Remember to check out all the available Cypress visual testing plugins to learn more!

Source code: Github