Testing for Visual Regressions with Percy

Celý článek

While this post is not sponsored (it is based on Paul’s own personal experience), we have worked with Percy before on a sponsored video, that also goes through the process of setting up Percy on a site. It’s fascinating, powerful, useful stuff and I highly recommend checking it out.

Let me set the stage:

1. You've pushed some code and all the meticulously unit tests you wrote pass. Fantastic! Let’s get this branch into QA.
2. You receive a Slack message from your QA team asking why a button is now floating off the screen?



3. "But... I didn't touch any code in that part of the application," you think to yourself.
4. Then you remember you did change some CSS.
5. Panic! What else has changed in the UI? Does this affect iPad? Will Firefox behave differently than Chrome? What about mobile?

This is a very common scenario many of us face when building large-scale applications that deal with different screen sizes and browsers. It’s a Herculean task to test UI for each and every change to code.

What now, throw in the towel and move to the mountains? Thankfully, no. We have Percy to help save the day! And it’s really the best friend we have for testing unexpected outcomes that impact design and layout. Percy has become an indispensable part of my development stack and I convinced CSS-Tricks to let me share some things about it that have made my code stronger and helped prevent errors from shipping.

Plus, it integrates well with other tooling and is a relative breeze to set up. So hang with me a bit as we walk through what Percy is and how to leverage it for your projects.

So, what exactly is Percy?

According to Percy’s site, it’s an “all in one visual review platform."

I’ve found that holds true. What it boils down to is that Percy provides a way to test visual regressions. That’s pretty awesome if you think about it. Many changes to a codebase — especially working with CSS — can introduce breaking changes to a site’s design. If you’ve ever inherited a large legacy stylesheet, modified a class, and hit Save, then you probably have a great idea of how nerve-wracking that can feel. Percy’s goal is to provide confidence in those types of situations where it’s difficult to know all of the UI that depends on the same line of code.

Excited? Let's get started.

Setting up an example site

Let’s set up a little site that Percy can hook into and test some UI we’re going to make together. These days, this couldn't be easier, thanks to Gatsby and Netlify. It is way beyond the scope of this article to do a deep dive into these technologies, but rest assured, they are wonderful as well and can get us online without a bunch of server setup.

Head over over to Netlify templates and click the "Deploy to Netlify" button, which will set up a git repo on your GitHub account and also deploy the app using Netlify.

After completing the setup steps, we should get something like this (after the site is deployed):

Magically, our site is now live! We will use this to get to grips with Percy.

Using CircleCI for automated testing

Percy works best in a continuous integration (CI) environment that will trigger testing based on an action. We will use CircleCI to make it happen by integrating with the example site’s GitHub repo and running the builds, allowing us to test every commit.

The first thing we need to do is clone down our new repo on GitHub, I clone mine as follows:

git clone https://github.com/PaulRyanStitcherAds/gatsby-starter-netlify-cms.git

With our repo now cloned, we can head over to CircleCI and sign up with a GitHub account.

We now need to add our project, so click "Add Projects" in the side navigation and you should see a screen like the following screenshot. Find the project we just cloned and click “Set Up Project."

In the Set Up Project area, we want to select Linux as our operating system and Ruby as our language (pery-cli is in Ruby). Here are the rest of the steps for this part:

CircleCI tells us that we need a .circleci directory and that we need a config.yml file in it. Create the following structure within your project.

CircleCI offers a configuration snippet to copy and paste for our configuration, but it is far too verbose for us; we only need a simple config.yml file.

Go ahead and use the following snippet. You’ll see that we install the percy-cli gem along with Percy in our tests:

version: 2
jobs:
  build:
    docker:
      - image: circleci/ruby:2.4.1-node-browsers
    working_directory: ~/repo
    steps:
      - checkout
      - run:
          name: install dependencies
          command: |
            npm install
            gem install percy-cli
      
      - run: 
          name: run our tests
          command: |
            npm run build
            percy snapshot public

This config is all we need.

At first, It took me a while to figure out why my build was failing and turned out I was trying to install percy-cli as an npm module. Yikes!

We now have the CircleCI configuration set up so finally we can start using Percy!

As a sanity check, comment out the run our tests step above and push your code to the master branch.

Now click the "Start building" button which will use the configuration you just pushed to create a build. Here's what you should see in the workflows section:

From here on out, CircleCI will create a build for us whenever we do a push.

Hooking Percy up to CircleCI

A Percy account is needed to use the service. Head over to Percy’s site and sign up with your GitHub account.

Once signed up, you can create a new organization (if you don't already have one) and call it whatever you want.

Next thing to do is add a project to the organization. It’s probably a good idea to call the project something matching the name of the repo so that it’s recognizable later.

Now we need to add a Percy token to CircleCI. Percy tokens are located under "Project Settings."

Alright, let’s add the token in CircleCI in Environment Variables under “Build Settings." You can find Build Settings by clicking the gear icon beside your project in the workflows section.

It’s time to run Percy! If you commented out the run our tests line in the config file earlier, then remove the comment because that command will run Percy. Push to master again and then head over to the Percy app — we will see Percy has started its own build for creating snapshots. If all goes well, this is what we should get:

If you click on this build, then you can see all the screens Percy has snapped of the application.

You might be wondering what the deal is with that empty left column. That's where the original screen normally is, but since this is the first test, Percy informs us that there are no previous snapshots to compare.

The final thing we need to do to wrap up this connection is link our repo to Percy. So, in Percy, click “Project Settings" then click on the “install an integration" link.

Select the organization and hit install for all repositories:

Finally! We can link to our repository.

Unlocking the true power of Percy

Since we now have everything set up, we can see how Percy can be used in a code review workflow! The first thing we will do is create a branch from master. I’m calling my branch "changing-color."

Go to the /src/components/all.sass file, change Line 5 to use the color pink, then push the change to the repo. This is what we’re going to evaluate in our visual test.

Create a pull request for the change in GitHub.

CircleCI is carrying out checks for us but the one we are focused on is the Percy step. It may need a minute or two for Percy to pop up:

Percy is letting us know that we need to review changes we made, which in this case, is the change from red to pink. We can see the impact of that change:

Although the changes on the right are red, that is highlighting the areas that have been changed to pink. In other words, red is indicating the change rather than the actual appearance.

We can give this a quick glance and then click the “Approve" button which will give us a green tick on GitHub indicating we’re free to merge the pull request.

This is the true power of Percy: allowing us to see the impact of a small change and giving us the option to approve the changes.

Fantastic! We have now taking a tour on how to set Percy up in our projects along with how to integrate CircleCI. I really hope this will save you many headaches in the future when managing your UI.

The post Testing for Visual Regressions with Percy appeared first on CSS-Tricks.