If you've been following my blog in the last few years, you might have noticed that I kind of like testing & automation. Mostly, I'm working with .NET and TypeScript in cloud and backend services, so plain old integration testing gets me pretty far. Recently, however, I've done a bit more on the frontend side and I wasn't happy with the way I setup my testing.

Basically, I had separate tests for the frontend and backend, with just very rudimentary end to end tests that were run on a pre-production slot in Azure. If these passed, the slot was switched live and a new version was deployed.

This approach worked, but had three big drawbacks:

1. It was slow. The app had to be built and deployed to the cloud. Results from this could take up to 10 minutes until I had feedback
2. It couldn't be run locally, since it really was more of a pre-deployment check than an actual test. This also meant I didn't get feedback on all code changes, only on those in a branch that got deployed
3. Testing was limited to operations that didn't change any production data, such as creating new users or assigning roles

It was a really uncomfortable situation to be in, and the solution was clear: Automate it, dockerize everything and have full end to end tests run on every single commit in a headless Chrome browser. It took me a bit to get started with all this, so in this article, I'll describe in detail how I did it.

What are end-to-end tests?

In automated testing, there are different types or categories of tests you write. The definitions are always a bit fuzzy, everyone has a different opinion on them. The most fundamental tests are called unit tests, checking the behavior of a small, independent unit. Integration tests verify that multiple components in conjunction work as expected, usually with a longer run time than unit tests. Finally, automated end-to-end or e2e tests are performed on the whole application, from a users perspective. Think browser or UI automation.

You should call them what's most appropriate in your situation. For example, is an automated test checking the REST API endpoints of your backend, hooked up to a local database, still an integration test or already end-to-end? I'd say it's an integration test, since your users typically don't access the API directly. Except when they're developers themselves and do... So, don't get caught up in semantics😊

For this article, end-to-end tests are tests that simulate the users behavior in a browser, they test both the frontend code and the backend together, simulating real user actions on your production system.

Overview

This post won't go into some details that are out of the focus here. I'll assume you have some understanding of using Jenkins to automate your tests (or any other CI service, like Azure DevOps or AppVeyor), are familiar with Docker as well as basic unit testing approaches and frameworks such as xUnit. Additionally, I'm using the NUKE build system to orchestrate the setup, but simple bash or PowerShell commands will be perfectly fine for you.

Docker Setup

The easiest way for me was to run all required services in Docker Compose. Let's take a quick look at the following script:

My setup consists of three images:

1. Dangl.Identity, which is the app under test. It's an OpenID server with user, rights & role management and related services, built on IdentityServer4, that we're using at DanglIT
2. Dangl.Icons, which is a really small service that's similar to Gravatar, it essentially generates user icons
3. SQL Server, to match the setup of the production environment

The setup is app specific and likely quite different in your case, but the gist is that we'll have a full featured app with all required services, mirroring the production environment, available at http://localhost:44848 when running this composition.

Test Configuration

To actually perform the test, a mix of technologies is used: While xUnit is the test framework itself, I'm using Selenium for browser automation. A minimal .csproj looks like this:

The main object you'll be using for e2e tests is the ChromeDriver. It's your context for interacting with the browser and really easy to setup:

You'll notice that I check if the debugger is attached to check if I'm running it locally in Visual Studio, otherwise I set it to headless and simulate a Full HD resolution. This example class makes use of xUnits Fixture, but that's just for performing some initialization for the test environment, you can skip that.

Test Implementation

Now you're ready to write tests! I'm usually separating them again at this point, one category represents simple, minimal tests while the other covers user stories, e.g. complex tasks from registration to login to performing some tasks on the site.

Let's take a look at a simple test, checking if the UI disables the Register button when the user missed filling in her username:

The code should be simple to follow, I hope😊 More complex tests aren't much different, it's just more code. You'll generally notice a trend here - while unit tests require very little code, e2e tests quickly grow into dozens of lines if you're doing more complex actions.

Running your Tests

Depending on how you're automating your pipeline, you're using bash or PowerShell or something similar. I'm a huge fan of the NUKE build system, so that's what I naturally use for all my automation needs. Let's take a look at the implementation:

Examining it closely, it does just two things: It runs docker-compose up to start the Docker environment in a non-blocking way and then executes dotnet test. xUnit test results are both printed to the console and saved as testresults.xml for further analyzing.

Running Tests in Jenkins

The Jenkins pipeline is configured via a Docker E2E Tests build step. It's configured to run on a Linux node, executes NUKE with the E2ETests target and then reports the test results:

You'll see it's worth the effort when a nice, all-green stage view greets you in Jenkins:

Summary

While there are quite a few steps involved in performing proper end-to-end testing, the payoff is worth it: You're able to validate a mirror of your production setup in exactly the same way your users are using the app. While this post has just scratched the surface of what you can do, it hopefully gave you a good starting point to implement your own strategy.

Happy testing!