Dangl.Blog();

Setting the Language in Chrome Headless for E2E Tests

Thu, 01 Jun 2023 10:17:26 Z

For End-to-End (E2E) testing of web apps, the headless mode of Chrome is a great choice. We're using it in our projects to ensure some true end to end test coverage, often for stories like happy paths for important actions. E2E tests are usually quite cumbersome to write, and tend to be brittle during upgrades, but they give you a lot of confidence when doing automated deployments, and free up valuable developer time from having to do repeated, manual tests and verifications. I've blogged before about them, for example hows to instrument them via Playwright or how you can use Selenium in a Docker container.

One thing that's been missing is the ability to set a language for the headless Chrome instance when testing. Especially if you have multilingual apps that automatically detect the users language, you might experience problems in testing when you're checking for text contents or trying to find Html elements via their labels. There was a long open issue with Chrome, where the language supplied via a command line argument was simply not set in headless mode.

But, this has changed with the new headless option for chrome!

Finally, when you start headless via --headless=new, you can simply set the language with another switch: --lang=en. This makes testing so much easier, and allows to provide a consistent developer experience, no matter the local language settings.

Happy testing!

Changing the Order of Parameters in Swagger / OpenAPI Documents when using NSwag to Generate the Swaggerfile

Sun, 16 Apr 2023 20:15:53 Z

Our AVACloud SaaS product has been available for a few years already. While that's a great achievement in itself, it also comes with drawbacks: The API is used by a lot of customers by now, which requires us to keep it stable and not introduce breaking changes or incompatibilities with existing implementations. Move fast still applies, but the break things part would now be a problem.

So, we've recently updated one endpoint to support additional parameters. It's a simple REST endpoint that takes a file, some options, and does a conversion. In the.NET backend, the controller receives three parameters: The file, and two options objects for the import respectively export configuration. That maps nicely, and introducing additional parameters is as easy as adding a new property to a view model used for the bindings. Many clients generated via Swagger can follow the same approach of using options objects for the client interface, thus no problem there.

However, the objects are transported as query parameters. This means that from a Swagger perspective, we've now added two new query parameters. This can cause problems when the new parameters are not sorted to the end, thus generating a new client version would break existing implementations and require attention and a rewrite from every consumer of the API.

Luckily, we're using NSwag in the backend, which allows a great deal of customization. When we set up Swagger generation, we can simply inject custom code in the PostProcess callback and sort the parameters however we want.

We're just getting the current parameters, apply our custom sort logic which ensures the new parameters are at the end, and then clear and refill the parameter list.

There are other options available, such as using custom x-position attributes, but I've found that relying on such non-standard features often causes problems later with some client generators. So, let's keep it simple and stupid😀

Happy customizing!

Updating Azure App Service on Linux for Docker via Webhooks from C#

Sun, 16 May 2021 17:43:01 Z

If you're using Azure App Service to run Docker images, you already know how easy it is to get your app running. Azure nicely abstracts everything for you - it may be a bit more expensive than hosting your own Kubernetes cluster on cloud VMs, but it makes more than up for it in simplicity.

Among the cooler features of using Azure App Service to run Docker ist that you automatically get a green/blue deployment. All you have to do is update the Docker image for the App Service and after a few moments, you're running the latest version, without any downtime. This is super convenient when you're deploying Docker to Azure Container Registry, setting up automatic updates is then just a single click.
However, it's a bit more complicated when you're using something else, e.g. GitHub Packages for hosting your Docker containers, since then you're responsible for calling a webhook in Azure to let it know there's a new container version available.

That's in itself not very complicated, but it has one little point that's worth mentionioning, as you see in the code below:

The webhook url in Azure includes an UserInfo part, meaning it's got username:password right there in the url. Most tools will support that out of the box, but in case you're using .NET to send the webhooks, you should be aware that you need to parse that information and use it with Http Basic Authentication.

By the way, for real life implementations, you probably don't want to hardcode the webhook uri in your build script😀

Happy deploying!

Improving ASP.NET Core End-to-End Tests with Selenium Docker Images

Wed, 20 May 2020 13:28:10 Z

Last year, I've blogged about automating end-to-end (E2E) tests with an ASP.NET Core application. Quickly summarized, you're setting up your environment locally via Docker and then use Seleniums ChromeDriver to automate UI tests for your web application. This worked well, but had one big drawback - when you're executing the tests, you need to have Chrome installed on the build agent or on your local machine. While most who do web development probably have Chrome available, it's a bit of a hassle with testing - you need to make sure that your Selenium version matches the current Chrome version.

Luckily, Selenium provides Docker containers with an embedded Chrome and their own RemoteWebDriver. This means to be able to access an automated browser via Selenium, you just have to spin up a container and connect to it. That's pretty great and makes versioning really easy!

Building on the previous post, you just have to follow these steps:

Docker Configuration

I'm using the selenium/standalone-chrome:3.141.59 image and make sure to assign it enough memory, as per the official recommendation.

Test Setup

In the test project, I simply check if the containerized Selenium driver should be used and return a RemoteWebDriver instead of a ChromeDriver. For context, I do sometimes let the tests run with my local version of Chrome and have the debugger attached, so I'm not using the headless mode there but instead can track the actions.

Browser Automation

Finally, here's a simple method that automates the sign up process. You'll note that writing E2E tests is quite cumbersome, because you're really just describing step-by-step what's happening. But you'll be able to test your actual project in real world conditions and have it run automatically, which is pretty much worth it!
You're now able to fully test your app in any environment, all you need is Docker.

Happy testing!

Performing End-to-End Tests for a .NET Core Web API with an Angular Frontend in Docker with Jenkins

Fri, 26 Jul 2019 18:12:04 Z

If you've been following my blog in the last few years, you might have noticed that I kind of like testing & automation. Mostly, I'm working with .NET and TypeScript in cloud and backend services, so plain old integration testing gets me pretty far. Recently, however, I've done a bit more on the frontend side and I wasn't happy with the way I setup my testing.

Basically, I had separate tests for the frontend and backend, with just very rudimentary end to end tests that were run on a pre-production slot in Azure. If these passed, the slot was switched live and a new version was deployed.

This approach worked, but had three big drawbacks:

It was slow. The app had to be built and deployed to the cloud. Results from this could take up to 10 minutes until I had feedback
It couldn't be run locally, since it really was more of a pre-deployment check than an actual test. This also meant I didn't get feedback on all code changes, only on those in a branch that got deployed
Testing was limited to operations that didn't change any production data, such as creating new users or assigning roles

It was a really uncomfortable situation to be in, and the solution was clear: Automate it, dockerize everything and have full end to end tests run on every single commit in a headless Chrome browser. It took me a bit to get started with all this, so in this article, I'll describe in detail how I did it.

What are end-to-end tests?

In automated testing, there are different types or categories of tests you write. The definitions are always a bit fuzzy, everyone has a different opinion on them. The most fundamental tests are called unit tests, checking the behavior of a small, independent unit. Integration tests verify that multiple components in conjunction work as expected, usually with a longer run time than unit tests. Finally, automated end-to-end or e2e tests are performed on the whole application, from a users perspective. Think browser or UI automation.

You should call them what's most appropriate in your situation. For example, is an automated test checking the REST API endpoints of your backend, hooked up to a local database, still an integration test or already end-to-end? I'd say it's an integration test, since your users typically don't access the API directly. Except when they're developers themselves and do... So, don't get caught up in semantics😊

For this article, end-to-end tests are tests that simulate the users behavior in a browser, they test both the frontend code and the backend together, simulating real user actions on your production system.

Overview

This post won't go into some details that are out of the focus here. I'll assume you have some understanding of using Jenkins to automate your tests (or any other CI service, like Azure DevOps or AppVeyor), are familiar with Docker as well as basic unit testing approaches and frameworks such as xUnit. Additionally, I'm using the NUKE build system to orchestrate the setup, but simple bash or PowerShell commands will be perfectly fine for you.

Docker Setup

The easiest way for me was to run all required services in Docker Compose. Let's take a quick look at the following script:

My setup consists of three images:

Dangl.Identity, which is the app under test. It's an OpenID server with user, rights & role management and related services, built on IdentityServer4, that we're using at DanglIT
Dangl.Icons, which is a really small service that's similar to Gravatar, it essentially generates user icons
SQL Server, to match the setup of the production environment

The setup is app specific and likely quite different in your case, but the gist is that we'll have a full featured app with all required services, mirroring the production environment, available at http://localhost:44848 when running this composition.

Test Configuration

To actually perform the test, a mix of technologies is used: While xUnit is the test framework itself, I'm using Selenium for browser automation. A minimal .csproj looks like this:

The main object you'll be using for e2e tests is the ChromeDriver. It's your context for interacting with the browser and really easy to setup:

You'll notice that I check if the debugger is attached to check if I'm running it locally in Visual Studio, otherwise I set it to headless and simulate a Full HD resolution. This example class makes use of xUnits Fixture, but that's just for performing some initialization for the test environment, you can skip that.

Test Implementation

Now you're ready to write tests! I'm usually separating them again at this point, one category represents simple, minimal tests while the other covers user stories, e.g. complex tasks from registration to login to performing some tasks on the site.

Let's take a look at a simple test, checking if the UI disables the Register button when the user missed filling in her username:

The code should be simple to follow, I hope😊 More complex tests aren't much different, it's just more code. You'll generally notice a trend here - while unit tests require very little code, e2e tests quickly grow into dozens of lines if you're doing more complex actions.

Running your Tests

Depending on how you're automating your pipeline, you're using bash or PowerShell or something similar. I'm a huge fan of the NUKE build system, so that's what I naturally use for all my automation needs. Let's take a look at the implementation:

Examining it closely, it does just two things: It runs docker-compose up to start the Docker environment in a non-blocking way and then executes dotnet test. xUnit test results are both printed to the console and saved as testresults.xml for further analyzing.

Running Tests in Jenkins

The Jenkins pipeline is configured via a Docker E2E Tests build step. It's configured to run on a Linux node, executes NUKE with the E2ETests target and then reports the test results:

You'll see it's worth the effort when a nice, all-green stage view greets you in Jenkins:

Summary

While there are quite a few steps involved in performing proper end-to-end testing, the payoff is worth it: You're able to validate a mirror of your production setup in exactly the same way your users are using the app. While this post has just scratched the surface of what you can do, it hopefully gave you a good starting point to implement your own strategy.

Happy testing!

Caching Bundled and Minified CSS and JavaScript with the Wrong File Extension in the Free Tier on CloudFlare with Umbraco

Fri, 11 Jan 2019 19:47:44 Z

I've been using CloudFlare for quite some time now, on this blog and on many of my other websites. It's a great service that offers lots of value even in it's free tier. Most useful, for me, is it's built-in CDN (Content Delivery Network) functionality. This means basically the following:

This blog here is hosted on an Azure server somewhere in Europe. That's great for people living near, but bad for all the visitors outside of Europe. Or even just a few hundred kilometers away from my server. It's bad in the sense that web requests take a lot of time when the physical locations of client and server are continents apart. Even worse, when the first request to my main page is finished, a second round of requests retrieves images, CSS and JavaScript files. That's where CloudFlare comes into play: They'll cache all my assets on their servers, of which they have many around the world. So they're kind of a proxy between this blog and it's visitors and they optimize delivery by caching my content, meaning typical page loads from North America, for example, no longer take 8 seconds to complete. Most web requests don't even hit my server, at all.

I'm using the free tier on this site, which works well enough but has some limitations. For example, they're only caching a handful of file extensions, so .js and .css files are cached but, .axd are not. Since I'm using Umbraco on this blog, the built-in ClientDependency package makes sure that all my JavaScript and CSS files are bundled and minified in production builds. The package internally works by routing the requests through a custom handler via a route similar to /DependencyHandler.axd?s=h4sh&t=JavaScript. Bundling and minification is great and what we want, but we also want CloudFlare to cache the requests. We'll have to get Umbraco to serve it via urls with the correct file extensions then!

First, you need to create a custom renderer that outputs the desired file endings in your website links to the bundles:

This new renderer should be registered in your ClientDependency.config file. Simply replace the old one:

Now, for the magic to work, we're using a rewrite rule in IIS' web.config file to map requests that come to either DependencyHandler.js or DependencyHandler.css internally to the route DependencyHandler.axd:

Happy caching!

Update, one week after the original post: With this approach, I managed to increase the cached bandwith via CloudFlare from about 20% to over 60%! Most of the uncached content now is the dynamic content generated by the GAEB converter. Virtually everything else doesn't cost me any more bandwith!

Deploy to Azure App Service with no Downtime by using Staging Slots

Tue, 13 Nov 2018 19:20:07 Z

My company provides the AVACloud webservice as a SaaS offering. It's used to integrate tendering workflows for the German GAEB standard in my customers' apps. In fact, it's also used to power the free GAEB Converter here on this blog.
When running software as a service, it's mandatory that you have no downtime - this means for deployment scenarios, you can't afford to shut down your app, update it and slowly restart it. In fact, even warmup times in the range of 10 to 20 seconds are sometimes considered service interruptions, and they certainly don't make your service look professional and your customers happy!

Traditionally, you'd host multiple instances of your app with a central load balancer in front of it. When you update the app, you drain single instances until they have no more open connections, update them and repeat this process until all your nodes are at the latest version. This works fine, but is often a bit tedious to manage, especially from a DevOps perspective. Luckily, Azure App Service has the concept of deployment slots built in. You're only managing a single site, deploy to a staging slot, wait for it to warm up and then seamlessly swap it with the production instance. This can even be done automatically, so all you have to do is deploy to your staging slot and you're good to go.

However, sometimes you want to do some integration tests after deployment to ensure that the site works correctly, so you can't use the auto swap feature. In the following snippet, I'll show you how to use the Microsoft.Azure.Management.Fluent package to manage your App Services via C#.

To ensure that the app is healthy and working as expected, I'm doing a simple check on the https://avacloud-api.dangl-it.com/status endpoint. The app would report an unhealthy status if anything's not right. In your real life scenario, you might do a bit more than checking a single http response.

Tip: If disaster strikes, keep calm and remember that with deployment slots in Azure, you're just one single action away from doing a rollback to your latest working versions!

To set this up, you need to create a service principal in Azure with access rights to your App Service instance. Just follow the official docs, they're describing it in great detail. Just remember to follow best security practices and only allow access to a single App Service, not your whole subscription!

If you're interested how to do more DevOps & build automation directly with C#, have a look at the introduction for NUKE.

Happy swapping!

Different Response Schemas in ASP.NET Core Swagger API Definition with NSwag

Sat, 27 Oct 2018 19:46:52 Z

I'm using automatically generated Swagger API definitions in some of my projects, such as AVACloud and Dangl.Identity. Against best practices, I've got some controllers that might return different types of response objects. For example, I've developed LightQuery to enablie pagination and sorting in ASP.NET API controllers with the single addition of a [LightQuery] attribute.
This means that, depending on the request, the response might be a simple JSON array of objects or it might be paginated and wrapped in a response container with information about the current subset, similar to how OData structures its responses.

On top of that, I'm a big fan of doing as little work as possible. In the context of web APIs, this means I'll use NSwag on the server side to automatically generate Swagger API definitions, which I subsequently use to drive the automatic generation of, for example, .NET or TypeScript clients for libraries and web frontends. To facilitate that, I'm annotating my controllers with a bit of metadata, such as the [ProducesResponseType] attribute.

As you can see, I've used the attribute twice - once to indicate the plain response, a second time to state what the paginated response will look like. Up until NSwag v11.19, this produced correct code - both schemas were included in the action descriptor of the API specification. This meant that my .NET clients did type the response as object and I had to handle it at the consumer side.

However, after upgrading NSwag to v11.20, I've noticed that it was only picking up the first defined response type, thus simply ignoring all others:

For obvious reasons, this is bad - the generated spec did not match with the actual behavior of my server. After doing a bit of digging around, I did not find anything about this change being intentional. However, I've also discovered that you should preferably use the attributes shipped in the NSwag.Annotations namespace.

Swapping the attribute leads to this action definition:

which promptly produces correct API definitions again:

You can spot the difference in the generated .NET / C# client, where it now correctly produces the response as object again:

Finally, to support my striving for having to do as little work as possible, I've added two tests to ensure I'll never run into this specific problem again.

The first test requests the generated Swagger specification from the In-Memory TestHost instance and just ensures that a particular action I know to have two response schemas actually has two schemas. The second one is just a safeguard - I'm using reflection to ensure I do never use two [ProducesResponseType] attributes on the same controller action to define valid responses.

In summary, I'm really happy with automatically generating Swagger specifications, but I'm learning something new about it every week. If you're relying on such generated documents, I'd recommend you have good integration tests for your projects and use them. Changes might come in suddenly when doing package upgrades without you necessarily noticing them early.

Happy coding!

Migrating an Umbraco Site from a Server to an Azure Web App

Fri, 14 Sep 2018 18:38:56 Z

Umbraco is a great choice for a .NET based CMS, in fact, this blog and my professional site both run on Umbraco. It is, however, still running on the full .NET Framework, with ASP.NET Core compatibility far, far out on the roadmap. This means that it's sometimes not as easy to handle as more modern solutions.

But this doesn't stop you from running it in Azure - in fact, as you'll see, it's quite easy to migrate from a regular, Windows Server hosted solution to an Azure Web App, with all the benefits included, such as auto scaling or easy blue-green deployments. My main reason for porting was to consolidate all my production infrastructure in Azure, as well as benefit from the much higher availability in contrast to a server I have to manage. Just last week, my virtual server was down for over an hour due to scheduled updates to mitigate the latest Intel processor flaws.

Migration Steps

You'll have to do three basic steps to migrate your Umbraco site while keeping all the content:

Migrate your SQL database to one hosted in Azure.
Copy your Umbraco media folder, which includes all your uploaded files, to an Azure Blob Storage.
Update your Umbraco repository and configure it to deploy to Azure.

SQL Migration

Microsoft has made it really easy to move On Premise SQL databases to Azure. There's a simple tool, the Data Migration Assistant, which should handle everything automatically. Just follow the linked tutorial and you're good to go. You will get some warnings when migrating the database structure, related to no longer supported Xml data types, but you can ignore those. Also, make sure your target database name does not contain a . in it's name, as that will cause a bug and make it unable to connect to your Azure SQL instance.

Media Migration

In Umbraco, all the files you upload are saved by default in the media folder, relative to the root of your website. In Azure, that's a big no no. For once, disk access in your Web App tends to be not the fastest. Also, you'll run into issues as soon as you scale to multiple instances - your uploaded media files might get out of sync between instances and even disappear completely when you scale down again.

The recommended approach is to use the Azure Blob Storage service for this. Simplified, it's just storage as a service, driven by a REST API. This means you have to configure a storage account in Azure, then install the Umbraco extension for Azure Storage as a NuGet package and you're done. The NuGet installation script will automatically update your config files to make use of the new provider. Keep the old config around, as you might need it for local debugging of your Umbraco instance. I've chosen to simply include both config files and swap the right one in via a build task, depending on the environment.

Now all that's left is to put the actual data you already have in your Blob Storage. There's AzCopy, which simply let's you do this via a PowerShell cmdlet. I've also written a really small console app for this, which does the same and is a simple .NET Core dll executed with two arguments.

Umbraco Configuration & Deployment

There's little you have to do different regarding Umbraco's configuration and deployment strategy. When you've configured Azure SQL and Blob Storage, your website works as before.

However, there's one tip when hosting in Azure: To mitigate the really slow startup times for fresh deployments or app restarts, you might want to configure a staging slot with auto swap enabled and configure initialization actions to warm your site up before it's getting traffic. This means you'll no longer have to wait up to two minutes until your site responds to traffic!

If you're looking for options to automate your deployments, you can take a look at the NUKE build tool. It's open source, flexible and comes with lots of stuff built in - I use it literally everywhere.

Happy blogging!

Securing ASP.NET Core Controllers with a Policy and Api Keys

Thu, 12 Jul 2018 19:45:05 Z

In a typical ASP.NET Core web service request pipeline, you'll find authentication pretty early on, then some authorization and finally the execution of the desired action.

For most cases, that's great. However, you surely have encountered situations where all you needed was a simple yes/no validation - often for services that don't return stored information but simply perform some action based on your inputs. Integrating that in full-blown approaches like OpenId flows can often be cumbersome. Imagine you'll want to store your build artifacts on some web service you own - passing a single, secret Api key through your CI pipeline is much easier than setting up complicated, multi-step authentication setups.

Generally speaking, when your action does neither retrieve information nor alter existing data, you're safe to use an Api key.

With ASP.NET Core, you'd do that by creating a custom Policy that guards certain actions. These components are essential:

An IAuthorizationRequirement, which defines the requirement
Accompanied by an AuthorizationHandler, which checks the requirement
A bit of plumbing in your app to make them work

Let's walk through the implementation!

First, define an ApiKeyRequirement:

It's an implementation of the IAuthorizationRequirement interface and simply stores the list of all valid Api keys in-memory.

The corresponding ApiKeyRequirementHandler is responsible for evaluating incoming requests for their compliance with the requirement, in this case by checking if the Api key is given in the request header.

This has to be wired together in your Startup class. First, you have to add a custom authorization policy with a unique name - ApiKeyPolicy in this case. An instance of the ApiKeyRequirement is added to it. Secondly, you have to register the ApiKeyRequirementHandler in your service collection.

When everything is correctly set up and configured, you can now reference this authorization policy in your controllers:

Please consider this post simply as an example! For production use, you'd want to implement at least storing the Api keys hashed in your database, with the option to revoke them and with some connection between users and Api keys

Happy Authenticating!

PS: If you're looking for an approach to implement custom authentication, you can take a look at how that is implemented with using Http Basic Authentication as example.

Edit, 30.01.2020: It's always great to get feedback, especially when a blog post was able to help somebody! Valentin from randommer.io used this example to quickly secure their API with keys.

Excluding Assemblies in ASP.NET Core MVC Controller Discovery

Tue, 10 Jul 2018 17:09:21 Z

I'm a big fan of using the Microsoft.AspNetCore.TestHost package to perform integration tests for ASP.NET Core web projects. Even so much that I've put my central OpenId server using IdentityServer4 in a NuGet package, hosting itself in-memory. This let's me perform my integration tests as close to the staging and production environments as possible.

In one project, however, I've had some undesired behavior: Both the OpenId server and the actual project under test defined a route GET /status, so when I tried to request that I got an exception stating that multiple routes matched. That was weird, since there's two TestHost instances in memory. It turns out that with the latest updates, ASP.NET Core scans all loaded assemblies for controllers, so my application under test did also include the controllers from my integration test authentication service.

It's quite an easy fix, though:

Simply make sure that when you're configuring MVC, explicitly exclude assemblies for controller discovery that you want to ignore.

Happy Testing!

Altering Outbound Traffic from IIS with Rewrite Rules

Wed, 04 Jul 2018 12:42:48 Z

Sometimes, you're hosting apps in IIS where you can't or won't access the source code. For example, I'm hosting a private Git server with Bonobo.Git, which I'm using to mirror repositories for customers that get source code access.

My main development happens on GitHub, but for some commercial projects I offer source code access. Since I can't do that via GitHub (it would be reselling of their services), I have set up a private instance of Bonobo Git.

While Bonobo Git is Open Source, I do use their prepackaged versions for deployments, instead of maintaining a fork. It works great out of the box and requires very little configuration. I had just one little requirement that was not offered: Hiding the list of users who have access to the source code since I don't have permission to share my customers data with each other.

While you could add some custom JavaScript or CSS, this wouldn't solve the issue - the server should not send the data at all, instead of relying it to be hidden by the client. That's a big No-No!

Luckily, there's a lot you can do with plan old IIS web.config entries, like using a rewrite rule that simply removes the section completely from the response before it's sent over the wire.

Take this little snippet:

There's an outboundRule called Remove Git Repository Contributors which does the following:

It's active only for urls like /Repository/Detail/{RepoId}
Only when the response content type starts with text/html
Text that matches a defined regular expression on the page is simply replaced with an empty string

This single rule takes care of preventing sensible information to ever be sent out from the server!

Additionally, there's one restriction with content-altering outbound rules: They work only for non-compressed content. This means, in case you have enabled compression, you must disable it for this to work:

Depending on your setup, you might have to tweak this a bit. Most of the time, you are probably fine with just disabling compression for certain url patterns or content types. In my case, since I'm hosting behind Cloudflare, it didn't matter much - my content will be compressed by their CDN anyways, so I was safe to turn it off.

Happy rewriting!

Serving Localized Angular Single Page Applications with Asp.Net Core

Wed, 23 May 2018 16:49:05 Z

I'm doing most of my web development with an ASP.NET Core backend and an Angular frontend these days, and I'm quite happy with how these two work together.

ASP.NET Core serves both my API endpoints as well as the static files, so all I have to deploy is just a single website. Angular files are in this case served from the wwwroot/dist folder. This works fine as long as you only have to support a single language, but causes a bit of trouble as soon as you need internationalization, or i18n for short.

With Angular, there's no possibility to switch languages at runtime. That's because so much of the templates is compiled internally when you do a production build. So when you need a multilanguage app, you're actually deploying two independent Angular apps and need some way to serve different language files at runtime. For a lot of sites, going with different domains, e.g. .com for English and .de for German, is a viable approach. But many projects require that the user can select a language, without having to change domains or paths. And who wants to see /de in their urls, anyways?

As always, there's lots of solutions for how to solve this. The two most prominent ones are probably using a dedicated SPA controller that would return the correct entry point to your SPA, or simple re-route all not served requests to the SPA entries index.html at the end of your pipeline. I've chosen to do the latter.

Configuring Angular for Multilanguage Builds

This post isn't going to detail how to set up i18n in Angular, but here's a short recap: At first, you have to configure multiple configurations in your angular.json, one for each locale. Make sure that each one has a distinct deployUrl parameter that maps to where the static files will be put later, because that's going to be used as path for the injected references in index.html.

Then, make sure to have build scripts that you run before publishing the app in project.json:

Adding a Localized SPA Middleware to ASP.NET Core

To be able to serve different files based on a users locale, you have to implement something like IUserLanguageService:

It's up to you how you implement it, but I've decided to either use a stored Cookie set by the Angular frontend (so I can have a language switcher in the app) or default to the Accept-Language http header. Make sure your UserLanguageService implements some default language as fallback, since not all clients will send the required header or have Cookies set.

Next, there needs to be some provider that can generate the different paths on disk to the localized files, like dist/en and dist/de:

The LocalizedSpaStaticFilePathProvider simply builds the path to where each localized Angular app is stored.

Now you have to wire this together, for example in an extension class:

The LocalizedSpaStaticFilesExtensions configures the services required for our implementation of a localized SPA middleware. We're just providing it the list of availableLocales, so that the UserLanguageService won't return a locale we don't have a translation for. The spaRootPath parameter just indicates where, relative to wwwroot, we put all the localized builds.

For the application builder, we define a middleware that reroutes all requests to the localized defaultFile, e.g. index.html, for the current user and then have the regular ASP.NET Core StaticFiles middleware handle it. It's important to note that the applicationBuilder.UseStaticFiles(); call does not replace the regular static files middleware, it's just being called a second time. You must put UseLocalizedSpaStaticFiles() at the end of your request pipeline, because it alters all requests to your SPA entry point. That's required because your SPA most likely has routes of its own defined that your backend doesn't know, so you want to just return your SPA and let it handle client side routing.

Finally, configure everything in your Startup class:

That's how you get a localized app!

Happy translating!

Simple Sorting and Pagination in ASP.NET Core with LightQuery

Tue, 13 Mar 2018 22:49:28 Z

With LightQuery, I've written a really small yet powerful library that assists you in paging and sorting ASP.NET Core web apps. It's a great alternative if you're looking for a simple substitution for OData or similar, heavyweight frameworks that come with great functionality but usually require a bit of setup. Additionally, LightQuery comes with clients both written in C# for .NET and also for Angular 5+ applications.

It's available both on NuGet as well as on npm.

Motivation

Some time ago, I was looking to implement pagination in an ASP.NET Core Web Api call. OData wasn't yet released for .NET Core (I think it still isn't?), and all the other solutions I found required quite a bit of setup. I didn't want to deal with complex setup scenarios and neither did I want to leak library code in my controller actions, for example by specifying custom arguments or requiring setup voodoo.

All I wanted was some simple [LightQuery] attribute that I could add to any controller action and have it take care of sorting & pagination. So here's how you have to annotate a controller action in LightQuery:

Just add the [LightQuery] attribute, everything else is handled by the library and optional.

I'm using it extensively for typical, CRUD / data entry style apps, like showing thousands of users in a paged table layout. It works fine for such use cases, but it's nowhere near some feature monsters like OData.

Tip: Use [AsyncLightQuery] if you're working with Entity Framework Core for async / await support.

How Does it Work

LightQuery is purposely kept very simple. It's just a few lines and inherits from ASP.NET Cores ActionFilterAttribute.

ASP.NET Core executes the attributes OnResultExecuting() (or OnResultExecutionAsync() with Entity Framework Core) and passes the current request context. It checks for query parameters like page, pageSize or sort in the request url. If there are any, or the attribute is set to always enforce pagination, it applies the query transformation to an ObjectResult.Value property on the response. If there's no ObjectResult returned by your controller action, or its Value is not of type IQueryable, nothing happens. Go check out the source!

Controlling Pagination & Sorting on the Client Side

For clients, it's just as easy to request paged results (and thus save bandwith and time).

To paginate, send page and pageSize as url parameters in the request, like this:

https://my-api.com/users?page=2&pageSize=3

There are clients both for .NET and Angular 5+ already available as prebuilt packages.

If you further want to sort by properties, you can add the sort parameter and optionally specify desc for descending sort order, otherwise it defaults to ascending. If you specify a non-existent property, there'll be no sorting. The following request returns the users, sorted by descending email property:

https://my-api.com/users?page=2&pageSize=3&sort=email desc

Tip: Title casing is taken care of, so email works both for Email and email properties.

Do you want to know more? Head over to the GitHub repository to find more examples.

Happy Paging!

Adding Custom Claims when Logging In with Asp.Net Core Identity Cookie

Sun, 24 Dec 2017 12:25:10 Z

When working with Asp.Net Core applications, Asp.Net Core Identity is a great and easy to use choice for managing app authentication and authorization. By default, logins happen via an application cookie. This means that on a successful login, a cookie is set in the users browser that can be verified by the server on each request, without requiring database roundtrips. It also stores a few claims (pieces of information about the logged in user) directly in the cookie, such as the username or roles. You can even easily define your own claims, and the default implementation will automatically store this information in the cookie as well, allowing you to access it in every request.

However, sometimes you don't want to store a claim as persistent in the database, but still attach some information to a login sessions or context. This could be for claims that are somehow stored not as regular claims, like a team the user is associated with, or even data generated on the fly, like if the login happened on a Tuesday. If you take a look at Asp.Net Core Identities SignInManagers source, you see it just generates a ClaimsPrincipal for the user and then performs the sign in action (generating the cookie and appending it to the http response as a Set-Cookie header). This means, if you want to intercept that action and append more claims to the generated principal, you can do that either by providing your own ClaimsFactory or, as what I'll show you, just provide a custom SignInHandler.

The CustomClaimsCookieSignInHelper<TIdentityUser> provides just a wrapping mechanism around the core sign in functionality. It's easy to use in your AccountControllers login action:

To add custom claims, simply pass them as arguments to the SignInHandler.SignInUserAsync() method. If you want to go further down that route, you should check out the official implementation for the UserClaimsPrincipalFactory.

Merry Christmas and happy authenticating!

Chasing a Bug when Working with an RxJS Observable

Thu, 21 Dec 2017 19:55:46 Z

I'm the author of LightQuery, a library that offers simple pagination and sorting functionalities for Asp.Net Core web services. I've also created client libraries to consume these services, both for .Net and TypeScript with Angular. It's useful for these ever-repeating tables of data that you find in almost any app and integrates nicely with popular table and pagination modules, such as from Angular Material or ngx-bootstrap.

Both client apps provide a generic implementation of PaginationBaseService<T>, which handle all the sorting and paging of list resources. In one of my apps, I've extended this service with methods like getElementById(id: string). They make use of the service caching the last retrieved pagination result from the server. If the requested item is locally available, I can directly return a resolved Promise instead of requerying over http. This is as simple as in the following snippet:

There's a detail view component as part of the app. You select an item from a list or table, then you can view it in detail. If desired, you can toggle an editor mode to modify the record. CRUD 101, you've written that yourself often enough. The detail component watches the editor. If it's being informed that the entity was updated, it forces the pagination service to update it's current record set and then follow on with a retrieval of the item by its id.

Now that implementation is not very efficient. When I do update a single record, it doesn't use the actual response from the server but just requeries the whole list again. But it worked as a first iteration. Or did it?

The snippet above is from the service implementation. It's of interest that the local cache variable lastPaginationResult is only updated after the Observable has emitted the latest result. And that's the catch. I didn't pay attention to the order of execution for subscribed actions to Observables, but I should have. As soon as an Observable emits a value, it's subscriber actions are invoked. Meaning that there's quite a lot of stuff happening between lines 12 and 13. Looking back at the subscribe() in the main component: It's querying for a single element as soon as the paginationResult is updated. Since an element with this id is still in the (stale!) cache, there's no asynchronous actions being invoked. The execution happens directly in a single chain, deterministic without interruption. And before updating the services local cache. I've since created a unit test to catch that behavior, and the fix was as easy as simply changing the order of two statements in my code.

This was a lucky catch for me, the bug was subtle but interesting, easy to diagnose and fix, yet it alerted me to pay closer attention when working with reactive patterns. The service was later refactored to use the server response to the PUT request to directly refresh its cache and list without additional network roundtrips. If I had done that right from the start, I wouldn't have caught this subtle bug. And I would also have continued to use reactive patterns wrong.

Happy Debugging and Merry Christmas!

Attention when working with the Asp.Net Core TestHost and its HttpMessageHandler

Tue, 21 Nov 2017 23:39:22 Z

I'm a big fan of the Asp.Net Core TestHost package, which allows you to easily write integration tests of your applications without the need to do actual network calls.

TestServers feature a nice method CreateClient() that returns an HttpClient instance which will interface with the TestServer but not make any actual network calls. So basically, to perform an integration test against one of your endpoints, you treat the client as you would treat a regular one. This can also be used where you have dependencies in your code that themselves need to call your server, for example a Json Web Token authority validation utility.

For these cases, you can supply your own HttpMessageHandler:

Did you spot the error?

Yeah, i actually made the handler to delegate it's call to a new HttpClient. This means a request travels like this: HttpClient -> IntegrationTestsHttpMessageHandler -> HttpClient. If you're familiar with how request are handled internally, you've noticed that the first HttpClient marks the request as sent and the second one then refuses to send the message again with a nice System.InvalidOperationException: Cannot send the same request message multiple times. Luckily, the TestServer has direct access to a HttpMessageHandler:

I've spent the better part of an evening figuring this out...

Happy testing!

Use Any Authentication Handler for all Actions in Asp.Net Core 2.0

Wed, 08 Nov 2017 22:03:50 Z

In Single Page Applications, it's common to use Bearer authentication, for example with OAuth2 integration, to make the API available to client applications. While this can also be used on the web frontend, I enjoy the simplicity of the Cookie based authentication that's the default for Asp.Net Core Identity projects.

Unfortunately, the latest release of Asp.Net Core 2.0 introduced a behavior that's different: There is only one default authentication scheme, others are only invoked and challenged when it's further specified, for example with an [Authorize] attribute or a policy. Shawn Wildermuth wrote a good post about that. I didn't want to clutter my code with annotations everywhere, and I also wanted to use multiple authentication schemes on non-authorized controllers to get user info, no matter if the request is done via Cookie or Bearer authentication.

Luckily, you can get the old behavior back: By defining a global catch-all policy that specifies all supported authentication schemes. It's simple to set up:

First, you configure all authentication methods in your Startups ConfigureServices method, in the example it's the regular Asp.Net Core Identity cookie authentication and IdentityServer4s OAuth2 functionality.

In the configuration for MVC,there's a defaultPolicy which specifies all authentication schemes used in your app. Since policies mandate at least one requirement, we'll add an assertion that simply returns true for all requests: .RequireAssertion(c => true). The effect of the policy is now that it doesn't enforce authentication, but tries to challenge all schemes. So without further configuration, your actions should now make use of either Cookie or Bearer authentication. If required, the policy can be further tweaked to define exceptions where it's not applicable.

Happy authenticating!

Blank Screen after Login to Umbraco Backoffice

Tue, 17 Oct 2017 21:10:52 Z

For the last two years, I've been using Umbraco CMS for a few sites. It's a great content management system, giving you much freedom and easy extensibility while still maintaining a reasonably learning curve. It's probably not suited for typical 1-click-end-user-install scenarios, but it integrates really well in a .Net / Microsoft technologies stack with source control and continuous integration.

Today, however, I wanted to log in to the backend of one site to change something. It didn't work. The login was successful, but then I only got 401 responses from the server, rendering only a blank, white screen instead of the expected Backoffice. Another site, the same problem. Every Umbraco site on that server showed the same symptoms. The front ends were rendering just fine, with everything working as expected, but I could not access the administration section of any of them. I tried quite a few things, from different browsers to different devices to different ISPs, but the problem persisted. Search results for this issue were not particularly helpful, most being outdated and the bugs been fixed since a few versions ago. The logs were no sources of wisdom either, just showing a lot of successful login attempts without follow up actions or failures. Nothing had been changed on the sites, so I didn't really suspect it to be an issue with Umbraco anyway, and I went on to check the server.

The Let's Encrypt extension did update two certificates in IIS that day, but that's happened before and shouldn't cause any problems. By chance, while being remote on the server, I noticed that the servers time was off. A good two hours. I fixed the time, tried to log in and - success!

Maybe I can save you some time digging through Umbraco issues when all you should do is check your servers time.

Happy logging in!

Using the StackExchange MiniProfiler with an Angular Single Page Application

Thu, 27 Jul 2017 10:49:57 Z

Profiling your application is important. While optimization is rarely the highest priority, you should at least have an idea of what your computing resources are being spent on. I've found the MiniProfiler, developed by the StackExchange team and extensively used at Stack Overflow, to be a great tool for that.

I love it mostly for two things:

You get an idea of how long each step of your requests take
It profiles and analyzes your SQL calls. This is great for finding slow queries and identifying where you have N+1 problems

The server side configuration doesn't differ when you're designing a Single Page Application (SPA), but the (current) client side snippet does not yet support Angular 2, Angular 4 or newer frameworks. Whenever you do a profiled request, MiniProfiler attaches a x-miniprofiler-ids Http header with ids to relate the profiling results. Now your client side code should request them and display them for you, but this doesn't integrate yet in Angular without implementing a custom hook in Angulars Http class.

The MiniProfilerHttp class just extends Angulars Http provider. It's checking every response for profiler ids and, if any are found, requests the results.

You have to alter your applications NgModule to provide the derived class instead of Angulars Http provider.

Now, your applications API requests should make MiniProfiler display you the results without having to do a full refresh of the site:

If you've configured the backend correctly and added the MiniProfiler includes to your website, you'll see the results being displayed in the top left of your page.

Happy profiling!

Validating Usernames in an Angular Frontend with Asp.Net Core Identity on the Server

Wed, 05 Jul 2017 22:06:55 Z

In Angular, you've got the built-in [pattern] form validation directive that makes it easy to validate an input with a regular expression. Today, I wanted to check usernames consistently across my app and I found out that the Asp.Net Core Identity backend uses just a long string which contains all allowed characters. This could look like this: "abcACB123._"

It's configured this way in your Startups ConfigureServices method:

That's not regex, so you've got two options: convert it into regex or create a custom directive that validates consistently with your backend. I've chosen option two, and want to share how quick that was:

The ValidateUsernameDirective simply checks the input string letter-by-letter for illegal chars. In template forms, you use it just like this:

Happy validating!

Mocking an Asp.Net Core HttpClient for Unit Tests with Moq

Sat, 03 Jun 2017 12:08:48 Z

When you design services around Http requests, you should really just inject your HttpClient as a dependency instead of wrapping it in a using statement in-place. This has the nice effect of making your code both easier to maintain and test. While doing a project recently, I decided to give Moq a try and use it to generate a HttpMessageHandler (that's the class the HttpClient uses internally for sending the network requests) for the HttpClient instead of providing my own HttpClient implementation. It's a really awesome mocking framework, but it took me a bit to set it up correctly. That's why I hope this snippet might save some of you a few minutes:

Happy mocking!

Http Basic Authentication in Asp.Net Core Projects

Thu, 23 Mar 2017 13:31:17 Z

There are many different ways how requests to an Asp.Net Core web application can be authenticated. Asp.Net Core Identity offers a great built in identity provider with some options for authentication, but there's no built in support for Http Basic authentication. The Microsoft docs for the old Web Api framework have a bit of background information about the disadvantages of Http Basic Authentication, especially that credentials are being sent with every request. Generally, it's not a great idea to support it, but unfortunately, you can't always enforce it when there are dependent services of your application. To support the Http Basic Authentication scheme, you've got to insert a small middleware in your app that validates Authorization headers, which can be done with the following code. I've split it up into a few classes to be more readable and easy to test, but here you go:

The BasicAuthenticationMiddleware checks if there's already an authenticated user for the request and if not, checks for Http Basic Authentication and tries to set the user identity for this request.

Http Basic headers are, essentially, passed as Base64("Username":"Password"). BasicAuthenticationHeaderValue decodes that.

A wrapper to perform the actual sign in action for the user. It's not persisting the login via, for example, a cookie.

The MiddlewareExtensions just give you a nicer syntax to add the middleware to your Startup class.

Just remember that when using this, there is absolutely no way you can use unencrypted traffic! While unencrypted login pages themselves are bad enough (just ask devGeorge), Http Basic sends the username and password with every request. This makes eavesdropping so much more likely since any intercepted request now contains your full login data. Just use Let's Encrypt. While you're at it, use it everywhere. Use it for your staging environment. Use it for static pages hosting demos. Really, use encryption! Make the lock 🔒 in the browser bar your minimum requirement for every website. And educate your users to stop requiring Http Basic.

Otherwise, happy authenticating!

NetCoreHeroes: Angular 2 with .Net Core in Visual Studio 2015, Part VI

Tue, 21 Feb 2017 21:25:20 Z

The source code for this project is available at GitHub and here's a live example of the website.

While the first five parts of the tutorial cover everything from setting up the development environment to finally deploying a unit-tested app, this part will briefly describe how an app that uses system.js for client side module loading can be set up to use webpack. The steps I'll be describing here will work nicely in Visual Studio for a great development experience. This commit covers the required steps for the demo project.

What's a module loader used for?

Apps that are written in JavaScript (or TypeScript, Coffescript...) are getting ever more complex. For this, just like in traditional object oriented frameworks like Java and the .Net system, using, referencing and managing module dependencies is becoming really important and quite hard to do by hand. Small apps might get away with just a few source files and then just merging them together (or even just loading them in the correct order), but this is simply too tedious and error prone for rich applications. Here come module bundlers into play. Basically, you give them your applications entry point (something like a main.js file) and a bit of configuration and they'll take care of dynamic loading of dependencies or even bundling.

Why prefer webpack over system.js?

webpack does, as it's name implies, pack your app for the web. This means you'll generate one or a few bundle files that are then loaded in the browser. In contrast, system.js might potentially load dozens of different JavaScript files from the server if you've got a lot of components. So while system.js is easier to set up, the webpack approach offers a lot of ways to optimize size and speed of your application. Since usual browser configurations do not allow for more than 5 concurrent requests to the same domain, loading many small files initially just takes a whole lot longer. Finally, webpack offers features like Hot Module Replacement which makes changes in your source code almost instantly appear in your debug sessions browser window.

npm dependencies in package.json

Mainly webpack and loaders for different file types are the novelty here. Note that systemjs is still around. I'm using that for the unit tests of my TypeScript code with Chutzpah and found it way simpler and faster to configure than webpack, so I'll still have it as a devDependency.

webpack configuration for your own modules

That's a very basic configuration for webpack. What's happening is that webpack enters the file defined in entry (on line 8) and from there on walks the tree of your dependencies to build your bundle. On line 16, you see that these two loaders are chained: ['awesome-typescript-loader?silent=true', 'angular2-template-loader'], meaning that every matching file is first fed into the angular2-template-loader and then into the awesome-typescript-loader. That's a really cool feature here, because the first loader will replace the template and style urls of your Angular 2 components with require() statements so they're being inlined in the generated bundle, massively lowering requests to the server in production! For this to work though, you must set the resolution of .ts file extensions before .js files on line 6, otherwise webpack will just use the (already compiled) JavaScript file and happily ignore the loaders you've specified for TypeScript.

webpack configuration for dependencies

There's not much to see here. You'll just have to specify all the modules that change not as frequent as your custom code in a separate bundle to save on compilation time and possible bandwidth if you're iterating fast while your app is already in use. Line 28 contains the configuration of the Dll plugin, which itself is referenced by the bundle of your code.

App entry point

Since zone.js and reflect-metadata will no longer be loaded separately from the browser, you'll need to import them at the entry point of your app so they're available. Additionally, I'm using the style-loader to load global CSS into the app. The style loader will insert the content of the file at runtime in the documents <head> section, making it available for the whole app.

Updating your Asp.Net Core view

Finally, you can remove all the single bundles and just include two scripts in your view: vendor.js and main.js. Just make sure to either defer loading them or put them below your <my-app> tag, otherwise you'll get an error from Angular 2 telling you that it can't find the app root element.

Result

Here's how much faster (and smaller!) the example app becomes after using webpack:

Happy bundling!

Update 28.05.2017: Updated Angular to 4.1.3 and Asp.Net Core to 1.1.2. Switched to Visual Studio 2017 csproj format

Signing in Users for Integration Testing with the Asp.Net Core TestHost

Mon, 12 Dec 2016 16:32:30 Z

When developing Asp.Net Core web apps, there's a really simple way to create integration tests for your project with the Microsoft.AspNetCore.TestHost package. In any web project, there's likely some form of authentication and authorization for specific users. Luckily, there's a really simple way that you can use to sign in users in an integration test scenario by means of evaluating a custom header without having to pass Asp.Net Core Identity cookies back and forth.

Place the following request handler method in the Startup class you're using for integration tests:

public async Task SignInIntegrationTestUser(HttpContext context)
{
    var integrationTestsUserHeader = context.Request.Headers["IntegrationTestLogin"];
    if (integrationTestsUserHeader.Count > 0)
    {
        var userName = integrationTestsUserHeader[0];
        var userManager = context.RequestServices.GetRequiredService<UserManager<ApplicationUser>>();
        var user = await userManager.FindByEmailAsync(userName);
        if (user == null)
        {
            return;
        }
        var signInManager = context.RequestServices.GetRequiredService<SignInManager<ApplicationUser>>();
        var userIdentity = await signInManager.CreateUserPrincipalAsync(user);
        context.User = userIdentity;
    }
}

Configure it in the request pipeline after app.UseIdentity() but before app.UseMvc() in your Startup.Configure() method.

public void Configure(IApplicationBuilder app, IHostingEnvironment env, ILoggerFactory loggerFactory)
{
    using (var serviceScope = app.ApplicationServices.GetRequiredService<IServiceScopeFactory>().CreateScope())
    {
        // Seed with integration tests specific data
        ServerInitialization.InitializeIntegrationTestsDatabase(serviceScope.ServiceProvider).Wait();
    }
    app.UseStaticFiles();
    app.UseIdentity();
    // To log in a user for integration tests
    // DO NOT USE IN PRODUCTION
    app.Use(next => async context =>
    {
        await SignInIntegrationTestUser(context);
        await next.Invoke(context);
    });
    app.UseMvc();
}

Generating authenticated clients is now as simple as attaching a custom header:

public static HttpClient AdminClient
{
    get
    {
        var client = _testServer.CreateClient();
        client.DefaultRequestHeaders.Add("IntegrationTestLogin", "admin@example.com");
        return client;
    }
}

You can use the client to perform web requests to the TestHost, all your controllers will be aware of the UserPrincipal.

[Fact]
public async Task StatusOkForAdminUser()
{
    var client = ServerInitialization.AdminClient;
    var response = await client.GetAsync(ACTION_ROUTE);
    Assert.Equal(System.Net.HttpStatusCode.OK, response.StatusCode);
}

Just be careful not to ever let this come near Production code!

Happy testing!