Dangl.Blog();

Installing .NET 9 Alongside Older Versions on Ubuntu 22.04

Fri, 29 Nov 2024 11:29:43 Z

With the recently release .NET 9, Microsoft has changed how they're publishing packages for Linux distributions. Previously, there was a Microsoft managed package repository available from which you could easily install different .NET versions, making it especially easy for CI agents to keep multiple .NET versions installed and updated.

However, starting with .NET 9, Microsoft no longer provides a feed for their distributions. So if you need the SDK or just the runtime, you need to use a feed managed by Canonical (the company behind Ubuntu). The problem is, there's no feed on which you can get multiple versions, e.g. .NET 8 and .NET 9 combined. Also, you can't just install from two different repositories, since the packages will have mutually exclusive dependencies, prohibiting you from installing .NET 8 and 9 at the same time. Also, the Ubuntu repositories seem to be a bit behind, so you could have to wait up to one week to get the latest updates for your SDKs.
I didn't properly dive into the reasons behind this change, I think it's mostly Microsoft wanting to integrate their .NET distribution into more official channels, but from a developer perspective, that doesn't seem very ideal for me.

So, yesterday evening, I tried to figure out how to get .NET 9 running while keeping all the other versions installed. Even though they're out of support, sometimes you just need to update an on-off CLI tool with a bugfix or new feature without having to update the whole project and the CI pipeline. Also, since both .NET 8 and .NET 9 are still in active support, we plan to also run tests for our products on both - so having them all on our CI agents was required.

There was no way that I found how to work with feeds, but luckily, you can just use the dotnet-install.sh script for manual SDK installations. I found it pretty easy to use, but there's a gotcha: If you've already got some .NET versions installed, they're probably by default in a different directory than what the installation script uses. In that case, though, before installation, you can simply run dotnet --list-sdks to get an output like this:
dotnet --list-sdks
6.0.428 [/usr/share/dotnet/sdk]
7.0.410 [/usr/share/dotnet/sdk]
8.0.404 [/usr/share/dotnet/sdk]

Here, we see that .NET is installed to /usr/share/dotnet, which we can then pass to the installation script as directory like this:
./dotnet-install.sh --install-dir /usr/share/dotnet --channel 9.0

Running dotnet --list-sdks again should now list .NET 9 as well, and you're good to go. There's no need to set any more environment variables. The only thing to keep in mind is that you now have to manually update .NET via the install script, since apt-get won't do those for the .NET 9 sdk now.

Happy developing!

Creating an API and Web UI for my District Heating System

Thu, 13 Jul 2023 19:19:25 Z

A few months ago, we had a new heating system installed. It's a district heating system, with the heat exchange being controlled by a Schneid MR 12 control unit. From a heating perspective, it's a pretty simple concept: Two pipes go in your hose, one delivers hot water to a heat exchanger, the other carries the slightly colder water away. We're now part of a small network of 15 houses that use a central energy source for their heating and warm water needs.

It's working great, and replaced our own wood chip fired system. Now, with a new system, I naturally wanted to be able to get some data out of it and have the ability to control it remotely. I looked into some options, and I have to say, it's all a bit less comfortable than what we software engineers are used to with modern cloud systems. Nevertheless, the manufacturer could provide me with a network card, which makes a Modbus TCP interface available. It's quite an ancient protocol, having no security features or really anything besides reading and writing raw "registers", but it's quite well established in electronic engineering and thus has a wide range of libraries and packages available. The security part was pretty easy to take care of, by isolating it in the network and only granting specific hosts access.

For the actual API part, I've gone for a regular ASP.NET Core backend with an Angular frontend. The Modbus TCP connection is handled by FluentModbus, which provides an easy wrapper around the Modbus protocol so we can just use it to read and write binary data.

Finally, I've built a dashboard as the main entry point:

It's great for a quick visualization of the data, and with a local SQLite database, I can even periodically log some data points and visualize them, like the outside temperature:

There are also some properties that can be edited, like target temperatures or the operation mode of the system. And, all of those values are exposed via a REST API, so you can integrate the sensors e.g. as RESTful sensors in Home Assistant.

You can find the project directly on GitHub.

Happy heating!

Accessing AVACloud Directly with User Accounts

Wed, 05 Jul 2023 11:42:51 Z

Disclaimer: This is a cross-post from my professional website. It's a tutorial for a product my company sells.

With AVACloud, you would usually use service based accounts to access the API. However, recently we've had an interesting use case: A client did the integration for AVACloud in an Excel AddIn, and for this, we wanted to go with individual Dangl.Identity user accounts instead of centrally managed OAuth2 clients. Since the default AvaCloudClientFactory from our Dangl.AVACloud package assumes that you're working with clients, you need to add some plumbing code to work with user accounts. It's pretty straightforward, but you need to follow a few steps to achieve it. So, here's how to do it:

We're first creating a class called UserAccountTokenHandler, which implements the ITokenHandler interface. This will try to obtain a Json Web Token (JWT) directly from AVACloud, with provided user credentials. It's used to replace the OpenID Connect based client credentials authentication flow used by default, and allows you to authenticate with AVACloud with a real user context. The implementation is pretty straightforward, getting a token is a single call to the AVACloud API.

We're then adding the TokenAccessChecker class, which just gets a token and checks if the user does have access to perform AVACloud operations. That way, we can show a notification in the UI if AVACloud access is denied, before waiting to see if service calls fail with a 403 - Forbidden status code response.

UserAccountHttpClientAccessor is a small utility class that's just a wrapper around HttpClient. We're using that to be able to provide a typed HttpClient with the HttpClientFactory pattern. We could also use named clients, but I usually prefer to wrap the Http client in a separate class.

Now that everything's set up, we want some more convenience. Working with REST APIs always has some overhead, like managing HttpClient lifecycle and ensuring each request is properly authenticated. Since I'm a big fan of using dependency injection, I usually create factory classes that can be used a singletons throughout the whole app lifecycle, and which do their internal service management. So, in our case, we're just initializing a single instance of AvaCloudUserClientFactory and then rely on it's internal service provider when we want to get a client class.

Finally, here's a test showing you a quick example that gets a token, checks if the user has access and then converts an Excel file using AVACloud:

Happy converting!

Setting the Language in Chrome Headless for E2E Tests

Thu, 01 Jun 2023 10:17:26 Z

For End-to-End (E2E) testing of web apps, the headless mode of Chrome is a great choice. We're using it in our projects to ensure some true end to end test coverage, often for stories like happy paths for important actions. E2E tests are usually quite cumbersome to write, and tend to be brittle during upgrades, but they give you a lot of confidence when doing automated deployments, and free up valuable developer time from having to do repeated, manual tests and verifications. I've blogged before about them, for example hows to instrument them via Playwright or how you can use Selenium in a Docker container.

One thing that's been missing is the ability to set a language for the headless Chrome instance when testing. Especially if you have multilingual apps that automatically detect the users language, you might experience problems in testing when you're checking for text contents or trying to find Html elements via their labels. There was a long open issue with Chrome, where the language supplied via a command line argument was simply not set in headless mode.

But, this has changed with the new headless option for chrome!

Finally, when you start headless via --headless=new, you can simply set the language with another switch: --lang=en. This makes testing so much easier, and allows to provide a consistent developer experience, no matter the local language settings.

Happy testing!

ASP.NET Core Locally Serving Outdated Dev Certificate

Wed, 17 May 2023 20:19:36 Z

Today, I was trying to run one of our apps locally. It's a fairly small service, that's only receiving updates every few months, and all the testing and deployment happens in CI, so it hasn't been run locally for quite some time. But, when I started it, it tried to serve an expired certificate, so Chrome showed me a warning about the site not being secure.

With ASP.NET Core, the development workflow is quite nice. You can easily use self signed certificates for working with localhost, and dotnet takes care of managing all that. Yet, somehow, it served me a certificate that expired almost three years ago. Even after regenerating the dev certificates locally, I had the same error, so I looked into the configuration.

Turns out, I had this configuration option set in the user secrets on that machine: kestrel:certificates:development:password

User secrets are a way of managing not really sensitive but developer or machine specific configs for ASP.NET Core projects. The option above, with using a set password for a certificate, resulted in the app not automatically choosing the current dev certificate but instead one that expired long ago. So, quick fix when you know where to look😀

Happy encrypting!

Changing the Order of Parameters in Swagger / OpenAPI Documents when using NSwag to Generate the Swaggerfile

Sun, 16 Apr 2023 20:15:53 Z

Our AVACloud SaaS product has been available for a few years already. While that's a great achievement in itself, it also comes with drawbacks: The API is used by a lot of customers by now, which requires us to keep it stable and not introduce breaking changes or incompatibilities with existing implementations. Move fast still applies, but the break things part would now be a problem.

So, we've recently updated one endpoint to support additional parameters. It's a simple REST endpoint that takes a file, some options, and does a conversion. In the.NET backend, the controller receives three parameters: The file, and two options objects for the import respectively export configuration. That maps nicely, and introducing additional parameters is as easy as adding a new property to a view model used for the bindings. Many clients generated via Swagger can follow the same approach of using options objects for the client interface, thus no problem there.

However, the objects are transported as query parameters. This means that from a Swagger perspective, we've now added two new query parameters. This can cause problems when the new parameters are not sorted to the end, thus generating a new client version would break existing implementations and require attention and a rewrite from every consumer of the API.

Luckily, we're using NSwag in the backend, which allows a great deal of customization. When we set up Swagger generation, we can simply inject custom code in the PostProcess callback and sort the parameters however we want.

We're just getting the current parameters, apply our custom sort logic which ensures the new parameters are at the end, and then clear and refill the parameter list.

There are other options available, such as using custom x-position attributes, but I've found that relying on such non-standard features often causes problems later with some client generators. So, let's keep it simple and stupid😀

Happy customizing!

Signing Electron Apps before Bundling with Azure Key Vault and EV Code Signing Certificates

Wed, 18 Aug 2021 20:42:54 Z

Just a few days ago, I've blogged about running E2E tests for an Electron app. But, once tested and verified, the next step is deployment.

If you've ever shipped an application for Windows users, you might be aware of the Windows SmartScreen filter. It's basically a check that might pop up and warn your users in case Microsoft doesn't have any data on your app being trustworthy. That's actually a great feature, since it encourages application developers to sign their programs and therefore increase overall app security. Except for when your build pipeline becomes compromised, as in the recent SolarWinds attack...

However, I'd like to focus on the technical aspects. For code signing certificates, there are two options available: Either a regular certificate, which has quite a low bar for verification and is available for less than 100,- € per year, or an Extended Validation (EV) certificate. The EV ones are much pricier, and do require the certificate issuer to perform a more in-depth check before handing it out. You pay more, but you get more: In contrast to regular certificates, which require a certain number of installs until Windows SmartScreen recognizes them as trustworthy, EV certificates work out of the box and don't show any warnings to your users. So, for anything commercial, you probably want to go with an EV certificate.

But it also comes with a catch: You usually need some hardware device to secure it, often an USB dongle or a Hardware Security Module (HSM). That's just a requirement for the heightened security about EV certificates, but this also means that it's harder to integrate it in automated build workflows. You don't want to be in a position where you need a dedicated build computer in your office where you manually need to insert some USB dongle to generate a release.

Here comes Azure Key Vault: It's a service available in the Azure Cloud which just kind of does everything around securing stuff, hence the name. It's typically used for things like secret management, e.g. to handle configuration for cloud services and the like, or just for plain SSL certificate management, e.g. with Let's Encrypt management via shibayan's key-vault-acme-bot. The part that's interesting for us in this article, however, is that the premium tier of Azure Key Vault allows you to store certificates on a HSM, securely on Microsoft infrastructure.

When going that way, you give up the option of exporting the certificate, e.g. temporary to your build server to sign applications. That would introduce too much of a security risk, so you need to remotely use the signing feature of Azure Key Vault to sign your apps.

And this is where this blog post is headed: When building an Electron app, you've got two steps. First, you build & publish the app, then it's getting packaged. Now, you want to sign both outputs, the actual *.exe files in the bundle as well as the installer itself. So, I'll show you some code that performs the build, calls a hook between the publish and package step to sign everything, and then finally signs the installer itself. All done via remote signing in Azure Key Vault.

I'm a big fan of the NUKE build system, so I'm using that for the actual build automation. The concept should be easily translatable to whatever system you're using. So, let's look at the code finally!

First, we're starting with this build script:

The part above is the target in NUKE that generates the Electron bundle. In this specific example, we're even using Electron.NET, since the app itself is really just a website that some customers want to use as a desktop application. I won't go into too much detail here, since the build process should be pretty straightforward. It's really just a wrapper around electron-builder with something injected here and there.

This second part is a bit more interesting here. The electron.manifest.json is your app definition, we're configuring, for example, that the Windows target should use the nsis installer. But the real fun is on line 11: We're defining a JavaScript file that is called during the build, in our case right after the built-in signing task is completed.

electronAfterPackHook.js, again, is a simple script. It's really just calling back out to the NUKE build system to run the SignExecutables target and passes the current output directory as an argument. The magic here is that this is now called during your build, after the executable has been created and patched with the icon, but before it's being bundled up by the installer.

Finally, we're using the AzureSign package to sign our executable with the certificate in Azure Key Vault.

After everything's done here, the build continues. At the end, the parent NUKE process just signs the installers and we're done. The next step would be publishing, for us that means generating the documentation and deploying the artifact.

So, you've seen we're really doing something like in the movie Inception: We're calling our build system from a hook from a build process initiated by the build system itself. It feels a bit Rube-Goldberg-y, but it works fine and is pretty simple to setup & understand.

Happy signing!

Running Fully Automated E2E Tests in Electron in a Docker Container with Playwright

Thu, 12 Aug 2021 07:03:26 Z

If you've read some of my previous articles, you probably know that I'm kind of obsessed with rigorous testing - I'm advocating for having a good, automated quality control for every project I encounter. It's probably one of the biggest time savers there is in software development, whether it's because you catch regressions early or you've got lots of infrastructure set up so that reproducing bugs is a matter of minutes instead of hours.

For one of our apps, we've got a bit of a complex setup. The backend is a traditional ASP.NET Core application with the usual moving parts, a relational database, some blob storage and the compute part of the app itself. That's easy to test on its own.

The frontend part deviates a bit from our regular architectures. Due to the nature of the app, it's built and deployed as an Electron application. Electron is a great tool that I've blogged about previously, but it comes at a cost. It gets the most flak for being inefficient, whether it's due to it having a large memory footprint (both in storage and memory), or being less fast than native applications. However, the upsides are great - since Electron is really just a wrapper around Chromium, it allows you to build desktop applications with all tried & tested web technologies. This makes it a natural choice when your team is building web applications usually and can therefore transition seamlessly to an Electron project.

However, Electron feels sometimes a bit like building on sand - it works really well, but there are a lot of moving parts under the hood. One obstacle we've encountered was around testing the full application in an end-to-end (E2E) way. To ensure such tests run in a controlled environment, independent of whatever host is currently executing, we're using a small Docker setup to spin up the entire infrastructure, test it and then tear it down again. That works pretty well with regular web apps, but not so much with Docker.

When we first started setting this up, we've ran into lots of problems. The biggest maybe was a lack of community information around this issue - it just seemed like not a lot of people are executing such tests, so there was not a lot to be found except small bits here and there. We've finally managed to get it up in the end, but even an external consultant specializing in that area was struggling to set it up. One of the major hurdles was that while there are lots of tools for automating browsers that work fine in Docker, there are not a lot of options when it comes to Electron.

Then came last week, when we updated the Electron version. That immediately broke our build. It turned out that Spectron, the test runner for Electron, doesn't have any active maintainers left. And it also looked like we're not the only ones affected by this.

Luckily, there's a new player around, trying to offer a modern and stable way for browser automation: Playwright by Microsoft. It does have official Electron support, is actively maintained and, due to it being just a bit over a year old, was able to use a modern and easy to use API approach.

After spending some hours on trying to get the original test setup fixed, we've decided to give Playwright a try. To our amazement, it just worked! It took just about an hour to set everything up and migrate the tests. And, our E2E tests started being green again🚀

I'll try to give you a condensed summary of what we did to get it running, and how it was set up. In reality, we're also spinning up the other services, like backend, database and blob storage, put all the Docker containers in the same network and therefore simulate the app as close to the production environment as possible. But, let's take a look at it file by file:

We start with a Dockerfile. That's a rather simple one, the few things worth mentioning are:

We're building from the node image, since a lot of dependencies are already present there
Some tools need to be installed around display drivers, most importantly xvfb which will act as a virtual display inside the Docker container
A custom entrypoint for the container is provided to execute a script at container start

The entrypoint itself isn't complicated, either. When looking around for running xvfb in Docker, you'll often find similar snippets. Ours was, in fact, also mostly copied from various sources. Its really just making sure that a virtual display is available before the actual command is run

The common-setup.ts file is a base for all our tests. Here, we're using playwright and launch a new instance of the app for every test. As you can see, the API could probably not be simpler here, yet it works flawlessly.

Finally, some tests. They're shortened, but give you a small indication what's possible with playwright, or automated E2E tests in general.
We do usually aim for two things with such tests:

We want to get quick feedback to see if the app is generally running, is it able to start, do we get any script errors in the console, can users log in and such
Additionally, we usually have full E2E tests for the critical paths in our applications. For example, can new users register, confirm their email, login, start a trial and then upgrade to a paid subscription? Generally, things a QA department would do but that are easy to automate

It's sometimes tedious to write such tests, and you most likely won't ever have your full app covered with them. But they're a huge confidence boost, especially when your pipeline automatically deploys right into production.

Finally, to actually run the tests, we're just spinning up the Docker container with a script like the one above. The important part here is that we're now calling docker run to execute end to end tests in the container. We're doing some more optimizations, like using tmpfs for the node_modules folder and mounting the app directory into the container. But in the end, the tests run and we're getting a test results file out that can be processed in CI systems.

So, happy testing!

Updating Azure App Service on Linux for Docker via Webhooks from C#

Sun, 16 May 2021 17:43:01 Z

If you're using Azure App Service to run Docker images, you already know how easy it is to get your app running. Azure nicely abstracts everything for you - it may be a bit more expensive than hosting your own Kubernetes cluster on cloud VMs, but it makes more than up for it in simplicity.

Among the cooler features of using Azure App Service to run Docker ist that you automatically get a green/blue deployment. All you have to do is update the Docker image for the App Service and after a few moments, you're running the latest version, without any downtime. This is super convenient when you're deploying Docker to Azure Container Registry, setting up automatic updates is then just a single click.
However, it's a bit more complicated when you're using something else, e.g. GitHub Packages for hosting your Docker containers, since then you're responsible for calling a webhook in Azure to let it know there's a new container version available.

That's in itself not very complicated, but it has one little point that's worth mentionioning, as you see in the code below:

The webhook url in Azure includes an UserInfo part, meaning it's got username:password right there in the url. Most tools will support that out of the box, but in case you're using .NET to send the webhooks, you should be aware that you need to parse that information and use it with Http Basic Authentication.

By the way, for real life implementations, you probably don't want to hardcode the webhook uri in your build script😀

Happy deploying!

Let's use NUKE to Quickly Deploy an App to Azure via Zip Deployment

Mon, 12 Oct 2020 20:15:01 Z

It's no secret I'm a big fan of the NUKE build system, it's making my life just so much more convenient! So here's a small real world example that demonstrates how easy it is to deploy a static website to an Azure App Service using NUKE and Kudu Zip Deployment. You can check out the repository here if you want to see it all.

Recently, I was going through some old repositories on my GitHub account for the occasional cleaning - just checking if something needs an update, or is no longer working. My antlr-calculator project hadn't been updated in a while, and the demo site was still hosted on a virtual server. So, a great task to kill an evening was found! I decided to update the build system from just some CLI commands to use NUKE and to additional move the hosting to an Azure App Service.

Many think that for using NUKE to automate your build, you need a .NET project. But while NUKE leverages C# to set up your build, you can actually use it to automate any task you want. For example, here's how you build an NPM package:

Or, in this case, build & deploy a static website to Azure. Let's go through this step by step. First, we define the Target DeployDemo in Nuke. It's set up to invoke the Clean target and specifies some required parameters:

Next, we use NPM to build the site and update a placeholder in index.html for the version:

Then we get the Base64 encoded authentication header value to deploy to Azure and zip the output into a Zip file:

Finally, we POST this zip file to the Kudu API and wait for a response:

That's it! Now you've deployed a small websites to Azure. With build automation servers, like Jenkins, GitHub Actions or Azure Pipelines, you can configure your build scripts to run on every commit.

Happy automating!

Running SQL Server Integration Tests in .NET Core Projects via Docker

Tue, 22 Sep 2020 19:51:36 Z

I'm a huge fan of test driven development. It helps me deliver fewer defects in my apps and gives me confidence that my changes don't break any existing features.

While I write mostly unit tests that run quickly, I try to also have lots of integrations tests for all API endpoints in my web applications. This typically involves code that hits a database, so I had to find some way to easily spin up database instances for my tests. I'm a firm believer that running tests should only involve a single command, so relying on any existing database on the build agent was a hard no for me. Additionally, multiple instances might run in parallel on the same machine, e.g. when different branches are built at the same time.

My initial approach has always involved using a SQLite in memory database, actually a fresh one for every single test. This worked well, was fast enough and dead easy to set up. However, it had one huge problem: I'm not running SQLite in production, so I wasn't really testing the system under the same circumstances and additionally, I could not test features that involved database specific behavior.

Typically in such a situation, most developers would use a containerized database, such as Microsoft SQL Server. But most approaches like this that I've seen were a bit ugly: the database usually had to be spun up manually before running tests, or at least via a build script. But I wanted a way that works independently of how the tests are run - whether via my build script, by running dotnet test or just by clicking Run in the Visual Studio Test Explorer.

Luckily, I've found the great Docker.DotNet library which allows you to access the Docker Daemon directly in your C# code, e.g. during test setup and teardown methods. The integration & setup for such integration tests is then pretty straightforward, but requires a bit of code to get it reliably working. Just follow the snippets below and you should be able to set up something similar!

The class DockerSqlDatabaseUtilities is where Docker.DotNet is referenced - it's purpose is to ensure that a Microsoft SQL Server Docker image is running and ready for connections.

Next, SqlServerDockerCollectionFixture is a test fixture that essentially just orchestrates the Docker setup and implements XUnit's IAsyncLifetime interface.

Further down, we're specifying a custom TestFramework in AssemblyInfo.cs, since we'll be running the test setup from the previous fixture class only once per assembly, and that's not supported out of the box in XUnit. You could refactor the code to internally track if the setup has already been performed, or simply use the Xunit.Extensions.Ordering as test framework😀

Another class I'm using is TestHelper. You can guess from the name that it's offering supporting methods during testing, such as providing an in memory instance of our OpenID Service Dangl.Identity and performing the actual setup of our in memory ASP.NET Core backend, which is what we actually want to test.

Finally, we've got an abstract base class IntegrationTestBase that all our tests inherit from. TestHelper is actually provided as a dedicated instance per test, which means that we can run all tests in parallel yet still fully isolated from each other.

Does this approach work for you? Tell me in the comments!

Happy testing!

Simple and Quick Way to Backup Jenkins to Azure Blob Storage

Sun, 14 Jun 2020 14:52:13 Z

For years, I've been a happy user of Jenkins to automate all our Continuous Integration & Continuous Delivery CI/CD steps. Just recently, I've been evaluating some other, more modern platforms. While trying out GitHub Actions and Azure DevOps, I've been generally satisfied, but found that both don't fit perfectly for what I want. GitHub Actions is still in it's early stages and lacks many features, and with Azure DevOps I've felt the setup to be a bit too complicated and tightly integrated with Azure services. Jenkins just could do everything quite well while giving you lots of freedom.

These findings resulted in me not changing the existing setup too much - two servers, one running Windows and one on Linux worked fine so far. However, I've been a bit unsatisfied with the server's performance characteristics. The instances were on some virtual servers that were approaching their 4 year mark, so I decided to switch to two up to date, dedicated machines.

This led me to rethink the backup process. Previously, a cronjob just backed up all the Jenkins data daily to a network share that was provided by the server hosting company, and that felt a bit too outdated and makes accessing backups actually more complicated than necessary. So I decided to just Backup the data to Azure Blob Storage. However, there was no ready made solution that I could find, so I had to roll my own.

Luckily, Jenkins uses just file storage in it's JENKINS_HOME directory for all configurations, from users to jobs to plugins, so backing up the configuration is actually pretty easy - just copy the parts you want to be backed up. For this, I decided to directly leverage Jenkins itself to run the backup jobs, with a simply script that backs up the data to a Zip file and uploads it to the cloud.

Essentially, it boils down to two parts, all of them are available directly on GitHub. First, the Jenkinsfile which configures the job itself:

It's really just doing two things: Specify a trigger that runs once a month and then invoke the actual backup script. If your master is not a Windows machine, just call build.cmd BackupInstance instead of the PowerShell version.

The script itself leverages the NUKE build automation tool, which itself is an awesome asset that I've blogged about previously. It's an engine that lets you write build scripts in .NET and execute them anywhere. This backup script is a great example - it doesn't require anything preinstalled, you just run build.cmd and it works! You can view the full script here.

Happy Automating!

Improving ASP.NET Core End-to-End Tests with Selenium Docker Images

Wed, 20 May 2020 13:28:10 Z

Last year, I've blogged about automating end-to-end (E2E) tests with an ASP.NET Core application. Quickly summarized, you're setting up your environment locally via Docker and then use Seleniums ChromeDriver to automate UI tests for your web application. This worked well, but had one big drawback - when you're executing the tests, you need to have Chrome installed on the build agent or on your local machine. While most who do web development probably have Chrome available, it's a bit of a hassle with testing - you need to make sure that your Selenium version matches the current Chrome version.

Luckily, Selenium provides Docker containers with an embedded Chrome and their own RemoteWebDriver. This means to be able to access an automated browser via Selenium, you just have to spin up a container and connect to it. That's pretty great and makes versioning really easy!

Building on the previous post, you just have to follow these steps:

Docker Configuration

I'm using the selenium/standalone-chrome:3.141.59 image and make sure to assign it enough memory, as per the official recommendation.

Test Setup

In the test project, I simply check if the containerized Selenium driver should be used and return a RemoteWebDriver instead of a ChromeDriver. For context, I do sometimes let the tests run with my local version of Chrome and have the debugger attached, so I'm not using the headless mode there but instead can track the actions.

Browser Automation

Finally, here's a simple method that automates the sign up process. You'll note that writing E2E tests is quite cumbersome, because you're really just describing step-by-step what's happening. But you'll be able to test your actual project in real world conditions and have it run automatically, which is pretty much worth it!
You're now able to fully test your app in any environment, all you need is Docker.

Happy testing!

Executing NUKE Build Scripts on Linux Machines with Correct File Permissions

Sun, 16 Feb 2020 16:04:04 Z

I've blogged before about the NUKE build system, a great tool written by Matthias Koch that allows you to define all your build steps in a C# project, complete with full IDE integration and lots of helpers to get you started quickly. Since work on NUKE has begun almost three years ago, it's grown to be a fully featured build system with a great community. We're using it in all of our projects at DanglIT, and the experience so far has been great!

One thing, though, that's always bugged us was that when we invoked the build on Linux machines, we always had to explicitly prefix it with the shell command, like bash build.sh Test -Configuration Debug

We could never directly invoke the build scripts comfortably on Linux, no matter if we were using a project that has had NUKE added over 2 years ago or for a fresh install. First, we thought it might have been related to line ending differences, since all our development happens on Windows machines. That could quickly be checked, and it wasn't the case - checkout worked as expected on Linux.

After some digging to find out what we were doing wrong, it turned out that the culprit was the executable permissions of the scripts. If you're on Linux or Mac, git will recognize scripts as executable files and mark them as such. If you're on Windows, however, they're simply added as regular files. The answer from Antwane at StackOverflow explains it quite nice, but in short, you just set the chmod permission via git: git update-index --chmod=+x build.cmd

Before, we saw that the file permissions were 644, or read/write:

After updating the index, we get the correct permissions, including executable 755:

After you've done that, commit your changes. Now, your scripts should also work on Linux as expected. At least, it worked for us!

Happy building!

Cancel Obsolete Http Requests in Rx.NET with the Switch Operator

Thu, 30 Jan 2020 19:51:54 Z

Using Observables in Rx.NET is a super fun and efficient way to organize data flows in your applications. While it's widely used for front end development - it's basically a core part of Angular - it's actually a great library to use in any projects where you're using loosely coupled services and state management.

One of these use cases is having a filtered list. Imagine you've got a small app that shows you some table, along with a single text field for filtering. Imagine also that this list is fed from some asynchronous source, so it's not an in memory filter operation but maybe the response from a Http call.

Now, when you start typing in this box, what should happen? In the simplest case, without delays or checks, every letter typed in that filter box sends a Http request. That's probably a bit of an unoptimized approach, but it should work. This could look something like the following:

What's actually happening is that you're creating a stream of events in your code. Whenever you change the url, you're sending a request. When that request finishes, you're updating the list. The problem now comes when the requests don't come in the order they've been sent. Your user might have typed pizza, deleted that and then again typed pasta. If the first request, for really any reason, was delayed, you might have the situation where you're sowing pizza results to pasta guys!

To resolve this, you want to track which requests have been sent and discard all but the latest one. In rxjs, you'd be using the switchMap operator. In Rx.NET, it's simply called Switch and expects you to either provide a Task or another IObservable:

What's happening here is that only the latest call will come through. So when you've got more than one request running at the same time, you're never again getting stale data.

If you want to dive deeper into this, you can take a look at the source code for LightQuery, which uses this approach in conjunction with CancellationTokens. I'd also like to thank Brandon for providing a great answer at StackOverflow, which really goes into detail!

Hope you're going to have fun with Rx.NET!

Accessing the Revit API in Callbacks from Other Threads

Tue, 21 Jan 2020 11:41:25 Z

Just this week, I've been doing some work with a plugin for Autodesk Revit. The general idea of the plugin is to show a window which basically just wraps a Chromium embedded browser to display a web UI and interact with the Revit API. The interaction itself is pretty straightforward - with CefSharp, you can bind JavaScript objects in the browser to .NET objects in the plugin and thus access Revit features.

We've implemented that by having a singleton on the .NET side that listens to commands from the browser. The first, possibly naive approach, was to simply call our code whenever we got a message:

However, this didn't work. While all the data was transferred correctly, whenever we accessed certain features of the Revit API, we got a SEHException from unmanaged code in Revit. A quick Google search yielded this bit of info: "SEHExceptions are always indications of a bug in Revit.". Ok, we thought, but that's super basic stuff we're accessing, there's probably not an undiscovered bug with Revit here.

Luckily, debugging this gave us a bit more info: The API is working correctly when accessed from the main thread, but throws when accessed by another one. So even though we weren't doing any UI operations, this seemed to be the root cause of our issue. Since you also can't invoke something on the main thread in Revit, with Application.Current.Dispatcher not being set, we fell back to using the UIApplication.Idling event and a stack to invoke our methods on the main thread:

This cost us a few hours, so I hope it's helpful to others!

Happy integrating!

Handling DateTimeOffset in SQLite with Entity Framework Core

Tue, 15 Oct 2019 12:07:18 Z

Recently, ASP.NET Core 3.0 was released, along with all it's supporting libraries like Entity Framework Core. In the process of migrating Dangl.Identity over to the new version, I discovered that some integration tests failed with this message:

System.NotSupportedException : SQLite cannot order by expressions of type 'DateTimeOffset'. Convert the values to a supported type or use LINQ to Objects to order the results.

The error message is pretty clear - SQLite with Entity Framework Core 3.0 does no longer support some operations when using DateTimeOffset properties in database models, as specified in the official Microsoft Guidelines on limitations with SQLite.

The recommendation to switch to a supported type is great, but what to use? Falling back to regular DateTime, you'll lose the time zone information. Even if you're storing only UTC dates, while ensuring you're never making an error anywhere you touch dates, Entity Framework will always return a DateTimeKind.Unspecified when retrieving values from the database. While you can work around that with some conversion via an EntityMaterializerSource, this feels awkward and error prone.

Luckily, aptly named user bugproof on GitHub posted a great snippet that attaches a built-in converter for all properties in your database model. Here's how I've implemented it in my database context class:

The only drawback is that the conversion only supports up to millisecond precision, but for most uses cases this is likely not a problem. In case you're comparing values, simply trim the last three digits from your values in your test code and you're good to go:

Happy modelling!

Performing End-to-End Tests for a .NET Core Web API with an Angular Frontend in Docker with Jenkins

Fri, 26 Jul 2019 18:12:04 Z

If you've been following my blog in the last few years, you might have noticed that I kind of like testing & automation. Mostly, I'm working with .NET and TypeScript in cloud and backend services, so plain old integration testing gets me pretty far. Recently, however, I've done a bit more on the frontend side and I wasn't happy with the way I setup my testing.

Basically, I had separate tests for the frontend and backend, with just very rudimentary end to end tests that were run on a pre-production slot in Azure. If these passed, the slot was switched live and a new version was deployed.

This approach worked, but had three big drawbacks:

It was slow. The app had to be built and deployed to the cloud. Results from this could take up to 10 minutes until I had feedback
It couldn't be run locally, since it really was more of a pre-deployment check than an actual test. This also meant I didn't get feedback on all code changes, only on those in a branch that got deployed
Testing was limited to operations that didn't change any production data, such as creating new users or assigning roles

It was a really uncomfortable situation to be in, and the solution was clear: Automate it, dockerize everything and have full end to end tests run on every single commit in a headless Chrome browser. It took me a bit to get started with all this, so in this article, I'll describe in detail how I did it.

What are end-to-end tests?

In automated testing, there are different types or categories of tests you write. The definitions are always a bit fuzzy, everyone has a different opinion on them. The most fundamental tests are called unit tests, checking the behavior of a small, independent unit. Integration tests verify that multiple components in conjunction work as expected, usually with a longer run time than unit tests. Finally, automated end-to-end or e2e tests are performed on the whole application, from a users perspective. Think browser or UI automation.

You should call them what's most appropriate in your situation. For example, is an automated test checking the REST API endpoints of your backend, hooked up to a local database, still an integration test or already end-to-end? I'd say it's an integration test, since your users typically don't access the API directly. Except when they're developers themselves and do... So, don't get caught up in semantics😊

For this article, end-to-end tests are tests that simulate the users behavior in a browser, they test both the frontend code and the backend together, simulating real user actions on your production system.

Overview

This post won't go into some details that are out of the focus here. I'll assume you have some understanding of using Jenkins to automate your tests (or any other CI service, like Azure DevOps or AppVeyor), are familiar with Docker as well as basic unit testing approaches and frameworks such as xUnit. Additionally, I'm using the NUKE build system to orchestrate the setup, but simple bash or PowerShell commands will be perfectly fine for you.

Docker Setup

The easiest way for me was to run all required services in Docker Compose. Let's take a quick look at the following script:

My setup consists of three images:

Dangl.Identity, which is the app under test. It's an OpenID server with user, rights & role management and related services, built on IdentityServer4, that we're using at DanglIT
Dangl.Icons, which is a really small service that's similar to Gravatar, it essentially generates user icons
SQL Server, to match the setup of the production environment

The setup is app specific and likely quite different in your case, but the gist is that we'll have a full featured app with all required services, mirroring the production environment, available at http://localhost:44848 when running this composition.

Test Configuration

To actually perform the test, a mix of technologies is used: While xUnit is the test framework itself, I'm using Selenium for browser automation. A minimal .csproj looks like this:

The main object you'll be using for e2e tests is the ChromeDriver. It's your context for interacting with the browser and really easy to setup:

You'll notice that I check if the debugger is attached to check if I'm running it locally in Visual Studio, otherwise I set it to headless and simulate a Full HD resolution. This example class makes use of xUnits Fixture, but that's just for performing some initialization for the test environment, you can skip that.

Test Implementation

Now you're ready to write tests! I'm usually separating them again at this point, one category represents simple, minimal tests while the other covers user stories, e.g. complex tasks from registration to login to performing some tasks on the site.

Let's take a look at a simple test, checking if the UI disables the Register button when the user missed filling in her username:

The code should be simple to follow, I hope😊 More complex tests aren't much different, it's just more code. You'll generally notice a trend here - while unit tests require very little code, e2e tests quickly grow into dozens of lines if you're doing more complex actions.

Running your Tests

Depending on how you're automating your pipeline, you're using bash or PowerShell or something similar. I'm a huge fan of the NUKE build system, so that's what I naturally use for all my automation needs. Let's take a look at the implementation:

Examining it closely, it does just two things: It runs docker-compose up to start the Docker environment in a non-blocking way and then executes dotnet test. xUnit test results are both printed to the console and saved as testresults.xml for further analyzing.

Running Tests in Jenkins

The Jenkins pipeline is configured via a Docker E2E Tests build step. It's configured to run on a Linux node, executes NUKE with the E2ETests target and then reports the test results:

You'll see it's worth the effort when a nice, all-green stage view greets you in Jenkins:

Summary

While there are quite a few steps involved in performing proper end-to-end testing, the payoff is worth it: You're able to validate a mirror of your production setup in exactly the same way your users are using the app. While this post has just scratched the surface of what you can do, it hopefully gave you a good starting point to implement your own strategy.

Happy testing!

Impact of O(n²) Runtime in Practice - With Before & After Results

Fri, 14 Jun 2019 13:44:33 Z

With AVACloud, we're providing a hosted SaaS service for our GAEB & AVA .Net Libraries, a project calculation module & data exchange format for the construction industry.

Among the paid service, there are some free offerings for AVACloud for limited use cases. With that, users often convert from the various data formats to Excel. In our monitoring, we noticed some irregularities from the free service. Conversions usually take just tens or a few hundreds of milliseconds. But we had some that took about half an hour, way to long for any one user to be happy.

Luckily, we've been contact by the user and were asked for assistance. After a bit of investigation and profiling, we quickly found the culprit - we had a part in the conversion process that had O(n²) runtime:

The graph above shows the run time of the complete conversion process by the amount of elements within a parent container. It's a pretty nice O(n²) graph, at least visually, not so much for actual users😊

The thing is, this wasn't really noticed anytime before. In practice, service specifications for construction projects look like this:

So while real construction projects often have up to a few hundred of positions, sometimes even a few thousand, they're structured in a tree so that each container itself only has at most tens of child elements.

The offending part of the code actually worked on the level of a single container. It did perform a validity check when adding a new element, by checking against all siblings. This wasn't a problem until one day, a customer had a single container with over 40.000 elements that really triggered this...

The fix was rather easy, it basically just amounted to introducing a cache. And thus, the performance quickly became linear:

In that case, the dotted red line is the previous, O(n²) runtime, while blue represents the situation after the fix. Even more impressive is a zoomed view, which shows just how massive the difference from such an oversight can be:

In hindsight, the error was very obvious. It wasn't noticed because typical data sets were always small enough so this never really was a problem, until it one day was... Luckily, the fix could be implemented very quickly, and I got a satisfied user and some material for a blog post out of it😊

Happy optimizing!

Recap from the BCF Hackathon in Helsinki and Formation of the CDE Group

Sun, 19 May 2019 10:29:31 Z

This past week, the BCF implementers group came together at the Solibri HQ in Helsinki, Finland, for their bi-annually Hackathon. Besides the regular conference calls, this is a great time to discuss all things related to the BIM Collaboration Format, a standard that should make it easier for users to collaborate on planning & building construction projects across teams. It's solving a key issue in the whole BIM process - the integration of various professionals and tools, often across companies and disciplines.

Photo © Oliver Günter, from left to right: Georg Dangl, Oliver Günter, Pieter Buts, Josephus Meulenkamp, Andrea Dallera, Eduard Mrazek, Pasi Paasiala, Henning Kongsgård, Jari Juntunen, Kristof Kerekes, Veni Lillkåll, Simon Daum, Nick Tindall, Rahul Sule

Past & Future of BCF

BCF currently stands at version 2.1, and it's available both as a file-based standard as well as an Http REST API specification. It's quite stable now and feedback from production shows little problems, except some minor issues every now and then.

Support by applications is steadily increasing, but still mostly focused on the file based Xml exchange format. One of the big goals of the group is to support implementations of an API-based workflow. While there are many options already on the server side, there are still way too few client applications available that use the BCF REST API. For this year, at least, the open source tool BCFier is planned to get some BCF API features implemented.

Another point is to get more end users involved. The BCF Group has always been a very technical group, and most of the standardization efforts were driven by software vendors. This caused long feedback cycles between the group and end users. Now that the standard is more stable, we're actively trying to include more designers, contractors and users into the decision processes. Starting with regional buildingSMART clusters, we're always open to any new members or input.

Formation of the CDE Group

Something that's been discussed lately is the formation of a group that focuses on standardizing APIs meant for Common Data Environments. Similar to what's being done with BCF, successfully utilizing a CDE in construction projects requires that the users - designers, contractors, stakeholders - are able to integrate their own tools in the collaborative workflow.

Common Data Environments are mostly cloud based applications that aggregate and manage data from various sources. Simply said, they're tools that support all or most digital aspects of the construction process in one central place. For this to work, it needs to integrate with existing workflows and applications.

The very basic roadmap for this group is as follows:

The standard for authenticating API requests, part of the BCF API, will be extracted and made its own specification
A basic API for managing documents between applications and CDEs
Further down, some kind of directory service is needed, to discover projects, manage users and roles across systems
In the long term, this group might also extend the web based APIs to support object and attribute level access in the context of digital building models (BIM)

The decision to form the group was made by Catenda, Graphisoft, Oracle Aconex, Solibri and think project! at or before the Hackathon in Helsinki. It will be part of the Technical Room at buildingSMART International Standards Program, and there have been already some early discussions about the document management part, published on GitHub.

Transform your ASP.NET Core Website into a Single File Executable Desktop App

Thu, 21 Mar 2019 16:34:07 Z

Using web technologies to create desktop apps has been around for quite some time now, with frameworks like Electron being mature and used in lots of products, such as Spotify and Slack. I've only ever used it for side projects, until finally this week a customer asked me if I could provide a desktop application of our popular WebGAEB converter. Of course I could! So I sat down at my computer and made an ASP.NET Core MVC app into a desktop app, bundled as a single executable file.

This approach works by bundling everything together - your frontend, sometimes even your ASP.NET Core backend server and it's own embedded browser. This has been a bit polarizing in the last years among various communities. It's great for developer productivity, but it's also a very wasteful way to use a computers resources.

However, I'm leaning heavily towards productivity to be able to quickly build features our users care about. I could have built a dedicated .NET WPF app from scratch, or I could just use Electron and be done in two hours. That's what I'd like to share with you in this blog post.

Build Script

The following is the complete build target for Nuke Build. You're free to use something else to orchestrate your build, but even if you're unfamiliar with this tool, you'll see that it's quite straightforward:

I'll explain the script step-by-step, but I will skip the trivial parts, like copying the files to the output directory. If anything is unclear, please leave a comment!

Configure Electron.NET

Electron.NET is an integration for the the Electron package with ASP.NET Core. It's got a great documentation at it's project site, so I'm not going to deeply into the details.

When you set up your ASP.NET Core project, you're using the Electron.NET package to instrument the interaction with the Electron shell. The result will be a self contained deployment that bundles the Kestrel server into your app.

The Dangl.WebGAEB solution uses a Standalone configuration which sets up some things differently in contrast to the web deployment. For example, I'm using embedded services instead of calling a REST API. This heavily depends on your app as a whole and is not covered in this post.

Build a Single Executable File with Warp

While Electron itself is cool and all, it's Node.js heritage really shows: After the app is created, you'll have a folder with tons of files. This isn't a problem if you're doing some big business app with an installer, but I simply want a single executable file, .exe for Windows in my case. I remembered the Warp project, which I've read about some time ago. This tool takes a folder, bundles it into a single executable file and lets you specify an entry point.

Change the Generated Exe to Hide its Console Window

One drawback to this approach is that warp internally starts your ASP.NET Core application, which itself is a console host. The final Electron window is then again launched from your ASP.NET Core server. This means there will be an additional console window present, besides your app. This won't fly with users, so it has to be disabled.

Truth is, I'm not really familiar with how this works. But there's some metadata in executable files on Windows. One of them is whether or not an .exe should show it's console host. Unfortunately, this isn't configurable until .NET Core 3.0, so we have to find another way. Fortunately, there's a great wiki page on the AvaloniaUI GitHub page. It's explain how the editbin.exe utility, which is included in Visual Studio, can be used to patch your executable. So just go find editbin.exe in your Visual Studio installation folder and you're good to go!

Upload your Artifacts

Your final result will be a single executable file. Which contains your ASP.NET Core backend, the Kestrel webserver, your frontend, and last but not least, a full web browser. I see where the critics saying this is wasteful are coming from, but it just works amazingly well😊

This example is pushing the artifacts, along with some documentation, to DanglDocu. There, my customers get notified about new releases and can download it. For you, that's probably different!

The Result

You can check out the online version of WebGAEB. It's basically the same as the GAEB converter on this site, except it's in German. The cleverly named "Desktop Edition" looks like this and runs 100% locally:

Happy packaging!

Dangl IT at the Gründerpreis Rosenheim 2019 (Founders Price) Finals

Wed, 13 Feb 2019 10:28:00 Z

I've grown up and lived ever since in Rosenheim, a beautiful, small city right at the foot of the alps. It's always been an economic center for the region, and in recent years, digital industries and services have increased a lot.

This is supported by the city and county. Among lots of offerings for businesses, they're hosting the bi-annual Gründerpreis Rosenheim (Founders Price). This competition is aimed to support local startups and small businesses to gain traction. Although it's not strictly aimed for digital only, over half of the participants in the current round are from this group.

Over the last months, many events have taken place where founders could learn the basics of starting and operating a successful business. Courses included topics such as finances, legal, but also marketing, corporate design and how to write a convincing business plan.

My company, DanglIT GmbH, which was founded just last year, currently takes part. DanglIT is focusing on providing solutions for the building industry, such as working with digital building models, data exchange and interoperability scenarios and cloud services.

Last Friday, the final company presentations of the business plans took place. There, I had the opportunity to present my company to a group of experienced professionals from the region. The slide in the picture shows the weekly active users of WebGAEB, where I managed to increase the users from around 200 per week to almost 500 in January (with a small dip around Christmas, where there's little activity for a B2B app).

Personally, I've learned a lot in the last weeks and made great, new contacts. I've gotten valuable feedback and insights into running a successful business.

Thanks to everyone involved for making such a great platform possible!

Integrating BIM & IFC References with Bills of Quantities and GAEB Files

Wed, 16 Jan 2019 20:09:49 Z

The creation of fully integrated and connected representations of construction projects with Building Information Modelling (BIM) is becoming ever more important. One key point of BIM is that data should no longer be split into individual, separate data islands but that everything should be connected. This leads to real, measurable benefits by avoiding duplication, transfer errors and a much better understanding of the building that's designed and operated.

The BIM process relies greatly on open data formats and a solid, technological foundation in data exchange. Unfortunately, we're still in very early stages of this, so there are many problems in today's industry when different applications talk to each other. At DanglIT, we're tackling this by providing individual software and finished products to deal with that. One of our specialties is the German GAEB data standard, which is a container format used to describe bills of quantities. For building models, the Industry Foundation Classes (IFC) files are often used to represent buildings in an open and well known data format.

Many of our customers ask about connecting these GAEB files with the actual building models. To showcase this with a practical example, let's say you've got multiple concrete walls in a building. Often, these walls are represented by different positions in the bill of quantities. An outer wall can be represented as Concrete, Casing and Reinforcing Steel. On the other hand, you will often only have a single position for all the concrete in all the walls in a project, so there's a mismatch between the data in the building model and the data in the bill of quantities - but both represent the same, real entity!

Linking such cost elements between GAEB and IFC is actually quite easy, and already fully supported with our GAEB & AVA .Net Libraries and SaaS offering. In GAEB, there's the concept of CatalogueReferences, which are typically used to reference some kind of classification but can also hold user defined data. In the example below, the GAEB file (click here to download) defines a custom catalog that references the IFC file which contains the building model:

Now, a single position element in the GAEB file can reference this catalog. The example below contains all the concrete for the project, but a Quantity Split links a sub quantity of 10 m³ to an element in the building model with the Id 04BUoi9FT31BWlyfND2MS2 (yes, that's how IfcGuids look!):

For reference, you can view the full GAEB XML file directly in the browser.

Such simple connections between GAEB and IFC give you real value. For example, here's a super simple web app we've put together - you can load the building model and the GAEB file to visualize the connections between them:

To create the GAEB XML file above with our GAEB & AVA .Net Libraries, here's some example code you can use. That's the complete code actually - you don't need a single line of code more than what is shown to produce the output.

If you have any questions about working with GAEB, IFC, BIM or other, related technologies we're always happy to help. Just contact us!

Oh, and there's one big bonus to this approach: The next version 3.3 of the GAEB XML standard is scheduled to be published by the second half of 2019. Among the new features is a native support for connecting IFC with GAEB. It's going to be working in basically the same way as the example shown above.

Happy Connecting!

Caching Bundled and Minified CSS and JavaScript with the Wrong File Extension in the Free Tier on CloudFlare with Umbraco

Fri, 11 Jan 2019 19:47:44 Z

I've been using CloudFlare for quite some time now, on this blog and on many of my other websites. It's a great service that offers lots of value even in it's free tier. Most useful, for me, is it's built-in CDN (Content Delivery Network) functionality. This means basically the following:

This blog here is hosted on an Azure server somewhere in Europe. That's great for people living near, but bad for all the visitors outside of Europe. Or even just a few hundred kilometers away from my server. It's bad in the sense that web requests take a lot of time when the physical locations of client and server are continents apart. Even worse, when the first request to my main page is finished, a second round of requests retrieves images, CSS and JavaScript files. That's where CloudFlare comes into play: They'll cache all my assets on their servers, of which they have many around the world. So they're kind of a proxy between this blog and it's visitors and they optimize delivery by caching my content, meaning typical page loads from North America, for example, no longer take 8 seconds to complete. Most web requests don't even hit my server, at all.

I'm using the free tier on this site, which works well enough but has some limitations. For example, they're only caching a handful of file extensions, so .js and .css files are cached but, .axd are not. Since I'm using Umbraco on this blog, the built-in ClientDependency package makes sure that all my JavaScript and CSS files are bundled and minified in production builds. The package internally works by routing the requests through a custom handler via a route similar to /DependencyHandler.axd?s=h4sh&t=JavaScript. Bundling and minification is great and what we want, but we also want CloudFlare to cache the requests. We'll have to get Umbraco to serve it via urls with the correct file extensions then!

First, you need to create a custom renderer that outputs the desired file endings in your website links to the bundles:

This new renderer should be registered in your ClientDependency.config file. Simply replace the old one:

Now, for the magic to work, we're using a rewrite rule in IIS' web.config file to map requests that come to either DependencyHandler.js or DependencyHandler.css internally to the route DependencyHandler.axd:

Happy caching!

Update, one week after the original post: With this approach, I managed to increase the cached bandwith via CloudFlare from about 20% to over 60%! Most of the uncached content now is the dynamic content generated by the GAEB converter. Virtually everything else doesn't cost me any more bandwith!

Remove Duplicate Enum Entries in Swagger Documents with NSwag in ASP.NET Core

Thu, 03 Jan 2019 20:18:36 Z

NSwag and NJsonSchema are great tools that make it super easy to integrate Swagger / OpenAPI specification documents in your ASP.NET Core apps.

Basically, you simply define your web server backend and then all your API endpoints and models are automatically available for anyone to generate clients in their favorite language.

In such scenarios, NSwag is often configured to use a StringEnumConverter, meaning that enum values are meant to be serialized as strings when they're sent over the wire. However, there's one nasty thing that's allowed by the C# specification when it comes to this:

You might actually have two different string representations that both match an internally used integer identifier. That becomes a problem when you're using string serialization for enumerations but track them internally by their integer id.

In NJsonSchema, for example, there's a small bug that would lead to TemporaryRedirect being serialized twice in this case. That's sometimes not a problem, but strongly typed languages like C# or Java will fail to compile the generated client code.

To solve this, you can simply remove these distinct values in a PostProcess action when you're setting up NSwag for your ASP.NET Core application:

Happy serializing!