As I’ve written before, I’m leaning heavily into immutability in the election site code. Until September 2025 (it’s taken a long time to get round to writing this blog post) that meant a combination of records, ImmutableList<T> and ImmutableDictionary<TKey, TValue>.

In an ECMA C# standards meeting, however, Joseph Musser passed on some really valuable feedback – that those immutable types are significantly less efficient than ImmutableArray<T> and FrozenDictionary<TKey, TValue> for the way that I use them.

Usage patterns of immutable collections

There are two very distinct patterns for how you might use immutable collections.

The pattern that ImmutableList<T> and ImmutableDictionary<TKey, TValue> support really well is “build over time, using the intermediate collections”. Both of these have Add methods which returns a new immutable collection based on the previous one, but with an additional entry. There are certainly plenty of places where this usage pattern is appropriate – but my election site hardly ever does this.

Instead, my code tends to create a collection (often using LINQ, but not exclusively) and then never “modify” it again. When something in the data changes, I create a new collection from scratch rather than basing a new collection on the existing one. This is the usage pattern which ImmutableArray<T> and FrozenDictionary<TKey, TValue> are designed for.

Migration

Migration from ImmutableDictionary to FrozenDictionary was really straightforward:

• Add imports for the System.Collections.Frozen namespace
• Rename my own extension methods (e.g. ToImmutableReferenceDictionary becomes ToFrozenReferenceDictionary)
• Call ToFrozenDictionary instead of ToImmutableDictionary
• Change declarations for a few properties and record parameters

That’s it! Aside from anything else, the simplicity of that change was validation that I really wasn’t using the ImmutableDictionary-specific methods for constructing new immutable dictionaries from existing ones.

Migration from ImmutableList to ImmutableArray was more fiddly, although never actually hard. There are two differences that are relevant:

• ImmutableArray is a value type (with a single field which is a reference to the backing array)
• ImmutableArray has a Length property (like arrays) as opposed to the Count property in ImmutableList

Again, looking over the commit, I can’t see anywhere that I was using the ImmutableList-specific Add method and had to change how I was constructing any lists. I didn’t have any helper methods to rename for ImmutableArray, but the value-type-ness did mean other helper method changes. For example, consider this declaration:

public static bool ElementsEqual<T>(ImmutableList<T>? left, ImmutableList<T>? right)

That’s fine even if the caller has non-nullable references, and you couldn’t overload it to the non-nullable equivalent parameters anyway – whereas with ImmutableArray you almost certainly don’t want to wrap a non-nullable ImmutableArray value in an ImmutableArray? just for it to be checked and unwrapped again, so I’ve ended up with overloads:

public static bool ElementsEqual<T>(ImmutableArray<T>? left, ImmutableArray<T>? right)
public static bool ElementsEqual<T>(ImmutableArray<T> left, ImmutableArray<T> right)

(The “only-one-is-nullable” possibilities aren’t worth adding in my codebase, although they’d be entirely valid.)

Likewise, in various places I would have code such as:

public void DoSomething(ImmutableList<string>? list)
{
    if (list is null)
    {
        // Whatever needs to happen for the null case
        return;
    }
    // Now use list in a non-nullable way. The compiler knows it's not null,
    // so I don't get warnings.
    // For example:
    foreach (var element in list)
    {
        ...
    }
}

That doesn’t work with the trivial migration to ImmutableArray because the parameter is a nullable value type. So there end up being two different options. We can “unwrap” to the non-nullable value type using the .Value property, which is usually the simplest option when we only use it once:

public void DoSomething(ImmutableArray<string>? array)
{
    if (array is null)
    {
        // Whatever needs to happen for the null case
        return;
    }
    // Use .Value to access the array
    foreach (var element in array.Value)
    {
        ...
    }
}

Or use the is operator to declare a new local variable:

public void DoSomething(ImmutableArray<string>? array)
{
    if (array is not ImmutableArray<string> arrayValue)
    {
        // Whatever needs to happen for the null case
        return;
    }
    // Now we have a new non-nullable local variable, arrayValue
    foreach (var element in arrayValue)
    {
        ...
    }
}

The latter is the approach I’ve taken in most places where I’m using the array multiple times in a single member.

Performance benefits

I was initially slightly skeptical about the possible benefits of changing collection types. How bad could the existing types be, really? The site was already performing “well enough” for me anyway.

At the time of migration, I only had three benchmarks:

• Constructing ConstituenciesVM
• Constructing CandidatesVM
• Validating ElectionContext

All of these three improved after migration, but the construction only improved a little bit… whereas ElectionContext.Validate improved from 5.5ms to 0.826ms – a huge improvement, due to it performing lots of read accesses.

Do I care that much about ElectionContext.Validate? Not really – on its own, that wouldn’t justify the change, particularly if the code ended up being more complex or hard to read. But it does demonstrate that the benefits aren’t purely theoretical. Given that my usage pattern is really, really closely correlated to the FrozenDictionary/ImmutableArray design, I figured I might as well take the plunge.

Realistically, even if I actually get a lot of traffic in the end (which is still a big “if”) almost all the traffic will end up being served from naturally-efficient paths without seeing much benefit from this. The relevant bits of “work” to be done are:

1. Loading the data from Firestore and converting it into the low-level models and the overall ElectionContext
2. Building view-specific view-models from the ElectionContext
3. Rendering Razor pages based on view-models
4. Serving the pages (and API requests)

Of these, only steps 3 and 4 happen in the serving path. Steps 1 and 2 only happen when data has actually changed, and even then they happen during health checks, so aren’t user-facing.

Even step 3 only happens once every 10 seconds per page, per instance – all pages (and most API responses) are behind a 10 second cache.

I should (and will) add more benchmarks for Razor page rendering and API serving. It may be possible to backport those benchmarks to a branch of the code before the migration. I’d expect to see modest gains from the migration in most cases, and potentially more significant gains in just a couple of cases. I don’t expect to see any regressions, but obviously half the point of benchmarking is that expectations are often confounded.

Conclusion

Overall, this was a simple change to make, and one that I’m glad I’ve done.

I’m left with two niggles though:

Firstly, having different name prefixes (and even namespaces) just feels a little untidy. It’s not like I’m using the types in very different ways, so why is one Immutable and the other Frozen? This is a purely aesthetic niggle, and one that I suspect would bother some other people, and leave others completely baffled as to why I’d even mention it.

Secondly, the value-type-ness of ImmutableArray definitely gets in the way a bit. This is mostly in terms of nullability, but it also feels a little wrong… because really we are dealing with a reference, just wrapped in an ImmutableArray type. It’s almost like we need a new kind of type: “this is just wrapping a single reference, so pass the reference around directly under the hood, but then use it via the members declared in the wrapper type (and don’t allow direct access to the real reference).” I’m sure I’ve written plenty of types like this in the past myself, and currently they always either need a pointless object involved (with the performance cost of constructing and garbage-collecting these wrappers) or you end up with the value type wrapper with accompanying weirdness. The additional language complexity probably isn’t worth even proposing this, but it would be an interesting thought experiment.

I could address both niggles to some extent by introducing my own FrozenList class, but that would suffer from the performance cost mentioned above, and the possibility (or probability, even) of Doing It Badly. The niggles aren’t severe enough to justify this.

For now, I’ll live with the niggles and enjoy the improved performance. In other projects, I’m still using ImmutableList and ImmutableDictionary where I really don’t care about performance, but it’s good to know about the alternatives. It’s possible that I should make FrozenDictionary and ImmutableArray my “default” immutable types, but I’m not quite there yet…

In my previous blog post I described some behaviour of C# record types which was unexpected to me, though entirely correct according to the documentation. This is a follow-up post to that one, so if you haven’t read that one yet, please do so – I won’t go over all the same ground.

Is this a problem or not?

The previous blog post made it to Hacker News, and the comments there, on Bluesky, and on the post itself have been decidedly mixed.

Several people believe this is working as intended, several people believe it’s a terrible decision on the part of the C# language team.

Perhaps unsurprisingly, the most insightful comment was from Eric Lippert, who referred to a post on the Programming Language Design and Implementation Stack Exchange site. Eric has answered the question thoroughly, as ever.

I believe the difference in opinions comes from interpretation about what “with” is requesting. Eric wrote:

The with makes a copy of the record, changing only the value of the property you identified as wishing to change, so it ought not to be surprising that nothing else changed.

That’s not how I’ve ever thought of with – I haven’t expected it to say “this object but with these different properties”, but instead “a new record, with the same parameters as the original one, but with these different parameters“. It’s a subtle distinction – sufficiently subtle that I hadn’t even bothered to think about it until running into this problem – but I suspect it explains how different people think about the same feature in different ways. I wouldn’t have thought of “setting the property” because I think of records as being immutable to start with: that the only way you can end up with a record where the property returns a different value is by providing that different value as part of construction. (Again, to be crystal clear: I don’t think I’ve found any bits of documentation which are incorrect. It’s my mental model that has been wrong.)

I haven’t gone back over previous YouTube videos describing the feature – either from the C# team itself or from other developers – to see whether a) it’s described in terms of setting properties rather than parameters; b) the videos describe the distinction in order to make it clear which is “right”.

In my defence, even when you do have a better mental model for how records work, this is a pretty easy mistake to make, and you need to be on the ball to spot it in code review. The language absolutely allows you to write records which aren’t just “lightweight data records” in the same way that you do for classes – so I don’t think it should be surprising that folks are going to do that.

So, after this introductory spiel, this post has two aspects to it:

• How am I going to stop myself from falling into the same trap again?
• What changes have I made within the Election 2029 code base?

Trap avoidance: Roslyn Analyzers

In the previous post, I mentioned writing a Roslyn analyzer as a possible way forward. My initial hope was to have a single analyzer which would just spot the use of with operators targeting any parameter which was used during initialization.

That initial attempt worked to some extent – it would have spotted the dangerous code from the original blog post – but it only worked when the source code for the record and the source code using the with operator were in the same project. I’ve now got a slightly better solution with two analyzers, which can even work with package references where you may not have access to the source code for the record at all… so long as the package author is using the same analyzers! (This will make more sense when you’ve seen the analyzers.)

The source code of the analyzers is on GitHub and the analyzers themselves are in the JonSkeet.RoslynAnalyzers NuGet package. To install them in a project, just add this to an item group in your project file:

<PackageReference Include="JonSkeet.RoslynAnalyzers" Version="1.0.0-beta.6"
        PrivateAssets="all"
        IncludeAssets="runtime; build; native; contentfiles; analyzers"/>

Obviously, it’s all very beta – and there are lots of corner cases it probably wouldn’t find at the moment. (Pull requests are welcome.) But it scratches my particular itch for now. (If someone else wants to take the idea and run with it in a more professional, supported way, ideally in a package with dozens of other useful analyzers, that’s great.)

As I mentioned, there are two analyzers, with IDs of JS0001 and JS0002. Let’s look at how they work by going back to the original demo code from the previous post. Here’s the complete buggy code:

// Record
public sealed record Number(int Value)
{
    public bool Even { get; } = (Value & 1) == 0;
}

// Use of record
var n2 = new Number(2);
var n3 = n2 with { Value = 3 };
Console.WriteLine(n2); // Output: Number { Value = 2, Even = True }
Console.WriteLine(n3); // Output: Number { Value = 3, Even = True }

Adding the analyzer package highlights the int Value parameter declaration in Number, with this warning:

JS0001 Record parameter ‘Value’ is used during initialization; it should be annotated with [DangerousWithTarget]

Currently, there’s no code fix, but we need to do two things:

• Declare an attribute called DangerousWithTargetAttribute
• Apply the attribute to the parameter

Here’s the complete attribute and record code with the fix applied:

[AttributeUsage(AttributeTargets.Parameter)]
internal sealed class DangerousWithTargetAttribute : Attribute;

public sealed record Number([DangerousWithTarget] int Value)
{
    public bool Even { get; } = (Value & 1) == 0;
}

The attribute doesn’t have to be internal, and indeed in my election code base it’s not. But it can be, even if you’re using the record from a different assembly. The analyzer doesn’t care what namespace it’s in or any other details (although it does currently have to be called DangerousWithTargetAttribute rather than just DangerousWithTarget).

At this point:

• The source code makes it clear to humans that we know it would be dangerous to set the Value property in a with operator with Number
• The compiled code makes that clear (to the other analyzer) as well

After applying the above change, we get a different warning – this time on n2 with { Value = 3 }:

JS0002: Record parameter ‘Value’ is annotated with [DangerousWithTarget]

(Both of these warnings have more detailed descriptions associated with them as well as the summary.)

Now you know the problem exists, it’s up to you to fix it… and there are multiple different ways you could do that. Let’s try to get warning-free by replacing our precomputed property with one which is computed on demand. The analyzers don’t try to tell you if [DangerousWithTarget] is applied where it doesn’t need to be, so this code compiles without any warnings, but it doesn’t remove our JS0002 warning:

// No warning here, but the expression 'n2 with { Value = 3 }' still warns.
public sealed record Number([DangerousWithTarget] int Value)
{
    public bool Even => (Value & 1) == 0;
}

As it happens, this has proved unexpectedly useful within the Election2029 code, where even though a parameter isn’t used in initialization, there’s an expected consistency between parameters which should discourage the use of the with operator to set one of them.

Once we remove the [DangerousWithTarget] attribute from the parameter though, all the warnings are gone:

public sealed record Number(int Value)
{
    public bool Even => (Value & 1) == 0;
}

The analyzer ignores the Even property because it doesn’t have an initializer – it’s fine to use Value for computed properties after initialization.

A new pattern for Election2029

So, what happened when I enabled the analyzers in my Election2029 project? (Let’s leave aside the bits where it didn’t work first time… there’s a reason the version number is 1.0.0-beta.6 already.)

Predictably enough, a bunch of records were flagged for not specifying the [DangerousWithTarget]… and when I’d applied the attribute, there were just one or two places where I was using the with operator in an unsafe way. (Of course, I checked whether the original bug which had highlighted the issue for me in the first place was caught by the analyzers – and it was.)

For most of the records, the precomputation feels okay to me. They’re still fundamentally pretty lightweight records, with a smattering of precomputation which would feel pointlessly inefficient if I made it on-demand. I like the functionality that I’m given automatically by virtue of them being records. I’ve chosen to leave those as records, knowing that at least if I try to use the with operator in a dangerous operator, I’ll be warned about it.

However, there are two types – ElectionCoreContext and ElectionContext, which I [wrote about earlier] – which have a lot of precomputation. They feel more reasonable to be classes. Initially, I converted them into just “normal” classes, with a primary constructor and properties. It felt okay, but not quite right somehow. I liked the idea of the record type for just the canonical information for the context… so I’ve transformed ElectionContext like this (there’s something similar for ElectionCoreContext):

public sealed class ElectionContext : IEquatable<ElectionContext>
{
    public ElectionContextSourceData SourceData { get; }

    // Bunch of properties proxying access
    public Instant Timestamp => SourceData.Timestamp;
    // ...

    public ElectionContext(ElectionContextSourceData sourceData)
    {
        // Initialization and validation
    }

    public sealed record ElectionContextSourceData(Instance Timestamp, ...)
    {
        // Equals and GetHashCode, but nothing else
    }
}

At this point:

• I’ve been able to add validation to the constructor. I couldn’t do that with a record in its primary constructor.
• It’s really clear what’s canonical information vs derived data – I could even potentially refactor the storage layer to only construct and consume the ElectionContextSourceData, for example. (I’m now tempted to try that. I suspect it would be somewhat inefficient though, as it uses the derived data to look things up when deserializing.)
• I can still use the with operator with the record, when I need to (which is handy in a few places)
• There’s no risk of the derived data being out of sync with the canonical data, because the ordering is very explicit

Ignoring the naming (and possibly the nesting), is this a useful pattern? I wouldn’t want to do it for every record, but for these two core, complex types it feels like it’s working well so far. It’s early days though.

Conclusion

I’m really pleased that I can now use records more safely, even if I’m using them in ways that other folks may not entirely condone. I may well change my mind and go back to using regular classes for all but the most cut-and-dried cases. But for now, the approaches I’ve got of “use records where it feels right, even if that means precomputation” and “use classes to wrap records where there’s enough behavior to justify it” are working reasonably well.

I don’t really expect other developers to use my analyzers (although you’re very welcome to do so, of course) – but the fact that they’re even feasible points to Roslyn being a bit of a miracle. I’m not recommending either my “careful use of records slightly beyond their intended use” or “class wrapping a record” approaches yet. I’ve got plenty of time to refactor if they don’t work out for the Election2029 project. But I’d still be interested in getting feedback on whether my decisions at least seem somewhat reasonable to others.

Postcodes

After a pretty practical previous post about records and collections, this post is less likely to give anyone ideas about how they might tackle a problem in their own project, and doesn’t have any feature requests for Microsoft either. It’s an area I’ve found really fun though.

An introduction to postcodes in the UK

I realise that most of the readers of this blog post will probably not be in the UK. Most countries have postal codes of some description, but the details vary a lot by country. UK postcodes are quite different in scale to US zipcodes, for example. A UK postcode is quite fine-grained – often just a part of a single street – so knowing a house/flat number an a postcode is usually enough to get to a precise address.

Importantly for my election site, every postcode is in a single constituency – and indeed that’s what I want to use them for. My constituencies page allows you to start typing a constituency name or postcode, and it filters the results as you type. I suspect that a significant proportion of the UK population knows their postcode but not the name of their constituency (particularly after the boundary changes in 2023) so it’s helpful to be able to specify either.

Wikipedia has a lot more information about UK postcodes but I’ll summarize it briefly here. Note that I only care about modern postcodes – the Wikipedia has details about the history of how things evolved, but my site doesn’t need to care about legacy postcodes.

A postcode consists of an outcode (or outward code) followed by incode (or inward code). The outcode is an area followed by a district, and the incode is a sector followed by a unit. As an example, my postcode is RG30 4TT: broken down into:

• Outcode: RG30
  • Area: RG
  • District: 30
• Incode: 4TT
  • Sector: 4
  • Unit: TT

Incodes are nice and uniform: they’re always a digit followed by two letters. (And those two letters are within a 20-character alphabet.) Outcodes are more “interesting” as they fall into one of the seven formats below, where ‘9’ represents “any digit” and ‘A’ represents “a letter” (although the alphabet varies):

• AA9
• AA99
• A9
• A99
• A9A
• AA9A

Note how the first two formats for outcodes only vary by whether they have one or two digits – and remember that an incode always starts with a digit. This isn’t a problem when parsing text that is intended to represent a complete postcode (as you can tell the length of the outcode by assuming the final three characters are the incode) – but when I need to parse an incomplete postcode, it can be ambiguous. For example, “RG31 4FG” and “RG3 1AA” are both valid postcodes, and as I don’t want to force users to type the space, “RG31” should display constituencies for both (“Reading West and Mid Berkshire” and “Reading Central” respectively).

Requirements

The requirements for my use of postcodes in the election site is pretty straightforward:

• Ingest data from an external source (the Office of National Statistics was what I found)
• Store everything we need in a compact format, ideally in a single Firestore document (so 1MB)
• Keep the data in memory, still in a reasonably compact format
• Provide an API with input of “a postcode prefix” and return “the set of all constituencies covered by postcodes starting with that prefix” sufficiently quickly for it to feel “instant”

Interestingly, Wikipedia refers to “a table of all 1.7 million postcodes” but the file I ingest currently has 2,712,507 entries. I filter out a few, but I still end up with 2,687,933 distinct postcodes. I don’t know whether the Wikipedia number is just a typo, or whether there’s a genuine discrepancy there.

Spoiler alert: the network latency easily dominates the latency for making a request to the API. Using airport wifi in Munich (which is pretty good for free wifi) I see a roundtrip time of about 60ms, and the page feels like it’s updating pretty much instantaneously as I type. But the time within ASP.NET Core is less than a millisecond (and that’s including filtering by constituency name as well).

Storage and in-memory formats

Logically, the input data is just a sequence of “postcode, constituency code” entries. A constituency code is 9 characters long. That means a very naive representation of 2712507 entries, each taking 16 characters (expecting an average of 7 characters for the postcode, and 9 characters for the constituency code) would be just under 42MB, and that’s before we take into account splitting the data into the entries. Surely we can do better.

There’s a huge amount of redundancy here – we’re bound to be able to do better. For a start, my naive calculation assumed using a whole byte per character, even though every character we need is in the range A-Z, 0-9 (so an alphabet of 36 characters) – and several of the characters within those values are required to just be digits.

Grouping by outcode

But before we start writing complex code to represent the entry string values in fewer bytes, there’s a lot more redundancy in the actual data. It’s not like postcodes are randomly allocated across all constituencies. Most postcodes within the same sector (e.g. “RG30 4”) will be in the same constituency. Even when we group by outcode (e.g. “RG30”) there are relatively few constituencies represented in each group.

At the time of writing, there are 3077 outcodes:

• 851 are all in 1 constituency
• 975 are in 2 constituencies
• 774 are in 3
• 348 are in 4
• 98 are in 5
• 21 are in 6
• 8 are in 7
• 2 are in 8

We also want to diffentiate between invalid and valid postcodes – so we can think of “no constituency at all” as an additional constituency in each of the lines above.

In each outcode, there are 4000 possible incodes, which are always the same (0AA, 0AB .. 9ZY, 9ZZ). So for each outcode, we just need an array of 4000 values.

A simple representation with a byte per value would be 4000 bytes per map, with 3077 such maps (one per outcode) – that’s 12,308,000 bytes.

Compressing the maps

Can we do better by using the fact that we don’t really need a whole byte for each value? For the 851 outcodes in a single constituency, each of those values would be 0 or 1. For the 975 outcodes in two constituencies, each value would be 0, 1 or 2 – and so on. Even without splitting things across bytes, we could store those maps as:

• 851 maps with a single bit per value (500 bytes per map)
• 975 + 774 maps with two bits per value (1000 bytes per map)
• The remaining (348 + 98 + 21 + 8 + 2) maps with four bits per value (2000 bytes per map)

That’s a total of (851*500) + ((975 + 774) * 1000) + ((348 + 98 + 21 + 8 + 2) * 2000) = 3,128,500 bytes. That sounds great – it’s saved three quarters of the space needed for the maps! But it’s still well over the maximum size of a Firestore document. And, of course, for each outcode we also need the outcode text, and the list of constituencies represented in the map. That shouldn’t be much, but it doesn’t help an already-blown size budget.

At this point, I tried multiple options, including grouping by sector instead of outcode (as few outcodes actually have all ten possible sectors, and within a sector there are likely to be fewer constituencies, so we’ll have more sectors which can be represented by one or two bits per value). I tried a hand-rolled run-length encoding and various options. I did manage to get things down quite a lot – but the code became increasingly complicated. Fundamentally, what I was doing was performing compression, and trying to apply some knowledge about the data to achieve a decent level of compression.

Around this point, I decided to stop trying to make the code smarter, and instead try making the code simpler and lean on existing code being smart. The simple 4000-byte maps contain a lot of redundancy. Rather than squeezing that redundancy out manually, I tried general-purpose compression – using DeflateStream and InflateStream. The result was beyond what I’d hoped: the total size of the compressed maps is just 500,602 bytes. That leaves plenty of room for the extra information on a per-outcode basis, while staying well under the 1MB document limit.

I would note that when I wrote about storage options, I mentioned the possibility of using Google Cloud Storage instead of Firestore – at which point there’d really be no need for this compression at all. While it would be nice to remove the compression code, it’s a very isolated piece of complexity – which is only actually about 15 lines of code in the end.

Processing a request

So with the storage requirements satisfied, are we done? Pretty much, it turns out. I keep everything uncompressed in memory, with each per-outcode map storing a byte per incode – leading to just over 12MB in memory, as shown earlier. That’s fine. Each per-outcode map is stored as an OutcodeMapping.

The process of converting a prefix into a set of possible constituencies is fiddly simply due to the ambiguity in the format. I don’t return anything for a single character – it’s reasonable to require users to type in two characters of a postcode before we start doing anything with it.

It’s easy to store a dictionary of “first two characters” to “list of possible OutcodeMapping entries”. That list is always reasonably short, and can be pruned down very quickly to “only the entries that actually match the supplied prefix”. For each matching OutcodeMapping, we then need to find the constituencies that might be represented by the incode part of whatever’s left of the supplied prefix.

For example, if the user has typed “RG30” then we’ll quickly filter down to “RG3” and “RG30” as matching outcodes. From that, we need to say:

• Which constituencies are represented by incodes for RG3 which start with “0”?
• Which constituencies are represented by incodes for RG30? (We’ve already consumed the whole of the supplied prefix for that outcode.)

By splitting it into outcode and then incode, it makes life much simpler because the incode format is so simpler. For example, if the user had typed in “RG30 4” then the questions above become:

• Which constituencies are represented by incodes for RG3 which start with “04”?
  • There can’t be any of those, because the second character of an incode is never a digit.
• Which constituencies are represented by incodes for RG30 which start with “4”?

We always have zero, one, two or three characters to process within an outcode mapping:

• Zero characters: the full set of constituencies for this outcode
• One character: the set of constituencies for that “sector” (which we can cache really easily; there are only ten sectors per outcode)
• Two characters: the fiddliest one – look through all 20 possibilities for that “half unit” in the map.
• Three characters: a single postcode, so just look it up in the map; the result will be either nothing (if the postcode is invalid) or a single-entry result

I suspect that there’s more efficiency that could be squeezed out of my implementation here – but it’s already fast enough that I really don’t care. At some point I might do a load test to see how many requests I can process at a time while still keeping the latency low… but I’d be very surprised if this caused a problem. (And hey, Cloud Run will just spin up some more instances if necessary.)

Conclusion

There are two things I love about this topic. Firstly, it’s so isolated. You don’t need to know anything about the rest of the site, election polls, parties etc to understand this small domain. You need to know about postcodes and constituencies, but that’s it. Aside from anything else, that makes it a pleasant topic to write about. Secondly, the results seem so wildly impressive, with relatively little effort. In around half a megabyte of data, we’re storing information about over 2.7 million postcodes, and the final lookup is dirt cheap.

There’s an interesting not-quite-contradiction in the approaches taken in the post, too.

Firstly, even though the data is sort of mapping a 6/7 character input (postcode) to a 9 character output (constituency code), it’s very far from random. There are only 650 constituencies, and the constituencies are obviously very “clustered” with respect to the postcodes (so two postcodes which only differ in the fifth or later character are very likely to belong to the same constiteuncy). The approach I’ve taken uses what we know about the data as a first pass – so that aspect is very domain-specific. Just compressing a file of “postcode constituency-code” lines only gets us down to just over 7MB.

In contrast, when we’ve got as far as “for each outcode, we have a map of 4000 entries, each to a number between 0 and 8 inclusive” (with awareness that for many outcodes, the numbers are either in the range 0-1 or 0-2 inclusive), trying to be “clever” didn’t work out terribly well. The code became increasingly complicated as I tried to pack as much information as possible into each byte… whereas when I applied domain-agnostic compression to each map, the results were great.

I don’t think it’s particularly rare to need to find that balance between writing code to try to take advantage of known data characteristics, and relying on existing domain-agnostic algorithms. This is just one of the clearest examples of it that I’ve come across.

Records and Collections

This post is to some extent a grab-bag of points of friction I’ve encountered when using records and collections within the election site.

Records recap

This may end up being the most generally useful blog post in this series. Although records have been in C# since version 10, I haven’t used them much myself. (I’ve been looking forward to using them for over a decade, but that’s a different matter.)

Having decided to make all my data models immutable, using records (always sealed records in my case) to implement those models in C# was pretty much a no-brainer. Just specify the properties you want using the same format as primary constructors, and the compiler does a bunch of boilerplate work for you.

As a simple example, consider the following record declaration:

public sealed record Candidate(int Id, string Name, int? MySocietyId, int? ParliamentId);

That generates code roughly equivalent to this:

public sealed class Candidate : IEquatable<Candidate>
{
    public int Id { get; }
    public string Name { get; }
    public int? MySocietyId { get; }
    public int? ParliamentId { get; }

    public Candidate(int id, string name, int? mySocietyId, int? parliamentId)
    {
        Id = id;
        Name = name;
        MySocietyId = mySocietyId;
        ParliamentId = parliamentId;
    }

    public override bool Equals(object? obj) => obj is Candidate other && Equals(other);

    public override int GetHashCode()
    {
        // The real code also uses EqualityContract, skipped here.
        int idHash = EqualityComparer<int>.Default.GetHashCode(Id);
        int hash = idHash * -1521134295;
        int nameHash = EqualityComparer<string>.Default.GetHashCode(Name);
        hash = (hash + nameHash) * -1521134295;
        int mySocietyIdHash = EqualityComparer<int?>.Default.GetHashCode(MySocietyId);
        hash = (hash + mySocietyIdHash) * -1521134295;
        int parliamentIdHash = EqualityComparer<int?>.Default.GetHashCode(ParliamentId);
        hash = (hash + parliamentIdHash) * -1521134295;
        return hash;
    }

    public bool Equals(Candidate? other)
    {
        if (ReferenceEquals(this, other))
        {
            return true;
        }
        if (other is null)
        {
            return false;
        }
        // The real code also uses EqualityContract, skipped here.
        return EqualityComparer<int>.Default.Equals(Id, other.Id) &&
            EqualityComparer<string>.Default.Equals(Name, other.Name) &&
            EqualityComparer<int?>.Default.Equals(MySocietyId, other.MySocietyId) &&
            EqualityComparer<int?>.Default.Equals(ParliamentId, other.ParliamentId);
    }

    public static bool operator==(Candidate? left, Candidate? right) =>
    {
        if (ReferenceEquals(left, right))
        {
            return true;
        }
        if (left is null)
        {
            return false;
        }
        return left.Equals(right);
    }

    public static bool operator!=(Candidate? left, Candidate? right) => !(left == right);

    public override string ToString() =>
        $"Candidate {{ Id = {Id}, Name = {Name}, MySocietyId = {MySocietyId}, ParliamentId = {ParliamentId} }}";

    public void Deconstruct(out int Id, out string Name, out int? MySocietyId, out int? ParliamentId) =>
        (Id, Name, MySocietyId, ParliamentId) = (this.Id, this.Name, this.MySocietyId, this.ParliamentId);
}

(This could be written a little more compactly using primary constructors, but I’ve kept to “old school” C# to avoid confusion.)

Additionally, the compiler allows the with operator to be used with records, to create a new instance based on an existing instance and some updated properties. For example:

var original = new Candidate(10, "Jon", 20, 30);
var updated = original with { Id = 40, Name = "Jonathan" };

That’s all great! Except when it’s not quite…

Record equality

As shown above, the default implementation of equality for records uses EqualityComparer<T>.Default for each of the properties. That’s fine when the default equality comparer for the property type is what you want – but that’s not always the case. In our election data model case, most of the types are fine – but ImmutableList<T> is not, and we use that quite a lot.

ImmutableList<T> doesn’t override Equals and GetHashCode itself – so it has reference equality semantics. What I really want is to use an equality comparer for the element type, and say that two immutable lists are equal if they have the same count, and the elements are equal when considered pairwise. That’s easy enough to implement – along with a suitable GetHashCode method. It could easily be wrapped in a type that implements IEqualityComparer<ImmutableList<T>>code>, although it so happens I haven’t done that yet.

Unfortunately, the way that records work in C#, there’s no way of specifying an equality comparer to be used for a given property. If you implement the Equals and GetHashCode methods directly, those are used instead of the generated versions (and the Equals(object) generated code will still use the version you’ve implemented) but it does mean you’ve got to implement it for all the properties. This in turn means that if you add a new property in the record, you need to remember to modify Equals and GetHashCode (something I’ve forgotten to do at least once) – whereas if you’re happy to use the default generated implementation, adding a property is trivial.

What I’d really like would be some way of indicating to the compiler that it should use a specified type to obtain the equality comparer (which could be assumed to be stateless) for a property. For example, imagine we have these types:

// Imagine this is in the framework...
public interface IEqualityComparerProvider
{
    static abstract IEqualityComparer<T> GetEqualityComparer<T>();
}

// As is this...
[AttributeUsage(AttributeTargets.Property)]
public sealed class EqualityComparerAttribute : Attribute
{
    public Type ProviderType { get; }

    public EqualityComparer(Type providerType)
    {
        ProviderType = providerType;
    }
}

Now I could implement the interface like this:

public sealed class CollectionEqualityProvider : IEqualityComparerProvider
{
    public static IEqualityComparer<T> GetEqualityComparer<T>()
    {
        var type = typeof(T);
        if (!type.IsGenericType)
        {
            throw new InvalidOperationException("Unsupported type");
        }
        var genericTypeDefinition = type.GetGenericTypeDefinition();
        if (genericTypeDefinition == typeof(ImmutableList<>))
        {
            // Instantiate and return an appropriate equality comparer
        }
        if (genericTypeDefinition == typeof(ImmutableDictionary<,>))
        {
            // Instantiate and return an appropriate equality comparer
        }
        // etc...
        throw new InvalidOperationException("Unsupported type");
    }
}

It’s unfortunate that the comments would involve further reflection – but it would certainly be feasible.

We could then declare a record like this:

public sealed record Ballot(
    Constituency Constituency,
    [IEqualityComparerProvider(typeof(CollectionEqualityProvider))] ImmutableList<Candidacy> Candidacies);

… and I’d expect the compiler to generate code such as:

public sealed class Ballot
{
    private static readonly IEqualityComparer<ImmutableList<Candidacy>> candidaciesComparer;

    // Skip code that would be generated as it is today.

    public bool Equals(Candidate? other)
    {
        if (ReferenceEquals(this, other))
        {
            return true;
        }
        if (other is null)
        {
            return false;
        }
        return EqualityComparer<Constituency>.Default.Equals(Constituency, other.Constituency) &&
            candidaciesComparer.Equals(Candidacies, other.Candidacies);
    }

    public override int GetHashCode()
    {
        int constituencyHash = EqualityComparer<Constituency>.Default.GetHashCode(Constituency);
        int hash = constituencyHash * -1521134295;
        int candidaciesHash = candidaciesComparer.GetHashCode(Candidacies);
        hash = (hash + candidaciesHash) * -1521134295;
        return hash;
    }
}

I’m sure there are other ways of doing this. The attribute could instead specify the name of a private static read-only property used to obtain the equality comparer, removing the interface. Or the GetEqualityComparer method could be non-generic with a Type parameter instead (leaving the compiler to generate a cast after calling it). I’ve barely thought about it – but the important thing is that the requirement of having a custom equality comparison for a single property becomes independent of all the other properties. If you already have a record with 9 properties where the default equality comparison is fine, then adding an 10th property which requires more customization is easy – whereas today, you’d need to implement Equals and GetHashCode including all 10 properties.

(The same could be said for string formatting for the properties, but it’s not an area that has bitten me yet.)

The next piece of friction I’ve encountered is also about equality, but in a different direction.

Reference equality

If you remember from my post about data models, within a single ElectionContext, reference equality for models is all we ever need. The site never needs to fetch (say) a constituency result from the 2024 election from one context by specifying a Constituency from a different context. Indeed, if I ever found code that did try to do that, it would probably indicate a bug: everything within any given web request should refer to the same ElectionContext.

Given that, when I’m creating an ImmutableDictionary<Constituency, Result>, I want to provide an IEqualityComparer<Constituency> which only performs reference comparisons. While this seems trivial, I found that it made a pretty significant difference to the time spent constructing view-models when the context is reloaded.

I’d expected it would be easy to find a reference equality comparer within the framework – but if there is one, I’ve missed it.

Update, 2025-03-27T21:04Z, thanks to Michael Damatov

As Michael pointed out in comments, there is one in the framework: System.Collections.Generic.ReferenceEqualityComparer – and I remember finding it when I first discovered I needed one. But I foolishly dismissed it. You see, it’s non-generic:

public sealed class ReferenceEqualityComparer :
    System.Collections.Generic.IEqualityComparer<object>,
    System.Collections.IEqualityComparer

That’s odd and not very useful, I thought at the time. Why would I only want IEqualityComparer<object> rather than a generic one?

Oh Jon. Foolish, foolish Jon.

IEqualityComparer<T> is contravariant in T – so there’s an implicit reference conversion from IEqualityComparer<object> to IEqualityComparer<X> for any class type X.

I have now removed my own generic ReferenceEqualityComparer<T> type… although it’s meant I’ve had to either cast or explicitly specify some type arguments where previously the types were inferred via the type of the comparer.

End of update

I’ve now made a habit of using reference equality comparisons everywhere within the data models, which has made it worth adding some extension methods – and these probably don’t make much sense to add to the framework (although they could easily be supplied by a NuGet package):

public static ImmutableDictionary<TKey, TValue> ToImmutableReferenceDictionary<TSource, TKey, TValue>(
    this IEnumerable<TSource> source,
    Func<TSource, TKey> keySelector,
    Func<TSource, TValue> elementSelector) where TKey : class =>
    source.ToImmutableDictionary(keySelector, elementSelector, ReferenceEqualityComparer<TKey>.Instance);

public static ImmutableDictionary<TKey, TSource> ToImmutableReferenceDictionary<TSource, TKey>(
    this IEnumerable<TSource> source,
    Func<TSource, TKey> keySelector) where TKey : class =>
    source.ToImmutableDictionary(keySelector, ReferenceEqualityComparer<TKey>.Instance);

public static ImmutableDictionary<TKey, TValue> ToImmutableReferenceDictionary<TKey, TValue>(
    this IDictionary<TKey, TValue> source) where TKey : class =>
    source.ToImmutableDictionary(ReferenceEqualityComparer<TKey>.Instance);

public static ImmutableDictionary<TKey, TValue> ToImmutableReferenceDictionary<TKey, TValue, TSourceValue>(
    this IDictionary<TKey, TSourceValue> source, Func<KeyValuePair<TKey, TSourceValue>, TValue> elementSelector) where TKey : class =>
    source.ToImmutableDictionary(pair => pair.Key, elementSelector, ReferenceEqualityComparer<TKey>.Instance);

(I could easily add similar methods for building lookups as well, of course.) Feel free to take issue with the names – while they’re only within the election repo, I’m not going to worry too much about them.

Interlude

Why not make reference equality the default?

I could potentially kill two birds with one stone here. If I often want reference equality, and “deep” equality is relatively hard to achieve, why not just provide Equals and GetHashCode methods that make all my records behave with reference equality comparisons?

That’s certainly an option – but I do lean on the deep equality comparison for testing purposes: if I load the same context twice for example, the results should be equal, otherwise there’s something wrong.

Moreover, as record types encourage deep equality, it feels like I’d be subverting their natural behaviour by specifying reference equality comparisons. While I’m not expecting anyone else to ever see this code, I don’t like writing code which would confuse readers who come with expectations based on how most code works.

End of interlude

Speaking of extension methods for commonly-used comparers…

Ordinal string comparisons

String comparisons make me nervous. I’m definitely not an internationalisation expert, but I know enough to know it’s complicated.

I also know enough to be reasonably confident that the default string comparisons are ordinal for Equals and GetHashCode, but culture-sensitive for CompareTo. As I say, I’m reasonably confident in that – but I always find it hard to validate, so given that I almost always want to use ordinal comparisons, I like to be explicit. Previously I’ve specified StringComparer.Ordinal (or StringComparer.OrdinalIgnoreCase just occasionally) but – just as above with the reference equality comparer – that gets irritating if you’re using it a lot.

I’ve therefore create another bunch of extension methods, just to make it clear that I want to use ordinal string comparisons – even if (in the case of equality) that would already be the default.

I won’t bore you with the full methods, but I’ve got:

• OrderByOrdinal
• OrderByOrdinalDescending
• ThenByOrdinal
• ThenByOrdinalDescending
• ToImmutableOrdinalDictionary (4 overloads, like the ones above for ToImmutableReferenceDictionary)
• ToOrdinalDictionary (4 overloads again)
• ToOrdinalLookup (2 overloads)

(I don’t actually use ToOrdinalLookup much, but it feels sensible to implement all of them.)

Would these be useful in the framework? Possibly. I can see why they’re not there – string is “just another type” really… but I bet a high proportion of uses of LINQ end up with strings as keys in some form or another. Possibly I should suggest this for MoreLINQ – although having started the project over 15 years ago, I haven’t contributed to it for over a decade…

Primary constructor and record “call hierarchy” niggle in VS

I use “call hierarchy” in Visual Studio all the time. Put your cursor on a member, then Ctrl-K, Ctrl-T and you can see everything that calls that member, and what calls the caller, etc.

For primary constructor and record parameters, “find references” works (Ctrl-K, Ctrl-R) but “call hierarchy” doesn’t. I’m okay with “call hierarchy” not working for primary constructor parameters, but as the record parameters become properties, I’d expect to see the call hierachy for them just as I can with any other property.

More frustrating though is the inability to see the call hierarchy for “calling the constructor”. Given that the declaration of the class/record sort of acts as the declaration of the constructor as well, I’d have thought that putting your cursor on the class/record declaration (in the name) would work. It’s not that it’s ambiguous – Visual Studio just complains that “Cursor must be on a member name”. You can get at the calls by expanding the source file entry in Solution Explorer, but it’s weird only to have to do that for this one case.

Feature requests (for the C# language, .NET, and Visual Studio)

In summary, I love records, and love the immutable collections – but some friction could be reduced with the introduction of:

• Some way of controlling (on a per-property basis) which equality comparer is used in the generated code
• Equality comparers for immutable collections, with the ability to specify the element comparisons to use
• An IEqualityComparer<T> implementation which performs reference comparisons
• “Call Hierarchy” showing the calls to the constructors for primary constructors and records

Conclusion

Some of the niggles I’ve found with records and collections are at least somewhat specific to my election site, although I strongly suspect that I’m not the only developer with immutable collections in their records, with a desire to use them in equality comparisons.

Overall, records have served me well so far in the site, and I’m definitely pleased that they’re available, even if there are still possible improvements to be made. Similarly, it’s lovely to have immutable collections just naturally available – but some help in performing comparisons with them would be welcome.