All posts by jonskeet

Mad props to @arcaderage for the "Princess Rescue" image - see https://toggl.com/programming-princess for the full original

Joe Duffy speaks as well as he writes

(This is being written on Thursday evening, but won’t be posted until Friday morning, just to explain any apparent oddities in timing. Quite appropriate, given the topic.)

I’m currently at TechEd in Barcelona, representing Iterative Training and generally trying to raise our profile a bit. (Mentioning the book to everyone I meet doesn’t hurt either.) All the sessions I’ve been to so far have been pretty good (or better) – but the real treat was hearing Joe Duffy speak. In a single session, he covered threading from why it’s necessary (and becoming more so) to some of the details of the memory model. Obviously this meant that for any single person, either some of the talk would be already known or some of the talk would be way beyond them – but even so, it had a really good feeling to it. We saw a little bit of what’s coming up in ParallelFX – and although Joe can’t say at the moment whether it has any of the Coordination and Concurrency Runtime stuff in it, what he presented certainly sounded pretty similar. Oh, and the CTP will be released really soon, with any luck. (I really hope so – I’ve got a sample in my book which I need to check before it goes to print!)

Anyway, the bottom line is that I’m still completely in awe of Joe. I don’t know his age, but I’m sure he’s not that much older than me – but his confidence and authority on what is fundamentally a massive and difficult topic is incredible.

Tomorrow should be a lot of fun – it’s a shorter day than today, but it looks like I might finally get to learn some F#…

I love LINQ: Simplifying a tedious task

As mentioned in my previous post, I’ve been putting together the code samples for C# in Depth. Now, these are spread across several projects in a few solutions. They’re referred to in the book as things like “Listing 6.2” but I’ve given the files “real” names in the projects. When you run any project with multiple executable listings in it, the project offers you options of which listing to run, showing both the type name and the listing, which is embedded using System.ComponentModel.DescriptionAttribute, e.g. [Description("Listing 12.4")]. A few listings have a description which is more than just “Listing x.y” – for instance, “Listing 6.1/6.2/6.3”. These are displayed with no problems.

Now, the problem for the reader would be finding a particular listing – it’s not always obvious from the code what the type name should be, particularly when there are many variations on a theme. Clearly some sort of map is required. Ideally it should be a file looking something like this:

Chapter 1:
1.1: Foo.cs
1.2: Bar.cs
1.3/1.4: Baz.cs

Chapter 2:
2.1: Gronkle.cs

It’s easy enough to work out the directory for any particular file – the projects are helpfully named “Chapter3” and the like. So, the next thing is to create this file. I really didn’t want to do that by hand. After all, there are about 150 listings in the book – and I’ve already done the work of attributing them all. Ah… we could do it programmatically. Sounds like a bit of a slog…

… but it’s a problem which is ideally suited to LINQ. It’s also fairly ideally suited to regular expressions, much as I hate to admit it. The regular expression in question is reasonably complex, but thanks to Jesse Houwing’s advice on adding comments to regular expressions, the results aren’t too bad. Here’s the finished code – which of course is part of the downloadable source code itself.

using System;
using System.IO;
using System.Linq;
using System.Text.RegularExpressions;

namespace Chapter11.Queries
{
    /// <summary>
    /// The listings are scattered around .cs files within various directories.
    /// This class uses LINQ to find all classes with a suitable Description
    /// attribute, groups them by chapters and orders them by chapter and listing number.
    /// </summary>
    class DisplayListingsMap
    {
        static readonly Regex ListingPattern = new Regex(
            @"# First match the start of the attribute, up to the bit we're interested in
            [Description(""Listing 
            # The 'text' group is the whole of the description after Listing
            (?<text>
            # The 'chapter' group is the first set of digits in the description, before a dot
            (?<chapter>d+).
            # The chapter group is the second set of digits in the description
            (?<listing>d+)
            # After that we don't care - stop the 'text' group at the double quote
            [^""]*)
            # Now match the end of the attribute
            "")]",
            RegexOptions.Compiled | RegexOptions.IgnorePatternWhitespace);

        static void Main()
        {
            DirectoryInfo directory = new DirectoryInfo(@"........");

            var query = from file in directory.GetFiles("*.cs", SearchOption.AllDirectories)
                        let match = ListingPattern.Match(File.ReadAllText(file.FullName))
                        where match.Success
                        let Details = new
                        {
                            Text = match.Groups["text"].Value,
                            Chapter = int.Parse(match.Groups["chapter"].Value),
                            Listing = int.Parse(match.Groups["listing"].Value)
                        }
                        orderby Details.Chapter, Details.Listing
                        group new { File = file, Description=Details.Text } by Details.Chapter;

            foreach (var chapter in query)
            {
                Console.WriteLine("Chapter {0}", chapter.Key);
                foreach (var listing in chapter)
                {
                    Console.WriteLine("{0}: {1}", listing.Description, listing.File.Name);
                }
                Console.WriteLine();                
            }
        }
    }
}

Isn’t it cool? The regex works out the listing number (first x.y part only) and sorts on that, grouping by chapter – then we just display the results. There are other ways of skinning the same cat – such as grouping and then ordering “inside” and “outside” a chapter separately – but they’ll all boil down to the same sort of thing.

Book news

The book is coming along well, and here are a few snippets which may be of interest:

  • It’s now on Amazon
  • All the chapters and the appendix have been written and given a first set of edits
  • We’re going to “final review” stage soon – that doesn’t mean the text is being finalized just yet, but it means this is probably the last round of peer review
  • I’ve been putting together the downloadable source code (see next post for some fun)
  • I’m hoping that the next couple of chapters will turn up in MEAP soon
  • Daniel Moth very kindly let me plug it at the recent UK MVP Open Day
  • There’s another plug on the flyers I’ll be giving out promoting Iterative Training at TechEd Developer in Barcelona next week
  • Manning are doing a 25% discount when you buy LINQ in Action and C# in Depth together

The last point is particularly cool – it’s something I’ve been suggesting for a while, as the books complement each other very nicely.

C# in Depth: Chapters 6 and 7 now in MEAP

Chapters 6 and 7 have now been included in the Manning Early Access Program. That means that the whole of the C# 2 part of the book is now available. Marc Gravell has been picking holes in it on the forum (and I mean that in a very positive way – it’s great to have more eyes running over it). Can you find more errors? Here’s a rundown of chapters 6 and 7:

 

Chapter 6: Implementing iterators the easy way

In C# 1, it was a pain to implement IEnumerable. C# 2 makes it easy with iterator blocks, and this can make it worthwhile introducing IEnumerable where you might not have done before. Aside from anything else, it’s fun just to watch the C# compiler build a state machine for you!

 

Chapter 7: Concluding C# 2: the final features

Confession: this is really “C# 2: the features which didn’t fit anywhere else”. It’s a round-up of features which didn’t deserve their own chapters, and which could easily wait until the “big” features had been explored before being mentioned. Ironically, C# 3 is exactly the opposite – the “little features” in C# 3 are pretty key to understanding the big features such as lambda expressions and query expressions, which is why they’re in chapter 8. Chapter 7 covers the following areas:

  • Partial types (including partial methods from C# 3)
  • Static classes
  • Separate getter/setter property access (e.g. public getter, private setter)
  • Namespace aliases (using ::, the global namespace alias, and extern aliases)
  • Pragma directives
  • Fixed size buffers

Fun with captured variables

 

I’m currently writing about readability, and discussing deferred parameter evaluation in LINQ on the C# newsgroup. It’s led me to come up with quite an evil application of captured variables.

Consider this C# 3 code:

using System;
using System.Linq;

public class Test
{
static void Main()
{
int max = int.MinValue;

int[] values = { 0, 5, 2, 3, 8, 4, -1, 10, 12, 8, 100 };

var query = from value in values
where value > max
select (max=value);

foreach (int value in query)
{
Console.WriteLine(value);
}
}
}

The max and values variables are easy enough to understand, and the foreach loop at the bottom just iterates through the query and dumps out the results – but what does the query itself do?

The answer is that it shows you all the values which are larger than any that have come before it. Each value encounters the “where” clause which is effectively “are you the biggest value I’ve seen so far?” and passing values then encounter the “select” clause which is “remember the current value as the highest we’ve seen, and return it”. So, the output is 0, 5, 8, 10, 12, 100.

It relies on the fact that the “select” clause for one passing value is executed before the next “where” clause. If you force it to evaluate the “where” clause for all the source values before performing the projection (e.g. with an “orderby” clause) then all the numbers are returned, because they’re all bigger than the initial value of max. If you try to use Parallel LINQ to evaluate it, all bets are off.

 Contrary to the normal advice, do try this at home. Just don’t try it in production code. (If you find a genuine use for it, at least comment it heavily!)

LINQ to Silliness: Generating a Mandelbrot with parallel potential

I’ve been writing about LINQ recently, and in particular I’ve written a small amount about Parallel LINQ. (Don’t get excited – it’s only about a page, just to mention it as a sort of “meta-provider” for LINQ.) I was wondering what to use to demonstrate it – what general task can we perform which could take a lot of CPU?

Well, I used to be quite into fractals, and I’ve written Mandelbrot set generators in various languages. I hadn’t done it in C# before now, however. Calculating the colour of each pixel is completely independent of all the other pixels – it’s an “embarrassingly parallelizable” task. So, a great task for PLINQ. Here’s the “normal LINQ” code:

 

var query = from row in Enumerable.Range(0, ImageHeight)
from col in Enumerable.Range(0, ImageWidth)
select ComputeMandelbrotIndex(row, col);

byte[] data = query.ToArray();

Changing this into a parallel query is really simple – although we do need to preserve the ordering of the results:

var query = from row in Enumerable.Range(0, ImageHeight).AsParallel(QueryOptions.PreserveOrdering)
from col in Enumerable.Range(0, ImageWidth)
select ComputeMandelbrotIndex(row, col);

byte[] data = query.ToArray();

Without being able to actually use PLINQ yet, I can’t tell how awful the order preservation is – Joe warns that it’s costly, but we’ll see. This is on a pretty giant sequence of data, of course… An alternative would be to parallelize a row at a time, but that loses some of the purity of the solution. This is a very, very silly way of parallelizing the task, but it’s got a certain quirky appeal.

Of course, there’s then the code for ComputeMandelBrotIndex and displaying a bitmap from it – the full code is available for download (it’s a single C# file – just compile and run). Enjoy.

Update!

This blog post has been picked up by Nick Palladinos, who has written his own Parallel LINQ provider (much kudos for that – unfortunately for me the blog is in Greek, which I don’t understand). Apparently on a dual core processor the parallelised version of the Mandelbrot generator is indeed about twice as fast – it works! Unfortunately I can’t tell as my laptop only has a single core… it’s very exciting though :)

Yeehaa! Framework source code, here we come!

It’s just been announced on Scott Guthrie’s blog that Visual Studio 2008 will allow debugging into the source code for the .NET framework – at least for some areas. I’ve wanted this to happen for a long time, and blogged about it before now – but I didn’t honestly expect it to happen, at least not for a long time.

This is a fabulous move, and one which MS should be commended for. I’m sure many people will spend a long time discussing whether the licence can be described as “open” (you can’t copy the code or recompile it) but to me that’s relatively unimportant. I will be able to view it, debug into it, and potentially spot/report bugs in it. I don’t really have much use for copying or recompiling it, personally.

Well done, Microsoft.

On a side note, I’d love to see the traffic logs for Reflector before and after this happens :)

C# in Depth: Chapters 4 and 5 now available in MEAP

Chapters 4 and 5 of the book have now been made available for early access.

 

Chapter 4 – Saying nothing with nullable types

Nullable types depend heavily on generics (described in chapter 3) and require both language and runtime changes. In this chapter I explore the problem they solve, the types involved (including runtime changes) and the C# changes (int? meaning Nullable<int> and the various operators and conversions available). I also cover a couple of uses of nullable types which haven’t necessarily hit the mainstream, but can prove useful – the comparisons I wrote about in this blog a little while ago, and using nullable types as an alternative to out parameters for the TryXXX pattern.

 

Chapter 5 – Fast-tracked delegates

C# 3 relies on delegates a lot. You can’t do any real LINQ work without them. C# 2 laid a lot of the groundwork for the lambda expressions available in C# 2 when it introduced anonymous methods. There are other changes in C# 2 which improve delegates, however – and there are more methods in the .NET 2.0 framework which take advantage of delegates than there were in .NET 1.1. (I’m thinking particularly of List<T>.)

I cover all the improvements in this chapter, but most of the chatper is given over to anonymous methods, and the handling of captured variables in particular. Without wishing to sound like a spoilsport, the use of captured variables can look like magic. Captured variables are still just as useful when the magic is explained away, and they’re somewhat less scary!

 

More chapters to come soon, I expect – when chapters 6 and 7 are released, that will cover the whole of C# 2.

Announcement: Partnership with Iterative Training

I’m delighted to announce my association with Iterative Training, a new .NET training company in the UK. It’s been founded by a colleague of mine and his girlfriend, and ran its first course earlier in the year – a WCF Master Class taught by Juval Löwy of IDesign. As you can tell from its starting point, Iterative Training is dedicated to providing really top notch training from genuine experts.

Where do I fit in? Well, I’m promoting the company on my website (the GoogleAds are now replaced with course adverts – but still without getting in the way of the content, I hope) and will be attending the last couple of days of TechEd in Barcelona to give some visibility to what we’re doing. Beyond that, I may well end up giving “taster talks” at user groups, and possibly even end up as a very occasional trainer. Who knows what the future holds?

I’m really excited about this opportunity to see more of the training side of development, and hopefully contribute to it. I have no idea what proportion of my readership comes from the UK, but I hope that if you do, and if you’re interested in some of the new .NET technologies, that you’ll have a look at the courses on offer and hopefully find something of interest. If you have suggestions for courses we should be offering but aren’t, we’d love to hear them – although we’re trying to keep to a relatively light schedule for the first year or so, focusing on quality rather than quantity.

None of this changes my existing situation: I will still keep up this blog and my articles, the book is still in progress, and I’m still in permanent employment. It’s expanding rather than replacing my range of activities :)

Java isn’t an acronym

Just a quickie while I remember. A pet peeve of mine has surfaced again recently, while reading some CVs.

Java, the programming language, is just written “Java”. It’s not an acronym. There’s no need to write it as “JAVA”. That just looks shouty and somewhat silly. Why do so many people get it wrong? While we’re at it, why does it irritate me so much to see it written the wrong way?