(Just to be clear: I hate the word "explicitness". It reminds me of Rowan Atkinson as Marcus Browning MP, saying we should be purposelessnessless. But I can’t think of anything better here.)

For the last few days, I’ve been thinking about context – in the context of C# 5’s proposed asynchronous functionality. Now many of the examples which have been presented have been around user interfaces, with the natural restriction of doing all UI operations on the UI thread. While that’s all very well, I’m more interested in server-side programming, which has a slightly different set of restrictions and emphases.

Back in my post about configuring waiting, I talked about the context of where execution would take place – both for tasks which require their own thread and for the continuations. However, thinking about it further, I suspect we could do with richer context.

What might be included in a context?

We’re already used to the idea of using context, but we’re not always aware of it. When trying to service a request on a server, some or any of the following may be part of our context:

• Authentication information: who are we acting as? (This may not be the end user, of course. It may be another service who we trust in some way.)
• Cultural information: how should text destined for an end user by rendered? What other regional information is relevant?
• Threading information: as mentioned before, what threads should be used both for "extra" tasks and continuations? Are we dealing with thread affinity?
• Deadlines and cancellation: the overall operation we’re trying to service may have a deadline, and operations we create may have their own deadlines too. Cancellation tokens in TPL can perform this role for us pretty easily.
• Logging information: if the logs need to tie everything together, there may be some ID generated which should be propagated.
• Other request information: very much dependent on what you’re doing, of course…

We’re used to some of this being available via properties such as CultureInfo.CurrentCulture and HttpContext.Current – but those are tied to a particular thread. Will they be propagated to threads used for new tasks or continuations? Historically I’ve found that documentation has been very poor around this area. It can be very difficult to work out what’s going to happen, even if you’re aware that there’s a potential problem in the first place.

Explicit or implicit?

It’s worth considering what the above items have in common. Why did I include those particular pieces of information but not others? How can we avoid treating them as ambient context in the first place?

Well, fairly obviously we can pass all the information we need along via method arguments. C# 5’s async feature actually makes this easier than it was before (and much easier that it would have been without anonymous functions) because the control flow is simpler. There should be fewer method calls, each of which would each require decoration with all the contextual information required.

However, in my experience that becomes quite problematic in terms of separation of concerns. If you imagine the request as a tree of asynchronous operations working down from a top node (whatever code initially handles the request), each node has to provide all the information required for all the nodes within its subtree. If some piece of information is only required 5 levels down, it still needs to be handed on at each level above that.

The alternative is to use an implicit context – typically via static methods or properties which have to do the right thing, typically based on something thread-local. The context code itself (in conjunction with whatever is distributing the work between threads) is responsible for keeping track of everything.

It’s easy to point out pros and cons to both approaches:

• Passing everything through methods makes the dependencies very obvious
• Changes to "lower" tasks (even for seemingly innocuous reasons such as logging) end up causing chains of changes higher up the task tree – possibly to developers working on completely different projects, depending on how your components work
• It feels like there’s a lot of work for very little benefit in passing everything explicitly through many layers of tasks
• Implicit context can be harder to unit test elegantly – as is true of so many things using static calls
• Implicit context requires everyone to use the same context. It’s no good high level code indicating which thread pool to use in one setting when some lower level code is going to use a different context

Ultimately it feels like a battle between purity and pragmatism: being explicit helps to keep your code purer, but it can mean a lot of fluff around your real logic, just to maintain the required information to pass onward. Different developers will have different approaches to this, but I suspect we want to at least keep the door open to both designs.

The place of Task/Task<T>

Even if Task/Task<T> can pass on the context for scheduling, what do we do about other information (authentication etc)? We have types like ThreadLocal<T> – in a world where threads are more likely to be reused, and aren’t really our unit of asynchrony, do we effectively need a TaskLocal<T>? Can context within a task be pushed and automatically popped, to allow one subtree to "override" the context for its nodes, while another subtree works with the original context?

I’ve been trying to think about whether this can be provided in "userland" code instead of in the TPL itself, but I’m not sure it can, easily… at least not without reinventing a lot of the existing code, which is never a good idea when it’s tricky parallelization code.

Should this be general support, or would it be okay to stick to just TaskScheduler.Current, leaving developers to pass other context explicitly?

Conclusion

These are thoughts which I’m hoping will be part of a bigger discussion. I think it’s something the community should think about and give feedback to Microsoft on well before C# 5 (and whatever framework it comes with) ships. I have lots of contradictory feelings about the right way to go, and I’m fully expecting comments to have mixed opinions too.

I’m sure I’ll be returning to this topic as time goes on.

Addendum (March 27th 2012)

Lucian Wischik recently mailed me about this post, to mention that F#’s support for async has had the ability to retain explicit context from the start. It’s also more flexible than the C# async support – effectively, it allows you to swap out AsyncTaskMethodBuilder etc for your own types, so you don’t always have to go via Task/Task<T>. I’ll take Lucian’s word for that, not knowing much about F# myself. One day…

I’ve had a wonderful day with Mads Torgersen, and amongst other things, we discussed multiple exceptions and the way that the default awaiter for Task<T> handles an AggregateException by taking the first exception and discarding the rest.

I now have a much clearer understanding of why this is the case, and also a workaround for the cases where you really want to avoid that truncation.

Why truncate in the first place?

(I’ll use the term "truncate" throughout this post to mean "when an AggregatedException with at least one nested exception is caught by EndAwait, throw the first nested exception instead". It’s just a shorthand.)

Yesterday’s post on multiple exceptions showed what you got if you called Wait() on a task returned from an async method. You still get an AggregateException, so why bother to truncate it?

Let’s consider a slightly different situation: where we’re awaiting an async method that throws an exception, and you want to be able to catch some specific exception that will be thrown by that asynchronous method. Imagine we used my NaiveAwaiter class. That would mean we would have to catch AggregateException, check whether one of those exceptions was actually present, and then handle that. There’d then be an open question about what to do if there were other exceptions as well… but that would be a relatively rare case. (Remember, we’re talking about multiple "top level" exceptions within the AggregateException – not just one exception nested in another, nested in another etc.)

With the current awaiter behaviour, you can catch the exception exactly as you would have done in synchronous code. Here’s an example:

Nice and clean exception handling… assuming that the task we awaited asynchronously didn’t have multiple exceptions. (Note the improved DelayedThrow method, by the way. Definitely cleaner than my previous version.)

This aspect of "the async code looks like the synchronous code" is the important bit. One of the key aims of the language feature is to make it easy to write asynchronous code as if it were synchronous – because that’s what we’re used to, and what we know how to reason about. We’re fairly used to the idea of catching one exception… not so much on the "multiple things can go wrong at the same time" front.

So that handles the primary case where we really expect to only have one exception (if any) because we’re only performing one job.

What about cases where multiple exceptions are somewhat expected?

Let’s go back to the case where we really to propagate multiple exceptions. I think it’s reasonable that this should be an explicit opt-in, so let’s think about an extension method. For the sake of simplicity I’ll use Task – in real life we’d want Task<T> as well, of course. So for example, this line:

would become this:

(Yes, the name is too long… but you get the idea.)

Now, there are two ways we could make this work:

• We could make the extension method return something which had a GetAwaiter method, returning something which in turn had BeginAwait and EndAwait methods. This means making sure we get all of the awaiter code right, of course – and the returned value has little meaning outside an await expression.
• We could wrap the task in another task, and use the existing awaiter code. We know that the EndAwait extension method associated with Task (and Task<T>) will go into a single level of AggregateException – but I don’t believe it will do any more than that. So if it’s going to strip one level of exception aggregation off, all we need to do is add another level.

According to Mads, the latter of these is easier. Let’s see if he’s right.

We need an extension method on Task, and we’re going to return Task too. How can we implement that?

• We can’t await the task, because that will strip the exception before we get to it.
• We can’t write an async task but call Wait() on the original task, because that will block immediately – we still want to be async.
• We can use a TaskCompletionSource<T> to build a task. We don’t care about the actual result, so we’ll use TaskCompletionSource<object>. This will actually build a Task<object>, but we’ll return it as a Task anyway, and use a null result if it completes with no exception. (This was Mads’ suggestion.)

So, we know how to build a Task, and we’ve been given a Task – how do we hook the two together? The answer is to ask the original task to call us back when it completes, via the ContinueWith method. We can then set the result of our task accordingly. Without further ado, here’s the code:

This was thrown together in 5 minutes (in the middle of a user group talk by Mads) so it’s probably not as robust as it might be… but the idea is that when the original task completes, we just piggy-back on the same thread very briefly to make our own task respond appropriately. Now when some code awaits our returned task, we’ll add an extra wrapper of AggregateException on top, ready to be unwrapped by the normal awaiter.

Note that the extra wrapper is actually added for us really, really easily – we just call TaskCompletionSource<T>.SetException with the original task’s AggregateException. Usually we’d call SetException with a single exception (like a BangException) and the method automatically wraps it in an AggregateException – which is exactly what we want.

So, how do we use it? Here’s a complete sample (just add the extension method above):

The result is what we were after:

The blanket catch (Exception e) block is there so you can experiment with what happens if you remove the call to PreserveMultipleExceptions – in that case we get the original behaviour of a single BangException being caught, and the others discarded.

Conclusion

So, we now have answers to both of my big questions around multiple exceptions with async:

• Why is the default awaiter truncating exceptions? To make asynchronous exception handling look like synchronous exception handling in the common case.
• What can we do if that’s not the behaviour we want? Either write our own awaiter (whether that’s invoked explicitly or implicitly via "extension method overriding" as shown yesterday) or wrap the task in another one to wrap exceptions.

I’m happy again. Thanks Mads :)

In an earlier post, I mentioned  that in the CTP, an asynchronous method will throw away anything other than the first exception in an AggregateException thrown by one of the tasks it’s waiting for. Reading the TAP documentation, it seems this is partly expected behaviour and partly not. TAP claims (in a section about how "await" is achieved by the compiler):

It is possible for a Task to fault due to multiple exceptions, in which case only one of these exceptions will be propagated; however, the Task’s Exception property will return an AggregateException containing all of the errors.

Unfortunately, that appears not to be the case. Here’s a test program demonstrating the difference between an async method and a somewhat-similar manually written method. The full code is slightly long, but here are the important methods:

The difference is that the async method is generating an extra task, instead of returning the task from TaskEx.WhenAll. It’s waiting for the result of WhenAll itself (via EndAwait). The results show one exception being swallowed:

The fact that the "manual" method still shows two exceptions means we can’t blame WhenAll – it must be something to do with the async code. Given the description in the TAP documentation, I’d expect (although not desire) the thrown exception to just be a single exception, but the returned task’s exception should have both in there. That’s clearly not the case at the moment.

Waiter! There’s an exception in my soup!

I can think of one reason why we’d perhaps want to trim down the exception to a single one: if we wanted to remove the aggregation aspect entirely. Given that the async method always returns a Task (or void), I can’t see how that’s feasible anyway… a Task will always throw an AggregateException if its underlying operation fails. If it’s already throwing an AggregateException, why restrict it to just one?

My guess is that this makes it easier to avoid the situation where one AggregateException would contain another, which would contain another, etc.

To demonstrate this, let’s try to write our own awaiting mechanism, instead of using the one built into the async CTP. GetAwaiter() is an extension method, so we can just make our own extension method which has priority over the original one. I’ll go into more detail about that in another post, but here’s the code:

Yes, it’s almost the simplest implementation you could come up with. (Hey, we do check whether the task is already completed…) There no scheduler or SynchronizationContext magic… and importantly, EndAwait does nothing with any exceptions. If the task throws an AggregateException when we wait for it, that exception is propagated to the generated code responsible for the async method.

So, what happens if we run exactly the same client code with these classes present? Well, the results for the first part are different:

We have to change the formatting somewhat to see exactly what’s going on – because we now have an AggregateException containing an AggregateException. The previous formatting code simply printed out how many exceptions there were, and their messages. That wasn’t an issue because we immediately got to the exceptions we were throwing. Now we’ve got an actual tree. Just printing out the exception itself results in huge gobbets of text which are unreadable, so here’s a quick and dirty hack to provide a bit more formatting:

Now we can see what’s going on better:

Hooray – we actually have all our exceptions, eventually… but they’re nested. Now if we introduce another level of nesting – for example by creating an async method which just waits on the task created by ThrowMultipleAsync – we end up with something like this:

You can imagine that for a deep stack trace of async methods, this could get messy really quickly.

However, I don’t think that losing the information is really the answer. There’s already the Flatten method in AggregateException which will flatten the tree appropriately. I’d be reasonably happy for the exceptions to be flattened at any stage, but I really don’t like the behaviour of losing them.

It does get complicated by how the async language feature has to handle exceptions, however. Only one exception can ever be thrown at a time, even though a task can have multiple exceptions set on it. One option would be for the autogenerated code to handle AggregateException differently, setting all the nested exceptions separately (in the single task which has been returned) rather than either setting the AggregateException which causes nesting (as we’ve seen above) or relying on the awaiter picking just one exception (as is currently the case). It’s definitely a decision I think the community should get involved with.

Conclusion

As we’ve seen, the current behaviour of async methods doesn’t match the TAP documentation or what I’d personally like.

This isn’t down to the language features, but it’s the default behaviour of the extension methods which provide the "awaiter" for Task. That doesn’t mean the language aspect can’t be changed, however – some responsibility could be moved from awaiters to the generated code. I’m sure there are pros and cons each way – but I don’t think losing information is the right approach.

Next up: using extension method resolution rules to add diagnostics to task awaiters.

One of the great things about working at Google is that almost all of my colleagues are smarter than me. True, they don’t generally know as much about C#, but they know about language design, and they sure as heck know about distributed/parallel/async computing.

One of the great things about having occasional contact with the C# team is that when Mads Torgersen visits London later in the week, I can introduce him to these smart colleagues. So, I’ve been spreading the word about C# 5’s async support and generally encouraging folks to think about the current proposal so they can give feedback to Mads.

One particularly insightful colleague has persistently expressed a deep concern over who gets to control how the asynchronous execution works. This afternoon, I found some extra information which looks like it hasn’t been covered much so far which may allay his fears somewhat. It’s detailed in the Task-based Asynchronous Pattern documentation, which I strongly recommend you download and read right now.

More than ever, this post is based on documentation rather than experimentation. Please take with an appropriately large grain of salt.

What’s the problem?

In a complex server handling multiple types of request and processing them asynchronously – with some local CPU-bound tasks and other IO-bound tasks – you may well not want everything to be treated equally. Some operations (health monitoring, for example) may require high priority and a dedicated thread pool, some may be latency sensitive but support load balancing easily (so it’s fine to have a small pool of medium/high priority tasks, trusting load balancing to avoid overloading the server) and some may be latency-insensitive but be server-specific – a pool of low-priority threads with a large queue may be suitable here, perhaps.

If all of this is going to work, you need to know for each asynchronous operation:

• Whether it will take up a thread
• What thread will be chosen, if one is required (a new one? one from a thread pool? which pool?)
• Where the continuation will run (on the thread which initiated the asynchronous operation? the thread the asynchronous operation ran on? a thread from a particular pool?)

In many cases reusable low-level code doesn’t know this context… but in the async model, it’s that low-level code which is responsible for actually starting the task. How can we reconcile the two requirements?

Controlling execution flow from the top down

Putting the points above into the concrete context of the async features of C# 5:

• When an async method is called, it will start on the caller’s thread
• When it creates a task (possibly as the target of an await expression) that task has control over how it will execute
• The awaiter created by an await expression has control (or at the very least significant influence) over how where the next part of the async method (the continuation) is executed
• The caller gets to decide what they will do with the returned task (assuming there is one) – it may be the target of another await expression, or it may be used more directly without any further use of the new language features

Whether a task requires an extra thread really is pretty much up to the task. Either a task will be IO-bound, CPU-bound, or a mixture (perhaps IO-bound to fetch data, and then CPU-bound to process it). As far as I can tell, it’s assumed that IO-bound asynchronous tasks will all use IO completion ports, leaving no real choice available. On other platforms, there may be other choices – there may be multiple IO channels for example, some reserved for higher priority traffic than others. Although the TAP doesn’t explicitly call this out, I suspect that other platforms could create a similar concept of context to the one described below, but for IO-specific operations.

The two concepts that TAP appears to rely on (and I should be absolutely clear that I could be misreading things; I don’t know as much about the TPL that all of this is based on as I’d like) are a SynchronizationContext and a TaskScheduler. The exact difference between the two remains slightly hazy to me, as both give control over which thread delegates are executed on – but I get the feeling that SynchronizationContext is aimed at describing the thread you should return to for callbacks (continuations) and TaskScheduler is aimed at describing the thread you should run work on – whether that’s new work or getting back for a continuation. (In other words, TaskScheduler is more general than SynchronizationContext -  so you can use it for continuations, but you can also use it for other things.)

One vital point is that although these aren’t enforced, they are designed to be the easiest way to carry out work. If there are any places where that isn’t true, that probably represents an issue. For example, the TaskEx.Run method (which will be Task.Run eventually) always uses the default TaskScheduler rather than the current TaskScheduler – so tasks started in that way will always run on the system thread pool. I have doubts about that decision, although it fits in with the general approach of TPL to use a single thread pool.

If everything representing an async operation follows the TAP, it should make it to control how things are scheduled "from this point downwards" in async methods.

ConfigureAwait, SwitchTo, Yield

Various "plain static" and extension methods have been provided to make it easy to change your context within an async method.

SwitchTo allows you to change your context to the ThreadPool or a particular TaskScheduler or Dispatcher. You may not need to do any more work on a particular high priority thread until you’ve actually got your final result – so you’re happy with the continuations being executed either "inline" with the asynchronous tasks you’re executing, or on a random thread pool thread (perhaps from some specific pool).  This may also allow the new CPU-bound tasks to be scheduled appropriately too (I thought it did, but I’m no longer sure). Once you’ve got all your ducks in a row, then you can switch back for the final continuation which needs to provide the results on your original thread.

ConfigureAwait takes an existing task and returns a TaskAwaiter – essentially allowing you to control just the continuation part.

Yield does exactly what it sounds like – yields control temporarily, basically allowing for cooperative multitasking by allowing other work to make progress before continuing. I’m not sure that this one will be particularly useful, personally – it feels a little too much like Application.DoEvents. I dare say there are specialist uses though – in particular, it’s cleaner than Application.DoEvents because it really is yielding, rather than running the message pump in the current stack.

All of these are likely to be used in conjunction with await. For example (these are not expected to all be in the same method, of course!):

Is this enough?

My gut feeling is that this will give enough control over the flow of the application if:

• The defaults in TAP are chosen appropriately so that the easiest way of starting a computation is also an easily "top-down-configurable" one
• The top-level application programmer pays attention to what they’re doing, and configures things appropriately
• Each component programmer lower down pays attention to the TAP and doesn’t do silly things like starting arbitrary threads themselves

In other words, everyone has to play nicely. Is that feasible in a complex system? I suspect it has to be really. If you have any "rogue" elements they’ll manage to screw things up in any system which is flexible enough to meet real-world requirements.

My colleague’s concern is (I think – I may be misrepresenting him) largely that the language shouldn’t be neutral about how the task and continuation are executed. It should allow or even force the caller to provide context. That would make the context hard to ignore lower down. The route I believe Microsoft has chosen is to do this implicitly by propagating context through the "current" SynchronizationContext and TaskScheduler, in the belief that developers will honour them.

We’ll see.

Conclusion

A complex asynchronous system is like a concerto played by an orchestra. Each musician is responsible for keeping time, but they are given direction from the conductor. It only takes one viola player who wants to play fast and loud to ruin the whole effect – so everyone has to behave. How do you force the musicians to watch the conductor? How much do you trust them? How easy is it to conduct in the first place? These are the questions which are hard to judge from documentation, frankly. I’m currently optimistic that by the time C# 5 is actually released, the appropriate balance will have been struck, the default tempo will be appropriate, and we can all listen to some beautiful music. In the key of C#, of course.

Yesterday I wrote about waiting for multiple tasks to complete. We had three asynchronous tasks running in parallel, fetching a user’s settings, reputation and recent activity. Broadly speaking, there were two approaches. First we could use TaskEx.WhenAll (which will almost certainly be folded into the Task class for release):

Second we could just wait for each result in turn:

These look very similar, but actually they behave differently if any of the tasks fails:

• In the first form we will always wait for all the tasks to complete; if the settings task fails within a millisecond but the recent activity task takes 5 minutes, we’ll be waiting 5 minutes. In the second form we only wait for one at a time, so if one task fails, we won’t wait for any currently-unawaited ones to complete. (Of course if the first two tasks both succeed and the last one fails, the total waiting time will be the same either way.)
• In the first form we should probably get to find out about the errors from all the asynchronous tasks; in the second form we only see the errors from whichever task fails first.

The second point is interesting, because in fact it looks like the CTP will throw away all but the first inner exception of an aggregated exception thrown by a Task that’s being awaited. That feels like a mistake to me, but I don’t know whether it’s by design or just due to the implementation not being finished yet. I’m pretty sure this is the same bit of code (in EndAwait for Task and Task<T>) which makes sure that we don’t get multiple levels of AggregateException wrapping the original exception as it bubbles up. Personally I’d like to at least be able to find all the errors that occurred in an asynchronous operation. Occasionally, that would be useful…

… but actually, in most cases I’d really like to just abort the whole operation as soon as any task fails. I think we’re missing a method – something like WhenAllSuccessful. If any operation is cancelled or faulted, the whole lot should end up being cancelled – with that cancellation propagating down the potential tree of async tasks involved, ideally. Now I still haven’t investigated cancellation properly, but I believe that the cancellation tokens of Parallel Extensions should make this all possible. In many cases we really need success for all of the operations – and we would like to communicate any failures back to our caller as soon as possible.

Now I believe that we could write this now – somewhat inefficiently. We could keep a collection of tasks which still haven’t completed, and wait for any of them to complete. At that point, look for all the completed ones in the set (because two could complete at the same time) and see whether any of them have faulted or been cancelled. If so, cancel the remaining operations and rethrow the exception (aka set our own task as faulted). If we ever get to the stage where all the tasks have completed – successfully – we just return so that the results can be fetched.

My guess is that this could be written more efficiently by the PFX team though. I’m actually surprised that there isn’t anything in the framework that does this. That usually means that either it’s there and I’ve missed it, or it’s not there for some terribly good reason that I’m too dim to spot. Either way, I’d really like to know.

Of course, all of this could still be implemented as extension methods on tuples of tasks, if we ever get language support for tuples. Hint hint.

Conclusion

It’s often easy to concentrate on the success path and ignore possible failure in code. Asynchronous operations make this even more of a problem, as different things could be succeeding and failing at the same time.

If you do need to write code like the second option above, consider ordering the various "await" statements so that the expected time taken in the failure case is minimized. Always consider whether you really need all the results in all cases… or whether any failure is enough to mess up your whole operation.

Oh, and if you know the reason for the lack of something like WhenAllSuccessful, please enlighten me in the comments :)

Warning: as ever, this is only the result of reading the spec and experimentation. I may well have misinterpreted everything. Eric Lippert has said that he’ll blog about exceptions soon, but I wanted to put down my thoughts first, partly to see the difference between what I’ve worked out and what the real story is.

So far, I’ve only covered "success" cases – where tasks complete without being cancelled or throwing exceptions. I’m leaving cancellation for another time, but let’s look at what happens when exceptions are thrown by async methods.

What happens when an async method throws an exception?

There are three types of async methods:

• Ones that are declared to return void
• Ones that are declared to return Task
• Ones that are declared to return Task<T> for some T

The distinction between Task and Task<T> isn’t important in terms of exceptions. I’ll call async methods that return Task or Task<T> taskful methods, and ones that return void taskless methods. These aren’t official terms and they’re not even nice terms, but I don’t have any better ones for the moment.

It’s actually pretty easy to state what happens when an exception is thrown – but the ramifications are slightly more complicated:

• If code in a taskless method throws an exception, the exception propagates up the stack
• If code in a taskful method throws an exception, the exception is stored in the task, which transitions to the faulted state
  • If we’re still in the original context, the task is then returned to the caller
  • If we’re in a continuation, the method just returns

The inner bullet points are important here. At any time it’s executing, an async method is either still in its original context – i.e. the caller is one level up the stack – or it’s in a continuation, which takes the form of an Action delegate. In the latter case, we must have previously returned control to the caller, usually returning a task (in the "taskful method" case).

This means that if you call a taskful method, you should expect to be given a task without an exception being thrown. An exception may well be thrown if you wait for the result of that task (possibly via an await operation) but the method itself will complete normally. (Of course, there’s always the possibility that we’ll run out of memory while constructing the task, or other horrible situations. I think it’s fair to classify those as pathological and ignore them for most applications.)

A taskless method is much more dangerous: not only might it throw an exception to the original caller, but it might alternatively throw an exception to whatever calls the continuation. Note that it’s the awaiter that gets to determine that for any await operation… it may be an awaiter which uses the current SynchronizationContext for example, or it may be one which always calls the continuation on a new thread… or anything else you care to think of. In some cases, that may be enough to bring down the process. Maybe that’s what you want… or maybe not. It’s worth being aware of.

Here’s a trivial app to demonstrate the more common taskful behaviour – although it’s unusual in that we have an async method with no await statements:

And here’s the result:

As you can see, the exception was only thrown when we waited for the asynchronous task to complete – and it was wrapped in an AggregateException which is the normal behaviour for tasks.

If an awaited task throws an exception, that is propagated to the async method which was awaiting it. You might expect this to result in an AggregateException wrapping the original AggregateException and so on, but it seems that something is smart enough to perform some unwrapping. I’m not sure what yet, but I’ll investigate further when I get more time. EDIT: I’m pretty sure it’s the EndAwait code used when you await a Task or Task<T>. There’s certainly no mention of AggregateException in the spec, so I don’t believe the compiler-generated code does any of this.

How eagerly can we validate arguments?

If you remember, iterator blocks have a bit of a usability problem when it comes to argument validation: because the iterator code is only run when the caller first starts iterating, it’s hard to get eager validation. You basically need to have one non-iterator-block method which validates the arguments, then calls the "real" implementation with known-to-be valid arguments. (If this doesn’t ring any bells, you might want to read this blog post, where I’m coming up with an implementation of LINQ’s Where method.)

We’re in a similar situation here, if we want arguments to be validated eagerly, causing an exception to be thrown directly to the caller. As an example of this, what would you expect this code to do? (Note that it doesn’t involve us writing any async methods at all.)

It could throw an exception eagerly, or it could set the exception into the return task. In many cases this will have a very similar effect – if you call DownloadStringTaskAsync as part of an await statement, for example. But it’s something you should be aware of anyway, as sometimes you may well want to call such methods outside the context of an async method.

In this particular case, the exception is thrown eagerly – so even though we’re not trying to wait on a task, the above program blows up. So, how could we achieve the same thing?

First let’s look at the code which wouldn’t work:

The problem is that we’re in an async method, so the compiler is writing code to catch any exceptions we throw, and propagate them through the task instead. We can get round this by using exactly the same trick as with iterator blocks – using a first non-async method which then calls an async method after validating the arguments:

There’s a nicer solution though – because C# 5 allows us to make anonymous functions (anonymous methods or lambda expressions) asynchronous too. So we can create a delegate which will return a task, and then call it:

This is slightly neater for methods which don’t need an awful lot of code. For more involved methods, it’s quite possibly worth using the "split the method in two" approach instead.

Conclusion

The general exception flow in asynchronous methods is actually reasonably straightforward – which is a good job, as normally error handling in asynchronous flows is a pain.

You need to be aware of the consequences of writing (or calling) a taskless asynchronous method… I expect that the vast majority of asynchronous methods and delegates will be taskful ones.

Finally, you also need to work out when you want exceptions to be thrown. If you want to perform argument validation, decide whether it should throw exceptions eagerly – and if so, use one of the patterns shown above. (I haven’t spent much time thinking about which approach is "better" yet – I generally like eager argument validation, but I also like the consistency of all errors being propagated through the task.)

Next up: dreaming of multiple possibly faulting tasks.

I apologise in advance if this post ends up slightly rambly. Unlike the previous entries, this only partly deals with existing features. Instead, it considers how we might like to handle some situations, and where the language and type system thwarts us a little. Most of the code will be incomplete, as producing something realistic and complete is tricky.

Almost everything I’ve written about so far has dealt with the situation where we just want to execute one asynchronous operation at a time. That’s the case which the await keyword makes particularly easy to work with. That’s certainly a useful case, but it’s not the only one. I’m not going to write about that case in this post. (At least, not much.)

At the other extreme, we’ve got the situation where you have a large number of items to deal with, possibly dynamically generated – something like "fetch all the URLs in this list". We may well be able to launch some or even all of those operations in parallel, but we’re likely to use the results as a general collection. We’re doing the same thing with each of multiple inputs. This is data parallelism. I’m not going to write about that in this post either.

This post is about task parallelism. We want to execute multiple tasks in parallel (which may or may not mean using multiple threads – there could be multiple asynchronous web service calls, for example) and get all the results back before we proceed. Some of the tasks may be the same, but in general they’re not.

Describing the sample scenario

To help make everything sound vaguely realistic, I’m going to use a potentially genuine scenario, based on Stack Overflow. I’d like to make it clear that I have no idea how Stack Overflow really works, so please don’t make any assumptions. In particular, we’re only dealing with a very small portion of what’s required to render a single page. Nevertheless, it gives us something to focus on. (This is a situation I’ve found myself in several times at work, but obviously the internal services at Google are confidential, so I can’t start talking about a really real example.)

As part of rendering a Stack Overflow page for a logged in user, let’s suppose we need to:

• Authenticate the user’s cookie (which gives us the user ID)
• Find out the preferences for the user (so we know which tags to ignore etc)
• Find out the user’s current reputation
• Find out if there have been any recent comments or badges

All of these can be asynchronous operations. The first one needs to be executed before any of the others, but the final three can all be executed in parallel. We need all the results before we can make any more progress.

I’m going to assume an appropriate abstraction which contains the relevant asynchronous methods. Something like this:

The simple "single-threaded" implementation

I’ve put "single-threaded" in quotes here, because this may actually run across multiple-threads, but only one operation will be executed at a time. This is really just here for reference – because it’s so easy, and it would be nice to get the same simplicity with more parallelism.

Just to be clear, this is still better than the obvious synchronous equivalent. While those asynchronous calls are executing, we won’t be sat in a blocked thread, taking very little CPU but hogging a decent chunk of stack space. The scheduler will have less work to do. Life will be all flowers and sunshine… except for the latency.

If each of those requests takes about 100ms, it will take 400ms for the whole lot to complete – when it could take just 200ms. We can’t do better than 200ms with the operations we’ve got to work with: we’ve got to get the user ID before we can perform any of the other operations – but we can do all the other three in parallel. Let’s try doing that using the tools we’ve got available to us and no neat tricks, to start with.

Declaring tasks and waiting for them

First, let’s talk about TaskEx.WhenAll(). This is a method provided in the CTP library, and I wouldn’t be surprised to see it move around a bit over time. There are a bunch of overloads for this – some taking multiple Task<TResult> items, and some being more weakly typed. It simply lets you wait for multiple tasks to complete – and because it returns a task itself, we can "await" it in the usual asyncrhonous way. In this case we have to use the weakly typed version, because our tasks are of different types. That’s fine though, because we’re not going to use the result anyway, except for waiting. (And in fact we’ll let the compiler deal with that for us.)

The code for this isn’t too bad, but it’s a bit more long-winded:

This is still nicer than the pre-C# 5 code to achieve the same results, but I’d like to think we can do better. Really we just want to express the tasks once, wait for them all to complete, and get the results into variables, just like we did in the one-at-a-time code. I’ve thought of two approaches for this: one using anonymous types, and one using tuples. Both require changes to be viable – although the tuple approach is probably more realistic. Let’s look at it.

EDIT: Just before we do, I’d like to include code from one of the comments. If we’re going to use all the results directly, we can just await them in turn rather than using WhenAll – it’s like joining one thread after another. That leads to code like this:

I definitely like that more. Not sure why I didn’t think of it before…

Now back to the original post…

An ideal world of tuples

I’m assuming you’re aware of the family of System.Tuple types. They were introduced in .NET 4, and are immutable and strongly typed, both of which are nice features. The downsides are that even with type inference they’re still slightly awkward to create, and extracting the component values requires using properties such as Item1, Item2 etc. The C# compiler is completely unaware of tuples, which is slightly annoying. I would like two new features in C# 5:

• Tuple literals: the ability to write something like var tuple = ("Foo", 10); to create a Tuple<string, int> – I’m not overly bothered with the exact syntax, so long as it’s concise.
• Assignment to multiple variables from a single tuple. For example: var (ok, value) = int.TryParseToTuple("10");. Assuming a method with signature Tuple<bool, int> TryParseToTuple(string text) this would make ok a variable of type bool, and value a variable of type int.

Just to pre-empt others, I’m aware that F# helps on this front already. C# could do with catching up :)

Anyway, imagine we’ve got those language features. Then imagine a set of extension methods looking like this, but with another overload for 2-value tuples, another for 4-value tuples etc:

It can look a bit confusing because of all the type arguments and calls to Result and ItemX properties… but essentially it takes a tuple of tasks, and returns a task of a tuple returning the component values. How does this help us? Well, take a look at our Stack Overflow code now:

If we knew we always wanted to wait for all the tasks, we could actually change our extension method to one called GetAwaiter which returned a TupleAwaiter or something like that – so we could get rid of the call to WhenAll completely. However, I’m not sure that would be a good thing. I like explicitly stating how we’re awaiting the completion of all of these tasks.

The real world of tuples

Back in the real world, we don’t have these language features on tuples. We can still use the extension method, but it’s not quite as nice:

We’ve got an extra local variable we don’t need, and the ugliness of the ItemX properties is back. Oh well. Maybe tuples aren’t the best approach. Let’s look at a closely related cousin, anonymous types…

An ideal world of anonymous types

Extension methods on anonymous types are somewhat evil. They’re potentially powerful, but they definitely have drawbacks. Aside from anything else, you can’t add a generic constraint to require that a type is anonymous, and you certainly can’t add a generic constraint to say that each member of the anonymous type must be a task (which is what we want here). But the difficulties go further than that. I would like to be able to use something like this:

Now in this magical world, we’d have an extension method on T where T : class which would check that all the properties were of type Task<TResult> (with a different TResult for each property, potentially) and return a task of a new anonymous type which had the same properties… but without the task part. Essentially, we’re trying to perform the same inversion that we did with tuples, moving where the task "decorator" bit comes. We can’t do that with anonymous types – there’s simply no way of expressing it in the language. We could potentially generate a new type at execution time, but there’s no way of getting compile-time safety.

These two problems suggest two different solutions though. Firstly – and more simply – if we’re happy to lose compile-time safety, we can use the dynamic typing from C# 4.

The real world of anonymous types and dynamic

We can fairly easily write an async extension method to create a Task<dynamic>. The code would involve reflection to extract the tasks from the instance, call TaskEx.WhenAll to wait for them to complete, and then populate an ExpandoObject. I haven’t included the extension method here because frankly reflection code is pretty boring, and the async part of it is what we’ve seen everywhere else. Here’s what the consuming code might look like though:

The extra local variables are back, because I don’t like being dynamic for more type than I can help. Here, once we’ve copied the results into our local variables, we can ignore the dynamic results variable for the rest of the code.

This is pretty ugly, but it would work. I’m not sure that it’s significantly better than the "works but uses ItemX" tuple version though.

Now, what about the second thought, about the difficulty of translating (Task<X>, Task<Y>) into a Task<(X, Y)>?

Monads

I’m scared. Writing this post has made me start to think I might be starting to grok monads. This is almost certainly an incorrect belief, but I’m sure I’m at least making progress. If we think of "wrapping a task around a type" as a sort of type decoration, it starts sounding similar to the description of monads that I’ve read before. The fact that async workflows in F# are one example of its monad support encourages me too. I have a sneaking suspicion that the async/await support in C# 5 is partial support for this specific monad – in particular, you express the result of an async method via a non-task-related return statement, but the declared return type is the corresponding "wrapped" type.

Now, C#’s major monadic support comes in the form of LINQ, and particularly SelectMany. Therefore – and I’m writing as I think here – I would like to end up being able to write something like this:

That feels like it should work, but as I write this I genuinely don’t know whether or not it will.

What I do know is that we only actually to write a single method to get that to work: SelectMany. We don’t even need to implement a Select method, as if there’s only a select clause following an extra from clause, the compiler just uses SelectMany and puts a projection at the end. We want to be able to take an existing task and a way of creating a new task from it, and somehow combine them.

Just to make it crystal clear, the way we’re going to use LINQ is not for sequences at all. It’s for tasks. So we don’t want to see IEnumerable<T> anywhere in our final signatures. Let’s see what we can do.

(10 minutes later.) Okay, wow. I’d expected it to be at least somewhat difficult to get it to compile. I’m not quite there yet in terms of parallelization, but I’ve worked out a way round that. Just getting it to work at all is straightforward. I started off by looking at the LINQ to Objects signature used by the compiler:

Now we want our tasks to end up being independent, but let’s start off simply, just changing IEnumerable to Task everywhere, and changing the type parameter names:

There’s still that nagging doubt about the dependency of the second task on the first, but let’s at least try to implement it.

We know we want to return a Task<TResult>, and we know that given a T1 and a T2 we can get a TResult. We also know that by writing an async method, we can ask the compiler to go from a return statement involving a TResult to a method with a declared return type of Task<TResult>. Once we’ve got that hint, the rest is really straightforward:

There it is. We asynchronously await the result of the first task, feed the result into taskSelector to get the second task, await that task to get a second value, and then combine the two values with the simple projection to give the result we want to return asynchronously.

In monadic terms as copied from Wikipedia, I believe that:

• The type constructor for the async monad is simply that T goes to Task<T> for any T.
• The unit function is essentially what the compiler does for us when we declare a method as async – it provides the "wrapping" to get from a return statement using T to a method with a return type of Task<T>.
• The binding operation is what we’ve got above – which should be no surprise, as SelectMany is the binding function in "normal" LINQ.

I’m breathless with the simultaneous simplicity, beauty and complexity of it all. It’s simple because once I’d worked out the method signature (which is essentially what the definition of the binding function requires) the method wrote itself. It’s beautiful because once I’d picked the right method to use, the compiler did everything else for me – despite it sounding really significantly different to LINQ. It’s complex because I’m still feeling my way through all of this.

It’s a shame that after all of this, we still haven’t actually got what we wanted. To do that, we have to fake it.

Improper monads

("Improper monads" isn’t a real term. It scores 0 hits on Google at the moment – by the time you read this, that count will probably be higher, but only because of this post.)

We wanted to execute the tasks in parallel. We’re not actually doing so. We’re executing one task, then another. Oops. The problem is that our monadic definition says that we’re going to rely on the result of one task to generate the other one. We don’t want to do that. We want to get both tasks, and execute them at the same time.

Unfortunately, I don’t think there’s anything in LINQ which represents that sort of operation. The closest I can think of is a join – but we’re not joining on anything. I’m pretty sure we could do this by implementing InnerJoin and just ignoring the key selectors, but if we’re going to cheat anyway, we might as well cheat with the signature we’ve got. In this cheating version of LINQ, we assume that the task selector (which produces the second task) doesn’t actually rely on the argument it’s given. So let’s just give it anything – the default value, for example. Then we’ve got two tasks which we can await together using WhenAll as before.

Okay, that was easy. But it looks like it’s only going to wait for two tasks at a time. We’ve got three in our example. What’s going to happen? Well, we’ll start waiting for the first two tasks when SelectMany is first called… but then we’ll return back to the caller with the result as a task. We’ll then call SelectMany again with the third task. We’ll then wait for [tasks 1 and 2] and [task 3]… which means waiting for all of them. Bingo! Admittedly I’ve a sneaking suspicion that if any task fails it might mean more deeply nested exceptions than we’d want, but I haven’t investigated that yet.

I believe that this implementation lets us basically do what we want… but like everything else, it’s ugly in its own way. In this case it’s ugly because it allows us to express something (a dependency from one task to another) that we then don’t honour. I don’t like that. We could express the fact that getting a user’s reputation depends on authenticating the user first – but we’d end up finding the reputation of user 0, because that’s the result we’d pass in. That sucks.

EDIT: Along the same lines of the previous edit, we can make this code neater and avoid using WhenAll:

Back to the original post…

Ironically, someone on Twitter mentioned a new term to me today, which seems strikingly relevant: joinads. They pointed to a research paper written by Tomas Petricek and Don Syme – which on first glance is quite possibly exactly what I’ve been mostly-independently coming up with here. The reason LINQ query expressions don’t quite fit what we want is that they’re based on monads – if they’d been based on joinads, maybe it would all have worked well. I’ll read the paper and see if that gives me the answer. Then I’ll watch Bart de Smet’s PDC 2010 presentation which I gather is rather good.

Conclusion

I find myself almost disappointed. Those of you who already understand monads are quite possibly shaking your heads, saying to yourself that it was about time I started to "get" them (and that I’ve got a long way to go). Those of you who didn’t understand them before almost certainly don’t understand them any better now, given the way this post has been written.

So I’m not sure whether I’ll have any readers left by now… and I’ve failed to come up with a good solution to the original problem. In my view the nicest approach by far is the one using tuples, and that requires more language support. (I’m going to nag Mads about that very shortly.) And yet I’m simultaneously on a huge high. I’m very aware of my own limitations when it comes to computer science theory, but today it feels like I’ve at least grasped the edge of something beautiful.

And now, I must stop blogging before my family life falls apart or my head explodes. Goodnight all.

Some of you may remember the bizarre query expressions I’ve come up with before now. These rely on LINQ finding the members it needs (Select, Where, SelectMany etc) statically but without relying on any particular interface. I was pleased to see that the C# 5 async support is based on the same idea. Here’s the relevant bit of the draft spec:

The expression t of an await-expression await t is called the task of the await expression. The task t is required to be awaitable, which means that all of the following must hold:

• (t).GetAwaiter() is a valid expression of type A.
• Given an expression a of type A and an expression r of type System.Action, (a).BeginAwait(r) is a valid boolean-expression.
• Given an expression a of type A, (a).EndAwait() is a valid expression.

A is called the awaiter type for the await expression. The GetAwaiter method is used to obtain an awaiter for the task.

The BeginAwait method is used to sign up a continuation on the awaited task. The continuation passed is the resumption delegate, which is further explained below.

The EndAwait method is used to obtain the outcome of the task once it is complete.

The method calls will be resolved syntactically, so all of GetAwaiter, BeginAwait and EndAwait can be either instance members or extension methods, or even bound dynamically, as long as the calls are valid in the context where the await expression appears. All of them are intended to be “non-blocking”; that is, not cause the calling thread to wait for a significant amount of time, e.g. for an operation to complete.

As far as I can tell, either the CTP release hasn’t fully implemented this, or I’ve interpreted a bit overly broadly. Still, let’s see what works and what doesn’t. For simplicity, each example is completely self-contained… and does absolutely nothing interesting. It’s only the resolution part which is interesting. (The fact that the Main method is async is quite amusing though, and takes advantage of the fact that async methods can return void instead of a task.)

Example 1: Boring instance members

This is closest to the examples I’ve given so far.

Hopefully this needs no further explanation. Obviously it works fine with the CTP. The compiler generates a call to GetAwaiter, then calls BeginAwait and EndAwait on the returned Awaiter.

Example 2: Extension methods

The CTP uses extension methods to get an awaiter for existing types such as Task – but I don’t think it uses them for BeginAwait/EndAwait. Fortunately, there’s nothing to stop us from using them for everything, and there’s nothing forcing you to put the extension methods on sensible types, either – as demonstrated below:

I should just emphasize that this code is purely for the sake of experimentation. If I ever see anyone actually extending int and string in this way in production code and blaming me for giving them the idea, I’ll be very cross.

However, it all does actually work. This example is silly but not particularly exotic. Let’s start going a bit further, using dynamic typing.

Example 3: Dynamic resolution

The spec explicitly says that the methods can be bound dynamically, so I’d expect this to work:

Unfortunately, in the CTP this doesn’t work – it fails at compile time with this error:

Test.cs(16,25): error CS1991: Cannot await ‘dynamic’

All is not lost, however. We may not be able to make GetAwaiter to be called dynamically, but what about BeginAwait/EndAwait? Let’s try again:

This time we get more errors:

Test.cs(22,25): error CS1061: ‘dynamic’ does not contain a definition for ‘BeginAwait’ and no extension method ‘BeginAwait’ accepting a first argument of type ‘dynamic’ could be found (are you missing a using directive or an assembly reference?)

Test.cs(22,25): error CS1061: ‘dynamic’ does not contain a definition for ‘EndAwait’ and no extension method ‘EndAwait’ accepting a first argument of type ‘dynamic’ could be found (are you missing a using directive or an assembly reference?)

Test.cs(22,25): error CS1986: The ‘await’ operator requires that its operand ‘DynamicAwaitable’ have a suitable public GetAwaiter method

This is actually worse than before: not only is it not working as I’d expect to, but even the error message has a bug. The await operator doesn’t require that its operand has a suitable public GetAwaiter method – it just has to be accessible. At least, that’s the case with the current CTP. In my control flow post for example, the methods were all internal. It’s possible that the error message is by design, and the compiler shouldn’t have allowed that code, of course – but it would seem a little odd.

Okay, so dynamic resolution doesn’t work. Oh well… let’s go back to static typing, but use delegates, fields and properties.

Example 4: Fields and properties of delegate types

This time we’re back to the style of my original "odd query expressions" post, using fields and properties returning delegates instead of methods:

Again, I believe this should work according to the spec. After all, this block of code compiles with no problems:

Unfortunately, nothing doing. The version using await fails with this error:

Test.cs(21,25): error CS1061: ‘PropertyAwaitable’ does not contain a definition for ‘GetAwaiter’ and no extension method ‘GetAwaiter’ accepting a first argument of type ‘PropertyAwaitable’ could be found (are you missing a using directive or an assembly reference?)

Test.cs(21,25): error CS1986: The ‘await’ operator requires that its operand ‘PropertyAwaitable’ have a suitable public GetAwaiter method

This wasn’t trying truly weird things like awaiting a class name. Oh well :(

Conclusion

Either I’ve misread the spec, or the CTP doesn’t fully comply to it. This should come as no surprise. It’s not a final release or even a beta. However, it’s fun to investigate the limits of what should be valid. The next question is whether the compiler should be changed, or the spec… I can’t immediately think of any really useful patterns involving returning delegates from properties, for example… so is it really worth changing the compiler to allow it?

Update (7th November 2010)

On Thursday I spoke to Mads Torgersen and Lucian Wischik about this. Some changes being considered:

• The C# spec being tightened up to explicitly say that GetAwaiter/BeginAwait/EndAwait have to be methods, at least when statically typed. In other words, the delegate/property version wouldn’t be expected to work.
• The BeginAwait pattern may be tightened up to require a return type of exactly Boolean or dynamic (rather than the call being "a boolean-expression" which is very slightly more lax)
• The dynamic version working – this is more complicated in terms of implementation than one might expect

Just to emphasize, these are changes under consideration rather than promises. They seem entirely reasonable to me. (Dynamic binding sounds potentially useful; property/field resolution definitely less so. Making the spec match the implementation is important to me though :)