It’s been a little while since I’ve blogged, and quite a lot has been going on. In fact, there are a few things I’d have blogged about already if it weren’t for “things” getting in the way.
Rather than writing a whole series of very short blog posts, I thought I’d wrap them all up here…
C# in Depth: next MEAP drop available soon – Code Contracts
Thanks to everyone who gave feedback on my writing dilemma. For the moment, the plan is to have a whole chapter about Code Contracts, but not include a chapter about Parallel Extensions. My argument for making this decision is that Code Contracts really change the feel of the code, making it almost like a language feature – and its applicability is almost ubiquitous, unlike PFX.
I may write a PFX chapter as a separate download, but I’m sensitive to those who (like me) appreciate slim books. I don’t want to “bulk out” the book with extra topics.
The Code Contracts chapter is in the final stages before becoming available to MEAP subscribers. (It’s been “nearly ready” for a couple of weeks, but I’ve been on holiday, amongst other things.) After that, I’m going back to the existing chapters and revising them.
Talking in Dublin – C# 4 and Parallel Extensions
Last week I gave two talks in Dublin at Epicenter. One was on C# 4, and the other on Code Contracts and Parallel Extensions. Both are now available in a slightly odd form on the Talks page of the C# in Depth web site. I no longer write “formal” PowerPoint slides, so the downloads are for simple bullet points of text, along with silly hand-drawn slides. No code yet – I want to tidy it up a bit before including it.
Podcasting with The Connected Show
I recently recorded a podcast episode with The Connected Show. I’m “on” for the second 2/3 of the show – about an hour of me blathering on about the new features of C# 4. If you can understand generic variance just by listening to me talking about it, you’re a smart cookie ;)
(Oh, and if you like it, please express your amusement on Digg / DZone / Shout / Kicks.)
Finishing up with Functional Programming for the Real World
Well, this hasn’t been taking much of my time recently (I bowed out of all the indexing etc!) but Functional Programming for the Real World is nearly ready to go. Hard copy should be available in the next couple of months… it’ll be really nice to see how it fares. Much kudos to Tomas for all his hard work – I’ve really just been helping out a little.
Starting on Groovy in Action, 2nd edition
No sooner does one book finish than another one starts. The second edition of Groovy in Action is in the works, which should prove interesting. To be honest, I haven’t played with Groovy much since the first edition of the book was finished, so it’ll be interesting to see what’s happened to the language in the meantime. I’ll be applying the same sort of spit and polish that I did in the first edition, and asking appropriately ignorant questions of the other authors.
Tech Reviewing C# 4.0 in a Nutshell
I liked C# 3.0 in a Nutshell, and I feel honoured that Joe asked me to be a tech reviewer for the next edition, which promises to be even better. There’s not a lot more I can say about it at the moment, other than it’ll be out in 2010 – and I still feel that C# in Depth is a good companion book.
MoreLINQ now at 1.0 beta
A while ago I started the MoreLINQ project, and it gained some developers with more time than I’ve got available :) Basically the idea is to add some more useful LINQ extension methods to LINQ to Object. Thanks to Atif Aziz, the first beta version has been released. This doesn’t mean we’re “done” though – just that we think we’ve got something useful. Any suggestions for other operators would be welcome.
Manning Pop Quiz and discounts
While I’m plugging books etc, it’s worth mentioning the Manning Pop Quiz – multiple choice questions on a wide variety of topics. Fabulous prizes available, as well as one-day discounts:
- Monday, Sept 7th: 50% of all print books (code: pop0907)
- Monday, Sept 14: 50% off all ebooks (code: pop0914)
- Thursday, Sept 17: $25 for C# in Depth, 2nd Edition MEAP print version (code: pop0917) + C# Pop Quiz question
- Monday, Sept 21: 50% off all books (code: pop0921)
- Thursday, Sept 24: $12 for C# in Depth, 2nd Edition MEAP ebook (code: pop0924) + another C# Pop Quiz question
Future speaking engagements
On September 16th I’m going to be speaking to Edge UG (formerly Vista Squad) in London about Code Contracts and Parallel Extensions. I’m already very much looking forward to the Stack Overflow DevDays London conference on October 28th, at which I’ll be talking about how humanity has screwed up computing.
Future potential blog posts
Some day I may get round to writing about:
- Revisiting StaticRandom with ThreadLocal<T>
- Volatile doesn’t mean what I thought it did
There’s a lot more writing than coding in that list… I’d like to spend some more time on MiniBench at some point, but you know what deadlines are like.
Anyway, that’s what I’ve been up to and what I’ll be doing for a little while…
Jon Skeet's coding blog 
It’s a pity about Parallel Extensions.
Modern language must contain features for multiprocessing support. So it _must_ be a language feature.
LikeLike
@Eric: I don’t think the language really *has* to have any multiprocessing features. For example, C# could have been designed without “lock” or “volatile” – volatility could have been done with attributes, and “lock” could have been done with a lock type which issues tokens implementing `IDisposable`.
Parallel Extensions is clearly a library rather than a language feature.
LikeLike
I was lucky enough to get a ticket to DevDays yesterday – woohoo! Looking forward to your talk, Jon.
LikeLike
Loved the connectedShow interview … one question however … you said that Linq to Objects can be a performance problem … “poor performace” I think you said .. anyway could you explain what you mean or just point to the relevant section of your book or a post on stack overflow, etc … lastly, you made the right choice regarding PFX and contracts, Contracts eliminate soooo much ‘defensive code’. Is this null? How did I pass null to this? Usually I find these after deployment. Contracts rule!!
kf
LikeLike
Personally, I want to wait to see how the concurrent/parallel frameworks play out (in both C# and Java) … Is PFW the best bet? Will CCR win out? … I dont know? … I’m not ready to bet on any one horse yet. And we may get something in the C# language.
I favor waiting.
kf
LikeLike
@keithflo: I didn’t say LINQ to Objects had poor performance – or at least I hope I didn’t. I at least *meant* to say that you can implement LINQ to Objects quite easily, if you don’t care about performance. It’s the design which is the clever part.
In fact you can implement an awful lot of it easily without worrying about performance – it’s things like sorting where a naive implementation will fall down a bit, as well as places where IList implementations can win out (e.g. Count() and Last()).
LikeLike
Unfortunately the BCL implementation of LINQ to Objects does have poor performance, as you demonstrated yourself in some earlier posts. Some of the Enumerable methods have exactly the defect you mentioned — they don’t bother to check for availability of a Count property which in-memory collections almost always have.
Such implementation shortcomings aside, LINQ queries are assembled from primitive operations that must communicate via intermediate collections or iterators. That’s another big performance penalty that custom code may be able to avoid.
But I haven’t listened to your podcast yet, so I apologize if you have already found a way to make LINQ to Objects ten times faster. :)
LikeLike
So I am taking it that you will not include anything in the book on the Rx Framework?
A.
LikeLike
@Anthony: No, I suspect I’ll use the Rx Framework as another example of in-process LINQ, although I doubt I’ll go into much detail on it.
LikeLike
I was looking at ToDelimitedString in MoreLINQ, and I noticed it uses a StringBuilder. I’ve found that calling ToArray on an IEnumerable and then passing that into String.Join consistently outperforms using a StringBuilder. I’d be curious to know if other people see the same results.
LikeLike
@Joel: We’ve looked at this a lot on StackOverflow, and I seem to remember coming to similar conclusions. There are certainly cases where a StringBuilder *will* outperform ToArray, admittedly, but we may well change the implementation there.
LikeLike
“Unfortunately the BCL implementation of LINQ to Objects does have poor performance, as you demonstrated yourself in some earlier posts. Some of the Enumerable methods have exactly the defect you mentioned — they don’t bother to check for availability of a Count property which in-memory collections almost always have.”
Can you tell me which methods you’re referring to? According to Reflector, the obvious LINQ to Objects methods (Count(), Last(), ElementAt(), LastOrDefault(), ElementAtOrDefault()) all do in fact attempt to cast the IEnumerable to an ICollection or IList as appropriate to do things the optimized way when possible. I’d like to know which methods fail to do this, since you seem to have specific ones in mind.
LikeLike
Joel: The critical method where I first discovered this issue was SequenceEquals for which I’ve opened a (fruitless) MSDN feedback item:
https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=361633
But there are a few more methods that don’t check for ICollection.Count. Looking through the class with Reflector, I found the following:
All and Any; Contains with a comparer; DefaultIfEmpty; ElementAt and ElementAtOrDefault (they only check for IList); SingleOrDefault (the simpler overload also checks only for IList); and, strangely enough, LongCount.
***
The fundamental problem with LINQ to Objects in general is that you cannot selectively replace a few methods with optimized ICollection/IList overloads because any intermediate result from another Enumerable method would be an IEnumerable and thus would not trigger the optimized overload.
Moreover, even those Enumerable methods that theoretically can use Count, indices, etc. won’t do so when operating on an intermediate LINQ result, because that intermediate result is usually a newly constructed hidden iterator class — even if the original source was an ICollection or IList!
So for a performant LINQ to Objects that always uses Count and perhaps indices, you would have to duplicate the *entire* Enumerable class with ICollection/IList-typed overloads that also return ICollection/IList instances when creating intermediate collections.
LikeLike
Ok, you are right that it’s no sense to hardcode the library in the C# language, because in future will be new libraries.
But consider the other meaning of the word ‘language’. Language as the style of programmer everyday coding. Not C# language, but the language of programmer that use it.
From this point of view, whether these features will be hardcoded or not is the detail of realization.
LikeLike
Apologies if this is a duplicate, I got an error on submission last time.
@Chris, Okay, I completely take your point on SequenceEquals – that seems like a fairly ridiculous oversight. Likewise, with LongCount.
I’m not sure I agree about the other methods you mention. All has to test every element until it finds an element that fails the test regardless, so I’m not sure what optimization you could do to that one. Any will stop iterating as soon as it gets a match, which is instantly if the enumerable is empty, so I don’t see much benefit from checking the Count. What do you expect Contains with a comparer to do? The ICollection.Contains method doesn’t support comparers. DefaultIfEmpty is arguable – it can’t take that long to call MoveNext exactly one time to find out if the sequence is empty. As far as the ElementAt methods and SingleOrDefault, what other types should they check for? IList will cover arrays and a whole lot of other indexable types – almost any that I’m personally likely to use with LINQ.
It’s true that intermediate results can’t be cast to the underlying IList or whatever, but I honestly don’t see this as a very big handicap. Sure, filtering a large IList with Where and then getting the Count won’t be very efficient, but it’s just barely less efficient than filtering the IList manually and accumulating a count as you go. Besides, if I just want to know if my filtered list is empty or not, I wouldn’t ask for the count, I’d use Any.
LikeLike