Stephen Colebourne’s comment on my last blog post (adding 1 month -1 day to January 29th) have knocked me for six. To avoid thinking about how I might implement the preferred behaviour in Noda Time while still using Joda Time’s "engine" I’ve decided to write about something else which has been niggling at me.
For a long time, I’ve espoused the idea of "design for inheritance or prohibit it" – in other words, default to sealing classes and making methods non-virtual unless you have a good reason to do otherwise. I’ve usually attributed this phrase to Josh Bloch writing in Effective Java, but it could well have been round for a long time.
Whenever I’ve espoused this in public, it’s caused disagreement. Not universal disagreement (which would be easy to cope with; if everyone else thinks I’m wrong, that’s a very strong indicator that I’m really missing something) but a fairly even distribution of "absolutely" and "hell no". Most people seem to feel passionately one way or the other. This has led me to water down my line of "the C# designers should have made classes sealed by default" to "the C# designers shouldn’t have included a default for classes being sealed or not – make the developer specify it".
One thing I’ve found interesting is that the split between "make everything non-virtual" and "make everything virtual" isn’t one of "smart" vs "dumb". There are plenty of publically admired developers on both sides of the fence, and my colleagues are reasonably evenly split too. However, I have detected a correlation in terms of programming preferences around type systems: I’ve generally found that those who are in favour of making everything virtual by default are generally more likely to also be enthusiastic around dynamic typing. That won’t be universally true of course, but I think one is likely to be a reasonably good predictor of the other.
Ultimately I think it’s about a balance, and different people place different amounts of value on the various pros and cons. It’s also about the relationship between different parties. Different pros and cons affect different parties in different ways.
A relatively inflexible API with a flexible implementation
I’m happy when I know everything that is going on in my code. I interact with other code through obvious dependencies: they are provided to me explicitly. You’re welcome to modify my code’s visible behaviour by implementing those dependencies in different ways, but my code should be fine as long as you abide by the contracts expressed in the dependencies (typically interfaces).
If I call one of my own non-virtual methods from within my code, I know what it’s going to do. If I have two non-virtual methods which could be implemented by one calling the other either way round, then it doesn’t matter which option I pick. I can change my mind later on, and no-one will be any the wiser. All the externally visible behaviour will be exactly the same. I don’t need to document which method calls which – just what the final results are.
If I create an immutable type and seal it, then all the immutability is within my control. If I’ve picked immutable types for my member variables, have Object as a base class, and make sure I don’t mutate anything myself, I’m fine. I can rely on my values being unchanging, and so can my callers. They can cache a value with impunity.
Basically, everything is simple… unless you want to make one of my types behave slightly differently.
Flexibility with the risk of breakage
The above section sounds all nice and rosy… but what if you want to just tweak my type slightly? You only want to override one method – how much harm can that do? You’ve looked at the implementation and seen that nothing actually relies on it working exactly the way it does… and it doesn’t call any other public members. If my type implements an interface including the member you want to tweak, then you could potentially implement the interface and delegate almost all calls to an instance of the original type, but implement that one call differently. Of course, delegation is great in many cases – but can fail when there are complex relationships involved (such as when the delegated instance passes itself to something else). Basically there are identity issues.
It would be much simpler in this case if you could override my method. That might help your testing, or allow you to use my type in a way I’d never anticipated, achieving fabulous things. As Stroustrup wrote, "I wouldn’t like to build a tool that could only do what I had been able to imagine for it." Now I believe there’s a big difference between imagining a "big picture" which my component may be some tiny part of, and imagining a crazy use for the type itself, but the sentiment is still worth considering. Creative freedom is a nice thing to have, after all… who am I to stop you from achieving greatness?
The downside is that you’re opening yourself to the risk of your code breaking if I change my implementation details in another release. Maybe it would only mean your tests breaking – maybe it’s your production code. (While I don’t want to put too many words in the mouths of those who hold the opposite opinion to me, I believe a lot of their reason for wanting to be able to override methods is to make testing easier. Personally I prefer to construct test doubles which implement interfaces directly, but I do understand that’s not always feasible – especially if the component in question hasn’t been designed with testability in mind to start with.)
In many cases there’s genuinely little risk of that actually happening… but how tolerant are you of that risk?
Risk evaluation and propagation
When I wrote about what might break if the library producer changes their code, I mentioned your production code and your test code. There’s a much nastier risk though: you break someone else’s code which is relying on yours.
Suppose I produce library X, and you use it in library Y. Now Joe Developer uses both of our libraries in his application… except he uses a new version of X. Maybe it’s a bug-fix version, which is meant to have no externally-visible changes… except it changes how it uses its own methods, in a way which will break if you’ve overridden some methods in a particular way… and you’ve done so in library Y. As far as Joe Developer is concerned, his combination of X + Y is broken. Who’s fault is it?
- Mine for changing the behaviour of X in a seemingly sensible way?
- Yours for overriding a member of X in Y in a way I hadn’t anticipated?
- Joe’s for using a version of X which you hadn’t developed Y against?
Maybe all three. The trouble is, as the developer of Y you have no way of knowing how likely it is that I’ll change the details of my implementation in "seemingly harmless" ways. Indeed, you may even have performed some testing of Y against the version of X that Joe is using… but maybe Joe’s overriding some other members of the types from X and Y in ways that neither you nor I expected… and the combination could be complex to work out.
Now this all sounds like doom and gloom – but you need to remember that there must have been reasons for overriding those members to start with. Achieving the same goals without using inheritance could certainly have been considerably more complex, and introduced other bugs along the way. Using inheritance could have been a big win all round, right up until the point where everything screwed up… at which point it may still be recoverable, or it could be a knot which can’t be untied. You probably won’t know until the breakage happens, and you probably can’t accurately gauge the likelihood of it happening in the first place. It may well never happen.
Three options as library providers and consumers
It seems to me that when you’re building an API, there are three broad options available (obviously with nuanced positions between them):
- Make every type unsealed, and every method virtual – but don’t make any guarantees about what will happen if someone overrides methods.
- Make every type unsealed and every method virtual – but document/guarantee every internal interaction, so that anyone deriving from your class can predict how it will behave.
- Make everything sealed or non-virtual unless you can foresee a reason for overriding it. Document what sort of overriding you expect to handle, and where the overridden methods will be called.
As the consumer of an API, you have various choices too:
- Rely on undocumented behaviour, betting that you’ll save more time by doing and fixing breakage later
- Only rely on documented behaviour when calling code, but rely on undocumented behaviour when overriding code, as this is typically less well documented anyway (very few APIs will specify exactly what’s deemed acceptable)
- Only rely on documented behaviour
While these options are reasonably easy to describe, they again miss the oh-so-common situation: I’m consuming someone else’s types, but providing my own types to other developers. This mixed behaviour is where a lot of the complexity comes in, increasing the risk of breakage and increasing the cost of fixing the problem.
Conclusion
I still believe that designing for inheritance or prohibiting it makes sense if you want to provide a robust library which makes it hard for the consumer to abuse. However, I appreciate that others want the ability to abuse a library – and are willing to pay the price for that down the line. I’m concerned by the "3 party" scenario though – where developer X can shoot your foot off by abusing my code.
Sadly, I can’t see this long-running argument coming any closer to resolution. Better mix-in support within C# would at least help, I believe – but delegation is no panacea either.
I want to be a pragmatic developer: I dislike the dogmatic prohibition of convenient practices for the sake of purely theoretical reasons as much as the next person… and I genuinely can see where it can be a pain not being able to override behaviour at times. However, I have yet to be convinced that a gung-ho, "It probably won’t break, honest!" attitude is really a good option in the long term. I hope I’m gaining an increasingly open mind though – and I hope that at least by discussing this from slightly less religious viewpoints from normal, both camps can learn something from each other.
Jon Skeet's coding blog 
Wow. Long article. :) (Oh, and when’s the last time _everyone_ disagreed with you? I have a hard time imagining that even happening!)
IMHO, a big difference between Java and C# is the defaults. In Java, the advice to design for inheritance or prohibit it makes more sense, because things are virtual by default. It’s easy to write a class that can be sub-classed in a broken way.
But in C#, you have to opt-in to polymorphism. Sub-classes can’t break base class behavior except where the base class was specifically written to allow that. This is especially true if one doesn’t include protected-accessible fields (which IMHO is a good rule to follow). In C#, I worry much less about failing to seal a class and still not designing specifically for inheritance than I do in Java.
In other words, it’s not sufficient to look at “sealed or not sealed by default” in isolation. It’s important to consider that question in the context of the other language defaults and features.
LikeLike
@Peter: I agree it’s worth looking at *all* the defaults at the same time… but theoretically, it shouldn’t make any difference, should it? If we all coded exactly the design we’d carefully pondered, then the defaults wouldn’t make any difference – we’d be able to express our design either way.
In other words, the discussion of what constitutes a good design is independent of the defaults – but should *inform* those defaults.
LikeLike
While a rule-follower by nature, I do try to maintain a sense of pragmatism as a programmer. It seems to me that in this discussion, it’s worth acknowledging that internally used libraries have a very difference set of design goals and constraints than a publicly used library. The latter needs to be much more concerned with documentation, consistent behavior, and of course anticipating inheritance.
(Where, of course, “internally” and “publicly” aren’t actually two discrete scenarios, but rather the poles of a continuum).
I also think that it’s important to keep in mind the question of chance of harm and seriousness of outcome. In particular, in this specific question it seems to me that the frequency with which inheritance leads to code breakage is relatively low, and the harm that usually comes of it winds up being that something simply doesn’t work. Subtle bugs are theoretically possible to be sure, but in my experience any bugs that might occur tend to be more obvious (and don’t happen often in any case, not specifically due to inheritance-related issues anyway).
In other words, the convenience seems (to me) to be balanced nicely against whatever minimal issues might come up, when it comes to being permissive about inheritance. And while I’m not suggesting that the potential for inheritance should be ignored altogether, I’m skeptical of the need to involve oneself in a highly active design and analysis process for any non-sealed class.
And speaking of convenience, I’m curious about your observation of the correlation between “seal everything” and “dynamic-enthusiastic”. I have to admit, in my own case it holds true. I’m a much bigger fan of statically-typed code, and at the same time don’t feel a need to seal by default. And yet, it’s not clear to me why such a correlation might exist. Any theories?
I would have thought that those in favor of dynamic code would be _more_ inheritance-friendly, so as to give them more flexibility as they write their code. If inheritance is good, extension by dynamic is even better, right? Or is it specifically that using dynamic typing, it’s much more important to be able to predict what’s in a class’s implementation, lest ambiguities arise in the resolution of dynamically invoked members?
LikeLike
“I agree it’s worth looking at *all* the defaults at the same time… but theoretically, it shouldn’t make any difference, should it? If we all coded exactly the design we’d carefully pondered, then the defaults wouldn’t make any difference – we’d be able to express our design either way.”
Sorry, but I think I’m not following what you mean. Bugs notwithstanding, I believe I _do_ code according to the design I pondered (ignoring for the moment, that the design is somewhat fluid, as one almost always find things during implementation that feed back into the design). And it seems to me that rather than the design informing the defaults, it’s the other way ’round. That is, the defaults inform design, at least to a degree, because the design has to take into account the most straight-forward way to implement the code in the language of choice.
In other words, for me I don’t design in a vacuum. The design will necessarily take into account the framework in which the design needs to be implemented. To do otherwise runs a serious risk of designing something that makes a lot of sense in one programming environment, but not in the environment that’s actually being used.
LikeLike
@Peter: The correlation *is* the other way round. I screwed up when writing it. Fixing now :)
LikeLike
Excellent post, Jon.
I’m one who’s in firm agreement with your “design for inheritance or prevent it” theory. But that’s because my background is (advanced) C++, and we have the capability to do what’s called “static polymorphism” in that language (like duck typing, but at compile time with complete static type-safetiness). Since C++ has that language feature, I find that most C++ programmers do gravitate towards the “design for inheritance or prevent it” theory.
In my spare time, I play with a C#-like language that I made up. One thing that I’ve given a lot of thought to is what you’re addressing in this blog post; in a way, it’s a problem right at the core of modern OOP. The current revision of my language doesn’t have a default of “sealed class” nor “inheritable class”, and “inheritable class” methods must be explicitly “virtual” or “sealed”. I’ve even gone so far to separate the “protected API” from the “public API” – the longer I consider it, the more it seems that “sealed class” and “inheritable class” are two completely different concepts.
You also hit the nail on the head re the Update Problem.
I think there’s two reasons people insist on “every class should be inheritable”:
1) Everyone learns OOP in school now, with the idea that inheritance is everything; therefore all classes should allow inheritance. Unfortunately, that type of “OOP” is misleading; the focus should be on polymorphism rather than inheritance. I’ve done a lot of OOP (using static polymorphism in C++) without a single class hierarchy.
2) C#, VB, Java, and most other statically-typed languages (including MS C++ until just a few years ago) simply cannot do static polymorphism, so it may be a case where people’s thinking is restricted by the language they use.
One last point: It is *really hard* to document a protected API, since one must take into consideration the full interactions with the public API. For example: HashAlgorithm. It’s just not possible to create an implementation without Reflector. :(
LikeLike
I actually enjoyed reading this article, despite getting no closer to a resolution at the end! Hypotheticals, hypotheticals…
LikeLike
>I dislike the dogmatic prohibition of convenient practices for the sake of purely theoretical reasons as much as the next person… and I genuinely can see where it can be a pain not being able to override behaviour at times.
In the vast majority of situations I avoid inheritance. In other words, very rarely I use inheritance ‘when it really makes sense’. As every engineering activity, programming is not all black or all white, developers needs experience to know when choosing inheritance won’t probably hurt.
Typically young developers loves inheritance, and it takes a lot of time and pain until they become experienced enough to be careful enough with this dangerous possibility.
I wrote an article on the topic 6 years ago, (but it is in French :)
http://www.dotnetguru.org/articles/dossiers/heritageimpl/FragileBaseClass.htm
The article enumerates many reasons against inheritance, including
-syntactic fragility
-semantic fragility and incoherence
-lack of flexibility
-poor code readability and complex code flow
-high coupling between base and derivatives
LikeLike
I think it depends mostly on who the consumer of the class is going to be. If it’s a group in a single company, where the use cases are known and specific, then close it up. If it’s the great wide world with potential applications that you’ve never considered, then by all means keep it open.
I have been strongly in the design-for-inheritance-or-prohibit-it (DFIOPI) camp ever since I learned the “sealed” keyword eight years ago, and have just in the last few months been starting to realize the error of my ways. I think it actually started with the Google App Engine. I wanted to do a quick web app for my company and decided to give it a shot. I have a background in web Java, but have been doing C# on embedded platforms for the last few years, so getting eclipse, the GAE plugin, the java runtime, and all that set up was a pain. Then integrating with GAE was a pain. What are all these configurations? What’s going on under the hood? Gah, google “python vs java google app engine” and realize a lot of people seem to think Python is the better platform and surprised to discover that Python was actually the original platform and Java was added only later. So, give it a try. Go through tutorial; use notepad++ to print “hello world”, upload, alas a perfectly working google app engine app in 5 minutes. And that was something of a revelation.
The freedom of a dynamic language is, well, freeing, and you learn to go through the library’s documentation and understand what should and should not be done, but you’re not prevented from at least trying anything you like. It’s like a convertible on the open road versus strongly-typed languages’ dusty clunker on the way to work. A life in, say, Montana, versus a life in, say, Jakarta.
So in the last few months I’ve gone from DFIOPI always, to “default to DFIOPI, but make-everything-virtual (MEV) if the situation lends itself to it”, to “default to MEV and DFIOPI only if necessary”. I still find it hard to force myself to leave classes open; “design intent! design intent!” my good conscience screams.
Anyway I digress and I’m not sure what my point is in the first place. But I think if you’ve never done it before, try to go through the tutorial for creating a GAE app in Java, and then do it again for Python, and see if it changes your viewpoint at all.
LikeLike
Where you intend to live on the stack plays a part in your implementation decisions. 3rd-party-Jodatime has more freedom than on-every-classpath-for-ever-JSR-310 will. When I look at what was done in ICU4J vs the platform, I can understand the trade-offs.
> I’m concerned by the “3 party” scenario though
In terms of complexity management, this is a bigger/wider problem for me than inheritance. Versioning/dependency management is an OO problem that tends to get glossed over.
(Adjust based on the type-visibility/versioning-support of your chosen language/platform. I work in the Java space.)
LikeLike
This is a complicated problem, and one that doesn’t have a clear right/wrong answer. There are many trade-offs and compromises that can be considered, in making a decision about how to embed extensibility in code that we expose to others.
In my opinion, one mitigating strategy is to ship conformance (acid) tests as part of your library or product.
These are essentially unit tests written in a way where each test accepts an instance of some type or interface T and runs a suite of verifications on it to ensure that any invariants or expectations in behavior are preserved by inheritors.
Unlike documentation, unit tests codify your expectation in machine-executable, unambiguous terms. The specification *IS* the verification process.
This approach doesn’t solve the problem entirely, but it does provide a bit of a safety net, as well as a body of documentation that can grow with the product that help future consumers understand the expectations and use-cases that a library they are relying on requires.
LikeLike
I always start off with sealed classes and non virtual methods. Even if I’ve got a factory method in my class that instantiates a dependant class, the method is private. Once the class is used in a real application I’ll make the factory method(s) protected and virtual so I can create stubs for testing purposes. In many cases I’ve waited for other developers to ask for a virtual method (or if I myself have needed a method to be virtual) before opening up classes for polymorphic behavior over methods that were not anticipated. I follow the same practice for framework code as well
If I override a method I’ll seal it as well and frankly, there are very few cases where these decisions have bitten me. On the contrary, there are many cases where class library/framework designer have regretted making a method virtual in version 1.0.
I think the notion that someone can do some wild and innovative things if the class was completely open is a fallacy. If the class was designed carefully/correctly then behavior changes (by descendants) would be possible, anything beyond that is not inheritance is it (Liskov’s substitution principle)?
>>
The downside is that you’re opening yourself to the risk of your code breaking if I change my implementation details in another release
>>
If code can break later in time, then maybe there is a design problem?
>>
I’ve generally found that those who are in favour of making everything virtual by default are generally more likely to also be enthusiastic around dynamic typing.
>>
Funny you say that, I’ve found that to be true as well. I think the common denominator here is “don’t make me think, just let me do what I want”.
LikeLike