Irrational Exuberance!

Hard and Soft, Fluffy and Brittle

February 3, 2009. Filed under software

At work we recently started a new product cycle, and we've been debating about design. Without much experience in the joint planning of software1, my chest swelled with strong feelings about the design, equaled only by my total inability to explain them convincingly. Then I spent a week flailing ineffectually against the XML specification, and--like a Neanderthal playing with a Zippo--I discovered fire.

In software design there is an ongoing battle between the implicit and explicit, between simple and specific, between soft and hard. It was between this rock and a soft place that my mind got stuck. Now that I am newly freed, let's explore this arbitrary distinction a bit and see what it has to offer.

The Hard Style

First is the explicit, hard style. XML--bane of the haphazard--is the epitome of this style of design2. The XML specification eradicates ambiguity with a plodding hatred:

Validating Processors MUST, at user option, report violations of the constraints expressed by the declarations in the DTD, and failures to fulfill the validity constraints given in this specification. To accomplish this, validating XML processors MUST read and process the entire DTD and all external parsed entities referenced in the document.

Not just XML, but JSON, HTML and anything with an explicit schema are all of the hard style (as a software engineer, most software you use likely falls into this category).

The hard style maximizes precision by applying restraints. Good hard design maximizes precision with a minimum of restraint.

Hard style of software. Hard versus brittle.

Software with restraints that cannot be justified by precision are not hard, but are instead brittle. Earlier I said XML is the epitome of the hard style, but it is also representative of brittle software. Let's look at an example of both hard and brittle in XML. The question to ask is "Is this restraint justified by precision?"

  • By design XML immediately breaks after observing malformed data. By breaking on malformed data you eliminate ambiguous parsing. What if the spec said The behavior upon reaching malformed tags is not defined.? Well, then you'd have to deal with ambiguity between different XML parsers, and we all know the path to hell is paved with competing specifications.

    This restraint makes it possible to make a precise assumption about how XML parsers will react to bad XML, and thus it is justified by precision.

  • Another restraint in the XML specification is to die on encountering a list of illegal entities. Restricting usage of < and > within elements makes perfect sense, but why forbid the vertical tab, the null bit, and a couple dozen others? Restricting those entities increases precision in one way and one way only: you know that XML won't contain those entities.

    This restraint doesn't have a clear gain in precision, making it a brittle restraint.

Now let's take a look at the hard style's complement.

The Soft Style

Where the hard style is interested in maximizing precision, the soft style is interested in minimizing complexity. For soft software, the interesting dynamic is between choices and options; good software should have more options than choices, and those with too many choices are fluff.

Soft style of software. Soft versus fluff.

You might suspect I'm pulling the standard make up new definitions for old words and then cry wolf game, but give me a second before campaigning against my abuse of sheep. Whether or not these meanings were real yesterday, they provide a useful tool in defining what constitutes good design for soft software.

Options are a subset of choices: the subset of useful choices. Where the precision and restraints of hard style are intentionally ignorant of their user, this soft distinction between choice and option depends upon the specific user (or specific category of users). There are two ways in which a choice can fail to be an option:

  1. Unpickable choices are not options. Consider a vegetarian at a pig roast.

  2. Poorly understood or unwanted choices are not options. Consider a text-editor that lets a user pick whether to save their files in UTF-8, UTF-16 or ascii and also in either 2003 format, 2004 format, 2005 format, txt or RTF3.

    For a "poweruser" these may be options (you need your mother to be able to open the file and you know she's using the 2003 version with an operating system that can only read Shift-JIS encoding), but for many people it's just an arbitrary choice, with a payload of failure lurking in the shadows.

    These poorly understood or unwanted choices force users who've never seen a gun to play Russian roulette.

Creating effective soft software is most dependent on deciding upon your target audience and ruthlessly eliminating choices which are not options for that specific audience. Those creating the software are almost always domain experts, and for them most choices will be options, and unchecked by user testing their software will be exactly what an expert wants, but perilous for a typical user4.

Picking Between Soft and Hard

Navigating the mire between choices and options makes soft options complex to design, and also why many engineers--left to their own devices--will exclusively create hard options. Actually, more than just engineers, all domain specialists prefer hard designs for their own domain. The unparalleled minds at the IRS have been creating very precise and unambiguous forms for decades. Precise and restrained to the point that an entire industry of soft solutions to taxes has grown up around those hard tax forms. Which is exactly how it should be: domain experts use hard solutions and create soft solutions for the inexperienced.

Picking between soft and hard styles.

The more domain knowledge that your customer has about the problem, the harder the appropriate solution. The less domain knowledge, the softer.

However, it's not only about the customer. It's about the problem as well. There is some cutoff where a problem becomes sufficiently complex--or requires a sufficiently high degree of precision--that a soft solution will be more complex and less precise than a hard solution. In that vein, it helps to remember that hard isn't intrinsically more difficult than soft, it just requires more user education and knowledge. In some situations, there can
be a higher payoff for training users than writing code5.

Whether hard or soft is appropriate for project my team is embarking upon, deliberately making this distinction between hard and soft has made it much clearer to me when to prefer one approach over the other. The interplay between precision/restraint and choice/option has also given me a cleaner mental framework for deciding whether features are actually features or just flaws dressed in tuxedos.

  1. A direct consequence of the "When your cathedral is a dog house, everyone is an architect." scenario.

  2. Actually, YAML has become another example of the hard style. Looking at snippets of the 1.2 spec is a bit overwhelming.

    unicode: "Sosa did fine.\u263A"
    control: "\b1998\t1999\t2000\n"
    hex esc: "\x0d\x0a is \r\n"
    sexagesimal: 3:25:45
    not a number: .NaN
    not-date: !!str 2002-04-28
    picture: !!binary |
    - [name, hr, avg]
    - [Mark McGquire, 65, 0.278]

    ...a bit overwhelming.

  3. Consider the constrast between Pages and Word. Word has an epic quantity of choices, which people continually screw up with. No one ever makes mistakes with Pages.

    Which is largely because no one has ever used Pages, but also because they have done an excellent job of turning choices into options by only showing them to truly interested users (by breaking Word's concept of saving into Saving and Exporting).

  4. This is, in my opinion, a bit part of why indie developers focus on creating expert software, rather than general use software. Writing software for yourself is easy, and if your team is only one person, despite your best intentions you will write the software for either yourself or an imagined but non-existant user.

    This is also why user testing and user experience researchers are the way to create software that non-experts can use. In the land of the blind a blind man is king, and the one-eyed man is a heretic. They killed the two-eyed guy a couple of months ago.

  5. Photoshop dominated the image editing market for a long time with this strategy. And a lot of book authors got rich... well... got another line in their resume.