Posts Tagged ‘quality’

O’Reilly gets it wrong

Friday, November 6th, 2009

O’Reilly’s Safari service provides access to thousands of books online, instantly, for a reasonable fixed monthly price. It’s a fantastic service for people like me who are frequently learning new technologies and working from a number of different locations. It’s such a good service, in fact, that even when they’ve ruined their web site and done everything they can to annoy their customers, I still can’t imagine giving it up.

Safari’s original service was to provide the text of the books in HTML form. Recently they launched a new version of the software, which does away with plain HTML and forces the reader to use a Flash-based viewer, providing reasonably faithful replication of the pages of the book but making plain text harder to read.

O’Reilly did just about everything wrong that they could have done with the launch. The new software is buggy to the point of being almost unusable, with navigation broken and corrupt and missing content commonplace. Despite the fact that this was a feature nobody asked for, the old (and bug-free) way of browsing the content as HTML was disabled as soon as the new feature went live. When customers protested that they found the new interface harder to use and preferred the old one, O’Reilly chose to argue with their customers and tell them why they were wrong. Reports of intermittent bugs are frequently met with “works for me”, with customers being forced to act as unpaid (in fact paying!) QA staff to reproduce bugs and provide evidence.

There’s a lesson in here somewhere. Safari isn’t Twitter, it’s a service that professionals pony up substantial amounts of money for on a monthly basis, and when people continue paying they expect to continue to receive the service. The fact that they have no direct competitor is the only thing saving them from a mass exodus at the moment, and most online services aren’t so lucky. The customer may not always be right, but they are never worth arguing with.

Workarounds are not good enough

Friday, October 9th, 2009

As part of my ongoing quest to introduce my girlfriend to the world of video games, last week I purchased the first episode in the series of Wallace and Gromit games, released by the studio Telltale Games. Précis: A harmless piece of “use wombat with lawnmower”-style point and click adventuring, that stays admirably true to the original source in tone and character animation, but with a fairly weak narrative drive.

I have just one problem with this game: It doesn’t run if you are connected to the internet. Not even slightly. It crashes hard. It’s 2009 and Telltale Games are apparently not testing their games on computers connected to the internet.

I don’t wish to be too hard on Telltale Games. Clearly it doesn’t have this problem on every computer; it’s probably down to the OS, or the video card, or the virus scanner, or one of many other similar variables. Testing on every combination is combinatorially impossible, so these problems are going to slip through QA and appear in production. The standards are low in computer games that retail below £10, and rightly so.

What bothers me is that once this bug was discovered in production, frequently enough to be discussed in their forums, their response has been nothing further than to suggest a workaround: disconnecting from the internet while you boot the game. Apparently in the months since the problem was discovered they’ve made no attempt to fix it, even in copies downloaded by new customers.

Again, I don’t wish to single them out. These problems are endemic to the industry, but I’d like to know why. We wouldn’t accept a kettle that emitted smoke or a watch that didn’t work on thursdays. Surely the bug must be simple to fix, and surely it will help to reassure customers who might be considering a follow-on purchase that they can expect a quality product?

I’m sure some will argue that this is a free market, and that the market is merely reflecting consumer preference. This is certainly true, and if the only way to change it were some form of coercion, I’d be happy to leave things as they are. But it seems to me that a world of poor expectations and shoddy code is just one stable solution to the differential equation that governs the software industry. I don’t believe that quality products and satisfied customers are an impossible goal, but the question is how to get from here to there.

I don’t have any answers as to what is holding us back, but one idea does occur to me. It’s worth noting that workarounds are much more prevalent in software than other industries, for purely technical reasons. I struggled even to come up with a hypothetical example where an engineering fault in an electrical appliance could be righted by an untrained user’s actions, but this is commonplace in software.

Maybe the presence of workarounds is part of the problem. Consider the classic 4-quadrant task planning diagram:

Workaround quadrants

The workaround doesn’t make the bug much less important: it still reflects badly on your company. However, it becomes much less urgent, and moves from quadrant 1 to 2. It’s well known that companies tend to spend too much time in quadrant 3 to the detriment of quadrant 2.

Even worse, it’s hard to tell when something is really important to your company, and I suspect a lot of time is wasted in quadrant 4 as well. Maybe thousands of these tiny misallocations of time are having a profoundly deleterious effect on the industry.