Book review: Knowledge for Action: A Guide to Overcoming Barriers to Organizational Change

March 27th, 2011

“The Human Side of Software Development” may just be a tacky slogan I came up with on the spur of the moment to make my WordPress install just a tad less generic, but the sentiment behind it is genuine, and something that I’ve always meant to expand more on in this blog. So here goes, with a review of a decidedly non-technical book.

One thing I’ve been convinced of since I first read Peopleware is that human factors are the cause of more of the problems in the average software team than technical issues. It’s not just that human problems exist in our teams and are difficult to solve, it’s that we don’t learn from our mistakes. Dev never talks to marketing. Engineers blame the testers, and testers blame engineers. Management write off all techies as being difficult to manage. These tropes are played out again and again in thousands of teams, and we still don’t seem to have a really clear idea what the underlying problem is, let alone what to do about it.

Nor is this just a matter of individual learning. Teams and whole organisations need to learn from their mistakes so that we don’t end up pulling in different directions, or even worse have the lone people who feel they have solutions feeling powerless to influence the herd.

Enter Chris Argyris, Professor Emeritus at Harvard Business School, who has spent a lifetime researching topics like these. His book Knowledge for Action attempts to tackle one of the most crucial barriers to this sort of organisation learning, namely the defensive habits and routines that make it impossible for organisations to change. Argyris paints an all-too-familiar picture of an organisation where everyone is overtly committed to effecting some change, but politics creeps in, fights break out and people tacitly cooperate in undermining their own efforts.

Argyris’s main contention is that attempting to change organisations throws up situations of embarrassment or threat, and that people respond to this by avoiding the difficult issues. Moreover, people silently collaborate on this because it’s in nobody’s interest to uncover the threatening material. The case study that’s central to the book develops the author’s hypothesis that by changing our fundamental internal model of the world (taking the focus off winning / losing and onto objectively verifying our beliefs about others) our individual and team behaviour will naturally follow.

I suspect that two aspects of this book will appeal to those of a technical persuasion. First of all, the book is research-based and as precise in its analysis as the subject matter allows. This is not some faddy airport self-help guide for middle managers. Secondly, the approach is the quintessentially nerdy technique of looking to change the second derivative of the problem: not dealing with things that are bad, or even with how to make them better, but how to improve the ‘making better’ process. Hopefully engineers will intuitively see the potential for huge leverage in getting this right.

Unfortunately, I can give this only a qualified recommendation for readers from a technical background. Yes it’s a good book, and a great contribution to the growing body of knowledge, but ultimately it’s still a piece of social science research and the author is clearly intending it to be read by other academics with a similar background. I probably read more “soft” science research papers than the average techie and I found it pretty hard going at times.

So this is really one for the enthusiasts, or those who’ve already read The Fifth Discipline and want to take it further.

How to tell if you’re in a method in C++

March 23rd, 2011

An issue came up at work that we wanted some generic logging code to be able to report the object it was being called from. The tricky part is that we wanted the same logging macro to work in non-method context as well. Any use of this is impossible because it will cause a compile error in the latter case.

In short, both the following instances ought to work, but the one in the class method should automatically know to print some extra gubbins about the object:

struct Something {
   void someMethod() {
      LOG("Some logging message");
   }
};
 
void someFunction() {
   LOG("Some other logging message");
}

I thought this was an interesting case, not so much for the eventual solution as for the thought process involved. One of the big problems with C++ is that although you have no shortage of powerful tools that enable metaprogramming, they were never designed to provide a coherent system. It takes a bit of experience even to know what’s possible, let alone how to go about finding a way to do something. With most metaprogramming the pattern is to find one language feature that yields the information you need (perhaps a type trait, a typedef, a function return value or whatever) and abuse some language feature (sizeof, SFINAE, template argument deduction etc.) to allow the original feature to indirectly feed through to generate a usable compile-time value such as an enum or typdef.

Obviously macros are going to be involved here: no other tool in C++ allows for code reuse and gives the repeated code access to the scope in which it is invoked. But macros are purely a lexical concept, so we’ll have to include something else that operates at a syntactic level to discriminate between the cases.

So what’s different in state between member and non-member functions in C++? There’s the this pointer, for a start. Also, a different namespace scope is available and potentially different accessibility to private and protected class members. Other than that, there’s not much else in the ISO standard (compiler extensions like __PRETTY_FUNCTION__ might help, though).

Using the this keyword is the most direct route to the information we need, since it pretty much is the information we need. Unfortunately, the definition of this is such that even mentioning it when outside of object context is an instant compile error. Since we can’t so much as mention it in our macro definition, we’re going to have to be a bit craftier.

The problem is that using this outside of a method is a syntax error, which will kill our compile immediately. Except there’s precisely one case where the compiler can recover from a syntax error and take a different path: instantiating a template. If the compiler attempts a template instantiation and gets a compile error, it may discard that template specialisation and try another possibility. This is surprisingly useful and even gets its own name, the rather snappy Substitution Failure Is Not An Error.

Unfortunately SFINAE isn’t going to help us here, because the area of expanded code where an error is allowable is within a different scope than the place we’re trying to log from: it’s either always in a free function scope, or always in a method scope (depending on whether we use a template function or a template class for SFINAE), but it will never depend on the calling context.

So we’re back to the drawing board, except that we know that we can’t mention this in our macro. This leaves us with two possible ways to discriminate the cases: class member accessibility and function overloads in scope. As an example of the former, consider the following:

#include <iostream>
 
using namespace std;
 
class Something {
public:
	static void func(...) {
		cout << "Ordinary function, anyone can see this" << endl;
	}
 
	void method() {
		Something::func(42);
	}
 
private:
	static void func(int n) {
		cout << "Private function, you need to be privileged" << endl;
	}
};
 
int main(int argc, char ** argv) {
	Something::func(42);
 
	Something something;
	something.method();
}

This is how you might like it to work: the call in main() gets the unprivileged overload, and the call in Something::method() gets the more specific overload that it can only see because it’s in the context of the class. This, by the way, works by a quirk of C++ that varargs functions are always used as a last resort and only if an overload that matches better is not available. Let’s ignore for the moment the fact that this only detects if we’re in a special sort of class (namely the one that defines func(), or a friend of it) and not the general distinction between class and non-class.

Unfortunately, we can’t use accessibility in this way because function overload resolution happens before accessibility is checked. Rather than getting a different result from the overload resolution in each case, you get the same result from overload resolution in each case and a compiler error if the preferred form is not accessible. You may be thinking that SFINAE could save us here, but in fact we have the same problem as before: the call will take place outside of the scope we’re trying to detect.

So the only thing we have left that distinguishes between free function context and method context is the different set of functions in scope:

#define LOG( x ) cout << (getThis() ? "In method" : "In function") \
   << ": " << x << endl;
 
void * getThis() {
   return NULL;
}
 
struct Fish {
   void * getThis() {
      return this;
   }
 
   void method() {
      LOG("Log line");
   }
};
 
void function() {
   LOG("Log line");
}

This is promising, actually. In fact, it’s pretty close to what we want except for the fact that we have to define this log method in every class that we want to be able to log from. This isn’t as bad as it first seems, since the only reason to get hold of the this pointer in practice is to call methods on it, and we can only do that if all the classes implement a common interface anyway.

In practice, the getThis() construction is a little weak. It only returns a void *, and in order to call any methods we’ll have to cast it back to the proper type, and we’re back to the problem of not knowing the type of the class we’re in (or even if we have a class type). Rather than trying to implement reusable code in a macro definition, any code that needs to know about types will have to be in a method that’s implemented for every class we care about, and corresponding stubs at global scope. This is pretty horrible, and will probably render this technique useless.

So in the end we don’t really have a solution, but at least we can be reasonably confident that we’re not missing anything.

Much of the above code doutless contains errors, as I haven’t tested a lot of it. Feel free to write in with corrections.

Job security

March 12th, 2011

A software developer reportedly wrote malicious code to ensure job security:

It wasn’t a club but a computer virus that shut down the Whac-A-Mole and more than 400 other games built by Bob’s Space Racers in Holly Hill. The company traced the problem back to computer programmer Marvin Wimberly.

Faced with a pay cut, police believe Wimberly programmed games to fail, ensuring he would be needed and keep making money.

[...]

Each game, after turning on and off a certain number of times, sometimes 50, sometimes 500, would fail. Wimberly would be paid to fix it, and police reports say, he would insert a new virus with a new countdown.

Of course, the big story here isn’t that the malicious action happened, but that it was discovered and traced back to the allegedly deliberate actions of an employee, and that police have been involved.

My guess would be that this type of thing goes on all the time, and is either not noticed or not taken seriously. I suspect it’s mostly not a deliberate sin of comission but a subtle sin of omission: developers don’t refactor, or don’t document, or don’t upgrade their design to integrate with new components. Quite probably they come up with ostensible reasons why it can’t or shouldn’t be done that are sufficiently convincing.

The irony, of course, is that doing a better job should make you more employable, not less. Even when unemployment was at its peak, the companies I’ve been working with have been terribly short of quality developers.

Here are some of factors that I think contribute to this:

  • People end up working in software development who don’t have the necessary interest or aptitude, but carry on with it because giving up would cause them to lose face, or because even a badly-paid software job is better than a great deal of alternatives.
  • Management in companies where software is only a small component of the business don’t necessarily understand the software development process, and as a result are overly deferential to the opinions of developers. Furthermore, they overestimate the cost of replacing an incompetent developer with a competent one, and underestimate the long-term benefit.
  • Bad developers are rewarded too well, and good developers too poorly. The best developers deliver something like 20 times the benefit of the worst developers, but are lucky to get paid twice as much. This blunts the incentive for a mediocre or poor developer to improve themselves, and encourages those at the bottom end of the scale to hang on rather than move to a different career to which they may be more suited.