Tao of Software Engineering

Scott Rosenberg's latest book "Dreaming in Code" contains a very readable history of Software Engineering. I will review the book in another post. Here, I want to talk about a couple of papers that are not mentioned in Scott Rosenberg's history that are parts of my Tao of Software Engineering.

The first is Melvin Conway's paper from 1968 called "How do Committees Invent". Fred Brooks references the paper in "The Mythical Man Month" and gave it the name "Conway's Law", however, only recently that my good friend Roger Shepherd pointed me to the original. The thesis of the paper is: "Any organization that designs a system (defined more broadly here than just information systems) will inevitably produce a design whose structure is a copy of the organization's communication structure."

While Conway's paper gives examples from several different branches of engineering, it is more applicable to Software Engineering which is the most plastic of the engineering arts. After all, a building has to have a floor, walls and a roof; a semiconductor chip is laid out on a flat surface; most mechanical devices start with a rotary power source and after that it is a question of packaging. On the other hand, software can take on any structure so it is absolutely natural that it should take of the structure of the organization producing it, whatever unsightly skew the organization may place on the software's structure.

It is not only the structure of the organization that influences a software architecture, it is the way the project is put together. For example, I have recently been working on a software project that started as a GUI demonstration. Because the GUI came first, it defined the client-server API and the data structures that passed through the API which then defined the database storage structure. We recently had to make significant changes to the API and data structures which has been extremely painful at such a late stage in the project.

This brings me to the second paper, Butler Lampson's "Hints for Computer System Design" from 1983. Like Conway's paper, it is not a difficult read. The paper is a collection of simple aphorisms on computer system design and examples to back up those aphorisms. While the aphorisms are still relevant to this day, some of the examples are creaky. For example, who remembers PL/1 or the SDS 940?

I reread this paper every few years and compare with my recent experience to see what I have done right and what could be done better next time. For example, one aphorism is "Keep basic interfaces stable". Several years ago, my very first act in a project to add a significant new feature to a large system was to go through the entire code base and change some basic APIs for the new feature. In my most recent project we have gone through months of pain because we decided to change the API just as the project was coming together.