While cross-posting here some comments I had written last year on the Wayland STEM site, I saw that I had written about my first two rules of programming. I left out the third rule.
The third rule of programming is simple:
If it takes you more than three sentences to describe to me why it works, it doesn’t.
This is a powerful rule. When someone is droning on about how their design can’t fail, you can just stop listening. It will fail, and probably fail catastrophically.
Remember the Titanic with the waterproof bulkheads? The designers could show that even when a compartment was filled with water, the ship would float. Even with several compartments flooded, the ship had enough buoyancy to remain above the water. It was designed to be absolutely unsinkable … until the hull plates riveted over three adjacent compartments were ripped out by an iceberg. Even then it floated … but the balance of the ship was off, water poured over the top of the watertight doors … and the ship filled with water like it was flowing from one hole to another in an ice cube tray.
Almost always, somewhere in the explanation the phrase “will never happen” appears.
May I tell a story?
When I was a new engineer at Intermetrics, Inc. in Cambridge, Massachusetts, my boss, Dr. James S. Miller, Jim, told me a story about a design session involving the Apollo guidance computer. I suggest reading the WikiPedia article on the Apollo Guidance Computer. The article describes a problem encountered during the Apollo 11 Lunar descent, when the guidance computer was overloaded. Jim was at the table during a design review. The software was too big, and they were looking for instructions to remove to make space. Some engineers were suggesting that the operating system code to cancel low-priority tasks was unneeded, since they had analyzed everything that could happen during every phase of the mission and were certain that there could not be a case where the computer would be overloaded. By removing that code, other important functions could be added, saving time, cost, and probably weight as well. One man, perhaps Dr. Woodrow Vandever, another former Apollo manager and member of this small company, refused to let the team take the simple approach, even though there was stack of paper proving it would work.
As the WikiPedia article describes in the section titled “PGNCS trouble”, an unexpected, unanticipated, but completely explainable condition developed. The computer was overloaded, but the unnecessary, wasted code shed processor load. The critical, real-time functions were executed on schedule. The mission succeeded. NASA, with their amazingly complete and persuasive analysis, had almost believed their lengthy analysis enough to destroy the mission.
From this story, and endless rationalizations for why one engineer or another didn’t want to do the right thing and simplify a problem until it was solved, I stumbled on this rule of explanation.
There are several ways to justify this rule, and I try to not fall victim of this same rule in describing why the three sentence rule works. Of course, that would be an interesting paradox: If rule takes more than three sentences to explain, it must not work. But if the three sentence rule is invalid, then it is permitted to use more than three sentences to prove it. Once proven, it must be proven in three sentences or be invalid. Godel would have loved this.
First justification, in three sentences:
Each sentence defines a string of contingent assumptions. Each assumption can be wrong. The probability of all assumptions being right is near zero.
Second justification, in one sentence:
When you explain with many sentences, only one needs to be wrong.
Third justification, in four words:
Each word invites Murphy.
BTW, there is a rich record of engineering tales from Apollo available on the web. The Report of Apollo 13 Review Board (referenced here as a web-reconstructed document) is a long but glorious read for anyone interested in failures, successes, management processes, engineering techniques, materials science, or detective stories.
Also, for those interested in more details of the Apollo-13 guidance software, especially the operating system design, read this report.