Beware of Fear-Driven Architecture (do you fear deployments?)

At Headspring, we joke about PDD, or pain-driven development.  We consider PDD to be a good thing.  If there is pain, we do a root-cause analysis and solve the pain.  If there is no pain, we call “yagni” and move on.  If we are in a design discussion and we realize that we are proposing a solution without pain existing, we stop and move on.

Pain can be a good indicator of a specific problem that needs to be solved.  Beware of fear, however.  Often, fear of something can color our technical decisions and cause us to suboptimize one part of our system or process.

Let’s take deployments, for instance.  If our deployment is an all-weekend affair, then we will start to fear it.  If a deployment takes several people yet always goes wrong, we will dread the deployment date.  This is a very real scenario.  Without an automated deployment, this is likely to happen.  There is a deployment document with a list of steps.  Even so, this list of steps doesn’t cover all the possible failure scenarios.  Worse yet, in this environment, the last deployment is probably more than a month back in history.  This makes it hard to remember the specific problems encountered the last time.

With this dysfunctional deployment environment, the team is literally afraid of deploying the software.  They will do anything to avoid one, include making design decisions base on it.  Business rules?  Hello rules engine with rules defined in XML files.  Configuration?  Hello complex config files and “magic records” in database tables.  With this fear, it leads the team to put anything that “could” change somewhere else besides in compiled code. 

Next, the problem compounds.  Because of so many external dependencies, the production deployment becomes a two-step process:  Get the system out there.  Then make sure all the external config files and “magic records” are just right.  However, because it’s been 2 months since the last deployment, those files have been changed.  The configuration records in the database have changed.  They were not merged back to the development environment, however.  As time goes one, there are actually two code-bases that have to be kept in sync:  1.  The code.  2.  The configuration (which has more lines of code that the system itself (maybe)).

Perhaps I’ve used a little hyperbole, but I’ve seen scenarios similar to this (and have experience one that will never leave my memory).  The team lets fear drive design decisions.

Solution:  Take the fear, and identify the pain that we are afraid of.  Then take swift action and resolve the pain.  Resolving the pain of deployments would lead us to automate the process, do it more frequently, and iron out any kinks.

Without fear driving our design decisions, we can make better choices given the system’s current requirements.