RJ Dudley has a great quickstart on getting my latest release of EZWeb up and running. If you need a website fast and easy, check out the quickstart and EZWeb. If you need a website easily modified by a family member or friend, this one is for you.
Monthly Archives: April 2006
Writing software is too EASY these days – 300
Writing software is too easy these days.
Since it’s so easy, there is a bunch of bad software out there.
It’s easy to write bad software. It’s hard to write good software.
Everyone needs software these days. From the home consumer to the large
enterprise. For the home consumer, we
have shrink-wrap software developed by large teams, such as Microsoft Office,
Quicken, etc. As a business grows,
however, it’s more likely to hire or contract some programmers to write some
custom software to drive efficiency in the business.
This is where the “software is too hard/easy” argument comes
in to play. Some complain that software
is too hard.
Paul
Reedman in 2004 blogged how he thinks software development is still too
hard. Paul focuses on Java
technologies. He complains that research
has focused on the language as the way to solve problems instead of the tools.
Steve at
furrygoat.com ponders whether writing software is too hard because it’s a
little difficult to learn how and it takes some effort.
Rocky
Lhotka declares that software is too hard.
He laments that software developers have to worry about similar tasks
from project to project (plumbing). In
this article, the tools are to blame because they provide too much flexibility
(require too many decision), whereas a good tool would reduce the creation of
software to merely configuration (my interpretation).
I declare that writing software is too EASY. For the semi-technical
folks that need to make something to help out the business, we have tools like
MS Access, Excel and even word macros.
You can do some pretty cool stuff with MS Access using linked tables to
SQL Server!
When it comes to custom
software, it requires a properly skilled person. Software is a craft. It’s bordering on engineering and overlaps as
well, but it still has some maturing to do.
An unskilled person is dangerous when attempting custom software. There are plenty of products a company can
buy and then let the semi-skilled office worker configure it, but custom
software is just that, custom. You
wouldn’t have a custom chopper built by someone who wasn’t very good, would
you?
Tools today make
custom software too easy to develop.
This statement may seem controversial, but I believe it. I’ve seen too many unskilled people take a
programming tool and throw together something that seems to work initially. The business folks can’t tell a fraud from an
expert or evaluate the work, so they use it and then feel the pain later when
they actually have to maintain it.
Software isn’t a toy. You don’t “play around” with writing
software. Companies rely heavily on the
custom software they pay for. My company
could be sued out of business if the software screwed up. Software is serious. From bank transactions to supply lines to
taking orders on the web. The business
relies on the software. If it’s buggy,
the company could be losing money and not even know it. If it was developed by an unskilled person,
it’ll be hell to maintain, and it’ll suck money out of the company later.
There’s been some talk on different types of “programmers” –
Mort/Elvis/Einstein (most recently by Scott
Bellware and 2nded by Sam
Gentile. This thought is that these
types of programmers aren’t valid anymore and that there needs to be one type (the good type)
The only type of
programmer should be the good type.
This is what it comes down too. Companies should not trust an unskilled
person to be a programmer for them.
Ultimately it’s the managers decision, but it also their
responsibility. A manager should know
what a good programmer is. That is a big
problem in a lot of companies. The
management has no way to know if a programmer is good or not. What happens is that an unskilled person
throws a piece of crap together that might work initially but after that person
leaves and another has to maintain the application, it is revealed that it
makes no sense how the ball of mess keeps working. At that point, management couldn’t bribe a good developer to maintain it –
and management has to pay for the software to be rewritten. What a
waste. This happens all too often,
though. How many times have we been on a
rewrite project?
If you have a rewrite
a piece of software from scratch, take the time to learn from past mistakes.
If you’ve ever been on a rewrite project, you’ve heard
project managers belabor the faults of the previous edition and claim that
“we’re going to do it right this time”.
What fails to happen is a critical retrospection of the method of
development of the initial package and what possibly caused the software to go
to hell? This step doesn’t happen, and
the rewrite occurs using the same methods.
To then expect a different result would be insanity.
It’s too easy.
It’s too easy for an unskilled person to throw a screen
together and deploy it. It’s too easy
for Joe blow to create a database application that pulls over entire tables to
the client for modifying one record (but it works – initially). It’s too easy for a newbie to get excited
about a new technology and completely screw up an application with web service calls to itself and overdo sending XML to Sql
Server 2000. It’s too easy for a database
guy to throw tons of business logic in stored procedures, call them from ASP
and call it an application (until a skilled programmer looks at it later and
has a heart attack).
If software is to be
easy, then it must also be disposable.
That’s right. If an
unskilled person throws together a piece of crap that is then used for a while
by a business, the management must know that if they need to change it and the
original author isn’t there, he’ll have to pay for a total rewrite. Like in the car insurance business, if the
cost to repair a vehicle is greater than the cost to replace, then the car is
totaled.
Apprentice,
Journeyman, Master
These are time-tested levels of skill in many
disciplines. An apprentice is not
trusted to work alone because the apprentice is unskilled. The apprentice could have the best of
intentions and know exactly what the customer wants, but he’ll make a mess of
things if left to work alone. Not all
apprentices make it to Journeyman (where they are trusted to produce good
work), and not all Journeymen make it to Master (where they are given the
responsibility of ensuring quality and training the others).
I’m not the only one
who thinks software is too easy:
- John
Reynolds - Joel
Spolsky - Michael
Feathers - David Gay (1/3 down the page)
- And a great threaded
discussion here
Code analysis from an XP project – level 300
I’ve posted on a retrospective of my team’s current release, and I’ve run a few code analysis numbers to get a baseline trend.
I normally don’t do code analysis since working code is our real goal, but here it is:
I analyzed our latest component, which is a part of a larger software product. This component delivers tremendous business value, and it was developed from scratch on my team using XP methods. Here’s the stats:
Statements: 6600
Productions statements: 2500. The rest is test code.
Number of classes 141 – 71 production classes. The rest are test classes.
7.5 methods per class.
About 5 lines of code per method on average.
Maximum cyclomatic complexity of any method: 6. Average is 1.5
We have a few methods that are close to 20 lines of code, but the number of those can be counted on 1 hand.
This release has seen very few bugs
I don’t see any value in using code metrics as a direct measurement of the quality of the code. It may be a trend, but there is no causality between the two. It is, however, interesting to look at the trends from time to time.
- We ended up with 2 times as much test code as production code.
- Our classes ended up very small. Our methods even smaller.
- Our method cyclomatic complexity averaged between 1 and 2.
- We ended up with about 5 actual bugs in the release. This might seem unreal, but I credit all the automated test coverage for this result.
I know some of you will cringe at the thought of writing two times as much test code as production code, but given the results we have achieved, I consider it worth it.
Release retrospective on extreme programming practices – level 300
Tomorrow, my team will make a software release that’s been in the works
since the end of January (2.5 months of work). We’ve been using exteme programming
practices, and I’ll lay out the good, the bad, and the ugly as I look
back on the release. First, here is a rundown of some things we
have been doing:
- Source control with Subversion.
Every developer also receives email notifications of every commit to
source control. Each developer is responsible for reviewing
commits. - Continuous integration with CruiseControl.Net. Every time code is committed to source control, CC.Net runs our automated build that is based on NAnt.
Each developer receives a success or failure notification in his
CCTray. Broken builds are not acceptable and must be fixed
immediately. Code cannot be checked into source control while the
build is broken. Our build runs unit tests, integration tests and
acceptance tests as well as creating a deployment package that’s used
for the install. - Automation. We automate everything. For example, we
have 8 development databases, and when the database scripts are updated
in source control, all 8 databases are dropped and recreated from the
changed scripts. Builds of components are propogated to
components that depend on them. Installation of the latest build
on the dev and test environment is completely automated allowing our
tester to pull the latest build with the click of a button. - Test-driven development (TDD)
– we drive the design of our classes through unit tests. The
byproducts are a loosely-coupled design and a large battery of unit
tests that are run continuously with every build. - Pair programming.
All production code is written by a pair of developers. We have
single tasks too (like tweaks to the CC.Net build, etc), but all new
code and code changes are written by two developers. We use VNC
for pairing because it allows us more comfort by having our own mouse,
keyboard and display even though we are working on a single workstation. - Collective code ownership.
We don’t have a concept of a single person’s code. If we need to
change some code, we change it. No need to consult someone for
“permission”. - The simplest design that will work.
Not the simplest design that will compile. “Work” is defined by
the customer. We will defer longer-range designs because in
practice we know that code written for 6 months down the road will
likely be wasted work because in 6 months, the customer will decide to
go in a different direction. If not, it’s just as easy to add the
same code 6 months from now. - Constant improvement. We are constantly improving the
system and the way in which it’s tested. We have an idea wall
(whiteboard) with a list of items we try to squeeze in that will allow
us to go faster. - Iterations.
We use 2 week iterations. We define and execute work two weeks at
a time. The customer is allowed to reprioritize every two
weeks. We know change is a part of software engineering, so we
don’t try to fight it.
So now, I’ll run down the good, the bad, and the ugly:
The good:
- We finished a week early. Our automated testing and
installs really paid off by allowing us to move very quickly.
From the time we changed some code, a new build could be on the test
environment within 30 minutes (after being confirmed by all automated
tests passing). - Very, very few bugs. Because of the heavy emphasis on
automated testing, bugs are squeezed out of existence. The only
place where bugs creeped in were places where we missed an opportunity
for an integration test or a small feature overlooked. The code
with tests had absolutely no bugs. - Easy deployment. Because of our emphasis on automation, our
system installs on development and test servers, and it’s super easy to
use the same process for deployment to our hosted servers. - No deathmarch at the end of the release cycle. We used a
sustainable pace throughout the release, and we avoided a last-minute
firedrill to make a date.
The bad:
- It was hard. To maintain the sustainable pace, we worked
hard all day every day. It was mentally tiring because of the
discipline required to work at this level of efficiency. - It makes it hard for the wife to be able to call at any
time. Because we all work in the same war room and are pairing on
all production code, when my wife calls, it’s never a good time to
talk, and I have to help her understand our way of working and that I
can’t have a phone conversation at any time. Family emergencies
obviously take precendence, but my wife likes to call and chat, and
that doesn’t work with extreme programming because we are engaged all
the time. - It takes a certain type of programmer to work in this disciplined
fashion, and not all programmers want to work this way. It is
taxing for us because sometimes we don’t want to pair program, but we
know we must.
The ugly:
- The code we inherited. We are improving a system with
inadequate test coverage. The test that are there are integration
tests and when they fail, it’s a mystery to discover what part of the
code is actually the problem. We made vast improvements on the
testability of the system, and we’ve employed acceptance tests with the
FitNesse wiki to allow for automated full system tests. - Technical debt and politics. We constantly made
improvements to the code and the way we worked. Sometimes this
wasn’t obvious to management. We had to make some changes to
facilitate some needed behavior, but management wanted the behavior
without paying for the prerequisite work. In this case, it was
our responsibility to push back and not allow technical debt to
accumulate just for political reasons. We try as much as possible
to keep technical decisions out of political ones. It’s our job
to take customer requirements and translate them into working
software. The manner in which we do that is our decision (the
decision of the technical players on the software team).
Overall, the employment of these extreme programming practices has been
wildly successful. We’ve improved the system in a very short
period of time while adding functionality, and we’ve paid down
technical debt we inherited without incurring more. We are
encouraged by these results, and we will continue these practices that
have been working for us. We’ll look for even more ways to go
faster by improving efficiency. We’ll increase our level of test
automation to free up our tester for more in-depth exploratory
testing. Overall, management is pleased because we’ve produced
the business value that matters. That’s all that matters:
delivering software that adds business value.
TDD makes refactoring easy – level 300
Here is the main reasoning for this: TDD states that a unit test
is written before a unit of code. Each unit of code will have a
unit test. When a unit of code needs to be refactored (changed
in structure without affecting behavior), the unit test will preserve
the behavior of the code. The unit can be refactored quickly, and
the unit tests will assert that the code still behaves as originally
intended.
Consider refactoring without unit tests or with only integration
tests. First, without tests. You refactor a piece of code
that doesn’t have a test. Now to ensure that you didn’t break
anything, you have to run your application with some scenarios that are
sure to exercise the changed code. This is more time-consuming
that merely running the unit tests for the unit in question.
Suppose you have _some_ tests for the unit, but they aren’t unit
tests. The tests involve several other pieces of code as
well. If all is well, the tests will still pass, but it they
fail, you would have to debug into the test to find out what’s really
going on since the test has a larger scope. A unit test’s scope
is only that of the code unit, and if the test fails, it pinpoints the
area of failure.
TDD reveals poor class design – level 300
If a class is well-designed, it will be easy to unit test. Good
design speeds up unit testing, and TDD reveals bad design. In
this way, TDD speeds up unit testing.
For a very simple example, consider a controller class that takes a
domain object and saves it to the database using a data access
class. When test-driving this code, it will be impossible to
create a unit test for this if it is poorly designed. Consider
this poor design: the SaveXX() method instantiates the data
access class and passes the domain object to it for saving. With
TDD it would be impossible to write a unit test for that because you
can’t fake the data access class, and you must test the controller
class apart from the dependency on the data access layer.
Because of the major roadblock in designing a test for code designed
poorly, it forces the developer to come up with a better design.
This better design will be a cinch to unit test because it will be
loosely coupled. Loose coupling is essential for unit testing.
To write a test for this controller class, I’ll have to define an
interface for this portion of the data access layer. I’ll inject
this interface into the constructor of the controller class and call it
in my SaveXX() method. With this design pending, I can fake the
interface in my test and assert that it was called with the correct
object. This will be my test. Next, I’ll write the code
that makes the test pass.
Poorly designed code is hard to test, and thinking about testability
will force a more thoughtful design. I would even go as far to
say that at the class level, a testable class is a well-designed class.
TDD speeds up unit testing – level 300
Before reading this post, I recommend reading a previous post of mine along with the comments and trackbacks to related posts.
First, you may wonder why I’m contending that TDD speeds up unit
testing if TDD itself stresses unit testing so much. One can unit
test without employing TDD. I can write some code and write a
unit test for it. If I use this method, I’m likely to have a hard
time unit testing the code because it may call out to a database or the
file system or a web service or some other object that is difficult to
create in isolation. If I think about how I’m going to test the
code as I’m writing it, I’ll end up with code that’s easier to test
because I’ll depend on interfaces and employ other techniques to ensure
my code is loosely coupled.
Because my goal was a piece of code for which it was easy to write a
unit test, I’ll end up with code designed for that. At this
point, I’m blurring the lines between just unit testing and TDD.
I didn’t write my unit test first, but I _thought_ about how I would
write my unit test first. I imagined the method of testing before
writing the code. From this point, it’s a snap to then actually
write the test that’s floating around in my head before coding the
production code.
From a pragmatic sense, it probably doesn’t matter which code is
actually typed first if you’ve already decided on the unit test in your
head. You’ve written the unit test in your head before the
production code. The typing is then just semantics.
If you aren’t thinking about unit testing at all (if you don’t
currently do unit testing), then this whole topic is worthless for you,
but if you are attempting to write unit tests after you design your
code, I’m sure you can relate that it’s often difficult because you
have to pull in other classes just to test a single class. If
this happens, then it’s not a unit test at all. It’s an
integration test because it tests multiple things. The difference
in unit testing and integration testing is a topic for another post,
but the difference is very important, and if the difference is not
understood, then arguments for/against TDD don’t make much sense at all.
The above is how TDD speeds up writing unit tests (if you have already been doing unit testing).
If the goal is code with unit tests, then the code will be designed for
easy unit testing. To design for easy unit testing, a developer
has to think about the test first. This leads to mental TDD, and
after the code is designed, the unit test can be written quickly.
The next step would naturally be to just go ahead and type the test
first instead of keeping it in your brain. If at this point you
prefer to continue to type the test after the production code, go
ahead. You will have benefited some by focusing more on how to
test the code.
Sam Gentile packed the house at ADNUG and preached the good news of Extreme Programming – level 200
Sam has already posted about it here, but I’ll continue.
Yesterday’s Austin .Net User Group meeting was a big success. We had 80 people, our largest crowd in a long time. Miguel Castro was pretty close with 75, but for the past several months we’ve had over 5 first-time guests, so people are spreading the word about the group. I try to keep the group focused on software concepts that are core to building sustainable software that can add value to the business now and for years to come. If you believe in “voting with your feet”, then these topics are getting a lot of votes.
Sam answered a question about what skills were needed in a developer, and he made a great point: A good software developer is one who can create software that’s easy to change. Not someone who’s an expert in SQL server, Indigo, ASP.NET, etc because those technologies will be the next legacy systems. A good developer hops from technology to technology but always keeps the software easy to change – after all, that’s the only constant.
We normally go until 7:30pm, but yesterday we went to 8:05 (Sam could have talked until midnight, but I had to cut him off), and 75% of the folks stayed to the very end (60 out of 80). It was obvious that everyone was very interested in what was being said.
We went to Rudy’s BBQ afterward and continued the discourse. 12 folks stayed until 10:30pm talking about TDD, iterations, FIT, estimating, how to handle a developer that’s not pulling his weight, etc. Great talk!
Sam was amazed that it was 80 degrees in the middle of the day and 70 degrees late at night. He’s used to it being bitterly cold. For us, that’s just what it’s like in Texas – of course we deal with 100+ in the summer also, and it only snows once a decade.
How to design a single method – level 200
What? I know how to design a single method! What can
Jeffrey Palerm possibly have to tell me about how to design a single
method?
Take a look at your product’s code base. Are any of the following true?:
* You have a method that you can’t view without scrolling.
* You have a method that takes 10 arguments.
* You have a method that not only returns a result but causes
disastrous things to happen if called more than once.
* You have a method that throws exceptions if you don’t set certain properties first.
* You have a method that has 5 nested foreach/if/try-catch code blocks.
* You have a method whose name doesn’t give you a clue as to what it does.
* You have a method that will not throw an exception because it swallows the exception without handling it.
I
could go on and on (ok, actually I’m out of ideas right now), but
chances are that you can find a method that meets one of the criteria
above. Find one of those methods, and fix it. Here are
general rules I try to live by when designing a single method:
* The name of the method should describe what the method actually does.
* A method should do one and only one thing.
* A method should either DO something or RETURN something. If it
returns something, you should be able to call the method over and over
with no negative consequences.
* The method
should be viewable without scrolling. Fewer than 10 lines of code
is desirable (my pain point).
* The method should
take few arguments. If the method needs more, refactor to require
and object that contains all the information required (parameter
object).
* The method depends only on method parameters or members passed into the class constructor.
* A method should only have one indented code block – don’t nest code blocks. Extract method first.
* Don’t try/catch in a method unless you can handle the exception or
add value. Let the exception bubble up to a point where you can
actually do something about the exception.
If you have any good points for method development, please feel free to leave a comment.
Sam Gentile on Extreme Programming at the Austin .Net User Group – level 300
Yesterday (Sunday), Sam Gentile came into town on his INETA speaker trip to the Austin .Net User Group. Since we had some extra time with Sam before the user group meeting, I organized a bbq at my house in Leander, TX.
We grilled some inch-thick ribeye steaks on the back patio and had a
good time talking about software for hours. We talked about the
merits of FIT, automation, and other things in play in the extreme
programming world. Some people you might know in attendance (and
if I didn’t list your name, get a blog) were: Sam Gentile, Steve Donie, Blake Caraway, Jeremy Miller, and Scott Bellware.
He said something that stands out in my mind, and I may change the way
I view and talk about the way I work on a day-to-day basis. I’ve
been using the term “Agile” to describe the group of practices that I
and my team uses. He has convinced me that the term “Extreme
Programming” is a more accurate term. It really is extreme.
There is a solid goal in sight, and we have to be very disciplined
about how we do things in order to keep our software
maintainable. One way to think about it is: How true is the
statment “grass is green”. Is that statement somewhat true, or is it
very true? Truth is extreme. It is not relative or
subjective. Truth is truth, and there is no room for negotiation
before it ceases to be true. With extreme programming, if you negotiation, it’s no longer extreme: it becomes moderate. Moderate programming.
How good can it get? It needs to be extremely good, not moderately good.
Sam is speaking to the user group as I write this post, and he
explains the principles of extreme programming extremely well.
Great job, Sam!