Jeffrey Palermo 4:29 pm on March 1, 2007

Just passed 1400 automated test cases – level 200

I just HAD to write this up. My company’s software has just crossed the 1400 automated test cases line. I still remember when the software was new and when I was coaching my team on why we should write unit tests. Now, the number of tests just climbs and climbs.

We have 1417 (as of right now) NUnit unit tests, NUnit integration tests and FIT integration tests. We test the following types of things:

That each public method of each class does the right thing.
That each database table is the correct structure and has the right permissions.
That each NHibernate mapping is configured correctly.
That each logical service does the right thing.
That the system as a whole operates when integrated.

Matt 5:28 pm on March 1, 2007

I’m incredibly jealous.
Derik Whittaker 5:50 pm on March 1, 2007

Thats cool…

I am pretty happy that we have about 550 tests that run and pass on our application.
Ross Neilson 5:54 pm on March 1, 2007

Is there any chance you might blog some examples of the second and third types of test? I have performed some integration testing with NHibernate before but never found a really good way of doing it.

Like Matt, I am jealous too. Where I work we have around 200 unit tests, all written by myself. Hopefully my colleagues will catch on someday.
rlewallen 6:10 pm on March 1, 2007

Congrats! Its a great feeling, isn’t it!
jdn 7:05 pm on March 1, 2007

How long does it take for the 1400 tests to run in a single pass?
jpalermo 8:25 pm on March 1, 2007

jdn,
It takes about 1.5 minutes.
jdn 9:11 pm on March 1, 2007

Sweet.
Jason Meridth 3:59 am on March 2, 2007

We’ve just surpassed 5000 unit tests. It takes about 5-6 minutes. We are currently in the process of possibly migrating/archiving/setting a daily time a majority of the current unit tests to speed up the check-in process. We currently require all developers to run the entire sweet before a check-in. It’s pretty amazing. This is from a 3 year project.
jpalermo 12:48 pm on March 2, 2007

Raymond,
It’s a secure feeling, but we have much more testing we need to do.

Ross,
That’s a good idea, but it’s pretty involved. I’ll give you the short of it: We have NUnit data access tests that do our CRUD operations with our domain objects through our NHibernate mappings to our database (which is created at the beginning of the build). In short this verifies the mappings are correct, the columns are correct, and that we have permissions to use the tables.

Remember that with database testing, the database has to be in a known state: so we delete all records from all tables at the beginning of each test and build up the data. We use some helper methods to trash the data. We never run these against a shared database, only against the local developer database and one on the build server.
Vikas Kerni 5:42 pm on March 2, 2007

Wow.

How are they organized ? Same project or different projects?
jpalermo 11:51 pm on March 4, 2007

We’ve found that organizing them according to fast/slow has worked well. That has led us to a .unittests and .integrationtests project for each production project. The unit tests projects usually reference Rhino Mocks, etc, and the integration tests projects reference structuremap since we are using that as our DI container.

Comments are closed.

Programming with Palermo

Jeffrey Palermo, Microsoft MVP, Author, Speaker, Clear Measure Chief Architect, Azure DevOps Expert

Just passed 1400 automated test cases – level 200

Related