How to integrate FitNesse acceptance tests into your CC.Net build – level 300

I’m doing a lot with FitNesse lately, and it’s going quite well.  Unit testing ensures that each component is doing what it is supposed to, and integration testing ensures that the components work well together.  Acceptance tests are a bit different.  They actually test that the software does what the customers think it’s doing.  They test that the developer understood the business need.  They give the customer (or product manager) assurance that the system actually works.

With FitNesse, our product manager (and crew) can  use a simple wiki to exercise the system.  The developers create test Fixtures that speak in the domain language but actually exercise the system under test.  A user may type:

!|fit.ActionFixture|
|start|Emailer|
|press|SendDefaultEmail|
|check|NumEmailsSent|

This simple table is enough to actually exercise the system under test and get it to do something.  If all is well, the Emailer will send a default email.  If this test fails, it saves the company from a black eye when a user encounters the problem.

It was a little tricky to get the FitNesse test integrated with our CC.Net build.  We wanted the acceptance test to be a current status report of where we are.  When all the acceptance tests are passing, we are done.  Consequently, we didn’t want failing acceptance tests to break the build.  Only NUnit tests break the build because they should _always_ be passing.  No problem.  We create a new target with some exec tasks:

<exec program=”<path to TestRunner.exe>” commandline=”-results <some directory>FitNesse-Results.html <FitNesse server> <port> <TestSuite>” failonerror=”false”/>

This will actually call the FitNesse server (with the wiki on it) and execute all the acceptance tests.  This will produce a file in the raw FitNesse format.  We’d like it in xml so we can incorporate it in the CC.Net build report.  Xml transforming isn’t a part of the .Net port, but it is in the Java version, so we’ll just use it.

<exec program=”java.exe” commandline=”-cp binfitfitnesse.jar fitnesse.runner.FormattingOption <some directory>FitNesse-Results.html xml <some directory>Fit-Results.xml <FitNesse server> <port> <TestSuite>” failonerror=”false”/>

This hooks into the transform logic on the FitNesse server to change our raw output file to a nice Xml format.  Now we have something that CruiseControl.Net can use for a build report and email.  We will have to make an Xsl, though.  Here’s a simple Xsl that will pull out the FitNesse summary information and through it out to your CC.Net build summary:


<?xml version=”1.0″?>

<xsl:stylesheet


xmlns:xsl=”http://www.w3.org/1999/XSL/Transform” version=”1.0″>


<xsl:output method=”html”/>


<xsl:variable name=”fit.result.list” select=”//testResults/result”/>


<xsl:variable name=”fit.wrongpagecount” select=”countU$fit.result.list/counts/wrong[text() > 0])” />


<xsl:variable name=”fit.ignorespagecount” select=”countU$fit.result.list/counts/ignores[text() > 0])” />


<xsl:variable name=”fit.exceptionspagecount” select=”countU$fit.result.list/counts/exceptions[text() > 0])” />


<xsl:variable name=”fit.correctpagecount” select=”countU$fit.result.list/counts)” />


<xsl:variable name=”fit.correctcount” select=”//testResults/finalCounts/right”/>


<xsl:variable name=”fit.failures” select=”//testResults/finalCounts/wrong”/>


<xsl:variable name=”fit.notrun” select=”//testResults/finalCounts/ignores”/>


<xsl:variable name=”fit.exceptions” select=”//testResults/finalCounts/exceptions”/>


<xsl:variable name=”fit.case.list” select=”$fit.result.list//test-case”/>


<xsl:variable name=”fit.suite.list” select=”$fit.result.list//test-suite”/>


<xsl:variable name=”fit.failure.list” select=”$fit.case.list//failure”/>


<xsl:variable name=”fit.notrun.list” select=”$fit.case.list//reason”/>


<xsl:variable name=”colorClass”>


<xsl:choose>


<xsl:when test=”$fit.exceptionspagecount > 0″>fiterror</xsl:when>


<xsl:when test=”$fit.ignorespagecount > 0″>fitignore</xsl:when>


<xsl:when test=”$fit.wrongpagecount > 0″ >fitfail</xsl:when>


<xsl:otherwise>fitpass</xsl:otherwise>


</xsl:choose>


</xsl:variable>


<xsl:variable name=”fit.tests.present” select=”countU//testResults/result) > 0 or count(/cruisecontrol/build/buildresults//testsuite) > 0″ />


<xsl:template match=”/”>


<xsl:choose>


<xsl:when test=”$fit.tests.present”>


<style>


*.fitpass{

background-color: #AAFFAA;

}

*.fitfail{

background-color: #FFAAAA;

}

*.fiterror

{

background-color: #FFFFAA;

}

*.fitignore

{

background-color: #CCCCCC;

}

*.fitheader{

border: solid 1px black;

margin: 1px;

padding: 2px;

}

*.line{

margin: 5px;

}

</style>


<div>


<div class=”{$colorClass} fitheader”>


<strong>FitNesse Summary — Test Pages:</strong> <xsl:value-of select=”$fit.correctpagecount”/> right, <xsl:value-of select=”$fit.wrongpagecount”/> wrong,

<xsl:value-of select=”$fit.ignorespagecount”/> ignored, <xsl:value-of select=”$fit.exceptionspagecount”/> exceptions

<strong>Assertions:</strong> <xsl:value-of select=”$fit.correctcount”/> right, <xsl:value-of select=”$fit.failures”/> wrong,

<xsl:value-of select=”$fit.notrun”/> ignored, <xsl:value-of select=”$fit.exceptions”/> exceptions

</div>


<xsl:for-each select=”$fit.result.list”>


<xsl:variable name=”colorClass”>


<xsl:choose>


<xsl:when test=”counts/exceptions > 0″>fiterror</xsl:when>


<xsl:when test=”counts/ignores > 0″>fitignore</xsl:when>


<xsl:when test=”counts/wrong > 0″ >fitfail</xsl:when>


<xsl:otherwise>fitpass</xsl:otherwise>


</xsl:choose>


</xsl:variable>


<div class=”line”>


<span class=”{$colorClass}” style=”padding:2px;” ><xsl:value-of select=”counts/right”/> right, <xsl:value-of select=”counts/wrong”/> wrong,

<xsl:value-of select=”counts/ignores”/> ignored, <xsl:value-of select=”counts/exceptions”/> exceptions </span>


<span style=”padding:2px;”><xsl:value-of select=”relativePageName”/></span>


</div>


</xsl:for-each>


</div>


</xsl:when>


</xsl:choose>

</xsl:template>

</xsl:stylesheet>

 

Hook this Xsl into your CC.Net configuration, and that’s as hard as it is!  FitNesse results “magically” show up in the build summary.  It’s really a big win for visibility into the state of the software.  The build summary is now the actual status report of the software, and it’s not subjective.  What does 83% actually mean?  With a list of acceptance tests to fulfill, we know exactly when we are done.

 

Response to EZWeb feedback – level 100

Stefano, in this post, gives some feedback about EZWeb 1.2 release 2.  He has set up EZWeb on his server here.  Here is Stefano’s comments (in italics) and my response.


I’ve setup the site on this address on my server and I have to say that the project has totally reached its 2 main purposes:



  • Easy to install: you’ve only to copy on a virtual directory of your IIS server the content of the package and set a special permission to a directory and all is done.
  • Easy to edit contents: the site uses the FreeTextBox control to edit pages. You can easy create how many pages you want and edit content is so easy as writing a document with MS Word (so it’s a task that can be improved also for a person that don’t know nothing about HTML).

However, this post is expecially to send feedback for future improvements, and I think that some more work must be done on these areas:



  • Site Templates: the templates that are present now on the package are really too much “poor”. Why not working on a graphical template, with images and CSS? The site’s appearance is not so good for eyes I think…

I definitely agree with this one.  I’ve concentrated my efforts on the internal framework, so the templates are very basic.  Templates are just a simple UserControl that is almost exactly like v2.0 master pages.  It is very easy to modify the markup of any of the templates to alter the look of the site.  I’ve included the full source for all templates and plugins – which are both just user controls.  I do plan on adding more attractive templates to choose out of the box, and if anyone wants to contribute a template, I’ll gladly include it.  With the advent of “Master Pages”, we’ll probably see online libraries of them.  It will be trivial to use any template or master page with EZWeb.



  • User Administration: adding a user to the site is really easy, but why not adding a more complete user interface, with the possibility to add user’s data for example?

I have big plans for my 2.0 version to coorespond with .Net 2.0, and one of the things is using the built-in User Management provider, so I didn’t want to invest in a custom user management piece at this point.  Even so, user attributes would only make sense in a situation where you extend EZWeb with a control that makes use of these attributes, like a control that checks roles.  For this, you can create your own User class at this point and change the web.config to use your class.  Without custom development, user attributes don’t make sense at this point, but I will comply with the Membership provider for my version 2.


Now two personal requests to Jeffrey:



  • How can I change user’s permission? If I want to create a user that has permissions to add pages just like an administrato, how can I do? I’ve not understood this…

To change user permissions, click on “Manage Page” when logged in as “admin”.  At the bottom of the screen, you see “Editors”.  All the users should appear in the drop-down.  Click Add, and then select what permissions to delegate for the current page.  For most cases, allowing all permissions is best, but you can give EditContent permission only if you really want.  Click “update” in that row, and then click “Save”.  Then, when logged in as that user, you can exercise that permission.  Permissions work down the web tree just like NTFS permission in the filesystem.  Permissions at the site root are inherited by all pages in the site. 



  • Is it possible to add a task list or a calendar feature to the site? I think that something like an “Outlook” calendar could be useful on a site like this.

It is possible if you have a calendar ASP.NET web control.  Any web control can be invoked dynamically by any page.  Use the “Control Loader” plugin.  It can load a user control or custom control.  For this version, the Content Editor is the only plugin with a GUI.  EZWeb is expandable by adding web controls that do the functionality desired.  Code to all plugins is included in the download.  If anyone has a control that would be useful for EZWeb, I’ll be happy to include it.


Also, at some point, I think an EZWebContrib open source project might be useful for Templates, Plugins, and Themes.

EZWeb 1.2 release 2 available for download – level 000

I’ve made a second release of version 1.2 on GotDotNet.  The reason for this is a bug in the configuration of FreeTextBox as well as simplification for hosting deployments.  GoDaddy has custom permission support for subfolders, but they don’t allow custom permissions inside the aspnet_client directory, so I’ve modified the configuration so that writeable files are in a new directory, EZWebFiles, and this plays better with GoDaddy.  I can only assume that some others might not like to grant change access to aspnet_client. 


The binaries haven’t changed, just the configuration has.


You can download it here.  I’m almost done with the EZWeb sandbox site, so in a bit, you all will be able to go to the sandbox and play.  I’ll be using this site as well to compile some FAQs and the beginnings of documentation.  After all, EZWeb allows the editing of content very simply!


If you’ve downloaded EZWeb and are using it for a community or personal site, put the word out.  Let everyone know how easy EZWeb is for getting a website up and running.

RealVNC works great for pair programming – level 200

RealVNC has been around for a long time, and it is a great tool for remote control of a computer.  I remember using it on servers back in the WinNT days.  It runs as a windows service, so using the small client, you can connect from anywhere over port 5900 (the default).


For pair programming, it’s essential that both developers are comfortable in front of their own keyboard, mouse, and monitor.  You CAN use a single computer, but the developer who is reviewing has a bad angle on the screen and just generally isn’t comfortable.  Another method is to attach two monitors, two keyboards and two mice to the same machine.  That would work (I’ve never done it), but after using RealVNC for two full days, I’m sold.  It doesn’t require a special computer configuration, and it’s super easy to get set up and working.  Your partner can know your VNC password, or (as I prefer), the partner turns on the listening client, and then I run the “Add a client” command, put in his computer name, and bingo!, we’re connected.  We’re running on my box, but do an F8 and Full Screen, and you can hardly tell the difference. 


I just finished 3 straight days of pairing, and we used VNC on the last 2 days.  It works really well.


VNC is also great for conference calls.  I was on a short conference call yesterday, and the folks on the other end needed to show us something.  I turned on my VNC listening client, they added me, and we had video!  Great tool!


If you aren’t already using it, download it now.  VNC goes beyond remote control well into collaboration.

EZWeb 1.2 downloads going strong – feedback is appreciated – level 000

Previously, I posted about the exciting release of EZWeb 1.2.  This is my personal project that I’ve been working on for 2 years now.  Many, many of you have downloaded it.  I’d like to get some feedback.  How many actually got it running (which should be super easy), and what did you think?


Please provide feedback either through a comment on this post, or through the contact form on my blog.


I do have plans to release a .Net 2.0 version (in fact I’ve already started).  If you have a super suggestion, please give it to me, and it may make the 2.0 release.  I will continue to improve usability as well as the programmer extension API.  I’ll have to get some programmer documentation together for the 2.0 release.