My current favorite private build script

I do pull in the Exec function from psake just because it was coded very well.  This build script is just powershell and is geared for .Net Core

. .\BuildFunctions.ps1

 

$projectName = “OnionDevOpsArchitecture”

$base_dir = resolve-path
.\

$source_dir =
$base_dir
\src”

$unitTestProjectPath =
$source_dir
\UnitTests”

$integrationTestProjectPath =
$source_dir
\IntegrationTests”

$uiProjectPath =
$source_dir
\UI”

$databaseProjectPath =
$source_dir
\Database”

$projectConfig = $env:BuildConfiguration

$version = $env:Version

$verbosity = “q”

 

$build_dir =
$base_dir
\build”

$test_dir =
$build_dir
\test”

 

$aliaSql =
$source_dir
\Database\scripts\AliaSql.exe”

$databaseAction = $env:DatabaseAction

if
([string]::IsNullOrEmpty($databaseAction))
{ $databaseAction = “Rebuild”}

$databaseName = $env:DatabaseName

if
([string]::IsNullOrEmpty($databaseName))
{ $databaseName =
$projectName}

$databaseServer = $env:DatabaseServer

if
([string]::IsNullOrEmpty($databaseServer))
{ $databaseServer =
“localhost\SQL2017”}

$databaseScripts =
$source_dir
\Database\scripts”

   

if
([string]::IsNullOrEmpty($version))
{ $version = “9.9.9”}

if
([string]::IsNullOrEmpty($projectConfig))
{$projectConfig = “Release”}

 

Function Init {

    rd
$build_dir -recurse -force  -ErrorAction
Ignore

       md $build_dir
>
$null

 

       exec {

              &
dotnet clean
$source_dir\$projectName.sln
-nologo -v
$verbosity

              }

       exec {

              &
dotnet restore
$source_dir\$projectName.sln
-nologo –interactive -v
$verbosity
 

              }

   

 

    #Write-Host
$projectConfig

    #Write-Host
$version

}

 

 

Function Compile{

       exec {

              &
dotnet build
$source_dir\$projectName.sln
-nologo –no-restore -v
$verbosity -maxcpucount –configuration
$projectConfig –no-incremental
/p:Version=$version /p:Authors=”Clear
Measure”
/p:Product=”Onion
DevOps Architecture”

       }

}

 

Function UnitTests{

       Push-Location -Path
$unitTestProjectPath

 

       try {

              exec {

                     &
dotnet test -nologo -v
$verbosity –logger:trx
–results-directory $test_dir –no-build
–no-restore –configuration
$projectConfig

              }

       }

       finally {

              Pop-Location

       }

}

 

Function IntegrationTest{

       Push-Location -Path
$integrationTestProjectPath

 

       try {

              exec {

                     &
dotnet test -nologo -v
$verbosity –logger:trx
–results-directory $test_dir –no-build
–no-restore –configuration
$projectConfig

              }

       }

       finally {

              Pop-Location

       }

}

 

Function MigrateDatabaseLocal {

       exec{

              &
$aliaSql $databaseAction $databaseServer
$databaseName
$databaseScripts

       }

}

 

Function MigrateDatabaseRemote{

       $appConfig =
$integrationTestProjectPath
\app.config”

    $injectedConnectionString =
“Server=tcp:$databaseServer,1433;Initial
Catalog=
$databaseName;Persist Security
Info=False;User
ID=
$env:DatabaseUser;Password=$env:DatabasePassword
;MultipleActiveResultSets=False;Encrypt=True;TrustServerCertificate=False;Connection
Timeout=30;”

   

       write-host “Using connection string:
$injectedConnectionString

    if (
Test-Path $appConfig )
{

        poke-xml
$appConfig “//add[@key=’ConnectionString’]/@value”
$injectedConnectionString

    }

 

       exec {

              &
$aliaSql $databaseAction $databaseServer
$databaseName $databaseScripts
$env:DatabaseUser
$env:DatabasePassword

       }

}

 

Function Pack{

       Write-Output “Packaging
nuget packages”

       exec{

              &
.\tools\octopack\Octo.exe pack –id
$projectName.UI” –version
$version –basePath $uiProjectPath
–outFolder
$build_dir

       }

       exec{

              &
.\tools\octopack\Octo.exe pack –id
$projectName.Database”
–version $version –basePath
$databaseProjectPath –outFolder
$build_dir

       }

       exec{

              &
.\tools\octopack\Octo.exe pack –id
$projectName.IntegrationTests”
–version $version –basePath
$integrationTestProjectPath –outFolder
$build_dir

       }

}

 

Function PrivateBuild{

       Init

       Compile

       UnitTests

       MigrateDatabaseLocal

       IntegrationTest

}

 

Function CIBuild{

       Init

       MigrateDatabaseRemote

       Compile

       UnitTests

       IntegrationTest

       Pack

}

 

 

Palermo Pamphlet launch – episode 001


My goal is to teach, inform, and have a little fun. But I want this to provide value for programmers shipping custom software using Microsoft tools. Here is the first episode of the Palermo Pamphlet, which I hope will be a valuable resource to you.

Here are the show notes

Performance tuning an Azure DevOps build configuration

We’ve all seen the culprits that constantly add time to builds.  One might observe that your NPM install or Nuget restore can take several minutes.  I remember back to the times of CC.Net in 2005 when a small application build could happen in 45 seconds, including unit tests.  And 10 minutes as a “thou shall not go over this” threshold.  So we cannot allow NPM or any other step to take minutes.  We have to ferret that out.
The answer is the same as code performance profiling.  Find out where every build is spending the same time doing work that adds no value or doesn’t vary often. Then we cache the result.  For so many builds, these are the culprits that take time but typically aren’t the changes that are being tested from build to build:
  1. Obtaining a build server (when choosing hosted build agents)
  2. Cloning the source
  3. Package restores
  4. Copying/archiving build artifacts
Here are my common solutions for reducing these common culprits (I’d be interested to know how others have eliminated these time sucks)
  1. Use our own Azure VMs as the build agents (running multiple agents on a single VM) – always available at a moments notice
  2. Let Azure Pipelines be a little less aggressive with cleaning source and instead have the build script delete the build directories at the beginning – removes need for a full clone and can just be a pull (works most of the time and requires probably a monthly purge for a clean clone, but saves SOOO much time)
  3. a) retain cloned working tree so that the previous package restore is used for subsequent builds or b) check in packages so that package restores are not necessary for every build
  4. Once builds are working and reliable, only archive the build artifacts that are directly used by the release pipeline (typically the nuget packages that house the application components)

Why I started the Azure DevOps Podcast

I wanted to share a little bit of my reasoning for starting the Azure DevOps Podcast.  The above video is about 4 minutes long.  Please take a look.  Feel free to play at double speed.  The gist of it is I like seeing developers having fun.  Busted releases are not fun.  All-weekend deployments are not fun.  New things breaking when you make a code change are not fun.  And software development can be SO MUCH FUN!  I’ve spent 21 years so far as a professional programmer – not counting the toy apps and websites outside that time. I still love it, and I would like to share that joy with anyone I can.

Here are some links I mention in the video:

 

Applying 4+1 Architecture Blueprints to Continuous Delivery

imageYou may just be learning about the iterative, emergent architecture method known as “4+1”.  You can read the original paper here.  Written by Philippe Kruchten in 1995, this 15-page paper lays out the views that are needed in order to communicate the architectural elements of a software system. Coursera has a good summary video in their course catalog here.

Architects in other professions go through similar thought processes as software architects, so it can be useful to borrow the graphical outputs that these other architects generate as illustrations of the decisions made while finding a solution suitable to the problem.

Continuous Delivery is an umbrella terms used to describe the process for an automated system that takes changes to the source and configuration of a system and flows those changes through a process and ultimately to a running production system while catching quality issues. Jez Humble maintains a very useful site dedicated to continuous delivery here.

I recently presented a sample 4+1 architecture to the Cloud Austin user group. Since I received several requests for the diagram, I’m posting it here. If you have any additional questions, contact me.  I’m always happy to help.  Additionally, I have a high-resolution PDF of this diagram (ARCH D in size, if you would like to print it on a plotter).

DevOps 4 1 Architecture Blueprints 200

How to incrementally adopt VSTS for devops automation

VSTS has fantastic devops-enabling features, but one of the common issues when planning to adopt it is how to plan the large migration of work and project plans from other tools.  The good news is that you don’t have to change anything about where you are tracking your work or projects in order to make use of builds, packaging, or automated deployments.  If you track your work in Jira, your code in GitHub, then you’ll want to configure VSTS like the following:

image

If my VSTS does not look like yours, you’ll want to turn on the new navigation features by selecting your profile picture in the top right corner.

image

With this configuration, you’ll be able to:

  • Configure continuous integration builds
  • Run unit tests & component-level integration tests
  • Package versioned release candidates
  • Architect release candidate packages in the onboard Nuget server
  • Create releases for a particular build
  • Map the progression of pre-production environments as well as production
  • Deploy release candidates
  • Run full-system tests against deployed instances of your builds
  • Trend statistics about your tests
  • Integrate static code analysis into your devops pipeline
  • Configure and run load tests

Many teams have systems to track projects and existing source code repositories in place.  When bringing online devops methods, it may be appropriate to focus the VSTS project on just the capabilities needed at the moment. 

Clear Measure 2016-Q3 update–#1 Austin Fast 50–small segment

a52f228c083a_99E2ClearMeasure200px_2A while back I wrote an update on Clear Measure. As time has passed, a lot has changed, and we’ve achieved a lot as well. You can read my previous update from April, 2016 here.

Austin Fast 50 Award

fast-50-logo-final-color750xx1500-844-0-113Just over a week ago, we were presented with a very big surprise. In the Austin, TX area, we were ranked as the #1 fastest growing small business across the board in any industry. They classify their small business category as one that posts less than $10M in revenue, and we closed out 2015 at $8.7M in revenue. This was the first year we were eligible to submit our numbers as the Austin Business Journal (ABJ) requires a complete 3 years of financials to be completed. From beginning operations on January 7, 2013 through the end of December, 2016, we grew at a erpace that was pretty hard to hold onto at times. I was a bit bummed that our remote employees around North America couldn’t come to the event, but we invited all of our local team to participate. While they give the award to the CEO, this award was for the whole company. 8bae7b28a822_CC89Fast 50 Clear Measure Team_2Everyone has worked extremely hard, and I am very proud of them. It’s a bit of a delayed award since it recognizes 2013-2015, but anyone who was on board during that period definitely remembers how hectic it was taking on customers and new employees at a high rate. We appreciate the ABJ for this award, and we are encouraged by it. And God has certainly blessed the Clear Measure family of employees and clients.

I also want to thank our 2013 clients. They are largely still with us, as we are a good portion or all of their software engineering core competency. Early in our relationship with these clients, they witnesses and tolerated the environment where we made the sale, mobilized the right team to tackle the problem and align with their organization, and then turned to mobilizing a team for the next client. More than once, a client asked for personal attention from me that was hard to give. I appreciate their patience, and I am so grateful for them for sticking with us. Our business has been built on long-term relationships rather than flash-in-the-pan projects, and our clients are some of the best around.

Honing our focus, driving our purpose

Rather than casting a wide net, I believe in going deep. And it seems business books everywhere talk about finding a market niche and “owning” it. We started with it in 2013, and ever since then, we have learned more about it and have found what works well and what doesn’t. We have used that learning to hone our mission statement.

“Clear Measure is a software engineering firm that brings a competitive advantage to growing companies”

We worked hard to make this statement short but specific. Since the beginning, we have been a software engineering firm. We aren’t a staffing firm. We aren’t a traveling consulting firm. And we don’t describe ourselves as a software development shop. We’ve chosen the words carefully.

Software

We are a company that writes code. We build software. We modernize software. We work in the flexible medium of business automation through computing. Digital record-keeping.

Engineering

We believe that the best returns on software investment come from the engineering discipline, not from art, where success is in the eye of the beholder. And as much as the craft movement produces great lessons, the demand of our city and the world has already far outstripped the capacity of the available craftsmen in our field. In order to provide predictable results at the necessary capacity, we take an engineering approach to software in order to drive out variability and increase certainty.

Firm

This may seem irrelevant, but organizations that provide a professional service are called firms. These organizations are more than just a LLC or Corporation. They are trusted partners with long-term client relationships. They provide necessary high-end services for clients – services that require the workers to be educated, well-trained, and experienced.

Competitive Advantage

Our clients are competing. They come from all industries. Rather than attempt to focus on multiple industries, we let our clients do that, and we focus on providing them with an enterprise-grade core competency in custom software. We have a horizontal focus on our custom software core competency. And we provide that to our clients so that they can gain an advantage when competing in their own industries against other players that have exploited computers and software for their advantage. This include digital record-keeping, automated business operations, and making workers highly efficient, sometimes to the tune of 400%

Growing Companies

We don’t solicit enterprises. There. I said it. A critical element in strategy is eliminating options. If we operated every day with infinite options, we would not have an identity at all, and every next client that came along would alter our direction. We have eliminated the enterprise as a target client. While we did do some work for some Fortune 500 companies in the first couple of years, we have chosen to focus ourselves on the companies that are in the middle. They are too big to be called small businesses, but they are not in the Fortune 5000 either. These companies are at a disadvantage in the marketplace amid this computing revolution. They don’t have the resources to invest millions are complete software departments, and they sometimes believe that they must choose a software package or ERP system in order to meet their needs – only to find out that while these systems may be great at basic and common business functions, they do not support the part of the business that is competitive – that is a differentiator in their market. Even with costly customizations, packaged products force business processes to confirm to the lowest common denominator rather than perfectly supporting that part of the business which is the true source of market differentiation. The enterprises know this and have custom software to allow them to execute exactly the way they need to . We level the playing field for growing companies. We bring our clients a core competency in custom software.

Impact of core values on our strategy

Our core values are: . .. . .

  • Empowered Employees
  • Building Relationships
  • Authenticity
  • Serving Others First
  • Excellence
  • Trusted Partner

We take them very seriously and weave them into every part of our company.  We are also continually improving how we apply them. When it comes to our core values of relationships and trusted partner, these directly impact how we execute our mission. We tell our prospective clients up front that we are not looking for a project, we are looking for a long-term relationship. While this might be risky if the client doesn’t desire the same thing, we believe that transparency will help us select the right clients and will help them properly qualify us. If we aren’t a match, there is no need to try to force it. Additionally, we behave as a true partner, and our desire is to become a trusted partner. This means long term focus. This means an understanding of and focus on operational metrics of the client to ensure a good return on investment is happening. And this means actually being responsible for something. We know of so many companies that will take on a client and assign some staff to that client. But then unfortunately the relationship ends. Obtaining productivity and results from those staff somehow become the responsibility of the client? That makes no sense. We insist on an actual outcome that we can be accountable for. And our company is structured for that type of client. Growing firms need to focus on their own industries, not technology. Healthcare companies need to excel at providing the best care to people, not keeping up with what technologies will best help secure information and automate time-consuming tasks. Whether our role is to provide all of the custom software needs of a client or to service only one department, we identify clients that want the long-term relationship, and we execute with the client’s best interest in mind to earn the role of trusted partner. This has led us at times to reduce or team or suggest that we not be the ones to perform a recommended action. Some might call it leaving money on the table. I call it doing the right thing. And I believe that doing the right thing for the client will build a very long relationship. And that is in our best interest.

New core value: empowered employees

In 2016, we adopted a new core value: empowered employees. On the surface, this might seem like a cliché, but we are very serious about it, and it guides individual behavior as well as the behavior of management. We are organized in a way that puts power in the hands of every individual. We have policies that push decision-making down to each employee. The person working hand-in-hand with the client has the most information about any situation, and is likely to make the best decisions in that context. And we expect them to. In our culture, which is defined by our core values, we are problem solvers. We are solutions people. Not only for our clients but for our own issues internally. We hire incredibly smart people. Not just our engineers and project managers, but across the board. Every single position is filled with an incredibly intelligent human being. And we don’t just want the work of their hands. We want their full brain engaged constantly. Our people know not to come with a problem without options for a solution. Coming with a problem isn’t productive. It’s just complaining. And very pessimistic. Our people come with facts about any issue at hand as well as some great possibilities for handling the issue. And managers know to always ask “how would you like to handle it?”. I do my best to set this example from the top when my management team discusses issues with me.

So while empowerment could seem like a buzzword, it’s the best word we found to describe a workforce which owns their day, owns their world, owns their project, and even owns their emotions. And it is the same best word that describes managers who get out of the way while providing work and challenges perfectly suited to the skills our our team. And it’s the best word to describe the way in which our sales team evaluates prospective clients, just as they evaluate us, so that if we sign a contract, all of our capabilities are set up for success so that we can push our new client forward in their market through the power of custom software applied to their business. The word also applies to me. I don’t always get it right. And in my core values briefing to every new employee who joins the firm, I invite them to confront me, to challenge me, to call me on the carpet if they catch me not being a good example of this core value or any others. For me specifically, my managers know that they can bluntly say “that’s not a good idea. We shouldn’t do that”, and it will be met with an accepting discussion rather than rebuke. And I tend to be an analytic person, so I’ve surprised several people by cutting off the challenge with a “you’re right. I’m convinced”. This company cannot run if anyone has a need to “be right”. When we can let go of the need to be right, it is so much fun to focus on making the client and each other succeed.

Executing our mission

The business world has changed a lot in the past few decades. More and more business are becoming decentralized. Peter Drucker’s prediction of the modern networked organization has already come true and is continuing to become more popular.

FedEx and UPS and other specialized shippers like UShip have changed the way we move goods from one place to another. Why would any new company today start out by building their own logistics or trucking fleet?

Payroll and benefits brokers like Insperity, ADP, and others have changed the way serve our employees through HR and benefits. Why would any new company today start our by building their own complete HR department?

Hosting companies and now cloud hosting providers like Google, Microsoft, and Rackspace have changed the way IT infrastructure is provisioned and leveraged from simple email, file storage, to putting applications online. Why would any new non-tech company today start out by owning servers in a data center facility?

And if you have read this far without clicking off to somewhere else, then perhaps you will tolerate a few paragraphs of marketing-speak.

Software engineering firms like Clear Measure are changing the way growing companies build and manage custom business systems by making world-class custom software accessible to non-enterprise companies. Why would any new non-technology company today start out by building an in-house software team or shackling it’s market differentiating processes to common ERP?

As the world becomes more decentralized and more specialized, we help our clients specialize in their field by bringing a custom software core competency where building one is cost-prohibitive and where individual practitioners are unpredictable. We are the leading custom software partner in central Texas. If you’d like to know more, please contact us.

How to have an open door policy with remote workers?

At Clear Measure, I have my office in the Austin headquarters building.  We have space on the 2nd floor and the 4th floor of the building with some separation for different activities.  For instance, the kitchen and lounge are separated and sound-insulated from our open-concept work area on one floor.  My office opens to this area where the Austin-based engineers and project managers work.  I have a door convention that I keep so that people in the office know when I’m available – without having to look at my calendar.

  • Closed door – I’m not available – send me a chat or knock if urgent
  • Cracked door – I’m busy but can be interrupted briefly
  • Open door – No matter what I’m doing, walk in and talk to me, my time is flexible.

IMG_7076I try to keep my door open as much as possible.  And it works out because if I’m in a meeting, I’m usually not in my office anyway. 

When we first moved into this space, I was determined to not have an office.  I made a desk for myself in the open area, but I found myself disturbing those around me, and I was a source of traffic, so now I have an office with a round table in it that doubles as a 2-3 person conference table.  In this way, I have all of my 1×1 meetings with my direct reports without stealing a conference room.

What I haven’t found a solution for is replicating this with our remote workers.  We have Google Hangouts, and Slack, and I have both apps on my iPhone, so I’ll get a notification if anyone contacts me, but there is no visualization on their side that replicates the physical door orientation.  How can I communicate to folks not in Austin if I’m available or I can be interrupted.  What has happened in practice is that the Austin-based people see the open door and just walk in; whereas, the remote folks tend to assume I’m busy and can’t be bothered.  I have much less interaction with the remote people as a result.  This saddens me because I want to have a great relationship with everyone. 

I’m writing this post as a question because I have to believe that someone has figured out this problem.  And my hope is that by sharing I might get some comments that will help me solve the problem.  Being the CEO of a 50-person company puts me in the position of being disconnected with the very people I worked hard to recruit.  I’m looking for a way to stay connected even as my job responsibilities take me away from my desk quite a bit during the day.  Have you, dear reader, seen someone solve this problem?