Improving upon continuous integration

I am quite a big fan of continuous integration and other automatable "agile practices" for improving performance in software teams. Roughly half a year ago the team I am the scrum master of started improving upon our practices, one of the first tasks was to create a script that made it able to create a release with a single command. With this done we added a continuous integration server (CruiseControl) and started down the path of automated tests & tasks. A couple of months later this has had measurable impact on the time it takes to make releases and as we've added compile & test tasks that run for each commit to SVN, this makes us more confident about the quality of the software we make.

I also hold the believe that nothing is ever perfect and it's always possible to improve and tune your software process. The problem I've been seeing lately is that, yes the continuous server helps us easily find compilation errors, bugs and broken tests. But once the code is in our SVN the damage is already done, we are working on a product with multiple teams in multiple time zones. It's very easy for someone to make a commit that breaks the build and then leave for the day, national holidays, other work assignments etc. I'll shortly outline our current process:

1. Write code & test
2. Check in your changes to the version control system (VCS), such as SVN, GIT etc.
3. Continuous integration server sees a change in the VCS and starts a build, tests and so on.
   
4. If your code fails to compile or have broken some tests you will be notified and hopefully you are able to fix it before other developers update from the VCS.
5. Other developers update from the VCS. They might update before you have had time to fix problems in the code you checked in.
6. The bad code propagates to the whole team or the team might be sitting idle waiting for a fix for the bad code that is already in the VCS.

If you look at this process the crucial part is step 3, this where we get our benefits by finding bugs early but why not move this step to take place before the commit is made? Many teams I've heard about have policies that all developers must run the build and the tests prior to a commit but this breaks down every once in a while. Developers are stressed, they forget, they are lazy (yes!) and make mistakes. So, I've been on the look out for a more robust solution to this problem. I was long aware of SVN various commit hooks but didn't really think of it as being a viable option since it would involve a lot of scripting, configuration and time that we do not just have. Luckily one of the team members pointed out TeamCity to me. TeamCity's pre-tested commit features solves exactly this kind of problem. See there homepage for some nice diagrams.

TeamCity allows us to have the following process:

1. Write code & test
2. Send your code to TeamCity, which can run the same checks and builds as the continuous integration server but before the code is committed to the VCS.
3. TeamCity automatically commits to the VCS if your code compile, pass all tests etc.
4. Other developers update as normally from the VCS but they only get good code.
   

Yes! And it actually works. Well it took two days to set it up and get everything working. The trick was that the build agent must run as the same user as I am normally using since our build process is using cygwin and various cross compilation tools that for some reason didn't work otherwise. I've only been running this for a limited time but I am impressed with TeamCity and it will probably help us in the long run to make better software and more effectively.

A couple of minor caveats:

• TeamCity has no command line tool, so there is no way to script commits. Everything is done with IDE plugins.
  
• Java centric, be ready for a lot of scripting & configuration if your build is complex using lots of different build systems (makefile, vcproj/sln, cygwin etc).