I really like continuous integration. Having come from a company that didn't practice it at all (this was a time I used to regularly work 12 to 16 hours a day), I was so happy when I got into a company that actually practiced it. I found it to be a really effective mechanism to ensure that developers are on the same page -- it encourages programmers to move in small steps (changes) with respect to the code base. Those in the team who like to check in lots of changes at a time are those who tend to break the build! Making smaller changes and checking it in quickly into the source control is a safer way of making things work with this system. Because people move in small steps, the system encourages developers to frequently sync with each other's changes. I couldn't do justice to it using mere words -- it takes doing it to actually appreciate continuous integration!

Of course I wasted no time applying the same process in my project at my current employer. However I encountered a few "hang-ups" with the process. I say "hang-up" because getting feedback after checking in my changes takes a loooong time. For the first few weeks, it was very fast to complete a build from the time a check-in is done, but now it takes us like 50 MINUTES for a single build to complete! I'd love to blame the single core 1 GB hardware for it, but sadly an upgrade to the build server is not in the horizon for our team.

In an effort to streamline our build, I realised that we were doing stuff that wasn't really necessary everytime a build is run by the server. First off, the server runs FxCop.  Then it runs unit tests via NCover even after it has already run unit tests -- meaning, each set of unit test DLL is run twice, first by NUnit, then second by NCover running each DLL through nunit-console! And finally it sends the source for analysis by SourceMonitor. While this produces the stats that our company bean counters like to pore over, it is not really needed for every build.  So what I did was to "cache" the NCover and NUnit XML output files for those assemblies that are not very much in flux -- I copied them to another directory. Ditto with the XML output of SourceMonitor. And instead of running NUnit tasks (via NAnt) and exec-ing NCover afterwards to get code coverage stats I just settled for NCover launching nunit-console, thus generating the NUnit and NCover XML files in one shot. Which is the way it should be anyway -- why should unit tests have to be run twice?!

So now...our build gets done in around 12 minutes. I'll still be looking for ways to speed it up, but this is already a marked improvement from the previous 50 minutes or so. To ensure that we get "fresh" SourceMonitor and NCover analysis everyday I put in a NAnt "if" task that would delete the "cached" XML files from NCover and SourceMonitor every 2 am, and enclosed the "exec" tasks for these in another "if" block so that if the cached files are missing, the tasks are run. CruiseControl.NET is then configured to force a build daily at 2 am (hopefully I don't have to be in the office when this happens). This way, we get to run SourceMonitor and other reports once a day while speeding up the builds for the rest of the day. No more continuous integration hang-ups for me!

JP Hamilton has posted a brief blog post on how he unit tests WinForms. Looks nifty, especially if you use a pattern like Model-View-Presenter for your forms. His rationale is that if you keep on testing only the presenter and mocking out the view in your unit tests, your forms code still gets zero code coverage.

He makes his extension methods available for download in the post. If you're using only VS 2005 or .NET 2.0, it's a simple matter to take out the "this" in the method signatures and just use them as static methods instead of cooler-looking extension methods. It is important to note though that these are especially useful if you use the MVP pattern; otherwise your forms would have functionality that this tool won't be able to test or cover.

It was my first time to join a community event in Singapore during yesterday's Heroes Community Launch and it was a blast!  It was also cool to see that of nine presentors during the event, three of us are Pinoys.  I definitely am looking forward to meeting more of the community people soon.

First-time jitters got to me though. Towards the end of my demonstration, I encountered a problem which I didn't encounter during my personal run-through while preparing for it.  And when I saw the "15 minutes left" sign from the organisers I knew I was in trouble! Fortunately the attendees were patient and didn't walk out on me, ha ha.

Anyway, as promised, here are the slides and demo code. The problem I encountered was because I was trying to trigger a UI event indirectly from the data bound to the control. I'll make sure I'll never do that again! Download the links below:

• Silverlight2UnitTestingSlides.zip (PowerPoint format)
• UIDemo.zip

I will be speaking tomorrow at the Heroes Community Launch 2008 here in Singapore with my favourite (boring?!) topic...Unit Testing!  Only this time it's about the testing framework that comes with Silverlight 2's beta release.

Kinda excited and nervous at the same time, since it'll be my first time to participate in a community event since arriving in Singapore over a year ago.  I'll be posting my slides and demo code tomorrow after the event.

I saw this somewhere in my daily reading list (I religiously browse my Google Reader at the start of the day).

I thought to myself that the blog author probably had negative experiences with programmers or architect types who place too much emphasis on design patterns.  I agree that people who give too much importance (or "cool factor") on design patterns will probably see the trees but not the forest.

Personally, I agree with the author that the problem has to be understood before a solution can be applied.  It was said that there was supposed to be this app whose developers wanted to implement all 23 GoF design patterns -- of what use is that?!  It's like putting the cart before the horse.  One has to feel the pain of a particular problem before a solution can become suitable and be appreciated.  That's why I believe in refactoring to patterns.

But that doesn't make me conclude that design patterns are from hell.  Just because one thing can be used negatively by some doesn't mean it's evil -- see for example the internet, or TV.  I had to eat my words not too long ago when I said that enums are evil. I see design patterns as useful in enabling developers to share a common design vocabulary ("you plan to use the state pattern there? that's too complicated, why not use strategy instead...") and it helps me identify possible solutions to design problems.  I wouldn't be proud of my code just because it implements this and that pattern -- I would be proud if it meets user expectations and runs relatively bug-free!

Wow, I've been in a black hole for months!  The usual excuse -- busy at work on projects.  Now that we've gotten our app into production, we're now into firefighting, er... I mean maintenance work for that app.  Here are some of the lessons I've learned in optimising and debugging our Windows forms app:

• Avoid overuse of threads and BackgroundWorkers
• Application splash screens are passe but they can be a good means of hiding application load activity such as caching data and loading stuff in the background -- preferably done in parallel.  The ThreadPool works very nicely here, and users with multi-core machines immediately feel the performance benefit.
• Most of the time, the asynchronous BeginInvoke() is much better than synchronous Invoke() when doing UI work from other threads
• Use the System.Threading.Timer class instead of rolling my own (stupid, I know)
• Use test-driven development as much as possible. Much as I am an advocate of TDD there are times I turn into a hypocrite and cut corners.  Then the app grows bigger and I realise I should've walked my talk instead!
• A great way of ensuring that our forms and controls are not riddled with dependencies is to create a separate Windows EXE project in the solution and launch the forms and controls from there -- when you do so you'll encounter lots of run-time exceptions.  Wow I didn't realise that so many of our forms and controls depended on singletons that in turn depended on services and remote calls...my bad!  It took quite a while to untangle the mess.
• In keeping with the TDD concept of eliminating context, doing the above in conjunction with making use of the System.Diagnostics.Stopwatch class helped me find the slow portions of our code.  Imagine taking more than four seconds just to instantiate a single form?!  We were able to reduce it to around 400 milliseconds after that.  The feedback cycle was quick since I didn't have to run the entire app just to do those performance measurements. (And my boss liked those figures too, so he can present in turn to his boss, I guess.)
• Our app forms made use of tab controls and pages, and some of our user controls made use of other controls that only appeared after certain events.  I realised that they were expensive to instantiate because the (composed) controls were already being created even though they weren't visible or active yet!  To address this, we lazy-loaded these controls only in response to certain events.  Performance radically improved after that, with only the penalty of a slight (barely noticeable, I hope) delay after the event is triggered as the composed controls are instantiated and initialised.  It was helpful to move the (initially) inactive controls to separate user controls so we still get the benefit of designer access from Visual Studio and ease of use when lazy-loading them.
• I don't want to repeat the tips mentioned in this article, so just take a look at it.
• ReSharper is simply indispensable.  Thank you very much, JetBrains.
• I wish we were already using C# 3.0 on this project.  My code is littered with all those anonymous delegates.
• Perception is everything when it comes to UI.

...when your arm hurts because you've started playing Wii Sports with your Nintendo Wii

Yesterday was a Saturday so it's no work day.  I spent half the day playing baseball and tennis with my kids...Wii Sports style. And when I woke up this morning my right arm was hurting a bit.  Looks like I need to go and radically do something about exercising...so my boy and I did a bit of basketball this afternoon.  Hopefully that can also go a little bit into reducing that bulging (pregnant!?) tummy of mine.

I've always joked that it's only my fingers that are getting exercise the past years, and I'm actually even getting less since I switched to the Dvorak keyboard layout.  Now it seems the joke's on me

After a long wait, I've finally completed the ISO download of Visual Studio 2008 from MSDN. After a slightly shorter wait upon launching the setup.exe, I was notified that I was short of disk space (it requires a minimum of 4 GB!). So I had to do a Disk Cleanup and do some delete/move/copy/whatever here and there to free enough space.

Installation took quite some time, good thing I didn't heed the warning to close all opened apps and windows, so I was browsing through 'net stuff while it was doing its install thing. Finally it was ready! Oops, not really ready as I had to restart Windows. I really hate this "feature" of Windows and hopefully Vista SP1 undoes this feature for good.

So after a reboot, I had to endure another pause as Vista applied configuration updates to the OS (meaning the VS 2008 install isn't really complete yet at this point). When I finally encountered the desktop logon prompt I hurriedly typed in my password (I use the Dvorak keyboard layout so I probably saved about 100 ms of time here) and immediately looked for the Microsoft Visual Studio 2008 entry in the Start menu.

Wonder of wonders, the thing came up instantly and presented me a blank IDE workspace. Now this is fast. And it offered to copy my VS 2005 settings too, including my preferred dark colour theme. So far so good. I proceeded to create a simple console app project so I can just get a feel of the new C# 3.0 language. Hmm...weird, Intellisense doesn't seem to be working. Then I realised that ReSharper doesn't work on VS 2008 and the settings that were copied from my VS 2005 probably had Visual Studio Intellisense disabled (since I prefer ReSharper's). So I got it to work after fiddling with the Options.

My first impression of VS 2008 is that it's fast. I haven't taken a look at memory usage yet, since all I did was a simple throwaway console app. I miss ReSharper and hope that they can come up with a release for VS 2008 (including all that C# 3.0 goodness) pretty soon.

The previous incarnation of that Microsoft article was heavily panned by critics and in a rare move, Microsoft actually pulled it out of its developer web site. I didn't realise it but Microsoft actually replaced it with one written by Jeffrey Palermo -- http://msdn2.microsoft.com/en-us/library/aa730844.aspx -- back in 2006. Most of you probably weren't aware of that either.

If you read the (updated) article though, try your very best to read how the TDD steps are actually done in VSTS. I couldn't find any!  I don't know about you but this to me is a very big indication that VSTS (at least the 2005 edition) was not actually designed to work with TDD. The tool is supposed to make it easier to create tests after you've created the code, not the other way around (which is "proper" TDD). Microsoft might as well just say that bluntly and be done with it.

I don't know how things have changed in VSTS 2008. As if I can afford it anyway. I guess the best .NET TDD tool for me (at present) is still the TestDriven.NET add-in to Visual Studio. I am also hoping that SharpDevelop can get its act together this coming year.

Have a Merry Christmas everyone!

The .NET world is eagerly anticipating the release of version 3.5 of the .NET Framework and Visual Studio 2008 later this month. Naturally there is a build-up of technical discussions, events and online material about the features the new release is adding to the table.

In the meantime, here I am studying Smalltalk, an object oriented language dating back to the 70s. I was still an avid watcher of Sesame Street that time! Will I just let the computing world pass me by as I dabble in programming nostalgia?

No way. In fact, many of the touted features for .NET 3.5 (and C# 3.0) as well as Java 7 have been in Smalltalk since it came out of Xerox PARC. LINQ, you say? What about

Extension methods? Hey, the entire Smalltalk virtual machine is extensible and you can modify the thing as it is running. I remember all the talk about "Edit and Continue" in Visual Studio 2005...Smalltalk oldies must have been yawning and saying, "Been here, done that."

Couldn't believe that your cutting-edge .NET technology has a signboard saying "Welcome to the 80s" around it? See it for yourself. Download Squeak, an open source implementation of Smalltalk. Take a look at the tutorials. Watch a video of Smalltalk in action.

.NET developers, it's time to come out of your shells. Welcome to the 80s.

We encountered a strange thing in our current project: there are times our client will bomb out with a web exception when a web service call is cancelled. We found it strange because we enclosed our web service call within a try-catch block, specifically looking for a WebException and testing if the WebException's Status is WebExceptionStatus.RequestCanceled. (Yep, that's a single "l" alright) If the status is RequestCanceled, that means that the exception was just a result of the cancellation of the web service call, not a real exception such as a timeout.

It turns out that the catch block isn't being called because what is being thrown is not a WebException but an InvalidOperationException! This exception, from my observation, is sent when the XML deserializer encounters an invalid XML fragment simply because the connection was cancelled and therefore the XML string is incomplete and malformed.

After discovering this, we just added another catch block to look for an InvalidOperationException. Inside this catch block we treat the situation similarly as if we caught a WebException with a Status of WebExceptionStatus.RequestCanceled. After the change, we didn't have problems with it anymore. I plan to clean this up by introducing a helper method where I can pass an anonymous delegate for performing the operation and an anonymous delegate for handling the call cancellation, just to get rid of the unsightly try-catch constructs.

So, could this be a .NET bug?

Yeah, I said enums are evil.

Now I'm taking them back. I was forced to swallow my pride and use an enum for my current project. This is for representing the privilege level of the user. I guess the simplicity of the enum won me over, since one can easily compare enums with each other. Besides, the data will be coming from a WCF service, so the data is just that -- data. It can't be an object complete with behaviour. So now, my position is to just use the simplest thing that will work. If it's an enum, go ahead and use them. If you outgrow the enum and realize that it is E-V-I-L in the context of the app you're making then just put the necessary measures in place so you can painlessly refactor later (stuff like TDD and regression tests).

Excuse me while I (grudgingly) eat my words...

This was a major announcement, and lots of people were already blogging about it when I turned on my PC this morning.

My initial reaction was -- what for? I think that for those who create custom libraries and controls, this will be a godsend. But for the rest of us developers, why do we need to debug through the .NET Framework code? Whatever happened to the concept of information hiding and encapsulation in OOP? While being able to dig deep into the .NET Framework source will be good for some, I can see that it can possibly be abused as well. By "abuse" I mean people might start coding against behaviour that's specific to .NET 3.5. What will happen once .NET 4.0 or newer comes out? Or if you target Mono, the open source .NET-compatible framework? The case with the latter is peculiar as well, because the current Mono contributor policy is that one cannot become a contributor if he/she is "tainted" -- if the person has viewed official Microsoft .NET code, either through the Rotor project or this new VS 2008 feature.

But then, I might be feeling this way because I am just a business software developer. I don't do low-level stuff with .NET and as long as I code against documented .NET Framework behaviour I'll be fine. Different strokes for different folks, as they say.  Though I can think of a particular scenario where debugging into .NET code might be handy -- whenever I'm doing ASP.NET Web Forms development. That thing's got lots of quirks and gotchas.

Anyway, one interesting implication of this is that .NET library code is written in .NET. It'll be interesting to find out how beautiful (or ugly?) official Microsoft code actually is. By the way, just to correct some wrong perceptions -- Microsoft is not open sourcing .NET. This will be under a reference license -- you can look at it but you can't touch it. The code is "open" but it is not open source.

This InfoQ link on pipelined CI got my attention because it was something that I can relate to. The project we're working on used to build on our CruiseControl.NET build server in only three minutes; now it's taking more than 18 minutes to complete. And it's a dual-core 2 GB machine at that!

Agile practices like test-driven development (TDD), continuous integration (CI) and the like are hinged on quick feedback.  The longer you put off the small tasks like testing and integration, the greater the risk of having things go wrong near the end of the project, the phase we dub as "crunchtime." There was one person who blogged that pipelined CI is a "pipeline of doom," meaning more things will go wrong if you pipeline or break your build into multiple but smaller builds.

I disagree. To me, the quicker one can do a check-in dance, the better. That doesn't mean one will have to do away with tests and go with half-hearted integration. It just means that you push them back so the more time-consuming ones are done later or less often. That way, the developers still get quick feedback even if the project code has grown considerably larger. For instance, in my current project, our architecture lead insisted on having just one solution (.sln) file for everything -- base code, server-side only code and UI code. I objected to this, since not everyone would be working on the server side code and it would only slow the build down. I proposed that we have two solutions (thus two builds) -- one for the server-side code and the other for the client-side code. But then, he's the lead so his opinion was the one taken by the whole team.

Now that our code has grown large and build times are five to six times longer, I realized that my opinion was correct. What I've noticed is that since the build takes longer, the other team members impatiently check in their code even if the current build is red. So bugs and broken behaviour pile up quickly. Other team members keep their code to themselves and only check in a few times a day. Our use of IBM/Rational ClearCase further aggravates this problem, since those devs who keep their code changes to themselves keep the source files exclusively checked out! Ah, it makes me pine for good old Subversion...

I'll be bringing this point up again in our next team meeting. I'll be suggesting that we break our solution down into server-side and client-side solutions so that one set of devs (those working on the server end of things) won't be slowing down the other (those working on the UI/client) and vice-versa. Also, I'll suggest that we move our integration tests out into a third solution, one that we can trigger anytime via a "force build" on the CCTray. So in case someone complains that the services aren't working properly, we'll just hit the "Force Build" button and see minutes later if things are green or red.

Have you heard of the term cyclomatic complexity?

I had my first encounter with it in the current project we're doing at work. One of the senior guys in our project set up our build server, with CruiseControl.NET invoking a tool called SourceMonitor and analyzing source code complexity. Basically, the tool counts the number of branches being made per method per source file.  An if statement, for example, generates an additional branch.  Having an else to that if statement adds another branch to the code.  You can imagine how many branches a massive switch statement would result in. The CruiseControl.NET generates statistics which can be viewed by the PHBs (or *NHBs, ha ha) in our organization to help them feel good about the project we're doing.

It's starting to have an effect on how I view my code. One effect is discussed in my previous blog post. My observation is that it meshes well with Test-Driven Development (TDD). With TDD, you are forced by the practice to create short methods that focus on specific tasks.  In order to keep method complexity to a minimum, it can be helpful to break the method into multiple methods.  Otherwise one can end up with a single method having a cyclomatic complexity of 54.  I've also observed that code with high complexity are those that are very procedural in nature...something I'd love to stick a (refactoring) knife to.

Of course, cyclomatic complexity is not perfect. Like I mentioned above, it measures procedural complexity. But it doesn't measure OOP complexity due to overuse of design patterns and abstractions. I hope I don't end up feeling good about my code because the highest complexity I have is 3 in most of my code, when in fact it's shifted into a different form of complexity (OOP), one that isn't measured by the tool we're using.

*NHB - "no-haired" boss