LINQed IN

I've become a fan of America's Test Kitchen, a TV and book series about cooking, where they not only show how to cook a great Pork roast, they will try it 10 different ways and see which one works best. They end simply by describing the best way (decided by a set of tasters normally) and publish that recipe with a pre-amble about all of the disasters.

It occurred to me that when describing what a Software Architect does to non-technical people, I might start by saying "I'm the test kitchen for software development in my company; I try to solve a problem in numerous ways and then describe the one that worked best."

I see a lot of Architects unable to get their hands dirty in code; How do they really understand a new technology unless they are able to cook the perfect integration pattern in their favorite kitchen: Visual Studio 2008! I think what America's Test Kitchen does right is they document the process really well - what worked, what didn't and why. I know my Red Wine Sauce has improved immensely by using their books; If I could just communicate to my team why my Service Software Factory was designed the way it is!

Troy.

Every now and then I come across a Windows Forms application that has memory leak, or graphic resource leak issues. In managed environments like .NET, some developers overlook that many components wrap un-managed finite resources.  I worked with a fellow developer Mike Zhang today, and because of the difficulties in replicating the issue, we decided that a full pass through the entire source code fixing these issues was our best shot at solving a niggling production issue. After today, here are our top 4 tips and the first steps to ensuring you have a well behaved Windows Forms .NET application.

Begin Update / End Update (Blank, unpainted regions of the screen)

Many Windows controls have a BeginUpdate and EndUpdate methods. These inhibit any Paint events being called when a lot of data is being added to a ListView for example. Big trouble occurs if an EndUpdate isn’t called matching a BeginUpdate! Most general case, is an exception occurs after the begin, but before the EndUpdate gets called.

To Fix: Do a solution wide search for “BeginUpdate”; Go through the result list (yes, EVERY one of them) and ensure that a the EndUpdate is wrapped in a Finally block.

BAD

   1:  this.BeginUpdate();

   2:  // lots of work, including work that may throw an exception!

   3:  // ...

   4:  this.EndUpdate();

GOOD

   1:  this.BeginUpdate();

   2:  try

   3:  {

   4:       // lots of work, including work that may throw an exception!

   5:  }

   6:  finally

   7:  {

   8:      // EndUpdate called even in the case of an Exception in the lots of work above.

   9:      this.EndUpdate();

  10:  }

Pen, Brush and Graphic Resources (Resource leak)

Even in .NET, a lot of managed code uses unmanaged resources. These managed wrappers implement the IDisposible pattern, where the Dispose method nicely releases all unmanaged resources. However – What if Dispose doesn’t get called? Either because Garbage Collection in .NET is happens on its own non-deterministic timetable, some external reference keeps an object alive, or an exception occurs and Dispose is never called. This manifests itself as areas that don’t draw properly or have an enclosed red cross.

To Fix: Do a solution wide search for “Graphic ”, “Pen “ and “Brush “; Go through the result list (yes, EVERY one of them) and ensure that the scope of the Brush, Pen or Graphic is within a using clause, or a try/finally block.

Read: "using" statement reference on MSDN: http://msdn2.microsoft.com/en-us/library/yh598w02.aspx

BAD

   1:      Graphics g = Graphics.FromImage(sourceImage);

   2:      Pen pen = new Pen();

   3:      // lots of work, including work that might fail

   4:      g.DrawRectangle(pen, 10, 10, 100, 100);

   5:      throw new Exception();

   6:      pen.Dispose(); // never called

   7:      g.Dispose(); // never called

GOOD

   1:      using(Graphics g = Graphics.FromImage(sourceImage))

   2:      {

   3:          using(Pen pen = new Pen())

   4:          {

   5:              // lots of work, including work that might fail

   6:              g.DrawRectangle(pen, 10, 10, 100, 100);

   7:              throw new Exception();

   8:           } // calls pen.Dispose() in all cases

   9:      } // calls g.Dispose() in ALL cases

Manual Event Wiring Up and Object Lifetime (memory and resource leak)

This is without a doubt the most common (and some say only common) memory leak on .NET applications. If your component subscribes to an event in ANOTHER component or class, you must –

a) Unhook yourself in your Dispose method for the form or class you subscribed FROM

b) Avoid multiple hookups to an event if your form is opened, or called multiple times

Tips:

a) Hookup events using the Forms designer when possible; This automatically handles events in the proper way

b) To be sure multiple hookups don’t accidently occur, remove yourself from an event before hooking your event up (this is safe, even the first time) –

BAD
ValidationGlobalManager.OnValidate += new My ValidationHandler(MyValidationFunction);

GOOD
  ValidationGlobalManager.OnValidate -= new My ValidationHandler(MyValidationFunction);
  ValidationGlobalManager.OnValidate += new My ValidationHandler(MyValidationFunction);

c) If you hook up an event outside of InitializeComponent (i.e. NOT through the forms designer), then YOU are responsible for unhooking your event in the Dispose() method of your form.

OnPaint and OnDraw Event Handlers - Call the base. implementation in all cases (unpredictable drawing behavior)

Many controls allow you to implement your own OnDraw and OnPaint event handlers. However, the base.OnDraw(); or base.OnPaint(); methods MUST be called under all conditions, even in the case your code fails with an exception.

To Fix: Do a solution wide search for “OnDraw ” and “OnPaint“; Go through the result list (yes, EVERY one of them) and ensure that

a) the base call is first, so it always runs, OR

b) the base call is protected with a try/finally block

Mike and Troy.

I was fortunate enough to attend Code Camp in Seattle last weekend. It was great to see the event attended by around 300 keen developers.

I took notes throughout some sessions, and posted full details the HookedOnLINQ.com website.

• Entity Framework Notes from CodeCamp Seattle
• Parallel LINQ CodeCamp Seattle Notes
• Rethinking Unit Testing Codecamp Seattle Notes
• SilverLight Notes from CodeCamp Seattle

By far, the Parallel LINQ session was a standout. I was surprised just how cleanly the ParallelFx team have committed to making multiple processor development mainstream. This is a library and feature set that is important to watch.

Troy.

1. Do a quick read through of all the items to set the room to a common definition and understanding of the terms and items being discussed
2. Clearly post the definition of each scale on a poster/whiteboard in clear view. If you are the moderator stand near it to re-affirm the definitions during discussion
3. People can assign the boundary conditions quicker than the middle territory. Often a quick pass through the list solely to assign the top rating or the bottom rating (in, out, will, won’t, etc). This process also warms the group up and gets the blood flowing. It is important that if there is ANY dissent, that item be skipped
4. The moderator should keep an eye on if someone isn’t contributing and focus a question for opinion in that person’s direction when the domain would suit (especially if it is an “obvious” yes or no). This is also a way of making sure a dominant personality gets slowed down in driving an agenda
5. People get better at the process through experience. At regular intervals, revisit a subset and ascertain the group still feels the same way
6.  After each break (at least every 1 hour), re-evaluate the last few items to “re-synch” people’s barometer. Depending on the time of day or the day of the week (or the amount of caffeine here in Seattle) – people can change their votes

The prioritization process is absolutely central to any agile process. Making sure you get accurately reflected priorities with large group can be very challenging.

Troy.

Answer: > 6

I was looking at a prototype web project today. The one point that struck me was the number of languages and different technologies that were used in assembling a solution:

Cleint Side -

• HTML for the page templates
• CSS for the stylesheets
• JavaScript for the client side interactive code (and understand the browser compatibility differences when coding)
• AJAX toolkit of your choice (still JavaScript, but you need to learn the intricacies of the one you are using)
• Flash or Silverlight for embedded interactive animations, streaming video, or rich media element

Server Side (this is technology choice dependent) -

• Ruby or C# or Java for the server side code
• ORM Tool of choice (might be in the same language as above, but configuration file formats are a language/skill of its own)
• SQL - at some stage you are going to have to create database tables, script domain data or debug an ORM generated query! Face it, you need to know at least some SQL!

Developers not only need to have some understanding of these languages, but they also have to know how to debug and deploy each technology. What is the chance of hiring an off-the-street developer who knows all of these skills used in a project today in 12 months time?

It seems like the web development tier is undergoing an explosion of little point tools. Pick an AJAX toolkit, pick an ORM solution, pick a MVC pattern to use. Many of our choices today in leveraging a tool to "expedite" initial development incurs a cost in maintainability, and some won't make the distance and be around or upgradable in a few years time.  

I'm simply not sure whether this is a problem at all. Whilst I think there are benefits in designing a solution without external dependencies and using as few technologies and languages as possible, I'm not sure how important it is.

Some other questions I have as I fall asleep -

1. Does Script# help? (write C#, generates JavaScript)

2. Does Silverlight offer hope? (write C# and have it hosted in either JavaScript or a cross-platform runtime)

3. Does using LINQ to SQL or the ADO.NET Entity Framework  and/or Astoria offer enough ORM/Rest features whilst using C# as the development language?

4. Does Volta help us at all? "First, design and build your application as a .NET client application, then assign the portions of the application to run on the server and the client tiers late in the development process. The compiler creates cross-browser JavaScript for the client tier, web services for the server tier, and communication, serialization, synchronization, security, and other boilerplate code to tie the tiers together."

5. Should we use Flash totally using Adobe Flex and go with the time tested solution?

Microsoft released the first public CTP of their Parallel Extensions Framework for .NET. This framework reduces the entry barriers and improves the developer experience when writing Multi-threaded applications. Why is this so important? Because, the laptop or desktop you are reading this post on probably has a dual-core processor. The one you get for Christmas will probably have a 4 core processor, and your IT department is probably getting sales calls from their reps offering then 8, 16, 32, 64 core or multi-processor machines.

I've had a play previously with OpenMP which is a standard syntax for marking up C++ code that then executes parallel (supported since visual Studio 2005, .NET 2.0), but if we wanted these benefits in managed code you are faced with rolling your own threading implementation, (and then getting it working :-) ) Possible, but by no means as simple as it could be.

These extensions will make it easier for everybody to write thread-safe, parallel operating code across as many processors or cores a machine has. You have got to love that.

Parallel Computing Microsite on Microsoft.com

December 2007 Parallel Extensions CTP for Visual Studio 2008 Download

Parallel Computing Blog for these Parallel Extensions

Go download them - What you can't multi-task at work???

Troy.

I've often heard and seen large teams struggle when implementing some agile project practices in large enterprises. I recently joined Corbis who were using many agile practices (introduced by my boss, David Anderson who wrote Agile Management book), but Continuous Integration and Automated Builds weren't on the list.

We discussed why CI was needed for a while, and then successfully implemented it. During this process, I kept careful notes on the barriers we encountered and how those barriers were overcome. On David's urging (I mentioned he was by boss right and does my reviews), I put together this draft article (attached to this post in PDF format). I'm keen to hear feeback on it.

Here is the Introduction -

Introduction

Continuous Integration and an Automated Build process are common practices employed by many high-functioning agile software teams. Many small teams are reporting high productivity improvements, but success stories of Continuous Integration and Automated Builds for Enterprise Scale software projects are much rarer. Some agile pundits have stated that there is a scale limitation for agile practices and these practices are in-appropriate for Enterprise Scale (or high staff count) projects.

This article aims to explore if and why the measurable benefits of Continuous Integration and Automated Build processes begin to decline as application size, and team size grows.

Our conclusion is that there is no inherent reason why Continuous Integration and Automate Build processes won’t scale to any size team. In fact, this article concludes that the problems these techniques solve are a higher pain point for Enterprise Scale applications and become more essential than ever. In addition, this article provides guidance on how to scale agile practices and how to make smart architectural choices up-front to ensure smooth adoption of Continuous Integration and Automated Builds leading to smoother projects and improved success rates.

Interested in your thoughts....

Troy.