.NET, Agile

100% code coverage might not be enough


I was recently working on a project where code coverage slipped and we needed to put some effort to catch up by writing few unit tests. During the process, we came across an interesting case which proved us that having a 100% code coverage was not  a sufficient criteria to stop writing unit tests.

Here is the method we were trying to cover:

public Status RetrieveInitialStatusForTicketType(RetrieveInitialStatusForTicketTypeRequest request)
        {
            try
            {
                var userProfile = base.CheckAuthorizationAndReturnValidUserProfile(request);

                var statusesWorkflow = StatusesRepository.RetrieveStatusesWorkflowForTicketType(userProfile, request.TicketTypeId);

                if (statusesWorkflow == null || statusesWorkflow.Where(s => s.IsInitialStatus).Count() != 1)
                    throw new Exception("Must have one and only one initial status");

                var initialStatuses = statusesWorkflow.Where(s => s.IsInitialStatus);
                Status status = initialStatuses.First();
                Status returnedStatus = (Status)status.Clone();
                returnedStatus.PossibleTransitions = null;
                return returnedStatus;
            }
            catch (SecurityException) { throw; }
            catch (InvalidTokenException) { throw; }
            catch (Exception e) { throw new InternalErrorException(e); }
        }

The whole method was covered except the line 10:

throw new Exception("Must have one and only one initial status");

and line 20 that catches the exception.

So we added a new unit test that expected an exception and in which we mocked the StatusesRepository (line 07) to return a null value. We ran the code coverage analyzer and, yeah!!! 100% of the code was covered. We thought we were done… but wait a minute!!!!

The unit testing is not about testing per se, but rather about validating the requirements implemented by a method. In other words, all the requirements the method implements must be validated by one or more unit tests.

However, by looking at our code, we clearly see, at line 09, that the returned workflow must contain one and only one status which flag “IsInitialStatus” is set to true. However, none of our tests was checking for this rule to be implemented although the code was 100% covered.

We decided to write a new unit test in which the workflow returned by the StatusesRepository (line 07) contains more than one initial status and another in which the returned workflow contains no status marked as the initial status. In both cases, the unit test expects and exception to be thrown. Only then will our code be really 100% covered functionaly speaking.

Along the way, we have split the condition of the line 09 into two different if statements in an attempt to make the code more comprehensive (lines 09 and 12).


public Status RetrieveInitialStatusForTicketType(RetrieveInitialStatusForTicketTypeRequest request)
        {
            try
            {
                var userProfile = base.CheckAuthorizationAndReturnValidUserProfile(request);

                var statusesWorkflow = StatusesRepository.RetrieveStatusesWorkflowForTicketType(userProfile, request.TicketTypeId);

                if (statusesWorkflow == null)
                    throw new Exception("No statuses workflow is defined for the specified ticket type id.");

                if (statusesWorkflow.Where(s => s.IsInitialStatus).Count() != 1)
                    throw new Exception("Must have one and only one initial status");

                var initialStatuses = statusesWorkflow.Where(s => s.IsInitialStatus);
                Status status = initialStatuses.First();
                Status returnedStatus = (Status)status.Clone();
                returnedStatus.PossibleTransitions = null;
                return returnedStatus;
            }
            catch (SecurityException) { throw; }
            catch (InvalidTokenException) { throw; }
            catch (Exception e) { throw new InternalErrorException(e); }
        }

Splitting the condition into two separate conditions had two advantages:

1 – Each condition correpsonds to a particular business rule. Line 09 means that a ticket type MUST be associated with a workflow and line 12 expresses the fact that a workflow must have only and only one initial status. It it now clearer that we do have two business rules and that the condition workflow == null was not a simple check to avoid null object exceptions.
2 – Since the two conditions are separated, we could throw a more specific exception in each case that explains more precisely the reason why it has been thrown (which is very helpful when debugging an application).

Conclusion

While writing unit tests, being in TDD or in TTCUBWWL (tests to catch up because we were lazy – to write them before the code -), we must pay attention to the fact that a method must be covered by unit tests that validate  each implemented requirement in the method and bear in mind that reaching a 100% code coverage of a method is not always a sufficient criteria to stop writing tests for the method.

My advise for the team was that they should write their tests before the code; first to avoid another cathing up effort but more importantly, if the developer wrote the code using TDD, he would certainly not miss the unit test to check the method behavior when a workflow has more than one initial status. But this is another debate: the efficiency of TDD in writing clean and more comprehensive code.

Advertisements
Agile, scrum

How SCRUM helped our team


In 2008, I began work with a client on a new project. The client was a airline and travel agency that needed to rewrite, from scratch, their online travel booking application. The new website had the following primary requirements:

  • Improve end user experience, including performance and security issues;
  • Offer new set of products on the website (Hotel booking, car hiring and insurance purchase); and
  • The new application had to connect to a brand new back-end system.

After learning about their process and project, I suggested that we try a new approach: Scrum. My clients did not know much about Scrum. In fact, the only Scrum-like practice my client had tried was daily meetings. I insisted that using Scrum could help us build software more quickly and build it with higher quality. This was not easy to sell. The client had a number of questions, such as:

  • How can you accurately estimate a project with an iterative process?
  • How can you determine the delivery date of a project if you re-estimate it after every sprint?
  • How can your customer agree on an analysis and “sign it” if you do not have an analysis phase?
  • Isn’t Scrum just a cowboy development process since we do not have a detailed design phase?

I answered their questions or worked with them to find answers. At the same time, I did not pretend that Scrum could solve all of their problems. I did point out what they already knew:  their existing waterfall methodology, with its detailed estimates and phases, only gave the illusion that it could deliver a high quality product with all required features on budget and on time. Scrum, on the other hand, could mitigate some of these risks. After much debate, we decided to give Scrum a try.

Read more… (http://www.scrumalliance.org/articles/128-how-scrum-helped-ourteam)

methodologies

Agile Design Artifacts


I recently got contracted by a company to be the technical lead for a web application development project. My first task was to come up with an architecture document for the application within 2 to 3 weeks. I rapidly realized that it was not realistic. I could eventually design the application, but I was convinced that it would not be efficient for the following reasons:

  • I was promised to get the full requirements a week after I started. I did not get them and still now, after 3 months,  there are still on going,
  • The web application was intended to connect to a back end bought from a third party provider. We did not know much about the API (actually, the API were planned to change – and still changing today after 3 months)

  • I always have been convinced that a too detailed architecture will be obsolete as soon as developers start to implement the functionality.

For these reasons, I told the team manager that I prefer not to spend my time drawing class diagrams or any sort of artifact that will be thrown away or, at best, be expensively kept up to date during the development process. What I suggested though was to:

  • Draw a high level design diagram; this diagram included all the providers and layers of the application.

  • Write few class stubs to give a direction to developers. The classes did not contain any implementation. They rather contained many “TODOs” with comments for developers; we also agreed with developers that the comments were there as a suggestion. They were allowed to go in another direction as long as they respect the overall design idea.

  • We focuse on only one type of artifact: the code itself. Well written code is the best artifact as it expresses the static view but also the dynamic view of the system (which is, from my point of view), the most important view to understand a system especially to support it.

  • We scheduled and took the necessary time to do few meetings with all developers to explain the design guidelines; this is far better than writing thick documents as developers came up with very interesting questions and suggestions from their past experiences. Once we agreed on the design, we started the development of what we called “the infrastructure” or the framework that will support the application.

During this phase, we decided to use SCRUM methodology; this was new to my client (they applied it partly in the past). I insisted on the fact that an iterative development process will help us to get things done and deliver high quality software.

We called this first phase the SPRINT 0; we did a lot of refactoring as the code was written and the application was growing in size. We decided that no interface (user interface) will be developed during this sprint; we wanted to focus only on the framework and the design.

Now, we are in the sprint 3. We delivered some functionality to the business and we had very positive comments so far. Even managers who first were skeptical regarding SCRUM methodology are now much more comfortable and confident that an iterative process can effectively help to deliver in time and with a high quality.