Uncategorized

New tools in App Insights to better understand your customers’ behaviour


I was trying to get some insights into our application and noticed a new section in App Insights which seems to be added on May 11th (probably announced in Build Conference). This section is in preview mode and looks very promising.

Basically, Microsoft is adding new analytics reports regarding users’ behavior like reports on sessions and users retention. You can also create cohorts and tracks users by groups (useful for A/B testing for instance).

Although these reports are very useful, I won’t see this replacing the use of Google Analytics and alike. But at the same time, I do not believe that this is Microsoft intent (maybe for the longer term). However, combining these reports with other information such like errors, performance and dependencies failures, the one can more easily figure out what’s potentially holding users from converting.

More details
https://azure.microsoft.com/en-us/blog/new-tools-for-understanding-user-behavior-with-application-insights/

.NET, Agile

100% code coverage might not be enough


I was recently working on a project where code coverage slipped and we needed to put some effort to catch up by writing few unit tests. During the process, we came across an interesting case which proved us that having a 100% code coverage was not  a sufficient criteria to stop writing unit tests.

Here is the method we were trying to cover:

public Status RetrieveInitialStatusForTicketType(RetrieveInitialStatusForTicketTypeRequest request)
        {
            try
            {
                var userProfile = base.CheckAuthorizationAndReturnValidUserProfile(request);

                var statusesWorkflow = StatusesRepository.RetrieveStatusesWorkflowForTicketType(userProfile, request.TicketTypeId);

                if (statusesWorkflow == null || statusesWorkflow.Where(s => s.IsInitialStatus).Count() != 1)
                    throw new Exception("Must have one and only one initial status");

                var initialStatuses = statusesWorkflow.Where(s => s.IsInitialStatus);
                Status status = initialStatuses.First();
                Status returnedStatus = (Status)status.Clone();
                returnedStatus.PossibleTransitions = null;
                return returnedStatus;
            }
            catch (SecurityException) { throw; }
            catch (InvalidTokenException) { throw; }
            catch (Exception e) { throw new InternalErrorException(e); }
        }

The whole method was covered except the line 10:

throw new Exception("Must have one and only one initial status");

and line 20 that catches the exception.

So we added a new unit test that expected an exception and in which we mocked the StatusesRepository (line 07) to return a null value. We ran the code coverage analyzer and, yeah!!! 100% of the code was covered. We thought we were done… but wait a minute!!!!

The unit testing is not about testing per se, but rather about validating the requirements implemented by a method. In other words, all the requirements the method implements must be validated by one or more unit tests.

However, by looking at our code, we clearly see, at line 09, that the returned workflow must contain one and only one status which flag “IsInitialStatus” is set to true. However, none of our tests was checking for this rule to be implemented although the code was 100% covered.

We decided to write a new unit test in which the workflow returned by the StatusesRepository (line 07) contains more than one initial status and another in which the returned workflow contains no status marked as the initial status. In both cases, the unit test expects and exception to be thrown. Only then will our code be really 100% covered functionaly speaking.

Along the way, we have split the condition of the line 09 into two different if statements in an attempt to make the code more comprehensive (lines 09 and 12).


public Status RetrieveInitialStatusForTicketType(RetrieveInitialStatusForTicketTypeRequest request)
        {
            try
            {
                var userProfile = base.CheckAuthorizationAndReturnValidUserProfile(request);

                var statusesWorkflow = StatusesRepository.RetrieveStatusesWorkflowForTicketType(userProfile, request.TicketTypeId);

                if (statusesWorkflow == null)
                    throw new Exception("No statuses workflow is defined for the specified ticket type id.");

                if (statusesWorkflow.Where(s => s.IsInitialStatus).Count() != 1)
                    throw new Exception("Must have one and only one initial status");

                var initialStatuses = statusesWorkflow.Where(s => s.IsInitialStatus);
                Status status = initialStatuses.First();
                Status returnedStatus = (Status)status.Clone();
                returnedStatus.PossibleTransitions = null;
                return returnedStatus;
            }
            catch (SecurityException) { throw; }
            catch (InvalidTokenException) { throw; }
            catch (Exception e) { throw new InternalErrorException(e); }
        }

Splitting the condition into two separate conditions had two advantages:

1 – Each condition correpsonds to a particular business rule. Line 09 means that a ticket type MUST be associated with a workflow and line 12 expresses the fact that a workflow must have only and only one initial status. It it now clearer that we do have two business rules and that the condition workflow == null was not a simple check to avoid null object exceptions.
2 – Since the two conditions are separated, we could throw a more specific exception in each case that explains more precisely the reason why it has been thrown (which is very helpful when debugging an application).

Conclusion

While writing unit tests, being in TDD or in TTCUBWWL (tests to catch up because we were lazy – to write them before the code -), we must pay attention to the fact that a method must be covered by unit tests that validate  each implemented requirement in the method and bear in mind that reaching a 100% code coverage of a method is not always a sufficient criteria to stop writing tests for the method.

My advise for the team was that they should write their tests before the code; first to avoid another cathing up effort but more importantly, if the developer wrote the code using TDD, he would certainly not miss the unit test to check the method behavior when a workflow has more than one initial status. But this is another debate: the efficiency of TDD in writing clean and more comprehensive code.

Agile, scrum

My thoughts on team leadership


Before I start, I would like to note that this post is about my thoughts on what is a good leader (nothing based on a particular study or article). It is based on my observation of what I considered good leaders and also what I considered bad leaders. In fact, throughout my career, I realized that I could learn from both types of leaders respectively what to do and what I should never do as a team leader.

Following are simple rules and values that I appreciate in the managers I worked with so far and which I try to apply and tie to as much as I can when I manage a team:

  1. Forget about hierarchy. It is an old concept.
  2. Gain the respect of your team not their fear (of eventually loosing their job)
  3. Be available to spend time out of work with your team (lunches are a very efficient way to strengthen your links with your team members)
  4. Bear in mind that you are not smarter or more competent than your team. You just have enough experience to facilitate the work of a team
  5. Convince and do not impose; if you are good enough, convincing your team is the only way to make them take a direction (the one you believe is the best in the circumstances)
  6. Be honest and transparent with your team.
  7. Never try to bullshit your team members. They are not stupid and every time you try to do so, you are losing a share of their respect
  8. Do not try to be a hero. You will be a hero anyway if your team succeeds and delivers
  9. From your standpoint, your team takes all the credit for successes and you are ultimately responsible of your team failures to deliver
  10. You work for your team and not the other way around
  11. NEVER YELL AT YOUR TEAM MEMBERS. NEITHER IN PRIVATE OR, WORSE, IN PUBLIC
  12. Bashing your team members is the same as someone bashing his kids and family; he does not have values
  13. Get the right people on the bus.
  14. Do not hire only people that think like you; hire people that will challenge you. This will help you to grow.
  15. You must learn from your team. If you believe that you cannot or there is nothing to learn from them, then do an introspection – there is something wrong and more likely with you than with your team –
  16. Help your team members to grow and be successful
  17. The best way to make someone do something for you is to give him the desire to do it (Dale C.)

If you think of some other values and/or rules that you believe are essential to be a team leader or team manager. Please share them.

Agile

Automated acceptance tests to get around requirement misunderstanding in a distributed teams


A year ago, I was part of an integration team who’s role was to implement new functionality in a travel web site based on a back end that was developed overseas. The functionality was about providing users with discounts when they select more than one product in their shopping basket. It may seem simple at first glance, but actually the business rules surrounding these discounts were everything but simple. They were based on dates (discounts were only available on certain dates for instance), products’ combinations (some products combined may give a discount but if you add a third product for instance, it will give another discount and the first one must be discarded etc) and destinations (some discounts are available only for certain destinations).

Before starting the development, we were given a business analysis document that described these business rules and we had QA team that was checking whether or not we implemented the right business rules. This functionality was crucial for the company and needed to be sure 100% that it was bug free… ok say 99% (perfect does not exist).

As we started to deliver the first iterations, the QA team started to raise bugs almost every day and these bugs were off 4 reasons:

  1. Business rules misunderstanding by the QA team
  2. Business rules misunderstanding by the development team
  3. Bugs in discounts calculations at the back end level
  4. Data issues (because all the rules were configured in the back end, if someone changed the rules, of course we would have different results and sometimes QA reported that as bugs)

The back end team was delivering a new version every week that was supposed to contain bug fixes and new functionality. However, it also contained few, not to say many, regressions.

This situation was frustrating. We were under the impression that no progress was made and fingers started to point out in all directions. The development team pointed the overseas team, the overseas team the business analysts and these latter the QA team.

We found out that the situation was due to 1 main factor: lack of requirements understanding.

Solution

We decided somewhat to “automate” the communication and comprehension of the requirements.

A senior developer and I started to write a kind of domain language (DSL). Basically it was a list of utility classes that abstracted our domain objects model and the interaction with the back end. These classes had methods such like ‘AddFlightTo(string destination, DateTime departureDate, DateTime returnDate)’ and ‘AddHotel(…)’ etc… and the most important method was ‘CalculateDiscount’. In the meantime, the QA team along with the business analysts were asked to write the requirements in terms of acceptance tests and not business rules. We (developers) asked them to provide us with tables which columns contain values and the expected results expressed with numbers (discount percentage & discount amount). By expressing requirements this way, we reduced the room of misinterpretation of business rules.

Using the DSL made writing automated tests far easiest and less time consuming in fact. We could write a bunch of scenarios according to the acceptance tests provided by the BA and QA teams and automated them using MS UNIT. These tests were scheduled to run as part of our night build which made regression bugs detection easiest and we no longer deployed a new version of the software than did not pass all the tests (note that we already had unit tests but they did not validate the requirements as these particular requirements were implemented on the third party back end).

We also provided the compiled automated tests to the overseas team ensuring that they always have the latest version. The back end team started to run them before they deliver a new version discovering regressions and misunderstanding of requirements far earlier than before. More important, they did no longer deliver bugs (especially regression bugs) or at least less regressions which had an impact not only on the productivity, since our back end had always a relatively stable version, but also it had an impact on the back end team reputation as well. Note that for safety reasons, we were validating back end deliveries by running our tests battery. If the number of bugs was not acceptable, the new version was simply not installed on our servers avoiding situation were developers and testers were blocked because of an unstable version.

Once all this in place, we started to see less frustration and more progress in the project. Of course, we still had some communication issues, sometime misunderstanding business rules, but it had nothing to do with the situation before the automated acceptance tests. We basically reduced the number of bugs from a dozen for each delivery to something like 2 to 3 bugs; it was a huge shift.

Conclusion

Even though we were not TDD oriented team and we never practiced acceptance tests driven development, we found out that a test is much more efficient to validate our understanding of the requirements than lengthy discussions and meetings. Making these tests automated took us less than 2 weeks efforts for 2 developers but saved us much more than the invested 2 weeks; it restored the confidence between different technical stakeholders.

The acceptance tests became the contract that bound the teams together and automating them removed any room for misunderstanding. Either the test passes and we could deploy or it did not pass and then we had either to fix the bug or discuss the test itself with the QA and BA teams. In both cases, the situation was clear.

.NET, Agile, ASP.NET, C#, design, Design Patterns

Guidelines for consistent exception handling


Exception handling is an essential concept in object oriented programming. It permits us to indicate program or method failure in way that it does not affect the method signature and thus, not affecting the overall design of our methods and classes.

However, exception handling is maybe the most misunderstood part of OOP as far as I have seen. Some think that exceptions are an evil we must live with; an evil that oblige us to decorate our code with a ‘Try/catch’ block whenever we think that an exception may arose. I do think though that exceptions, when used appropriately, can be our ally in making our code more robust and error proof. Exception handling is also a major part of an application design and we should give it the necessary thinking time that it deserves.

In the following article, I hopefully try to put some light on good practices in throwing, catching and designing methods to deal with exceptions.

What is considered an execution failure?

Often, we wrongly consider that we should throw an exception only at exceptional circumstances. The rule actually is “If the method does not what its names suggests (aka what it is supposed to do), it should throw an exception to notify it” (1). A good example is the .NET framework : if you try to open a file for writing but the file is a readonly file, the method will throw an exception because it could not open the file as the method name suggests it. That is said, it is not an unhandled case in terms of the code of the Open method, but it is not a normal execution path neither.

Exception handling versus returning error codes

This is probably a wide topic but I am going to give the main reasons why we should not use error codes instead or even in conjunction with exceptions (by using an error code to identify the nature of the exception).

  • When using error code, we must usually find a way to also return whatever the method is supposed to return. That either makes us change the signature and use (out) parameters or (worse) embed the error code and the returned type into another class/structure.
  • Another reason, and probably most important one, is that whenever we call a method that returns an error code, we must test it otherwise, we may let the code execute in an unstable state and eventually, propagate the error to upper layers of the application. On the other hand, if we throw an exception, it should be either explicitly caught or the application will simply terminate. This makes the code more robust since even if we forget to catch an exception, we will notice it while testing which is not the case if we use error codes and forget to check returned codes.
  • An exception contains much more information than an error code. It contains the exact location of the encountered issue and it should contain, in the message, the exact reason of failure and, for well designed exceptions, how to fix the problem. We no longer need to document each error code and manage error codes duplications. The exception contains all the information developers need to know.
  • If we want an error to propagate to upper callers in the stack, we just let the exception unhandled and it will be propagated to upper layers. Whereas using codes, we must test for the code, return it and do this for all the callers of the method at all levels.
  • Exceptions can be instrumented and monitored using standard applications in Windows.
  • When debugging, we can set the debugger to start when a certain type of exception is thrown.

Sometimes we see developers returning an exception instead of an error code (return an exception type as the return parameter of a method and not throwing an exception) as in the following code.

     public InvalidPayDateException CalculatePayAmount(out double pay) {}

Actually, this type of error handling defeats most of exceptions’ benefits that we just have seen. Thus, it is better not to return exceptions instead of error codes. That is pretty much the same thing as returning a simple code.

 

Throwing exceptions

As mentioned before, an exception should be thrown whenever the method can not complete its execution normally even if the scenario could be predictable in some way. Opening a file that does not exist should raise an exception for example even though we probably are testing for the existence of the file before opening it. Throwing an exception is like saying ‘Everybody above! There is an issue here and I cannot continue to execute normally!’ and this message will go up until someone (code) knows what to do about it to stabilize the code and continue.

A good place to raise exceptions is in the method pre conditions. Pre conditions are a set of tests on the passed parameters (or execution context) to ensure that all parameters are valid and we can continue to execute the method.  If we look at the following code, chances are that the method is called with an employee object set to null which will cause an exception thrown by the CLR.

    public class PayManager
    {
        public double CalculatePayAmount(Employee employee)
        {
            double payAmount = 0;
            if (employee.PayMode == PayMode.BiWeekly &&
                employee.LastPaid > DateTime.Now.AddDays(-14))
                throw new InvalidPayDateException("This employee is not supposed to be paid this week");
        
            payAmount = 2000; // of course, not everyone is paid 2000$
            return payAmount;
        }
    }

Now the same method with pre-conditions will throw an exception with more details (customized details that could contain much more information for developers).

     public class PayManager
    {
        public double CalculatePayAmount(Employee employee)
        {
            if (employee == null)
                throw new ArgumentNullException("The employee object must never be null");
            double payAmount = 0;
            if (employee.PayMode == PayMode.BiWeekly &&
                employee.LastPaid > DateTime.Now.AddDays(-14))
                throw new InvalidPayDateException("This employee is not supposed to be paid this week");
           
            payAmount = 2000; // of course, not everyone is paid 2000$

            return payAmount;
        }
    }

Of course, this is only valid for things we can check before the method does what it should do. In the core of the method, we may need to throw exceptions as well and it is very important to pick up the right exception type to throw being it an existing type or a custom type we create for the purpose of the application.

Choosing the right type to throw

One way to choose the right type of exception to throw is to use the caller perspective. In other words, we need to answer the question  “what type of information should the caller know in order to be able to do something if it catches the exception?”.

For instance, a method can throw several custom exceptions to indicate different issues and, on the caller side, even though these exceptions will be caught separatly (each type has its own catch statement), the treatment will be exactly the same. In such scenario, we will end up with a lot of code duplication as shown in the following code.

        public void DoSomething()
        {
            try
            {
                callMethod();
            }
            catch(customOneException ex)
            {
                log(ex);
                RollBackTransaction();
            }
            catch(customTwoException ex)
            {
                log(ex);
                RollBackTransaction();
            }
            catch(customThreeException ex)
            {
                log(ex);
                RollBackTransaction();
            }
        }

If we have considered the caller in our ‘doThisThing()’ method design, we would have thrown only one type of exception instead of the three and the caller code would be cleaner. This is not always possible, however we should always keep in mind how the caller will catch and treat different exceptions we may throw.

Creating custom Exceptions

I like a definition I have read in the ‘Framework Design Guidelines’ book: “Do not create an exception type if it can not be handled differently from existing .NET framework exceptions’. That makes the focus on how the exception is caught and not only on the nature of the exception as we may tend to do usually. In other words, if there is no added value for the caller in having a new exception type, do not create one and use standard exception types instead.

There are many “generic” exception types that can be used in different scenarios:

  • ArgumentInvalidException, ArgumentNullException and ArgumentOutOfRangeException
  • InvalidOperationException

If we consider these exceptions, we figure out that they may cover many cases of invalid arguments or, for the latest, an operation that we attempt to execute on an object in an invalid state for instance.

There are other exceptions that we should avoid using such as those thrown by the CLR (OutOfMemoryException, StackOverflowException and alike). Only the CLR should be able to throw these types. Another particular case is the System.Exception: this exception should not be thrown because it does not tell anything about the encountered issue. Remember that the type of the exception is actually the first indication of what caused it and using one custom type for the whole application is as inefficient as using the System.Exception base class.

Performance considerations

We must be careful not to have very common scenarios that provoque too many exceptions which will impact the performance. In such case, we should use one of the following two patterns:

Tester-Doer Pattern

Let’s say we do have a pay calculator that calculates the pay amount for an employee. To make things a bit more complicated, let’s consider that we do have weekly and bi-weekly paid employees. The PayManager may throw an exception in the case we try to calculate a pay for an employee that is not supposed to be paid that day.

Now, if we look at the following code:

    public enum PayMode
    {
        Weekly,
        BiWeekly
    }

    public class Employee
    {
        public DateTime LastPaid
        {
            get;
            set;
        }

        public PayMode PayMode
        {
            get;
            set;
        }
    }
   
    public class InvalidPayDateException : Exception
    {
        public InvalidPayDateException() : base() { }
        public InvalidPayDateException(string message) : base(message) { }
    }

    public class PayManager
    {
        public double CalculatePayAmount(Employee employee)
        {
            double payAmount = 0;
            if (employee.PayMode == PayMode.BiWeekly &&
                employee.LastPaid > DateTime.Now.AddDays(-14))
                throw new InvalidPayDateException("This employee is not supposed to be paid this week");
           
            payAmount = 2000; // of course, not everyone is paid 2000$

            return payAmount;

        }       
       
        public void CalculateEmployeesPay(ICollection<Employee> employees)
        {
            foreach(var employee in employees)
            {
                try
                {
                    var pay = CalculatePayAmount(employee);
                    //do something with the pay.
                }
                catch(InvalidPayDateException)
                {
                    //Do something
                }
            }
        }
    }

The first thing we notice is that every two weeks, there will be a number of cases where the InvalidPayDateException will be thrown because many employees should only be paid on a bi-weekly basis. This may have an impact on the performance and since it is a predictable scenario, we certainly should do something to avoid it.

One solution would be adding a method ‘IsValidPayDay(Employee)’ which will indicate whether or not the current date is valid for the current employee. Then, we will modify the method CalculateEmployeesPay in order to first check whether the current date is valid for the current employee and only in this case, do a pay calculation.


    ...
    public class PayManager
    {
        public double CalculatePayAmount(Employee employee)
        {
            double payAmount = 0;
            if (!IsValidPayDate(employee))
                throw new InvalidPayDateException("This employee is not supposed to be paid this week");
           
            payAmount = 2000; // of course, not everyone is paid 2000$

            return payAmount;
        }

        public void CalculateEmployeesPay(ICollection<Employee> employees)
        {
            foreach(var employee in employees)
            {
                if (IsValidPayDate(employee))
                {
                    var pay = CalculatePayAmount(employee);
                    //do something with the pay.
                }               
            }
        }
      
        public bool IsValidPayDate(Employee employee)
        {
            if (employee.PayMode == PayMode.BiWeekly &&
                employee.LastPaid > DateTime.Now.AddDays(-14))
                  return false;

            return true;
        }

   }

In this case, using the tester-doer pattern is legitimate and in fact, it may save us a number of unnecessary exceptions. However, there might be cases where the tester will be too expensive in itself; for instance a tester that will have to grab information from the database. Eventually, such tester will cost more than throwing exceptions and one way to avoid both, too many exceptions and an expensive tester, we should use the Try-Parse pattern.

Try-Parse Pattern

The Try Parse pattern consists in exposing a method and an exception safe version of the same method so that other programmers will be able to call it without paying attention to the exception. In .NET framework such pattern is used for the DateTime.Parse method. In fact, there is another version of this method called DateTime.TryParse which returns a boolean indicating whether the method succeeded or not and an OUT parameter that contains the resulting DateTime object in case it is successfull.

Generally speaking, when we use this pattern, we should provide both methods: with and without exceptions and both methods have the same name with a “Try” prefix for the exception safe one.

In our example, if we use the Try-Parse pattern, we add a TryCalculatePayAmount (same name with the Try as prefix) and as a result we will have the following code:

    public class PayManager
    {
        public double CalculatePayAmount(Employee employee)
        {
            double payAmount = 0;
            if (employee.PayMode == PayMode.BiWeekly &&
                employee.LastPaid > DateTime.Now.AddDays(-14)))
                throw new InvalidPayDateException("This employee is not supposed to be paid this week");

            payAmount = 2000; // of course, not everyone is paid 2000$

            return payAmount;
        }

        public bool TryCalculatePayAmount(Employee employee, out double pay)
        {
            bool result = true;
            try
            {
                pay = CalculatePayAmount(employee);
            }
            catch (InvalidPayDateException)
            {
                result = false;
            }

            return result;
        }

        public void CalculateEmployeesPay(ICollection<Employee> employees)
        {
            foreach (var employee in employees)
            {
                    double pay = 0;
                    if (TryCalculatePayAmount(employee, out pay))
                    {
                        //do something with the pay.
                    }
            }
        }
    }

Catching exceptions

Now, let’s talk about the other side of the exceptions. The catching. There are three main points to consider:

  • When do we catch an exception
  • What exceptions’ types should we catch
  • What should we consider when rethrowing and wrapping exceptions.

When do we catch?

We catch an exception only when we do something about it. The something about it can be anything like:

  • Solving the problem so that the program can continue to run.
  • Wrap the exception in another exception type to add more details about the context and rethrow it.
  • Stabilize the program and rethrow the same exception
  • Log the exception and continue as if nothing happened.

That is said, we should always avoid a Try-Catch-Finalize block with an empty Catch section. This will only hide potential issues. However, having a catch section that just rethrows the same exception can only be useful if the finalize section does something otherwise, we better remove the try-catch block at all.

What do we catch?

We should never catch the System.Exception exception (too generic) neither should we catch CLR exceptions (StackOverflowException, OutofMemoryException etc…). For the first, the reason is that catching the exception base type does not give us any information about what really happened. It is ‘blind’ catching which will hide all other sorts of exceptions (including CLR ones). For the second, the reason is pretty obvious: we cannot do anything to solve a stack overflow problem for example. In very rare circumstances, we may need to catch an OutofMemoryException but in most scenarios, we should not do it and let the exception go up.

The most precise in our catch statement we are, the more robust is our code because it will show us, in testing, what are the potential issues (exception types) that may arise from a particular method. If we ‘swalow’ all the exceptions, despite their type, we may feel that the code is robust but in reality, we’re just hiding real potential issues.

Rethrowing and wrapping

  • When rethrowing the same exception, we better use ‘throw’ (without arguments) otherwise, we will loose stack information.
  • When wrapping an exception, it might be a good idea to include the original exception as the inner exception (to keep some information about the original context).
  • Consider wrapping exceptions to give more details about the context. For instance, a caching component (that does file caching) may have a FileNotFoundException thrown when it tries to access a cache file. However, for its users, that might not have enough information to understand the exception. In this case, we should wrap it in a CacheFileNotFoundException (custom type) that contains information on the query we have done to the caching component for instance.

Conclusion

Exception handling is an important part of any software programming and it is very important to take the time to think about an approach before starting a new project. In this article I tried to go through the most important aspects we should take into consideration without going in too much details in some cases. For this reason I added the references below of two excellent books that, among other things, cover exception handling and programming rules in general.

References

  • (1) Framework Design Guidelines Second Edition (Conventions, Idioms, and Patterns for Reusable .NET Libraries) – Addison Wesley –
  • Clean Code (A Handbook of Agile Software Craftsmanship) – Prentice HALL –
ASP.NET, Uncategorized

10 simple rules to write better ASP.NET applications


When writing ASP.NET application, we usually do a lot of design on the back end isolating different application layers, focusing on domain related design etc… However, as soon as we get close to the UI layer programming, the code becomes a bit messy. My explanation of this phenomenon is that the closer we are to the UI, the less the code is reusable or at least, meant to be reusable. Although, that may be true (that the code is not aimed to be reused), UI code must be at least clean and maintainable.

UI code should use OOP principles such like Inheritance and encapsulation as the code in any other layer. Moreover, some of the SOLID principles can, and should, be applied when designing our user controls and web pages. As  web forms are still widely used, and I believe it will remain the case for a long time, in this article, I am exposing some design and programming rules that I use in my ASP.NET projects. Hopefully, following these rules will help having a cleaner code which will ease your UI code maintenance.

1. Use a base class for the UI classes

It is a good habit to start your project by creating at least a BasePage and BaseUserControl classes and make all your forms and user controls inherit from them. The reason is that you will certainly need to put some code in common to all or part of your web forms or user controls.
For instance, all pages of our site will have a title and eventually some meta-tags. Adding a PageID property and a method that retrieves the title and the meta-tags from a CMS based on the PageID will be achievable pretty easily using inheritance and the maintenance of the code will obviously easiest.
Another common scenario is the need to show/hide elements on a web form depending on a certain state. Adding a virtual method that is called by the OnPreRender event of the base class will make the mechanism of hiding and showing elements of the base centralized in one method overwritten by all the web forms. This is an implementation of the Open Close principle using template method design pattern.

Example:

using System;using System.Collections.Generic;
using System.Linq;
using System.Web;
using System.Web.UI;
using System.Web.UI.WebControls;
using System.Collections.Specialized;
using System.Data;
public class BasePage : System.Web.UI.Page
{
    protected override void OnPreRender(EventArgs e)
    {
        base.OnPreRender(e);
        DoEnableDisableControls();
        SetTitleFromCMS();
        SetMetaTagsFromCMS();
    }
    public string PageID
    {
        get;
        set;
    }
    /// &lt;summary&gt;
    /// Overwrite this method to enable or disable controls
    /// depending on the state of the page.
    /// &lt;/summary&gt;
    protected virtual void DoEnableDisableControls()
    {
    }

    private void SetTitleFromCMS()
    {
        if (String.IsNullOrEmpty(PageID))
            return;
        //Queries the CMS for the page title
        this.Title = "Whatever we got from the CMS";
    }
    private void SetMetaTagsFromCMS()
    {
        //Queries the CMS for the MetaTags
        //add the meta tags
    }
 

A page that derives from the base page will just need to set the PageID property and it will have a title and meta-tags (note the PageID property assigned in the page directive).

<%@ Page Language="C#" PageID="DefaultPage" AutoEventWireup="true" CodeBehind="Default.aspx.cs" Inherits="ExtRefWebApp._Default" %>

Using such a base class will let you inject common code at any time in the development process that will benefit all inheriting pages in a consistent way.

2. Encapsulate session variables

Following are the reasons why I never use session variables directly in the code but I rather encapsulate them into classes:

  • Session variables are widely used in web applications. They are very helpful but we must not forget that a session variable is nothing else than a global variable from an OOP perspective. 
  • When debugging or maintaining a Web application, we often come across a piece of code that makes usage of a session variable and we need to check where this session variable has been set. It is a bit difficult to find out.
  • Session is a property bag and it can hold any type. If we change the type stored for a certain key, we will need to check all the code and change the castings (we do this using the text search in the code).
  • A session variable either exists or not (null value). There is no way to have a default value for instance and thus, whenever we use a session variable, we must check its value (check whether it is a null value) before using it.

The best way to avoid these issues is to have a set of classes that hold session variables and expose them through properties. Using the ‘Find all references’ of a property setter will find all the objects that set the session variable. On the other hand, if we change the type held in the session variable, the solution will eventually not compile and therefore, it will be very easy to make all the changes or to estimate the amount of work necessary to make the change. It makes the code more type safe.
Session variables should be encapsulated in different classes if we have too many and these classes which, preferably, should belong to the same namespace and eventually be in the same folder to ease their maintenance.

There is certainly much more to say about Session variables usage than what is here. For instance, using sessions in other logical layers of the application than the UI is common; a business layer class that makes usage of the session. Needless to say that, in this case, we violate the encapsulation principle and the application layers isolation since the business layer depends on the HTTP context (we no longer can reuse it in a non HTTP application type).

3. Do not create a deep UI controls hierarchies

User controls are a very good mean to re-use UI code. However, when we start to embed user controls into other user controls without a limit, we end up with an endless tree of nested controls. This makes the maintenance of the application a real hell.

Usually, it is a good habit to limit the tree to 2 or 3 levels maximum. The page contains controls that may contain controls. But the inner controls must contain only .NET Framework basic controls (or a server control). And even at the same level, we must limit the number of user controls included in the same user control. For instance, creating one big control that contains many inner controls is not always a good idea. Bear in mind that a page can contain controls as well: I have seen very often developers who do not have any ASP.NET controls in the page but embed everything in a ‘super’ user control that in turn, contains child controls.  Why should we add this level of indirection and complexity? A page is a perfect container for controls.

4. Not everything should be embedded in a separate user control

If we do not pay enough attention, we may quickly have a huge number of controls in a web application. To avoid this, I usually follow the rules below in order to determine whether or not UI functionality must be in a separate control:

  • Reusability: if we know that this control is going to be reused somewhere else, we must embed it in a user control,
  • Complex rules: Sometimes it is better to separate certain parts of a user interface because we want to encapsulate some of its complexity into a separate class and file. This is nothing else the single responsibility principle applied to UI components.

For instance, a large form that contains multiple sections, that may communicate eventually, will be better implemented if we isolate the sections in different controls and make them communicate using events as explained in the next point.

5. Use events to communicate between user controls and their containers

Using a property in a user control to communicate a state with its container control is a common pattern we see in web development. In most cases, developers will set a session variable when something happens in the user control and the container (being it the page or another user control) will check the value of the session variable in order to determine whether or not something happened. This is usually done in the ‘OnLoad’ event. You will notice that pretty quickly, we end up with a pretty messy code in this event handler.
First, we must ensure that the code that sets the session variable runs before the code that checks its value. Also, we must ensure that the value is reset otherwise, next time we hit the same page, we may consider the event even though, it did not happen yet.
One elegant way to notify a status change from one control to another is to expose an event to which the container will subscribe. There are many advantages to this approach:

  • Clean code that reacts to a specific event from the inner control,
  • The same mechanism can be used to notify as many controls as we want,
  • We are sure that when the event happens we will be notified whereas, using session variables adds an uncertainty since we do not know which code is executed first: the code that sets the session variable or the code that checks it.
  • We do not have to care about the session variable and reset it after the event is over since we no longer use it.
  • We do not use a session variable just to hold a flag (save memory and respect basic OOP principles).

6. Avoid embedding code in the ASPX files

The best place to add code to a web form is the code behind file. Embedded code in the ASPX files makes the code messy and very difficult to maintain as in the old ASP days.

7 . Use declarative code as much as possible

If you need to set a user control, or any server control, property do it in the ASPX file instead of the code behind. There case where we do not have the choice (conditional values for instance), but in general this makes the HTML code clearer and increases the maintainability since we know much about the control just by looking at one file (no need to check the code behind).

8. Be careful when using static variables

Static members are shared and leave as long as the ASP.NET process runs. This means whenever you use static members, you are consuming memory that won’t be released unless done explicitly.

One example I have seen in a past project is storing user specific information in a static dictionary (it could be any type of collection by the way). When load tested, the application memory usage grew up to 12 GB for a couple of hundreds of users. As the dictionary was static, its size was just adding up as new users came in and at the end, we had an ‘Out of memory exception’.

In this particular scenario, we should have put in place a timer and a method that cleans up from time to time the dictionary in order to avoid the situation described above. Though, we removed the dictionary and got the information from the database each time the code needed it (no caching).

9. Consider the performance from the client side perspective

In terms of performance, we often focus only on the code that executes on the server and neglect the bandwidth usage and number of downloads that a page contains. While CPUs and memory is cheaper, the bandwidth on the other hand is more expansive and, unless we are implementing an intranet in a controlled environment, we do not have any control on the available bandwidth for our web site users.

There are many tools and ways to make the page performance better on the client side (a good starting point would be this article: http://www.codeproject.com/KB/aspnet/PeformanceAspnet.aspx).

On the server side code, we should take care of not using too much ViewState. In fact, storing an object tree in the ViewState may look helpful and more efficient than storing it in the session variables since the object will not be held indefinitely in memory. Performance will be worse actually since your object tree is serialized and sent as part of the response to the client browser. And when the client submits a form, the serialized object is sent back again to the server.

Another issue is the number of Javascript files that are loaded in a page. We must combine them using the ScriptManage (see https://bellouti.wordpress.com/2008/09/14/combining-javascript-files-with-ajax-toolkit-library/ for more details).

There are too many topics on the client side performance to be all covered in this article and the two links cited above will give you a very good starting point if you never considered this particular point.

10. Use caching whenever possible

User controls can be cached using the OutputCache directive. User controls that display content grabbed from a CMS for instance, should consider using this directive.
We can also cache the content from the CMS in static members or using EnterpriseLibrary caching objects. If we do so, we must ensure that we do cache the closer form of the content to the UI. For instance if you grab XML from a CMS and transform it with an XSLT to generate the HTML to be displayed, you better cache the resulting HTML rather than the XML. Caching the HTML will avoid retransforming the XML the next time we hit the page.

.NET, ASP.NET, C#, multithreading

Efficiently synchronize threads’ access to shared data


Accessing shared members, or static data, in an ASP.NET context must be done carefully. Actually, we must ensure that the code is thread-safe by using a synchronization mechanism. In other words, we must ensure that the static data is accessed by one thread at the same time only.

Usually, we achieve that using ‘lock’ keyword (C#). For instance, in an application we do have a cache manager class that stores some data in a memory cache shared by all the threads. If the cache manager has the data in its dictionary, it returns it otherwise; we will get it from the database and store it in the cache manager (this is for illustration only. Do not bother with the design).

The code would look like:

        public string RetrieveSingleContent(string key)
        {        
                string cacheValue = CacheManager.ReadFromCache(key);
                if (cacheValue != null)
                    return cacheValue;
                lock (_lockObjectSingleContent)
                {
                        using (IDataReader content = GetFromDB(key))
                        {
                            cacheValue = content.GetString(0);
                        }
                        CacheManager.AddToCache(key, cacheValue);
                }
               return cacheValue;
        }

What’s wrong with this code?

If we do have many threads requesting the same content (same key), we will end up with a queue of threads wanting to access the locked section of the code. Once we have retrieved the content from the database (first thread), the next thread will request the same data from the database even though the first thread did it already.

Checking the cacheValue before getting the database is not a solution either because cacheValue is a local variable and thus, it is not shared by all the threads. Therefore, the most efficient solution in the case is to query the content manager outside the lock section and also inside the lock section. In this case, whenever a queued thread will enter the section, it will first recheck whether the requested content has been retrieved in the meantime (between the moment it was waiting for its turn till it gets inside the locked section).

The new method will therefore look like:

        public override string RetrieveSingleContent(string key)
        {
                string cacheValue = CacheManager.ReadFromCache(key);
                if (cacheValue != null)
                    return cacheValue;
                lock (_lockObjectSingleContent)
                {
                    cacheValue = CacheManager.ReadFromCache(key);
                    if (cacheValue == null)
                    {
                        using (IDataReader content = GetFromDB(key))
                        {
                            cacheValue = content.GetString(0);
                        }
                        CacheManager.AddToCache(key, cacheValue);
                    }
                }
                return cacheValue;
        }

Obviously, if querying the cache manager for data is more expensing than getting the data from the database, then this will no longer be valid. But anyway, in this case, you should not use the cache at all.

.NET, C#, Generics

Generics constraints


It has been a long time since the generics are around and most of developers are using them in their code. In this post, I would like to talk about how to use generics more effectively by using constraints.

 What are constraints?

Constraints are conditions applied on the parameter type that are checked at compile time.

Why constraints?

We use constraints to avoid runtime errors when we create generic types.

For instance, if we write a generic class that calculates an employee salary. We want this class to be generic because we may have different types of employee classes (this is for illustration only):

 

public class Employee
    {
        public string Name
        {
            get;
            set;
        }

        public doubleHourlyRate { get;
            set;
        }

        public double NumberOfWorkedHours
        {
            get;
            set;
        }
    }

    class PayCalculator<T>
    {
        public double CalculateEmployeeSalary(T employee)
        {
            //We need an employee from the T type
            Employeeemp = employee as Employee;
            returnemp.HourlyRate * emp.NumberOfWorkedHours; 
         } 
       }    

What’s wrong with this code?

At runtime, there is no guarantee that the type passed to PayCalculator is an Employee type. If it is not an Employee, the ’emp’ variable will have a null value which will make the last line of the CalculateEmployeeSalary method raise a null value exception. We can add a test to check whether or not the ’emp’ variable is null before we calculate the salary. This will fix a runtime error but still, a developer could instantiate our PayCalculator with a non valid type which will lead to an unexpected result.

To avoid this situation, we must add a constraint to our generic class. which will look like:

class PayCalculator<T> : where T:Employee
    {
        public double CalculateEmployeeSalary(T employee)
        {
            //We need an employee from the T type
            Employeeemp = employee as Employee;
            returnemp.HourlyRate * emp.NumberOfWorkedHours;
        }
    }

This means that whenever we instantiate a PayCalculator with a type, the compiler will check whether the passed type is an Employee type or not (actually, it can be a derivative of Employee). If it is not the case, it won’t compile at all.

Types of constraints:

There are 6 types of constraints:

  1. where T : <base class> this is the one we just have seen.
  2. where T: <interface> the type must implement a certain interface.
  3. where T: new() the type must have a parmeterless constructor
  4. where T: U in this case our generic has two parameters and the T one must inherit from type U.
  5. where T: struct T must be a value type
  6. where T:class T must be a reference type.

Note that constraints can be combined: you can have more than one constraints for a class. In this case, the ‘struct’ and ‘class’ constraints (respectively 5 and 6) must be the first constraints. The ‘new()’ constraint, on the other hand,  is always the last one to be specified.

You can find more details on MSDN: http://msdn.microsoft.com/en-us/library/d5x73970(VS.80).aspx

There is also good examples in the book “More effective C#”

 

.NET, Agile, OOP

The S in SOLID – The hell of Helper classes and helper namespaces and helper…


I recently came across a very interesting podcast about SOLID principles and wanted to write a very brief blog entry regarding one particular principe. The Single Responsibility principle which is the S of SOLID.

Sometimes, even experienced developers seem to forget about some OOP basics and one of the most common behavior I have seen is the lazyness (or something else… don’t know) to correctly name a class, a namespace or a project. Have you ever seen someone adding an Utilities project? Or another developer creating a “Helpers” namespace that contains “Helper” classes…

Well, if you look at these classes, they usually contain everything and their opposites. While I agree that finding good names is not always straightforward, I admit that my hair stand when I come across such classes or, worse,  whole projects because it is the starting point for the mess… “I could not find a container for my method, I put it in the XYZHelper class. After all, it helps achieving a goal, so it pertains to a helper class. I never came across a HELPER namesace or class in the whole .NET framework even though, there are helper methods which are not isolated in a helper class).” 🙂

The Single Responsibility principle helps to solve this common issue. Simply, check your class and find out what are the reasons that can make it change. If you found more than one reason, then, you should split it into two or more classes. And once split, the name of the new classes is more obvious because you have split the methods into subsets based on their responsibilities which makes them have something in common (which is generally a good starting point for naming a class).

That’s the S principle (very simplified).

For more information, check out the podcast on this link: http://www.hanselminutes.com/default.aspx?showID=163

.NET, AJAX, ASP.NET, C#

ASP.NET AJAX Collapsible form section


There are many cases where we may need to implement complex forms that contain multiple sections that can be hidden or shown depending on the user’s entries. The issue we encounter in such scenarios is that the section we want to hide or show may contain validators that must be disabled in order to let the form to post back. This can be easily achieved using a server side code, but it involves a round trip to the server which we would like to avoid for performance reasons.

I have written a generic AJAX enabled control that will help you, very easily, to hide or show a form section with client side code (Javascript) so that you no longer have to worry about all the plumbing in JS that manages validators.

You can find the article that describes the solution along with the source code on the following link: http://www.codeproject.com/KB/ajax/AJAX_Collapsible_form.aspx