Digest Authentication on a WCF REST Service

Featured

Introduction

This is a follow-up on my earlier article that described how to use BASIC Authentication with a WCF REST Service. The disadvantage of that solution was that you need a https tunnel to really secure the username password verification process. Although possible, this is not always a feasible situation, for example, if you don’t want to invest in certificates or loose performance when using a https tunnel.

This article explains a different authentication mechanism called Digest Authentication which provides an alternative. This security mechanism is more secure than Basic Authentication and does not have the drawbacks from using a https tunnel.

Digest Authentication is available on multiple web servers and supported by multiple internet browsers. The drawback when using Digest Authentication with Internet Information server is that it automatically authenticates credentials against active directory. This article describes an implementation which enables you to secure a WCF REST service with Digest Authentication and authenticate against any back-end.

Digest Authentication was first described in RFC 2069 as an extension to HTTP Basic Authentication. Later, the verification algorithm and security was improved by RFC 2617. This is the current stable specification. The implementation in this article is based on that RFC 2617 specification. Digest Authentication is more secure because it uses MD5 cryptographic hashing and the use of a nonce to discourage cryptanalysis.

Overview of Digest Communication

Digest communication starts with a client that requests a resource from a web server. If the resource is secured with Digest Authentication, the server will respond with the http status code 401, which means Unauthorized.

In the same response, the server indicates in the HTTP header with which mechanism the resource is secured. The HTTP header contains the following “WWW-Authenticate: Digest realm=”realm”, nonce=”IVjZjc3Yg==”, qop=”auth”. The first thing you should notice is the string Digest in the response, here the server indicates that the resource that was requested by the client is secured using Digest Authentication. Secondly, the server indicates the type of Digest Authentication algorithm to use by the client with Quality Of Protection (QOP) and the string called nonce which I will explain later in this article.

An internet browser responds to this by presenting the user a dialog, in this dialog the user is able to enter his username and password. Note, that this dialog does not show the warning about transmitting the credentials in clear text as with a Basic Authentication secured site.

When the user enters the credentials in this dialog, the browser requests the resource from the server again. This time, the client adds additional information to the HTTP header regarding Digest Authentication.

The server validates the information and returns the requested resource to the client. The details of the response from the server and the additional request of the client will be described in the following part of this article.

Digest Authentication

When the server responds to an unauthenticated client request, the server adds a nonce and a qop key to the header of the HTTP response. Both are typical for Digest Authentication. First, the nonce will be described and second the QOP quality of protection.

The Nonce

The Nonce stands for “Number used Once”, this is a pseudo random number that ensures that old communications between a client and a server cannot be reused in replay attacks. A replay attack is a network attack in which previous valid data transmission is repeated. This is done by an adversary who intercepts the data and retransmit it. According to the RFC 2716 specification, the Nonce is a server specified data string which should be uniquely generated each time a 401 response is returned by the server. The 401 response that is sent back to the client includes the Nonce generated by the server. According to RFC 2716, the client should add this nonce to the header of next requests.

Generating the Nonce

The format of the nonce depends on the implementation. Each RFC 2617 digest authentication implementation may define their own nonce format. However, one should carefully design the format of the nonce as it is a part of the quality of the security. For my implementation, I choose to include a date time stamp and the IP address of the client into the nonce. The implementation generates the nonce as follows.

Nonce = Base64( TimeStamp : PrivateHash)

The nonce is generated by base64 encoding the string that is constructed by concatenating the time stamp, a colon and a generated private hash. In the source code, this is handled by the NonceGenerator class which has a Generate method that generates the Nonce string.

public string Generate(string ipAddress)
{
   double dateTimeInMilliSeconds =
      (DateTime.UtcNow - DateTime.MinValue).TotalMilliseconds;

   string dateTimeInMilliSecondsString =
      dateTimeInMilliSeconds.ToString();

   string privateHash = privateHashEncoder.Encode(
      dateTimeInMilliSecondsString,
      ipAddress);

   string stringToBase64Encode =
      string.Format("{0}:{1}", dateTimeInMilliSecondsString,
       privateHash);

   return base64Converter.Encode(stringToBase64Encode);
}

MD5 is used to generate the private hash of the string that is constructed by concatenating the time stamp, a colon, the IP address of the client, a colon and a private key that is only known to the server. As MD5 is used, the generation is one-way. It is not possible to reconstruct this information from the private hash.

PrivateHash = MD5Hash( TimeStamp : IP Address : Private key)

In the source code, generating the private hash is handled by the method Encode in the PrivateHashEncoder class. It uses the MD5Encoder class to actually generate the MD5 hash.

public string Encode(string dateTimeInMilliSecondsString,
    string ipAddress)
{
  string stringToEncode = string.Format(
     "{0}:{1}:{2}",
     dateTimeInMilliSecondsString,
     ipAddress,
     privateKey);

  return md5Encoder.Encode(stringToEncode);
}

Validating the Nonce

Every time the client sends the nonce to the server, the server validates if this is the nonce that the server sends to the client. The server validates the nonce in two steps:

The first thing that this implementation on the server does is validate if this PrivateHash was generated by this server and returned to this client. The server does this by generating the PrivateHash with the time stamp that is available in the Nonce and the IP address of the client. If this does not deliver the same PrivateHash as in the nonce from the client, the nonce is incorrect and the server responds with a 401. The NonceValidator is responsible in the source code for validating this nonce.

public virtual bool Validate(string nonce,
   string ipAddress)
{
   string[] decodedParts = GetDecodedParts(nonce);
   string md5EncodedString = privateHashEncoder.Encode(
      decodedParts[0],
      ipAddress);

   return string.CompareOrdinal(
      decodedParts[1],
      md5EncodedString) == 0;
}

Secondly, the server checks if the Time Stamp is too old. The server holds a certain time-out for a nonce. For example, the time-out is 300 seconds. The server validates if the time stamp in the nonce is not older than 300 seconds. If the nonce is older than 300 seconds, the server returns an indication in the HTTP header that the received nonce is too old together with a new nonce. RFC 2617 uses a special key called Stale in the header that is sets to true when the Nonce is too old. The NonceValidator is also responsible for checking if the time stamp is too old.

public virtual bool IsStale(string nonce)
{
   string[] decodedParts = GetDecodedParts(nonce);

   DateTime dateTimeFromNonce =
     nonceTimeStampParser.Parse(decodedParts[0]);

   return dateTimeFromNonce.AddSeconds(
      staleTimeOutInSeconds) < DateTime.UtcNow;
}

By using a time stamp and the IP address in the nonce, we make sure that the request is recent and comes from the client that requested the resource.

Quality Of Protection

Digest Authentication allows the server to ask which algorithm the client should use to encrypt the credentials of the user. Digest Authentication allows the following Quality Of Protection.

  • none = Default protection compatible with RFC 2069
  • auth = Increased protection that includes a client nonce and a client nonce counter
  • auth-int = Increased protection and integrity that included all of auth and a hash of the contents of the body

Note that it is a request from the server, the client itself is allowed to choose a lesser secure qop algorithm. If the server requests for auth, it is ok for the client to start communicating with the default or none qop.

The implementation with this article supports both default/none and auth. The class DefaultDigestEncoder and the class AuthDigestEncoder implement the default and the auth type of quality of protection. Both classes derive from DigestEncodeBase which holds common functionality.

At runtime, the server instantiates both types of encoders and stores them in a dictionary with the qop algorithm as the key. This enables the server to easily switch between different types of encoders at runtime.

internal class DigestEncoders :
   Dictionary
{
 public DigestEncoders(MD5Encoder md5Encoder)
 {
  Add(DigestQop.None, new DefaultDigestEncoder(md5Encoder));
  Add(DigestQop.Auth, new AuthDigestEncoder(md5Encoder));
 }

 public virtual DigestEncoderBase GetEncoder(
    DigestQop digestQop)
 {
  return this[digestQop];
 }
}

None or Default QOP

When an internet browser receives 401 HTTP status code with Digest in the authentication header, it will show a dialog for entering the username and password. When the client uses the default qop which is compatible with RFC 2069, the client encrypts the user name and password as follows.

HA1 = MD5( username : realm : password)

 

HA2 = MD5( method : digestURI)

 

response = MD5( HA1 : nonce : HA2)

An MD5 hash is created from the user name, realm and password, a separate MD5 hash is created from the HTTP method and the URI of the resource that the client requests. The response is created through a MD5 hash that combines the previous two MD5 hashes and the server generated nonce. The DigestEncoderBase class holds the functionality to generate both the HA1 and HA2 hashes.

private string CreateHa1(DigestHeader digestHeader,
   string password)
{
  return md5Encoder.Encode(
    string.Format(
    "{0}:{1}:{2}",
    digestHeader.UserName,
    digestHeader.Realm,
    password));
}

private string CreateHa2(DigestHeader digestHeader)
{
  return md5Encoder.Encode(
    string.Format(
    "{0}:{1}",
    digestHeader.Method,
    digestHeader.Uri));
}

The base classes AuthDigestEncoder and DefaultDigestEncoder are responsible for generating the response. This last step, generating the response, is what differs in the two derived classes. The response of the Auth algorithm should be generated differently. The Auth algorithm includes a nonceCount and a client generated Nonce in the response. Also, the actual qop string is concatenated before the hash is calculated.

response = MD5(HA1:nonce:nonceCount:clientNonce:qop:HA2)

This is why the Auth algorithm is more secure than Default, the server performs an additional check to see if the nonceCount is incremented by the client with every request. The CreateResponse method of the AuthDigestEncoder generates the Auth response.

public override string CreateResponse(
   DigestHeader digestHeader,
   string ha1,
   string ha2)
{
  return
   md5Encoder.Encode(
     string.Format(
     "{0}:{1}:{2}:{3}:{4}:{5}",
     ha1,
     digestHeader.Nonce,
     digestHeader.NounceCounter,
     digestHeader.Cnonce,
     digestHeader.Qop.ToString(),
     ha2));
}

Extending WCF REST

To be able to integrate Digest Authentication with WCF REST, the WCF REST framework has to be extended. This is done by creating a custom RequestInterceptor. For more information, take a look at my previous article on CodeProject which explains this extension in more detail.

Retrieving and Storing User Credentials

The password of the user is transmitted as part of the response generated by the client to the server. It is not possible for the server to extract the password from the response. The server generates a response and checks if the response is equal to the response that was given by the client. This means that there are two options for storing and retrieval of user credentials using Digest Authentication.

  • The first and most secure option is for every user to store the HA1 key in the credentials data storage and validate using the stored HA1 key. This has a disadvantage because you have to change the HA1 key in the data storage if the username, password or realm changes.
  • The second option is to store the password of the use in the credentials data storage in such a way that it is possible to retrieve the original password. This is obviously less secure than the first option.

Using the Source Code

If you want to secure your own WCF REST service with Basic Authentication using the provided source code, you need to execute the following steps:

  • Add a reference to the DigestAuthenticationUsingWCF assembly
  • Create a custom membership provider derived from MembershipProvider
  • Implement the ValidateUser method against your back-end security storage
  • Create a custom membership user derived from Membership user
  • Implement the GetUser method against your back-end security storage
  • Create a custom DigestAuthenticationHostFactory, see the example in the provided source code
  • Add the new DigestAuthenticationHostFactory to the markup of the .svc file

Points of Interest

The provided source code is developed using TDD and uses the NUnit framework for creating and executing tests. Rhino mocks is used as a mocking framework inside the unit tests.

The business case for a DI framework

Every Architecture Design decision should be based on one or more Non-Functional requirements. Each Non-Functional requirement in its case should be based on a business need. Recently I asked myself what are the business needs for using a dependency injection (DI) framework?

Which visible, tangible benefit would be reached using a DI framework? This blog post describes the arguments we used to successfully convince management.

Dependency Injection Framework

A dependency injection framework is a software module that manages the dependencies of your application. Dependency Injection is a specific form of inversion of control where the concern being inverted is the process of obtaining the needed dependency. An excellent discussion on how to use a DI container can be found here by Uncle Bob. Inversion of Control is one of the SOLID design principles. There are many types of dependency injection frameworks, for a comparison take a look at this and this blog post from Andrey Shchekin.

SOLID Dependency Inversion Principle

Convincing management

Recently at LetsGrow we started using a dependency injection or DI framework for the LetsGrow.com platform. We are using Test Driven Development (TDD) and using a DI framework seemed to be the next logical step. This would remove the implementation of several builder patterns and would make implementing tests easier. It would also make the LetsGrow.com platform more flexible. Although these benefits were clear to me, I had a hard time convincing management of the business benefits of a DI framework.

Who makes the design decision?

One could argue that the design decision of using a DI Framework should be made by an architect. I think that introducing a DI framework also comes at a cost. Surely the benefits will be greater in the long run, but the development of new functionality will be slower during the introduction of a DI framework. The development team has to learn how to use such a framework, some existing code will change with no visible benefit for the user of the application. Therefore, I think that such a decision should be made in cooperation with management.

Business benefits

The primary benefits of using a DI framework are that it makes the source code of an application easier to understand, to maintain and to extend. Using a DI framework will reduce the amount of source code which will decrease the time it takes to implement new functionality in your application. In the long-term, this results in reduced maintenance costs, lower bug rates and thus increased customer satisfaction.

Secondary benefits are that configuration of an application will be more obvious as it sits in a central place. All instances of objects will be created the same way (consistency). When using a DI framework it becomes possible to switch implementations at run-time. For end-to-end tests it becomes possible to replace key components of the application with fake implementations.

Risks

We also looked at the possible risks of introducing a DI framework. We identified the following risks:

  • increased memory usage
  • performance overhead
  • complexity in configuration
  • less transparent source code

After some research we found that there was indeed an increase in memory and some performance overhead but they were marginal. I think that everyone that wants to use a DI framework should also do this research themselves because this could differ per application.

Complexity in configuration is a real risk. If you describe each dependency of each class in the configuration file,  the configuration file becomes too large and complex. Only these dependencies that should be managed by the framework should be described using the configuration file. Hartmut Wilms also discusses this risk in this blog post.  The last risk, less transparent source code we think will be temporary. Developers will have to get used to using a DI framework and the fact that the DI framework takes care of creating instances of objects.

Conclusion

My conclusion is that with the arguments mentioned in this post you should have no trouble convincing management of the business benefits of using a DI framework for your application. If you have more benefits or convinced your organization using a totally different arguments let me know. I would love to hear it.

Basic Authentication On WCF Rest

This article explains a method to secure a REST based service using basic authentication. The service itself is implemented using Microsoft Windows Communication Foundation. The authentication of the credentials should be possible against any type of back-end.

For example authenticate against active directory, custom file or a database. Default, Basic Authentication is possible in combination with WCF and IIS, authentication is only possible then against active directory. This article and provided source code can be used in two ways. First just download the code, go down to the use the code part and implement your WCF service using Basic authentication. Second you can use the article to learn about what exactly is Basic Authentication and how can WCF be extended in a simple way.

Basic Authentication

Basic authentication is a standard protocol defined within Http 1.0 that defines an authentication scheme. In this scheme the client must authenticate itself with a user-id and password. Basic authentication is described in RFC 2617. When a client requests a resource from a site that is protected using Basic Authentication the server returns a 401 “Not authorized” response. Inside this response the server has added an sign that the site is protected using Basic Authentication. The server adds WWW-Authenticate: Basic realm=”site” to the header of the response, Basic indicates that the authentication scheme is Basic Authentication. Realm is an string that indicates which part of the site is protected. It has no further usage in the actual authentication mechanism itself.

An internet browser generates a dialog based on this response message, this dialog shows the realm and allows a user to enter a user name and password.

Request Response Flow Basic Authentication

When a user enters the user name and password, the client resents the request but adds “Authorization: Basic SGVsbG8gQmFzZTY0″ to the header of the request. The strange characters after Basic are the user name and password separated by a single colon “:” in a base64 encoded string. The server decodes the string, extracts the credentials and validates them against the back-end. When the credentials are correctly validated the server returns the requested content to the client.

Request Response Flow Basic Authentication

Extending WCF Rest

WCF REST is part of the WCF Rest starter kit. Unlike standard WCF, the WCF starter kit provides a simple way to apply behaviour to all server operations. Adding this behavior to normal WCF can get fairly complex. WCF Rest uses Request Interceptors. Request interceptors shield you from the more complex extensibility points. Request Interceptors are executed at the WCF channel level and enable you to interpret the incoming request and generate an appropriate response.

public class MyRequestInterceptor : RequestInterceptor
{
  public override void ProcessRequest
        (ref RequestContext requestContext)
  {
    //Access request with requextContext.RequestMessage
  }
}

By deriving a new class from the RequestInterceptor base class you create a custom request interceptor. The method ProcessRequest is executed for every request that arrives at the service. Inside this method we read the header of the request and decode the base64 encoded string if it is present. If it is not present we generate a response that includes WWW-Authenticate: Basic realm=”site” in the header.

Linking the custom RequestInterceptor to the service

The RequestInterceptor can be linked to the service by creating a custom ServiceHostFactory. The ServiceHostFactory is responsible for provides instances of ServiceHost in managed hosting environments. By Managed hosting environments Microsoft means IIS hosted services. The CreateServiceHost method of the ServiceHostFactory creates a new WebServiceHost and adds the new RequestInceptor to the Interceptors collection.

public class BasicAuthenticationHostFactory :
   ServiceHostFactory
{
  protected override ServiceHost CreateServiceHost
         (Type serviceType, Uri[] baseAddresses)
  {
    var serviceHost = new WebServiceHost2(serviceType,
        true, baseAddresses);
    serviceHost.Interceptors.Add(
        requestInterceptorFactory.Create("DataWebService",
            new CustomMembershipProvider()));
    return serviceHost;
  }
}

This new ServiceHostFactory is linked to the service through the markup file (.svc) of the service. Inside the markup, you override the host factory of the service.

ServiceHost Language="C#" Debug="true"
   Service="WCFServer.Service"
   Factory="WCFServer.BasicAuthenticationHostFactory"

Custom MembershipProvider

The provided source code uses a MembershipProvider to actually authenticate the credentials of the client. You can write your own MembershipProvider by deriving from MembershipProvider. The only method that needs to be implemented is the ValidateUser method. This is the method the code uses to validate the user. This customer MembershipProvider is linked to the RequestInterceptor in the CreateServiceHost method of the ServiceHostFactory.

public class CustomMembershipProvider : MembershipProvider
{
  public override bool ValidateUser(string username,
        string password)
  {
    //perform validation return false;
  }
  ...

Security context

After we have authenticated the incoming request, we have to create and set the security context. This enables the usage of the client credentials inside a service method. WCF uses the thread local ServiceSecurityContext for this. After the request has been authenticated, a new ServiceSecurityContext is created and added to the incoming request. This enables retrieving the user name inside the service method with ServiceSecurityContext.Current.PrimaryIdentity.Name

Using the code

If you want to secure your own WCF REST service with Basic Authentication, you need to execute the following steps:

  • Add a reference to the BasicAuthenticationUsingWCF assembly
  • Create a new membership provider derived from MembershipProvider
  • Implement the ValidateUser method against your back-end security storage
  • Create a custom BasicAuthenticationHostFactory
  • Add the new BasicAuthenticationHostFactory to the markup of the .svc file

Points of Interest

Although Basic Authentication is a method to secure a web site or service, the authentication mechanism itself is not secure. The user name and password are send base64 encoded over the internet. To make this more secure, the server should offer the service using https. Https secures the channel so that the base64 encoded user name and password cannot be decrypted.

This article and source code are based on the example of Pablo M. Cibraro who deserves the credit for the solution. The provided sourcecode is developed using TDD and uses the NUnit framework for creating and executing tests. Rhino mocks is used as a mocking framework inside the unit tests.

Source Code

The source can be found at CodeProject

SOLID Design Principles

Recently two colleagues and I did a workshop in which we explained SOLID Software development. SOLID is an acronym that stand for five principles for object-oriented development. SOLID stands for Single ResponsibilityOpen ClosedLiskov SubstitutionInterface Segregation and the Dependency Inversion Principle.

These principles can help a software developer manage dependencies between software components. Bob Martin the author of the principles writes “So dependency management, and therefore these principles, are at the foundation of the -ilities that software developers desire”. You can read the original article here.

These -ilities that Bob Martin refers to are the non functional requirements of a software product. I think that a software architecture is responsible for implementing the non functional or quality attributes of a software product. Therefore, I think that the SOLID design principles are one of the most fundamental rules for designing a robust software architecture. During the workshop we showed the following pictures to introduce a single principle. Can you spot the principle just by looking at a picture? The pictures are from this site which also explains the SOLID design principles.

SOLID Single Responsibility Principle

SOLID Open Close Principle

SOLID Dependency Inversion Principle

LetsGrow SOLID Software development

The nice thing about the SOLID principles that they can be used on many levels. For example, the Single Responsibility principle can be used on method level, class level, library level and even on (sub)system level. The same goes for the other principles. The slide show below shows the slides on slide-share that we used for the workshop.



Discover Non-functional Requirements

I talked about the importance of Non-functional requirements in my earlier post “Non-functional requirements define a Software Architecture”. What I did not describe was how you can discover the Non-functional requirements?

My preferred method when it comes to requirements gathering, functional or Non-functional, is through interactive workshops. During these workshops you facilitate the requirements gathering using interactive methods. I find that workshops with the main stakeholders are very time effective and create a common ground.

You should decide a standard set of Non-functional requirements for your company. This list of standard Non-functional requirements will function as your checklist at the start of a project. By using a standard you create consistency through all of your projects. You can choose from a number of existing standards or you could create your own. The thing is that not all of these standard will fit your organization, some will be too small, too complex or too large.

You can use any of the following standards or a combination as your Non-functional requirements checklist

A workshop is the process in which you find the Non-functional requirements of your application. If you have done a number of these requirements workshops, you often find a stable set that are important for all the applications that you develop. For example, in industrial applications, reliability and time behavior are often the most important. What would be the contents of such a workshop?

Which non-functional requirement

Depending on the maturity of your organization, you first need to create awareness about the importance of Non-functional requirements. Next,  invite the stakeholders of your application to a QUINT workshop. In this first workshop you are going to find the most important Non functional requirements of your application. For this first step I often use the main categories of QUINT (Functionality, Reliability, Usability, Efficiency, Maintainability and Portability). In the invitation of the workshop add some home-work for the participants which explains these categories.

At the start of the workshop explain the main categories and answer any questions about the home-work. The goal of this workshop is to determine a prioritized list with the most important QUINT categories for your application. An effective method to create this list is dot-voting. Each participant gets three dots that he or she can use to vote on a category. The total number of votes on a category determines the order.

After a first voting round, the stakeholders need to discuss the outcome, is this what they expected? Reassure the stakeholders that all Non-functional requirements are important and will get a place in the software architecture. This prioritized list is used as a basis for our design decisions. After the discussion, do another voting round to create the final list.

This workshop, which can be done in one or two hours creates a prioritized list of Non-functional requirements. This list will be the basis of the software architecture of your application. In a follow-up of this workshop you could descend to the real attributes of QUINT.

LetsGrow WordPress Weather Plugin

The sidebar of this page shows the LetsGrow.com WordPress Weather plug-in. This WordPress plug-in shows a map of the Netherlands together with real-time temperature and radiation data. The plug-in retrieves the data directly from LetsGrow.com using LetsGrow Services.

LetsGrow Services is a module that enables customers or partners of LetsGrow.com to develop third-party applications that use the data from the LetsGrow platform. The LetsGrow.com WordPress Weather plug-in shows the power and flexibility of the LetsGrow.com platform.

LetsGrow ServicesLetsGrow has been collecting data from different brands of climate computers and plant sensors for over five years. A customer can use and integrate this information into custom applications through LetsGrow Services.

As the figure shows LetsGrow Services run on machines in the LetsGrow Data Centre. Businesses can create applications that consume these services and offer new services to their customers.

Here are some examples of the kinds of applications that could be built using LetsGrow Services.

  • A research institute or university could combine the large amount of historically data that is available in LetsGrow. This may lead to better understanding and insights in the growing process of crops that can help reduce the energy demand of a greenhouse. Note that a customer maintains ownership of the data in LetsGrow. First, a customer gives explicit consent to use the data. Then a  research institute can use the data.
  • A horticultural advisor of a customer could extract the data of the customer and perform calculations on the data. The results of these calculations could be stored in LetsGrow using LetsGrow Services. A customer can see the original data combined with the results of the calculation in one single view.
  • Affiliates or partners of LetsGrow could create custom modules using LetsGrow data. For example, a flowering prediction module that has a model that predicts the flowering date of a crop. These extra modules may even lead to automatic changes in the greenhouse settings to aim for a specific flowering date.  These extra modules could be offered to the customers of LetsGrow.

For more information about LetsGrow Services take a look at the document pdf documentIntroducing LetsGrow Services (PDF, 750K) which describes LetsGrow services in more detail.

Semantic Architecture?

Ok, but how is this related to Semantic Architecture and the Semantic Web? It isn’t! Currently the LetsGrow platform doesn’t use a Semantic Architecture or Semantic Web technologies. In the next post I will explain how a Semantic Architecture could be used to solve the problems I had during the implementation. Also I will describe the details about how I created this WordPress plug-in.

Calculate sunrise, sunset and maximum solar radiation

I got some requests for releasing the library that calculates the sunrise, sunset and maximum solar radiation of the LetsGrow WordPress Weather plug-in. I extracted the algorithms from the implementation on the platform LetsGrow.com and decided to release it.

It is possible to calculate the sunrise, sunset and maximum solar radiation using some well-known formulas. For those of you who are interested in these formulas take a look at the following pages at Wikipedia (Declination of the Sun, Sunrise, Sunset). If you just want to calculate the sunrise, sunset and maximum solar radiation take a look below on how to use the code.

I wrote the code in C# and packed it in a Visual Studio 2008 solution. It has two assemblies, Astronomy and AstronomyTest. The assembly Astronomy has the SolarCalculator class which performs the real calculation. The assembly AstronomyTest has several unit-tests that validates the calculations against external sources. The SolarCalculator class needs the longitude, latitude, time-zone and whether to use daylight savings. You can create  an instance of the SolarCalculator like this:

const Double Longitute = 5.127869;
const Double Latitude = 52.108192;
const int LongituteTimeZone = 15;
const bool UseSummerTime = true;
SunCalculator calculator = new
  SunCalculator(Longitute, Latitude,
  LongituteTimeZone, UseSummerTime);

The actual calculation of sunrise, sunset and maximum solar radiation can be seen below.

DateTime sunRise = calculator.CalculateSunRise(
   new DateTime(2010, 4, 1));
DateTime sunSet = calculator.CalculateSunSet(
   new DateTime(2010, 4, 1));
Double maximumSolarRadiation =
   calculator.CalculateMaximumSolarRadiation(
      new DateTime(2010, 1, 1, 1, 3, 0));

The DateTime returned from CalculateSunRise and CalculateSunSet includes the sunrise and sunset time. For more information take a look at the unit-tests in the assembly AstronomyTest which explain the usage in more detail.

I added an article to The Code Project which also explains the usage of the library including a download link of the source code.

Code Project Logo

Dynamic WCF proxy

Most programmers that start with Windows Communication Foundation (WCF) use the provided samples and tutorials from Microsoft. Most of these samples are based on static generated service references or proxies.

When you want to connect from a client to a server, you have to create a reference from the client to the server project using the Visual StudioAdd Service Reference function. Visual Studio then connect to the server, extracts the meta data from the server and generates a proxy. The client uses this proxy to actually connect and call methods on the server.

Most programmers don’t know, that there is also a way for WCF to generate this proxy dynamically. This dynamic generation of a proxy has a lot of advantages.

Background

As we all now, sometimes during development an interface of a service changes. The same is possible for the configuration of a service, for example the port number on which a service listens could change. Now say that a developer changes the interface or the configuration of a service and somehow he or she forgets to change the client (it has happened to me more than once). With a statically generated proxy you would notice the error late in the development cycle, probably during the first test.

This scenario has happened to me many times and lead me thinking that there had to be another way. Wouldn’t it be better to tackle this problem during compilation on your local workstation or during a build on your build server? With dynamically generated proxy this is possible.

Structuring your assemblies

To be able to use the dynamic proxy and it’s advantages, you have to structure your assemblies or classes somewhat different. Both the client and the server reference and use a separate assembly that contains the interface and the configuration of the WCF service. The picture below shows the idea.

Structure Assemblies

The assembly called “Interface + Configuration” has the WCF configuration of the server, including the interface itself. By structuring your assemblies this way it is not possible to forget changing the client when the interface changes. It simply won’t compile.

Using the code

The source code contains three assemblies, the server, the client and the interface / configuration assembly all within a single Visual Studio 2008 solution. When you run the application both the server and client will start. The client sends a couple of messages to the server.

Generating the dynamic proxy using WCF is not difficult, it uses the ChannelFactory from WCF. The source code to generate a dynamic proxy is shown below

The Client

EndpointAddress endpointAddress =
   new EndpointAddress(Configuration.CreateServerUriString());
ChannelFactory channelFactory =
   new ChannelFactory(Configuration.CreateBinding(),
     endpointAddress);
IMessageSender messageSender =
   channelFactory.CreateChannel(endpointAddress);
messageSender.ShowMessage("Test message");
((IChannel)messageSender).Close();

The ChannelFactory uses the interface IMessageSender to create a channel through the method CreateChannel. The interface IMessageSender and the Configuration are both from the shared assembly. The server side also uses the information from the Configurationclass.

The server

Server singleServerInstance = new Server();
Uri baseUri = new Uri(Configuration.CreateServerUriString());
ServiceHost serviceHost =
  new ServiceHost(singleServerInstance, baseUri);
serviceHost.AddServiceEndpoint(typeof(IMessageSender),
   Configuration.CreateBinding(), baseUri);
serviceHost.Open();
Console.ReadLine();
serviceHost.Close();

The method Configuration.CreateBinding() creates the binding. This method returns an instance of a specific binding which is configured in the configuration class. If you change the binding of the server. For example, from http to tcp, both the client and the server will automatically use it.

Points of Interest

This solution is not exactly rocket science, but it has helped me and my company save time in the past. Which is the reason that I want to share this information with you. When someone forgets to change the client after an interface change, the solution won’t compile. This way it fails fast which is a good thing. Another advantage is that you could stop publishing meta data of your service, this will increase the security of your service.

Disadvantages

This solution has its disadvantages, it works only if you control both sides of the wire. If you publish a service for usage outside your company, it’s a different scenario. You could publish the interface and configuration assembly, but I think that enabling meta-data on your service would be easier for such a situation.

The source-code of this article is available on the CodeProject.

Posted in WCF

PDF reporting using ASP.NET MVC

Introduction

Almost every web application (that I have built), needs a reporting option. Many times the client wants to generate a PDF report that corresponds with the web page he or she is viewing. Using iTextSharp, a free C# PDF library this is possible. This article describes and includes a solution for creating reports using iTextSharp and ASP.NET MVC3.

Background

iTextSharp is a free C# PDF library that is ported from the Java-PDF Library iText. iText was launched in 2000 and is a popular open source Java library for programmatic creation and manipulation of PDF. As with many successful open-source Java libraries it was ported to C# under the name iTextSharp. iTextSharp has been under development since 2008 and is distributed under the GNU Affero General Public License version 3.

At its core iTextSharp is an api to manipulate PDF creation. For a recent project I was looking for a solution that would enable me to create PDF documents from existing HTML documents or Views used in an ASP.NET MVC3 project. The ideal solution for me would be to throw an existing ASP.NET MVC3 view at it and let it generate a PDF that is exactly the same as the HTML rendered by a browser.

The solution described here uses the HTMLWorker class from iTextSharp to generate a PDF from an HTML view. HTMLWorker is a iTextSharp class that is able to parse an HTML document and generate a PDF using the what is called the SimpleParser of iTextSharp. Notice the name SimpleParser which indicates that not all HTML elements and css styles are supported. However, I was able to get good results using this solution.

To add reporting to your ASP.NET MVC3 project, first add the PDFReportGenerator project to your solution and add the iTextSharp.dll binary to your solution. PDFReportGenerator needs a reference to the binary. By adding the binary to the solution you are able to build the project directly from source control. Check the demo src for an example.

Creating a report.

Follow the following steps to generate an actual report from your web application.

  1. Create a controller that derives from PdfViewController.
  2. Create a view that generates the HTML which should be translated to a PDF report.
  3. Create an action on a controller which calls the ViewPDF method on the PdfViewController.
  4. Create a link to trigger the action on the controller.

The following paragraph describes these steps in more detail.

Create a controller that derives from PdfViewController

Create a new or reuse an existing Controller and let it derive from PdfViewController from the PdfReportGenerator project.

This enables your controller to call the ViewPDF method of the PDFViewController which generates the real PDF. In the demo project this is the HomeController.

Create a view that generates the HTML

Create a View that should be translated to a report. This could be an existing view or a new view specially for reporting. I usually create a new view as it lets me control the HTML markup for the report. As stated earlier the report generator does not support all the HTML markup. In the demo project this is the PrintDemo view.

Below the PrintDemo view from the demo project is shown. This is just a simple ASP.NET Razor view with a table and some rows. It uses a strongly typed model but that it not necessary. A tip when trying to design your report is to add borders to your table or div. Using these borders when looking at the generated PDF you can clearly see the start and end of the areas of your report.

@using MvcReportGeneratorDemo.Models
@model CustomerList
<br />
<table cellpadding="3" cellspacing="3">
    <tr border="1" bgcolor="#777777" color="#ffffff">
        <td>Name</td>
        <td>Address</td>
        <td>Place</td>
    </tr>
    @foreach (Customer customer in Model)
    {
        <tr border="1">
            <td>@customer.Name</td>
            <td>@customer.Address</td>
            <td>@customer.Place</td>
        </tr>
    }
</table>

Create an action which calls the ViewPDF method

The PdfViewController class from which your controller derives, contains a ViewPDF method. This method has the following signature.

protected ActionResult ViewPdf(string pageTitle,
            string viewName, object model)

Parameters

pageTitle
Type: System.StringThe title of the report which appears on the header of the page.
viewName
Type: System.StringThe name of the view which should be converterd to a report.
model
Type: System.Object

The model that is rendered by the view.

This methods generates the HTML view and converts into a PDF report and sends this PDF as a binary stream back to the client. This means that when the client has a PDF plugin installed, the PDF is shown inside the browser.

From an action inside your controller this method should be called to generate the report and send it to the cliënt. The following action from the Demo application generates the PDF. “Customer report” is the title of the report, “PrintDemo is the name of the view and the model is returned by the CreateCustomerList() which as its name implies generates a dummy list with customers.

public ActionResult PrintCustomers()
{
   return this.ViewPdf("Customer report",
        "PrintDemo", CreateCustomerList());
}

The last step is to create a link on a page that calls this action to actually print the report.

Trigger the action on the controller

A simple method to create a link to trigger the action on the controller is by using an ActionLink. This link calls the action that we defined on the controller.

@Html.ActionLink("Print customers",
      "PrintCustomers", null,
      new { target = "_blank" })

These are the steps you need to create PDF reports from your ASP.NET MVC3 projects.

Read the next part of the article if you are interested how into the details of how the PdfReportGenerator actually converts the ASP.NET MVC3 view into a report.

Detailed Reporting Project Overview

The PdfReportGenerator project consist of 6 classes which can be seen in the image below. The PdfReportGenerator assembly works by rendering the ASP.NET MVC3 view into a string and converting this string with HTML using the iTextSharp into a PDF report.

Class model used for PDF reporting

 

Rendering an ASP.NET MVC3 view into a string

The class HtmlViewRenderer is responsible for rendering the ASP.NET view into a string. The method RenderViewToString has the following signature.

public string RenderViewToString(Controller controller,
      string viewName, object viewData)

The arguments are viewName which is the name of the view that should get rendered to a string including the model viewData that is needed by the view. The controller is necessary

to be able to use to render the view using the view engine of ASP.NET MVC.

Convert the html string into a PDF byte array

Once we have the html in a string the StandardPdfRenderer converts the html string into a PDF byte array. The class StandardPdfRenderer has a method Render with

the following signature.

public byte[] Render(string htmlText, string reportTitle)

The htmlText is the rendered view as a string that was produced by the HtmlViewRender, the pageTitle is as the name suggests the title of the report.

Send the byte array back to the client as a stream

The last step is to convert the byte array into an instance of the BinaryContentResult class. The class BinaryContentResult derives from the ASP.NET MVC ActionResult. It overrides the ExecuteResult and returns the content as a binary stream.

The class PdfViewController is the class that combines these classes. The method ViewPdf uses all the three previously mentioned classes to generate the PDF as shown in the code below

protected ActionResult ViewPdf(string pageTitle,
                    string viewName, object model)
{
 // Render the view html to a string.
 string htmlText =
       this.htmlViewRenderer.RenderViewToString(this,
           viewName, model);
 // Let the html be rendered into a PDF document
 // through iTextSharp.
 byte[] buffer = standardPdfRenderer.Render(htmlText,
           pageTitle);
 // Return the PDF as a binary stream to the client.
 return new BinaryContentResult(buffer,
           "application/pdf");
}

Points of Interest

Colors

iTextsharp supports color out of the box, in the demo application the background colors of the rows are alternated using different colors. These colors are visible in the report.

New page support

One thing that I needed with my project that was not supported by the HTML conversion in iTextSharp was functionality to force a page break.

Most of the time with reporting you need to be able to force a page break, for example if you want a graph to start always on a new page. The way I solved this was to add support for it to iTextSharp.

As it is open-source you are able to add new features to it. I added support for a non existing HTML tag called <np /> which forces iTextSharp to create a new page. I created a patch so that it could be committed to the iTextSharp project.

I do not know if it will be included in the main trunk of the project as it feels somewhat strange to invent new html tags just to support a page break. But if you need it, you can use the patch to compile iTextSharp with new page support.

Images

It is possible to add images to the report by using an <img src="" /> tag. I had no success with dynamically generated images.

So therefore, I generated the images that I needed before the action conversion process took place.

ASP.Net Support

It should be possible to use the same kind of solution using older versions of ASP.NET MVC. However, I have not tried it.

The source code of the solution can be found on CodeProject

Other Options

After I wrote this post several people mentioned other alternatives for generating PDF reports using ASP.NET MVC.

Michael Johnson wrote a blog post about using WkHtmlToPDF and someone mentioned using Flying Saucer for generating PDF’s.

10 Tips to improve your Software Architecture Design

You can improve the quality of your Software Architecture Design by using the following 10 tips. I use these 10 tips or guidelines daily and they have helped me creating high quality Software Architectures. Describing your software architecture design is useful for any type of project, it will share the design of the system among your stakeholders.

My 10 ways:

  1. Based on non functional requirements
  2. Rationale, rationale, rationale
  3. Don’t Repeat Yourself
  4. Slice the cake
  5. Prototype
  6. Quantify
  7. Get it working, Get it right, Get it optimized
  8. Focus on the boundaries and interfaces
  9. The Perfect is the enemy of the Good
  10. Align with your stakeholders

1. Based on requirements

You should base your software architecture design on the requirements of your stakeholders. An architecture focuses on the non-functional requirements. I see many software architecture designs based on purely technical motives. Each part of your design should be based on business requirements. You as an architect should translate these requirements into the right architectural design decisions. If the stakeholder values maintainability, you could use the layer pattern to separate several parts of the application. If performance is important, maybe layering is not a good solution. An exhaustive list of non-functional requirements can be found at ISO 9126 and at QUINT. If you do not use Non-functional requirements in your organization but want to introduce them,  take a look at this post.

Base architecture on requirements From the Non-functional requirements or quality attributes you have to create the right design. While you could create this from scratch there are many examples in the form of design patterns or architectural patterns. A design or architectural pattern expresses a relation between a problem and a solution. Although we often think that our problem is unique this is often not the case. If you take a step back you will see that many of our problems already have been solved using existing patterns. Two books that I can recommend are “Pattern-Oriented Software Architecture” and “Design Patterns”. Both books contain a catalog patterns. Each pattern describes the problem it solves and in which context it can be used. There are also many online pattern source on the web such as this one on Wikipedia and this from The OpenGROUP

2. Rationale, rationale, rationale

The most important aspect of your architecture description is the recording of your rationale behind your design decisions. It is important for a reader of the architecture description to understand the reason why you made a specific decision. Make your assumptions explicit and add them to the description. Assumptions may be invalid now or later but at least it will be clear how you came to that decision. It make communicating with your team (you do communicate do you?) that much easier if you share your rationale.

Note that recording your rationale becomes much easier if your non-functional requirements are explicit. It will be much clearer if you describe that you created several components to increase the testability because testability is the most important requirements. Do describe the Why and How in your software architecture design!

3. Don’t Repeat Yourself (DRY)

Don’t Repeat Yourself (DRY) or Duplication Is Evil (DIE) comes from software-engineering in general. The DRY principle is stated as “Every piece of knowledge must have a single, unambiguous, authoritative representation within a system”. You can apply this principle on many levels; Architecture, Design, Testing, Source Code and Data. For me this is one of the most difficult things to uphold. You have to fight the repetition because it will slow you and your project down. The difficult part of this Repetition Creep as I call it is that it is introduced very slowly. The first repetition won’t hurt you directly, it will even gain some time. You are able to release the first version of the application somewhat quicker, but as I found it always shows up later and makes something else more difficult. At that moment you regret the decision to introduce repetition.

 

If you absolutely must add another copy of information make sure that you automatically generate that copy of the information. It will make your live so much easier in the future. One thing that helps to fight repetition is to store the data where it belongs. This seems logical and is the basis of object oriented design but I often see this violated with regards to system architectures. For example take packaging an application for deployment. The process in which you filter the build of your software to include the components that are necessary in a package. Where would you store the information which component should be included in the package? You could create a list that includes the names of the components that should be packaged. That means you introduce your first repetition! You now have two places where component names are mentioned. A better solution would be to add that information to the component itself.

When the first list in any format shows up in or around an application, alarm bells should sound and you should be on the lookout for repetition!

4. Slice the cake

I struggled with naming this, but found Slicing the cake as it is called in Agile development the best description. By slicing the cake I mean that you design your architecture iterative in vertical slices. An architect implements or prototypes each vertical slice to confirm if it actually works. You should do this because architectures cannot be created on paper. It does not mean that you cannot use horizontal layering or any other pattern in your architecture. In the case of layering the horizontal layers are smaller. The picture below shows the principle.

Say you use layering in your architecture design because your stakeholders expect that the components that you develop for this system will be used in other systems as well. During the first iteration you design a small part of the User Interface (UI), a small part of the Business Layer (BL) and a small part of the Data Layer (DL). You make sure that this works as expected by proving it with a prototype or by actually implementing it. In the second iteration you add new functionality and expand each layer horizontally with the needed functionality. The existing UI, BL and DL are combined with the new UI, BL and DL to form the new layers.

The difficulty with slicing is how to slice the cake so that the next slice will properly align with the previous.

5. Prototype

When creating a software architecture design make sure that you prototype your design. Validate your assumptions, do that performance test and make sure that the security architecture is valid. A prototype will give you the opportunity to fail fast which is a good thing.

6. Quantify

This principle extends the first principle “Based on Requirements”. To be able to create a proper software architecture design you need to quantify your Non-functional requirements. It should be “fast” cannot be a requirement neither is maintainable or testable. How will you know if you have met these requirements? You won’t.

ISO 9126 and QUINT both describe ways to quantify the non-functional requirements. For example testability specifies an indicator “number of test cases per unit volume”. QUINT also specifies how you can actually measure an indicator for example the indicator “Ratio Reused Parts” from the quality attribute Reusability which you can measure using the following protocol:

  1. Measure the size of each reused part;
  2. Measure the size of the entire software product;
  3. Calculate the ratio of reused parts, which is the sum of reused parts divided by (2).

7. Get it working, Get it right, Get it optimized

In many projects I have seen architects and developers design software architectures that focus on creating general purpose libraries, services or infrastructure. These are created without a direct reference to a concrete application. Instead they are designing for tomorrow. This for me is like walking backwards, generality cannot be designed up-front! If you think, well… stop! you actually can’t. Today’s businesses change way too fast to design for generality up-front.

You should always start with a concrete implementation for a specific problem. At the time you start working on the next application and find similarities, that’s the time to think about generalizing. This makes the first solution simpler, which should be your design goal.

8. Focus on the Boundaries and Interfaces

When creating your software architecture design you should focus on the boundaries of your system and components. When starting blank you should think about separation of concerns. What component or system has which responsibility? Between the components or system design explicit interfaces. Don’t separate a system of component when a lot of communication is necessary between these components or systems.

9. The Perfect is the enemy of the Good

The phrase “The perfect is the enemy of the good” from Voltaire is also valid for software architecture design. How many times have you started a new project and thought I want this project to be perfect? And how many times have you actually found out that the project wasn’t perfect. Well, guess what – a project will never be perfect. There will always be problems, forgotten requirements or wrong design decisions.

 

Perfection is never possible. However you are able to create a good software architecture design. Do not try to analyze everything during the start of the project it will slow you down. Watch out for Analysis Paralysis.

10. Align with your stakeholders

Before you can create any type of system you need to identify your stakeholders. Each stakeholder has different needs of your software architecture and may require a different view. Software developers may need descriptions using Unified Modeling Language (UML) while business sponsors need a description in natural language. Operations and support staff for example may need other view such as context diagrams.

There is a tension between creating all these views for stakeholders and principle 3. Don’t Repeat Yourself. Each view essentially describe the same system and adds repetition. Therefore you should only add those descriptions that adds value for a specific stakeholder.


Well there you have it, my 10 tips to improve your Software Architecture Design. If you have another tip that you use to improve your architecture design let me know in the comment!