Abstract form

Mutually Dependent Systems

Posted in Architecture, Development, Modeling by Homam Hosseini on March 11, 2010

In this post I am discussing a problem that I have faced several times in the past year. Simplicity is always a goal in design as it saves resources during development and maintenance. But it’s not always clear which design is simpler. Sometimes a seemingly complex design turns out to be simpler to develop, maintain and extend.

In a master-slave architecture, assume S1 is the master. It produces one or more tasks form a given job and transfers them to S2 (the slave); S2 does the tasks and return the results back to S1. S2, the slave, depends on S1, the master.

If the next time that S1 assigns a task to S2 it uses the information that exists in the result of a previous task that had been assigned to S2 then S1 also depends on S2 and we have a mutually dependent couple.

In our terminology the systems are mutually dependent if and only if S1 uses the information it gained as a result of a previous task that it had already assigned to S2. It doesn’t matter if S2 has completed the previous task or not, but it should have reported something to S1 that is useful for S1 for a next assignment of a task to S2.

If S1 is only using the fact that S2 is busy or free then we don’t call it a mutual dependency. S1 must use the information that is generated by processing a task at S2. For example a MapReduce system is not a mutually dependent system.

Why is it important? You should have already guessed that S2 is the name of a class of slave systems that work with S1. There could be many instances of S2. Let’s define a homogenous mutually dependent system as a system that in which all slaves of S1 are in the same class.

Two slaves are of the same class if they share a common interface for communicating with S1.

Now assume that S3 is also a slave for S1. S3 is in a different class other than S2 if either its input or its output interface is different from S2’s.

When designing mutual dependent systems we have to always decide whether to keep the mutual dependencies or to break them by introducing new nodes. It’s mainly a decision over complexity. The other factor that may affect your decision is the swiftness of the system. Introducing a new node will usually reduce the responsiveness.

Breaking the mutual dependency by using S4 node. Note that S3 is another class and uses a different interface to communicate with S4.

For instance a new node must not be added if S1 waits for S2 to return. Generally you should try to keep the number of nodes as small as possible if the operations are not asynchronous.

Homogenous mutual dependency is OK (when the systems are simple and synchronous) but things get much dirtier as we introduce new classes to the system. On the other hand if extensibility is a goal you should try to avoid mutual dependencies.

For a conclusion, use mutual dependent systems in live systems, when a rapid response is required, and try to avoid them by introducing middle nodes if you have many classes of slaves or if extensibility is a goal.

Accessing Remote ASP.NET Web Services using JSONP

Posted in Coding, Development, JavaScript by Homam Hosseini on October 12, 2009

The problem:

You cannot call remote ASP.NET web service methods from a JavaScript, AJAX client.

Example:

You have a web service, at this address: http://a.com/service.asmx and you’ve configured the service to work with AJAX clients:

[WebService
(Namespace = "http://www.hyzonia.com/gametypes/PopNDropLikeGame/WS2")]
[WebServiceBinding(ConformsTo = WsiProfiles.BasicProfile1_1)]
[System.Web.Script.Services.ScriptService]
public class GameService : System.Web.Services.WebService
{
    [WebMethod(EnableSession = true)]
    public GameSessionResponse CreateGameSession(Guid questId)
    {
...
    }
}
}

And it works fine when you call its methods from a web page that is in this address: http://a.com/page.htm:

$.ajax({
        type: "POST",
        url: "GameService.asmx/CreateGameSession",
        data: "{questId: '" + questId + "'}",
        cache: false,
        contentType: "application/json; charset=utf-8",
        dataType: "json",
        success: function(response) {
            Game._onSessionGot(response.d);
        }
    });

But the very same client-side code doesn’t work from this address: http://b.clom/page.htm

The problem in depth:

At first I cannot hold myself and not say that it is a silly problem. web services are meant to be called by remote clients. The fact that browsers block access to web services by AJAX calls is clearly contrary to the purpose of web services.

Interestingly browser extensions like Flash and Silverlight also by default block remote web services, but they provide a work around. Unfortunately no browser by date supports this work around for XMLHTTPRequests. This “security measure” seems odder when we notice that it is perfectly correct to import a JavaScript code snippet from another domain by a script tag:

<script
src="http://ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.min.js"
type="text/javascript"></script>

 

The solution:

As it was said, Flash and Silverlight both support remote calls. You just need a clientaccesspolicy file to be hosted at the root of a.com (http://a.com/clientaccesspolicy.xml):

<?xml version="1.0" encoding="utf-8"?>
<access-policy>
  <cross-domain-access>
    <policy>
      <allow-from http-request-headers="SOAPAction">
        <domain uri="*"/>
      </allow-from>
      <grant-to>
        <resource path="/" include-subpaths="true"/>
      </grant-to>
    </policy>
  </cross-domain-access>
</access-policy>

This file allows remote calls to be made from any other domain.

But in many situations we want to call the web service methods directly by AJAX clients. This need was the cause of to the development of JSONP (JSON with padding) protocol. As it was discussed it is correct to have a <script> element that loads a script from another domain. On the other hand you may know that it is possible to load scripts dynamically by a simple JavaScript trick (writing<script> tags) or using this jQuery plug in. Now the bulbs are flickering! The solution is to access the JSON web service by the src attribute of a <script> element. This is the whole idea behind JSONP.

But there are a couple of problems needed to be solved for ASP.NET ASMX web services before we can use them in a JSONP scenario.

  1. ASP.NET web services by default only accept POST requests, a <script src=””> element, produces a GET request.
  2. The result of the web method call must conform to JSONP, and you guess, ASP.NET 3.5 by default doesn’t support it.

The solution to the first problem may seem trivial, we can easily enable GET calls to web methods using [ScriptMethod(UseHttpGet = true)] attribute. The immediate problem is that when we mark a web method by this attribute it only can be called by GET requests. And remember, other clients (actually anything other than JSONP clients) are supposed to communicate with the web service by POST requests. I usually end up inheriting from the original web service and marking web methods by [ScriptMethod(UseHttpGet = true)] attribute in the derived class. Therefore I will have two ASMX web services, one using the original class (expecting POST request) and the other using the derived class (expecting GET requests).

[WebMethod(), ScriptMethod(UseHttpGet = true)]
public override GameSessionResponse CreateGameSession(Guid questId)
{
   return base.CreateGameSession(questId);
}

Note you may need to add this code snippet in web.config:

<system.web>
 <webServices>
   <protocols>
     <add name="HttpGet"/>
   </protocols>
 </webServices>
…
</system.web>

There’s another problem to be addressed in the client side. The client should call the web method by a correct URL (it has to pass a correct query string that could be deserialized back to .NET objects in the server side). In case of POST requests, I’m used to JSON2 library to post data to ASP.NET ASMX web services. Jquery $.AJAX method (when it is configured to use JSONP, by dataType: “jsonp”) creates query string parameters for the data object it receives. But the result is not usable for ASMX web services.

Luckily there’s a ready to use JQuery plug in (jMsAjax) that has the required algorithms for serializing a JavaScript object into a query string that can be parsed by ASP.NET web services.

Using the plug in I created this function to serialize JavaScript objects into query strings:

$.jmsajaxurl = function(options) {
    var url = options.url;
    url += "/" + options.method;
    if (options.data) {
       var data = ""; for (var i in options.data) {
       if (data != "")
         data += "&"; data += i + "=" + msJSON.stringify(options.data[i]);
       }
       url += "?" + data; data = null; options.data = "{}";
   }
   return url;
};

You will need jMsAjax for this code snippet to work.

Finally this is a sample of a client side code using JQuery that calls an ASMX web service using JSONP:

var url = $.jmsajaxurl({
    url: "http://hiddenobjects.hyzonia.com/services/GameService3.asmx",
    method: "Login",
    data: { email: "myemail@mydomain.com", password: "mypassword" }
});

$.ajax({
    cache: false,
    dataType: "jsonp",
    success: function(d) { console.log(d); },
    url: url + "&format=json"
});

Or equivalently:

$.getJSON(url + "&callback=?&format=json", function(data) {
    console.log(data);
});

When you call an ASP.NET web service method (that is configured to receive GET requests) using a code similar to the above, it returns in XML. The problem is that the web service expects to receive a request that has a content type of “application/json; charset=utf-8” and <script> element simply doesn’t add this content type to the request. There’s a little thing we can do at the client side. The easiest way to resolve this problem is to use a HTTP module. The HTTP module should add this content type to the requests before they are processed by the web service handler.

On the other hand a JSONP client expects that the web service return the call by a string like this:

nameOfACallBackFunction(JSON_OBJECT_WEB_METHOD_RETURNED)

nameOfACallBackFunction must be given to the server by a parameter in the query string. Different JSONP compatible web services use different names for this parameter, but usually it is named ‘callback’. At least this is what $.ajax() automatically adds to the request in JSONP mode.

I grabbed this HTTP module from a post in elegantcode.com.

public class JsonHttpModule : IHttpModule
{
    private const string JSON_CONTENT_TYPE = "application/json; charset=utf-8";

    #region IHttpModule Members
    public void Dispose()
    {
    }

    public void Init(HttpApplication app)
    {
        app.BeginRequest += OnBeginRequest;
        app.ReleaseRequestState += OnReleaseRequestState;
    }
    #endregion

    bool _Apply(HttpRequest request)
    {
        if (!request.Url.AbsolutePath.Contains(".asmx")) return false;
        if ("json" != request.QueryString.Get("format")) return false;
        return true;
    }

    public void OnBeginRequest(object sender, EventArgs e)
    {
        HttpApplication app = (HttpApplication)sender;

        if (!_Apply(app.Context.Request)) return;

        if (string.IsNullOrEmpty(app.Context.Request.ContentType))
        {
            app.Context.Request.ContentType = JSON_CONTENT_TYPE;
        }
    }

    public void OnReleaseRequestState(object sender, EventArgs e)
    {
        HttpApplication app = (HttpApplication)sender;

        if (!_Apply(app.Context.Request)) return;

        app.Context.Response.Filter = new JsonResponseFilter(app.Context.Response.Filter, app.Context);
    }
}

public class JsonResponseFilter : Stream
{
    private readonly Stream _responseStream;
    private HttpContext _context;

    public JsonResponseFilter(Stream responseStream, HttpContext context)
    {
        _responseStream = responseStream;
        _context = context;
    }

    public override bool CanRead { get { return true; } }

    public override bool CanSeek { get { return true; } }

    public override bool CanWrite { get { return true; } }

    public override long Length { get { return 0; } }

    public override long Position { get; set; }

    public override void Write(byte[] buffer, int offset, int count)
    {
        var b1 = Encoding.UTF8.GetBytes(_context.Request.Params["callback"] + "(");
        _responseStream.Write(b1, 0, b1.Length);
        _responseStream.Write(buffer, offset, count);
        var b2 = Encoding.UTF8.GetBytes(");");
        _responseStream.Write(b2, 0, b2.Length);
    }

    public override void Close()
    {
        _responseStream.Close();
    }

    public override void Flush()
    {
        _responseStream.Flush();
    }

    public override long Seek(long offset, SeekOrigin origin)
    {
        return _responseStream.Seek(offset, origin);
    }

    public override void SetLength(long length)
    {
        _responseStream.SetLength(length);
    }

    public override int Read(byte[] buffer, int offset, int count)
    {
        return _responseStream.Read(buffer, offset, count);
    }
}

This HTTP module will be applied to each request to a .asmx file that has a format=json in its query string.

Note that you have to update web.config:

<system.web>
…
  <httpModules>
    …
    <add name="JSONAsmx"/>
  </httpModules>
</system.web>

For IIS6 and

<system.webServer>
  <modules>
  …
    <add name="JSONAsmx"/>
  </modules>
  …
</system.webServer>

For IIS7.

Now to test is open your web service in your browser, in my example:

http://hiddenobjects.hyzonia.com/services/GameService3.asmx/Login?email=e@e.com&password=p

It should return in XML

And

http://hiddenobjects.hyzonia.com/services/GameService3.asmx/Login?email=”e@e.com”&password=”p”&format=json&callback=myCallBackFunc

Will return:

myCallBackFunc({"d":{"__type":"HLoginResponse",
"isSuccessful":false,"error":false,"authSessionId":null,"nickName":null,"score":0}});

Don’t worry about myCallBackFunc, JQuery nicely manages it, so that the whole business is behind the scene and you can use $.ajax success callback the very same way you use it for a normal AJAX call.

We should note that JSONP has its own problems, especially… yes… in IE! All versions of Internet Explorer has a 2083 character limit for the URL of a request. It means that you cannot send large data in GET requests to the server. Sometime this limitation leaves us with no choice but to use Flash or create a proxy to the remote web service in the local domain.

Service Provider Pattern Part II

Posted in Architecture, Coding, Development by Homam Hosseini on May 29, 2009

Read the first part

At first I found it is generally a good advice to store the configuration in a static, read-only property somewhere. And there’s no better place other than the configuraiton class itself:

static MyServicesConfigurationSection _Value;
public static MyServicesConfigurationSection GetValue()
{
   if (_Value == null)
   {
     _Value = (MyServicesConfigurationSection)
        System.Configuration.ConfigurationManager.GetSection("myServices");
   }
   return _Value;
}

Here we assume that we always define the matching element to be “myServices” in the configuration file. It makes sense because it makes it easier for us to find the element in different configuration files. We can always access the value using:

var config = Configuration.MyServicesConfigurationSection.GetValue();

Modularizing is the obvious upgrade we are goging to make to the Service Provider Pattern Part I. I can find three distinct components:

Service-Provider-Pattern-Ar 

  • Service Provider Pattern contains all the logic required by the pattern to work. You alway reference it everywhere you want to use the pattern.
  • Service Definitions is where we define our services.
  • Implementation of the Services contains  concrete classes and implementations of (some of) the services that have been defined in Service Definitions module. In a real application you may end up with many Implementation components each containing the implementation of a one or few number of services.

As before the configuration file of the Host Application maps the definitions (abstract services) to implementations (concrete services).

It’s clear that using this architecture we can have host applications that are hosting different implementations of the services.

Configuring the Services

Till now the only purpose of the configuration file was to store the mapping between the definitions and the implementations. But we can use it in order to actually configure the services. For an example it’s very likely that the FTPFileStorageService that we defined in the previous part, needs a FTP account credential to work. In this case the <add> element (that maps FileStorageService to FTPFileStorageService) may look like:

<add def="MyServiceDefinitions.FileStorageService"
     impl="MyServiceImplementations.FTPFileStorageService,
                                      MyServiceImplementations"
     host="ftp://ftp.wingooli.com" username="wingooloi"
                                         password="password" />

Both the abstract service definition and the concrete implementation can implement IServiceBase interface:

public interface IServiceBase
{
    void Initialize(Configuration.ServiceProviderSettings settings);
}

This interface just has one method, Initialize, that takes a ServiceProviderSettings (the <add> element). We can read the additional attributes of the <add> element using a code like this:

settings.Properties["host"]

The logics in the Service Provider Pattern component automatically call the Initialize method if this interface has been implemented in the definition the  implementation of the service.

Here is the design:

Service-Provider-Pattern

Now our Service Provider Pattern has nothing less than ASP.NET Provider Pattern but it is easier to implement.

Using all these modules we can reduce the code needed to instantiate a service (this code snippet should be placed in the abstract service definition class):

static FileStorageService _Instance;
static object _InstanceLocker = new object();
public static FileStorageService Instance
{
 get {
    if (_Instance == null) {
      lock (_InstanceLocker) {
        _Instance =  ServiceModel.ServicesHelper
                       .GetServiceInterface<FileStorageService>();
      }
    }
    return _Instance;
  }
}

Because it is a repeatable pattern and to make it all more developer friendly, I created a C# code snippet that generates a similar code automatically.

You can download the whole things here.

Service Provider Pattern, Part I

Posted in Architecture, Coding by Homam Hosseini on May 20, 2009

Read the Second and final part

Hyzonia is made of dozens of services running in many machines and app domains all over the internet. One problem we faced early in designing Hyzonia was the need to have different implementations for the a service and the ability to switch between the implementation as the need arises in the run time.

ASP.NET has already had a well-established Provider Pattern. But soon it turns out that it is not the best solution in our case. It seems ASP.NET Provider Pattern is not that developer friendly especially when you have dozens and increasing number of service definitions; because you have to code a new configuration class, create an abstract class, its implementation and a static helper class and it means you have to write the same methods in at least three places.

I wanted to let our developers to be able to add new services very fast. Here is a description of the provider pattern I came with.

Here I assume that services are stateless singleton objects, that is a true assumption in the context of Hyzonia and most of service oriented applications. That’s also true about ASP.NET providers. The main reason lies in the SOA paradigm, as in service oriented architecture we think of an operation as something that when it is given its required input arguments it does all the job and returns a success or a failure message. When a client calls an operation it passed all the job to the service and basically it disconnects and waits until the operation returns. The operations that perform an a business object should perform a meaningful business process. It means that those operations must transfer a business object to consistent state.

It is guaranteed that by obeying such those rules you will have a clear separation between your service/business logic and the clients (that are the users of the services).

How to

For adding a new service you need to create the service definition that is an abstract class. Here as a rule we store the service definitions in a seperate assembly that could be shared among many app domains. For our pattern to work we need to add a static getter member to the abstract definitions, just like what you may do for creating a singleton object. Let’s name it Instance, that is a getter-only property of the type of the abstract class itself. It’s obvious that we are going to load, instantiate and return the concrete implementation here. Let’s talk about it a bit later.

For an example, suppose that we are building a File Storage Service. There’s no doubt that the main functionalists of such this service are Storing and Deleting the files. So here is the service definition:

public abstract class FileStorageService {
   public static FileStorageService Instance {
     get { // I told you, we talk about it later }
   }
   public abstract string Store(Byte[] bytes, string extension);
   public abstract bool Delete(string url);
}

Suppose we have a FTP implementation the service: FTPFileStorageService:

public class FTPFileStorageService : FileStorageService {
//...
}

Now if we want to instantiate the FTPFileStorageSerivce in a host application, we need to modify the configuration file of the host app in a way to relate the service definition to our desired implementation. Here is the way we do it:

<service def="FileStorageService" impl="FTPFileStorageService"/>

Now let’s back to where we left our discussion about the static Instance property in the service definition class. We need to somehow read the configuration file, find the active implementation, instantiate and return it here. Here is the code:

static FileStorageService _Instance;
public static FileStorageService Instance {
  get {
    if (_Instance == null)
    {
        var name = (typeof(FileStorageService)).FullName;

        var myAppConfiguration =
            (MyApp.Configuration.MyAppServiceConfigurationSection)
            System.Configuration.ConfigurationManager
                              .GetSection("myAppConfig");

        var services = myAppConfiguration.Services;

        ServiceProviderSettings config = null;

        foreach (var s in services)
        {
            var serviceConfig = (ServiceProviderSettings)s;
            if (serviceConfig.Definition == name)
            {
                config = serviceConfig;
                break;
            }
        }

        if (config == null)
            throw new Exception(@"A service mapping is
                        missed in the configuration file");

        var type = Type.GetType(config.Implementation);
        _Instance = (FileStorageService)Activator.CreateInstance(type);
    }
    return _Instance;
  }
}

You may want to download a demo project here.

If you’re new to .NET configuration you’ll find the download a good demonstration for starting working with custom configuration classes.

In the second and the final part of this discussion you’ll see how we convert a good idea to a working one. The most obvious issue with the code above is here

System.Configuration.ConfigurationManager.GetSection("myAppConfig")

where we are coupling the abstract class (the service definition) and the configuration of the host application. I’m advocating the idea that service definitions must be and are in nature independent of their host.

I will talk about such these issues and making the whole thing more developer friendly in the next part. We will also add more features and try to make it really like a pattern.

Read the Second and final part