Mar
12
2012

Some background on why I like CQRS

It's funny that it seems that ideas recycle themselves.  When I first got into web development, we were using ASP, JavaScript, and COM+ (Remember MTS?).  I remember having long discussions with some talented coworkers about the best way to organize our logic.  We put as much business logic as possible into the COM+ layer which was a separate, hosted part of the application.  We built real separated business objects that we made available through services and which were hosted in a runtime.  We had a separate model for querying our data store.  We segregated all of our UI code into ASP and had the vast majority of our biz logic in the components.  We quickly realized how "spaghettiesque" the ASP code became so we started making heavy use of JavaScript and made out-of-band calls back to the server and manipulated the DOM to create a "sophisticated" user experience.  It was really cutting edge at the time.  But it had a lot of problems as well.  We were using VB6 for our COM+ objects, remember single threaded apartments?  And all of our COM+ transactions were serializable...yow.  Talk about headaches...there was no JQuery and yep...Netscape (hah) and IE 5/5.5 were both really hard to simultaneously support.  Despite our best efforts, ASP still became spaghetti in some places.  Unit testing?  Lol.  So yeah there were serious problems but there were some good ideas there. 

We adopted ASP.Net as early as possible.  In fact while it was still in beta we started rewriting our entire project.  We all saw the issues early on.  So many full postbacks and viewstate?  That sounds expensive.  Yet we suspended disbelief and trudged bravely forward.  We loved true object-oriented languages, but there was no separate server infrastructure unless you wanted to keep using COM+.  We wrote our biz logic on the web server and segregated it as best we could.  Typed datasets wrapped in objects and data adapters to return complex datasets.  We still exposed web services, which we thought was really cool, and made JavaScript calls to the server.  We got used to Web Forms and being separated from the DOM and HTML in many ways, even if we didn't like it.  We even wrote services for many functions and used MSMQ to create messaging to those services.  We did some cool stuff here...but interests creeped together.  There were a lot of business interests that snuck into the UI and M$ seemed to encourage a lot of this with all of the momentum that they were putting behind designers and drag-and-drop programming.  We succumbed to a lot of that even though we had serious reservations about it.  But there was real reuse, inheritance, real design patterns, unit testing!  Well .Net evolved and a lot of cool stuff happened....I'd never go back.  But M$ really got behind the curve with some bad decisions they made on a lot of stuff.  Competing with open-source instead of supporting it, continuing to think that designers were better than solid frameworks.  Luckily Alt.Net and influence from Java started to make a huge impact.  Some of the folks at M$ saw the light and started to move in the right direction.  The Alt ecosystem got huge because the core framework itself was so awesome and a lot of cool stuff happened and is still happening.  Just like in music, Alt became mainstream because there was no other real choice for people that truly love development.

So here we are now.....

What I love about CQRS is that it takes the best of my old (ancient) experiences and meshes them with the best of Alt.Net and open source initiatives. 

  • Separation of UI, Querying/Reporting, and business logic
  • Freedom to use a wide array of technologies from storage to messaging to UI
  • Lots of low cost open source options
  • Very testable code
  • If done correctly, the capability to scale massively
  • A small but growing energetic community

You'll hear a lot of people say that the physical separation of concerns is to complicated and hard to ramp up.  I never heard that in the old COM+ days (cause we had no choice).  That's just how we did things.  We physically separated our business logic from our presentation and querying.  But now it's so much better.  True mature .Net support in all areas, service buses, guaranteed messaging, real scalability, and frankly it's easier.  I remember how much energy I put into learning MTS and COM+ back in the day.  It was complicated to set up and run properly.  So yeah, there is some learning curve to using CQRS, but there's so much to gain.  I think we're finally to the point where the right ideas are meeting mature(ing) technologies.

There's lots of information out there:

http://skillsmatter.com/podcast/open-source-dot-net/ian-cooper-will-give-a-tutorial-at-the-progressivedot-net-tutorials

http://elegantcode.com/2009/11/11/cqrs-la-greg-young/

http://www.udidahan.com/2009/12/09/clarified-cqrs/

http://ncqrs.org/

https://github.com/joliver/EventStore

So what is the problem?

  • A lot of shops only do what M$ releases.  So it's still considered somewhat edgy.
  • Documentation, while present, is sparse and the frameworks, including CQRS, Service Bus, etc... have very little documentation.
  • It's very intimidating to get rolling, you mostly have to depend on yourself to make the right decisions.

So hopefully I can help to bridge this gap a bit and help others learn through my learning experiences.  In my previous post, I explain what I'm trying to accomplish.  Hope some people will join my attempts to get this right and steer me in the right direction when I get off track.

Happy coding!

Mar
9
2012

Using MassTransit 2.1 with StructureMap

There's not a lot of documentation out there on setting up MassTransit with StructureMap.  It's not that difficult, but figuring out the right combination to set things up left me pulling out my considerably thinning hair.  I've started with two basic projects:

1) Domain.Service - This actually holds all of the service logic, command handlers etc.... It's basically a MassTransit endpoint.

2) Domain.Service.Host - This is just a wrapper for TopShelf to host the service.  I try to put as little logic in here as possible except what is necessary to start up and dispose the project.

The most important thing that happens in Domain.Service.Host is the following which occurs in the Start() method.  This method is called by TopShelf to start up the service. 

private void Start()
{
    //Set up log4net    
    XmlConfigurator.Configure();
    Logger.Info("setting up domain service, installing components");

    _container = Bootstrapper.Bootstrap();
    _bus = _container.GetInstance<IServiceBus>();

    Logger.Info("application configured, started running");
}

Basically I get my StructureMap container from my Bootstrapper and then tell it to give me an instance of the bus.  OK, so the important stuff is actually happening in the Bootstrapper.  This and all other code displayed in this sample exists in the Domain.Service project.  There's not that much to the Bootstrapper.

public static class Bootstrapper
    {
        public static IContainer Bootstrap()
        {
            ObjectFactory.Initialize(x => x.Scan(s =>
            {
                s.AssembliesFromApplicationBaseDirectory(y => y.FullName.StartsWith("AthLog"));
                s.WithDefaultConventions();
                s.LookForRegistries();
            }));

            ObjectFactory.Configure(x => x.For<IContainer>().Use(ObjectFactory.Container));

            return ObjectFactory.Container;
        }
    }

So what is this doing?  It's initializing Structuremap's main container.  In doing so, it's telling the container to scan the application base directory for assemblies that start with our applications root namespace, in this case "AthLog".  It also tells Structuremap to use default conventions.  This means if it finds an interface called IFoo, it will automatically try to map it to an implementation class called Foo.  This gets rid of a lot of unnecessary registration effort.  Then it will also scan for Structuremap registries which we'll see further down and attempt to register anything that is specified in one of the registries.  Finally I go through the odd step of registering the container with itself as the thing to return when an IContainer is asked for.  You'll see the reason for this a little further down when we examine how the consumers (message handlers) are registered.  There may be a more elegant way to get to this without performing this registration, I just don't know how.

So how does the actual service bus get registered?  I mentioned above that the Bootstrapper scan configuration tells Structuremap to look for registries.  This is an easy way to divide out your registrations into functional units so there's not so much registration code in one place to wade through.  Just a way of modularizing the registration process.  So also in the Domain.Service project, we have a registry class called BusRegistry.

public class BusRegistry : Registry
    {
        public BusRegistry()
        {
            For<IServiceBus>().Use(context => ServiceBusFactory.New(sbc =>
            {
                sbc.UseRabbitMq();
                sbc.ReceiveFrom(ConfigurationManager.AppSettings[Constants.BusEndPointUri]);
                sbc.UseRabbitMqRouting(); 
                sbc.Subscribe(c => c.LoadFrom(context.GetInstance<IContainer>()));
            }));

            Scan(x =>
                     {
                         x.TheCallingAssembly();
                         x.AddAllTypesOf<IConsumer>();
                     });
        }
    }

The bus registry performs two basic tasks.  The first is that it tells Structuremap how to build our bus for us.  Here I'm setting it up to use RabbitMQ, setting it's receive queue and then telling it what messages it should subscribe to.  Here is where that container self-registration from above comes in handy.  MassTransit has built in helpers for all the major containers (available via Nuget no less).  So when configured, you can tell it to set up the subscriptions by asking the container to see what kind of consumers are registered in Structuremap (or the container of your choice) when it builds a service bus for you.  Because this registration code is being processed by the container, I had to put in a way to access the container itself during resolution, hence the previous self-registration.  As I mentioned, there may be a better way to do this, I just don't know what it is.

Then you'll notice a second scan statement below.  This is the second part of the equation that tells Structuremap to register all IConsumers (aka MassTransit message handlers) from the current assembly.  The scan could be broader or more refined depending on your needs, but the idea is that all the consumers are registered with your container, then when the bus factory creates a bus it sets up all the necessary subscriptions for you.  Logically this scan would make more reading sense before the service bus registration, but it doesn't matter because execution is deferred until you actually as for an instance of the IServiceBus.

So now when you ask for your bus everything is ready to go!

 

Mar
6
2012

Getting RabbitMQ Management Plugin Working on Windows

I'm running RabbitMQ as a Windows service and was having some issues getting the management plugin up and running.  Some of the documentation in the message boards states that as long as the plugins are in the plugin directory it should work if you reinstall the service.  There's a little bit more to it than that, so I'll add a few things:

  1. Download and install RabbitMQ server (http://www.rabbitmq.com/download.html)
  2. Open a command prompt, make sure you open it as Administrator.
  3. Go to the directory: Program Files (x86)\RabbitMQ Server\rabbitmq_server-2.7.1\sbin
  4. Run the following command lines:
    1. rabbitmq-plugins.bat enable rabbitmq-management
    2. rabbitmq-service.bat stop
    3. rabbitmq-service.bat install
    4. rabbitmq-service.bat start
  5. Navigate to localhost:http://localhost:55672
  6. Login as guest/guest

Happy managing!

 

 

Mar
1
2012

MassTransit, EventStore, CommonDomain, StructureMap, MVC4, KendoUI

I'm working on a new demo project.  I've been wanting to get more familiar with MassTransit and EventStore for a while.  I looked a MassTransit a while back and the 1.0 version was just hard for me to get traction on, especially when NServiceBus was still free.  But things have changed.  MassTransit 2.0 looks pretty cool and has much better documentation than the 1.0 version.  I'm also psyched about using the RabbitMQ support.  It's clear a lot of work has gone into it.  In addition, the free version of NServiceBus is now somewhat hobbled.  I've played with Rhino Service Bus and while I don't see any issues with it, it just seems like more love it going into MassTransit at the moment and the guys that develop it are closer to home.

I like the ideas instilled in JOliver's EventStore and CommonDomain.  The problem I've had is their usage pattern is somewhat complex and the documentation is pretty minimal.  I get the ideas behind them and definitely agree with the approach, but you have to do some bushwhacking on your own to get it all wired up.  I was lucky enough to find a couple of examples that helped tremendously.  The first was an example on GitHub that isn't easy to find but very informative.  However this example uses NanoServiceBus and AutoFac.  The next is the Documently example.  Great example, a bit more complex and this one uses MassTransit but has Windsor as the DI container.  I know it doesn't sound like a major hurdle but when you factor in configuration for all of the disparate technologies, it's easy to get confounded by all of the wiring.  I'm using StructureMap as the container mostly because it's the one I almost always use.  It seems to perform pretty well in most benchmarks compared to the other full-featured containers and I already know most of it's tricks and limitations.

For the front end, I was undecided between a pure MVC front end or trying the Javascript client route.  If I use Kendo UI, I get both of those, so I'm going to experiment with using this framework.  I'm going to try to go for the MVVM pattern using JavaScript in the browser and treating the server as just that in almost all cases.  I'm going to try to leverage the MVC 4 Web API as my services interface.  I'm going to feel free to backtrack on both of these if necessary.  I'm somewhat conflicted with the Kendo UI.  The new beta Q1 2012 looks nice with the inclusion of the MVVM support.  It's got an open source version but it's restrictive in it's use so it might evolve to where I need to purchase a license if this demo ever goes anywhere.  Outside of Kendo, everything else has a liberal open source policy.  I had considerend Ext JS, but I've messed with Telerik's libraries in the past for MVC and Kendo UI borrows a lot from that so the learning curve is a little less steep and they both have the same open source policy so there's no real advantage with Ext JS other than it's more mature.

For the event store and the read store, i'm leaning towards SQL Server.  Both examples mentioned above use Raven DB.  I've played with Raven and it looks very cool.  I'm might substitute it in for the event store at some point, but if I'm working towards something that eventually pans out, I'll probably need SQL Server anyway for reporting and I can get it pretty cheap with a hosting plan somewhere. I may change my mind on this as I go forward.  I'll be using NHibernate as my ORM with Fluent NHibernate.  I've played with a lot of commercial ORMs, but NHib just works in almost all situations out of the box and I've got a lot of experience with it, so hard to argue with that.  If I decide to use Raven for the read model, I'll probably jettison NHib.  Update: I've decided to go with RavenDB...I don't think it's cost prohibitive if any of this pans out and I sure do like it in a lot of ways.

I'm going to follow up with a post on basic project structure and wiring, should be coming in a few days.

 

Mar
1
2012

They let me back on the web

Hello World!  Just giving a shout-out from my new blog location.  Content to follow!

Hello World!

Eric's Mug