Category Archives: Software Architecture

Writing a WS-Federation based STS using WIF

Even though SAML and WS-* have started to be looked upon as the old guard of security protocols with the popularity of OAuth 2, they are not without their merits. For one, they are inherently more secure than OAuth (in fact, you need to rely on a separate underlying secure transport for OAuth to be considered secure- and if you are someone who believes SSL is broken, then OAuth is practically insecure). It just so happens that their demerits are very visible to someone trying to implement or integrate them. OAuth, by contrast, is much simpler and easier to implement- and for most purposes, secure enough. If you have settled on WS-Federation as your protocol of choice, Windows Identity Foundation (WIF) is most likely going to be your de-facto choice. While powerful, WIF as a library is not what one would call “easy to use”. If it’s cumbersome when you use it as a relying party, the complexity is ten-fold if you try to build a security token service (STS) based on it.

I decided to take on the challenge and embarked on building a central login system (i.e. passive STS) that I could use for all of my applications. I mostly don’t like to maintain my own user database, so the central login system would then provide a hub for the user to login using any of the various identity providers out there such as Google or Facebook. The main advantage would be that the user would need to login once and be able to use all my applications. The initial plan was to make it protocol agnostic – i.e. something that will look at the incoming request, figure out what kind of request it is, and then delegate to an appropriate protocol handler. This way, the application would be able to support WS-Federation, SAML 2, OAuth 2, what-have-you, as needed. However, I saw what a great job IdentityServer has done in terms of providing a library you can easily build an OAuth based STS with – so that made me not want to pursue the OAuth path at all for this instance. My plan is to someday just build the next version of this as an OAuth STS using IdentityServer3.

With that said, if you look at the source code (available here), you will see that there is a protocol-agnostic pattern with a WS-Federation implementation. The application uses Windows Identity Foundation (WIF) as its core component. It acts as a passive STS (Security Token Service) while dividing the role of IP (Identity Provider) between the target application (or “Relying Party“) and one or more third-party providers such as Google or Facebook. The third-party providers are used for authentication, but the responsibility of storing whatever user information is needed by each application is the responsibility of that application (thus my statement that the identity provider role is divided between the Relying Party and the third-party Identity Providers). The entry point of the WS-Federation communication logic is in the WsFedLoginRequestProcessor class, specifically the ProcessSignIn method.

Each Relying Party is registered with the application through configuration – needs to have three settings populated: the realm URI (a unique identifier for the party – an example being urn:ApplicationName), the reply-to URL (the URL that this application will redirect to once the user is authenticated. This is usually the main entry point URL for the Relying Party application) and the “login service URL”. The Relying Party needs to implement a WCF service with a predefined contract (defined in ILoginService – available in the Commons Library. The service is responsible for providing information about the application as well as any given user to the STS, as well as exposing a way to create new users. The Relying Party application then needs to be configured for WIF and with the STS designated as the token issuer. There are methods available in the Commons Library that facilitate this. Communication between the Relying Party and the STS is encrypted and signed using a shared X.509 certificate.

When you navigate to a protected endpoint in the Relying Party, and are not authenticated, you are redirected to the login page hosted by the STS. This redirection request is generated by WIF and follows standard WS-Federation protocol. The STS then uses the realm information passed in the request to look up information about the party. It gets more information from the Relying Party using the ILoginService WCF endpoint. It uses this to display application-specific information in the login page. From the login page, you can use Google (using it’s OAuth API) or Facebook (using its JavaScript API) to authenticate yourself. The STS then communicates with the Relying Party using the ILoginService WCF endpoint to see if a user matching credentials just provided by Google or Facebook exists. If so, it uses this information to instruct WIF to write an encrypted session token cookie, and redirects back to the Relying Party reply-to URL – where it is now authenticated thanks to the encrypted session token cookie.

If a user is not found, the STS prompts you to enter a display name for the user that is going to be newly created. Once you provide the information, the ILoginService WCF endpoint is again used to create a new user record in the Relying Party application, and the rest of the process is same as above. When you logout from the Relying Party application, WIF sends a logout WS-Federation message to the STS which takes care of processing the logout operation.

Two Types of Domain Events

You can find a good primer on domain events in this post by Udi Dahan. There are some issues with his approach, though that Jimmy Bogard raises and addresses in his post. However, I was left with two questions:

  1. Shouldn’t the domain event be dispatched/handled only when the transaction or the unit-of-work commits? Because whatever changes have been made to the state of the domain isn’t really permanent until that happens.
  2. There may be cases when domain events need to trigger changes to other domain objects in the same bounded context – and all of that needs to be persisted transactionally. In other words, in this scenario – it makes sense to have the event be dispatched just before the transaction commits. However, in this case, whatever ends up handling that event also needs access to the current transaction or unit-of-work that is in play – so that all the changes make it to persistence in one fell swoop of a commit.

That leads me to conclude that there are really two types of domain events that need to be handled differently. The first type as listed above would either be infrastructure-y things like sending out e-mails and such, or sending messages to other bounded contexts or external systems. The second type would be within the same bounded context but maintain certain kinds of relations within the domain that could not be modeled within the same aggregate (simply put, they take the place of database triggers in the DDD metaphor).

At this point, I have no further design level ideas on how they would be modeled. More on that later, hopefully. I do know that the thing raising the event should not need to know what kind of domain event it is going to raise. If I am a Foo and my Bar changes, all I care about is I need to raise a BarChanged event. I do not need to know what kind of domain event BarChanged is. I will let this percolate a bit.

DDD, meet SOA

There is a lot of discussion online around whether DDD and SOA can co-exist, and if so, what that looks like. I am of the opinion that they can co-exist and have arrived at a model that seems to work for me. Consider a complex DDD system with several bounded contexts and contrast it to an SOA system – and I am including the flavor of SOA that I describe in this post.

So, how do the two come together? As I have mentioned before, one problem I have with straight-up SOA is that there is no identifiable “core” of the system where the “meat” is. This “meat” is in fact – the domain logic. Now, add to this the fact that the domain is the only thing that deals directly with the database (technically, it’s the infrastructure layer that handles persistence – but you get the idea – it is the domain constructs that are involved in persistence to/from a data store via repositories).

What if we modeled the entire domain as a service? It does not need to be strictly hosted – think of it as a service that is consumed by other services as a library. The repositories combined with the domain objects themselves can replace the accessor/persistor class of services. This makes sense in terms of volatility as well – since the domain really needs to be the part that is least volatile. You can then think of the application layer and the infrastructure layer as being made up of all the other services – and these would still follow the whole utility / computational service / use case handler hierarchy. You would not allow the domain layer to call utility services though – but that is something you would handle through domain events and the like. The presentation layer is still the presentation layer – and is where the UI lives, or the external hosting interface to your services.

The mode of communication between bounded contexts does not change. You can now think of each bounded context as its own SOA subsystem. Communication within bounded contexts would still happen through queues, messages and subscriptions. The application and infrastructure layer would also have “domain event handler” services that handle domain events – I guess you could argue that is just one type of a use case handler. The communication between bounded contexts, as mentioned earlier, stays the same.

A Method for Service Oriented Architecture (SOA)

When you adopt service oriented architecture (SOA), the most important aspect of your architecture and high-level design step when building a new system is obviously decomposition of the system into the right services. A prudent way to decompose a system into services is to first identity what parts of the system is more likely to change more frequently. Thus, you decompose by volatility and set up dependencies such that you always have more volatile services calling less volatile services. Within the same level of volatility, of course, you would further decompose services by function if needed.

This, it is said, makes for a much more maintainable system that is much more adaptable to business requirement changes – which are obviously the norm in our industry. There is a school of thought, however, that goes one level beyond this. It classifies a service as one of the following four types (in increasing order of volatility): utilities, accessors/persistors, computational services and use case handlers.

A utility service is something that deals solely with cross cutting concerns – throughout the system (such as security, logging, configuration), or throughout a given domain (e.g. persisting of documents in a larger system of which documents are a part). Utilities are mostly purely technical and can be called by all services.

An accessor/persistor is the service that sits closest to the data at hand – be it some external resource, or as is more likely to be the case, the database. These services deal primarily with retrieval and storage, but also logical grouping of the data in a manner that makes sense to the rest of the application (think aggregates in DDD terms), data-level validation, redaction for security, etc. Since these services are closest to the metal (that being the database), and we don’t want volatility to sneak in there, these need to be the least volatile services apart from utilities. These are the only services that can talk to data sources, and can be called by use case handlers and computational services. Utilities cannot call them. Other accessors/persistors cannot call them either.

A computational service will have hardcore computational logic that can change more often than, say, your database schema, but less often than specific use cases. They can call utilities and accessors/persistors; but usually it is advised that they be purely computational – in that they get the data they need to work with handed to them, they do their thing, and then they return their output. Only use case handlers can call them.

A use case handler is most volatile and executes business use cases by mostly orchestrating calls to less volatile services. They can call utilities, computational services and accessors/persistors as needed. They also handle operation level authentication and authorization if applicable. They are the external interface to the “service platform” in that all consumers, be it the UI or external systems – can only talk to use case handlers. Less volatile services are thus shielded. What if a use case handler needs to call another use case handler? Within a contiguous subsystem, that should not happen. If it happens, that is a design smell. That could become necessary to facilitate communication between two subsystems – in which case, it is a better idea to do it through queues, messages and subscriptions (think bounded contexts in DDD terms).

There are a few more guidelines to top these off:

  • Name your services so their role in the above taxonomy becomes clear. Instead of just “FooService”, think “FooUtility”, or “FooRepository” for accessors/persistors, “FooCalculator” or “FooEngine” for computational services, and “FooHandler” or “FooManager” for use case handlers.
  • As you are performing detailed design, keep an eye on the number of operations in a service. If it is starting to feel like a lot, perhaps it is time to decompose a bit further.
  • Treat each service as “the” unit of development for your system. This means – each service is a self-contained unit complete with tests and such. This also means that while you want to stay DRY within a service within reasonable limits, it may not be the best idea to strive for perfect DRYness across services.
  • Each operation should validate its requests and handle authentication and authorization if needed in a self-contained manner.

All of this also aligns pretty well with SOLID design principles. Now, all of this sounds great on paper; however, there are a few pain points to consider. If you strictly adhere to these restrictions, you will end up creating a lot more services than if you didn’t. For example, there are a lot of cases where the use case simply is to go get the data and go store the data. In such situations, you are forced to create a use case handler that is largely pass-through to an accessor/persistor. You could relax these rules in such situations, but then you have a dependency from your consumer to your accessor/persistor. If, at some time, the use case evolves, you need to make sure an use case handler is inserted at that point rather than the so-called “non-volatile” accessor/persistor being modified.

The other obvious pain point with this approach is code bloat. More services means more code, and that means more code to maintain. I think when you get to a system of a certain size, that volume of code becomes justifiable. So, there is a system size below which a lot of this is too much overhead. It is wise to identify that point and react accordingly. Perhaps better tooling could help, too – something tailored for building systems this way.

One problem I have with this – and in fact with SOA in general is that your system is made up of all these services that have logic distributed within them. If you decomposed by volatility and then by function, then granted – your logic is distributed properly. There still is no identiable “core” of the system where the “meat” is – so to speak. That is something that DDD addresses in my opinion. Hence my increasing interest in meshing DDD and SOA. More on that later.

Getting on the Domain Driven Design Bandwagon

Domain driven design has been around for quite a while. I believe the definitive book on it by Eric Evans came out first in 2004. For whatever reason, I had not been exposed to it in places I worked. I had been hearing about it for enough time and from enough smart people to give it a try. I researched it online a bit and went through quite a few articles. Especially, the set of articles on DDD by Jimmy Bogard (Los Techies) was quite helpful. Finally, I ended up buying Evans’ book and reading it cover to cover.

I liked what I saw. The whole idea behind keeping your domain logic encapsulated within your domain objects appealed to me. There were questions, obviously, but I figured it was worth trying out. So that is what I am deep into currently. The idea of entities, value objects, aggregates and aggregate roots makes sense, but at the same time, also raises questions – especially with regards to database performance. I am hoping I will arrive at satisfactory answers.

As things get more complex, other concepts such as bounded contexts and domain events enter the picture. While I get them in theory, my idea for now is to stay away from actually getting hands-on with those until I have a good handle on “vanilla” DDD. Another question I have is how this meshes with SOA – whether the two are complimentary or exclusive. I would hate to have to give up SOA to stick with DDD. In any case, it feels exciting – and I can’t believe it has been around for so many years and I never got into it.

For anyone getting into DDD, I strongly recommend reading Evans’ book. In software timescale, it was written aeons ago (when Java was the state-of-the-art, mind you). But all of it still applies, and if you’re working with something like C#, as I am, things become even easier since you have so much more power with these modern languages.

So, for the moment, let’s say I am on the bandwagon. Hopefully I don’t have to get off.

On Code Generation and Its Application to Aspect Orientation

TL;DR

The CodeDOM is a cool library within .NET that can be used for structured code generation and compilation. When combined with Reflection, one neat application is to be able to inject aspects into your code at run-time. I have created Aspects for .NET, a library that does just that, and also tries to bring AOP to MEF.

Whether it be generating boilerplate, generating proxy classes or processing DSL, code generation has numerous applications. There are a few different options for code generation in .NET:

T4: With T4 you create text templates that consist of literal text and “special” text that gets compiled, run and the output of the run inserted into the template to create the final output (a-la ASPX/Razor, etc.) If you need to generate code at design time off a model or DSL, this is your best bet. You can extend the T4 library and write stuff that does runtime code generation too. In that case though, you would be playing around with code in string literals. Not ideal.

CodeDOM: I feel that CodeDOM is a somewhat under-appreciated part of the .NET framework in that it does not get a lot of buzz (or at least that has been my impression). I am not sure what impact the introduction of Roslyn will have on it, but as of right now I love it the way it is. It is a DOM based approach to working with programmatically constructed code. The entire .NET type system is represented as different node types in the DOM. You can construct a DOM from scratch, or parse existing code into a DOM. Given a DOM, you can generate code in a given language (provided you have the code provider), or compile the DOM on-the-fly into an assembly that you can then load up and use. That is cool in my book any day.

Another cool approach is to combine the two- write classes that use CodeDOM to generate code as a string, and then use that in T4 templates. In any case, I think the choice depends on whether you want your code generated at design time or run-time.

Design Time: You would need to do this if there is non-generated code in your application that references constructs from within the generated code. The burden of having to keep generated code around in source control could be somewhat alleviated by having the generation take place as part of the build. In that case, also, however, until you build, the rest of your code is going to have a lot of squiggly red lines. Now, if the rest of your code only referenced interfaces and the implementations were generated, assuming you were using some sort of dependency injection mechanism, that would be a solution. However, that is not the norm with T4 (i.e. generating only on build).

Run-Time: With this strategy, you would generate code at run-time (most probably using CodeDOM to generate an in-memory assembly), load that up using Reflection or MEF or what-have-you and then use it. Preferably you would want this to happen at startup or very infrequently for performance reasons. In this case, also, you would need a minimal set of constructs that the rest of your code can reference that provides a bridge to the generated code. An example of this would again be non-generated interfaces with generated implementations.

As I think about it, there are two applications that I have been wanting to use run-time code generation for:

  • Generating WCF proxies when we have a reference to the service/data contracts (although increasingly I have been thinking that might be unnecessary, that’s another blog post to come).
  • Aspect oriented programming, or more accurately introducing aspects into my implementation code.

Now, aspects as a concept is quite good. To be able to remove cross-cutting stuff from your business logic code is always good. The attribute model in .NET is a very good fit for applying aspects. There are a few issues with what prevails in terms of AOP in .NET though:

  • PostSharp: They’re pretty popular within the AOP crowd, but I don’t like my code messed with post-compilation.
  • There are IoC containers that support AOP out of the box, but I don’t like to tie myself to any one container just because they support AOP; besides I have been using MEF for everything.
  • MEF does not have AOP support.
  • If you peruse through NuGet, there are quite a few AOP libraries out there. If you think about it, though, this is a perfect application for run-time code generation. I already have CodeDOM and Reflection in my toolbox. Why not build it in the spirit of rolling my own? I can then build it the way I think makes sense and build it with MEF support in mind.

To that effect, I ended up writing Aspects for .NET. It is a library that provides interfaces that you can implement to build your own aspects which you can then apply to your implementation classes. It provides a mechanism to wrap, at runtime, your implementation with aspects. There is also integrated support for MEF, albeit with some limitations.

You can visit the project page or the GitHub repo for more detailed documentation, but to summarize, you can “wrap” a class with aspects at run-time. When you do so, it inspects the contract interface and implementation class using reflection. It uses CodeDOM to generate an on-the-fly implementation of the contract interface, which expects an instance of the implementation to be initialized, and where each member simply calls out to the implementation (i.e. a hollow wrapper). However, in the process, it also inspects the implementation member to see if any aspect attributes are applied and adds code at appropriate places before or after the invocation to execute those aspects. The type that is generated on the fly is cached in memory.

As with everything else, there is always room for improvement and feedback is welcome.

Modeling, DSL and T4: Ramblings

UPDATE (Jan 10, 2015)
There have been changes in my thoughts about how one should go about this. Consequently, I have abandoned the modeling library that I speak of in this blog post. Understandably, the link to which that points no longer works.

I absolutely loathe writing repetitive code. That is what machines are for. My philosophy therefore is to try to generate as much of these kinds of code as possible. With .NET, T4 gives you a pretty neat code generation mechanism for generating code that follows a given pattern (the first example that comes to mind are POCOs from a domain model). If you think about it though, most multi-tier enterprise type applications have quite a bit of code that can be generated and that derives from the original domain model. How nice would it be to be able to generate a huge chunk of the codebase so that you only have to think about, write and test what you absolutely need to? I guess Entity Framework does some of it for you if you’re into it. If you don’t like the heavy-handedness of it (like me), you could opt for keeping your model as an EDMX file but then writing a bunch of T4 around it to generate various layers of code based on it.

What I think is really neat is to be able to have a domain specific language (DSL) that is capable of describing your model with all of its unique requirements and nuances that EDMX does not give you. You could then have something based on T4 around that model – all packaged into one neat library/add-in that you would use for development. A basic workflow would then involve creating the model, generating the code, and then filling in the blanks where actual logic is required.

That is the ultimate goal, anyway. That is my plan for this modeling library. For now, though, my DSL is a rudimentary JSON description of my entities, and all I have in there is a T4 based library that generates entities and repositories based on those JSON descriptions. But even having built this much of it has made my job so much easier when I am building applications. If you haven’t dabbled with T4, I would strongly recommend it. As far as a full-blown modeling kit is concerned, I believe this is what the big boys play with. I’m not there yet, but I plan to get into it fairly soon, situations permitting.