Writing your own LINQ provider, part 3

This is the third in a short series of posts on writing your own LINQ provider. A quick outline of the series:

  1. A primer
  2. Provider basics
  3. A simple, pointless solution (this post)
  4. A tiny ORM of our own

A simple, pointless solution

In the previous post, we took a look at what happens when you call LINQ methods on IQueryable<T>, and how you can use that to build your own provider. We take that a step further this time by building an actual provider – albeit a somewhat pointless one, in that it adds LINQ support to something that doesn’t really need it. The point, though, is to keep it simple and try to understand how the process works.

The best way to understand is to take a look at the source code first:

Now, a quick summary of what this is.

We have an interface, INextProvider. It has one method, GetNext that is supposed to get the next one in a sequence of items. An example implementation that uses a simple array as the underlying store is also included. Once you have an instance of INextProvider<T>, say, callednextProvider, you can then extract an IQueryable<T> out of it with this call:

var query = nextProvider.AsQueryable();

You can then use standard LINQ on top of it. Now, I know what you’re thinking: this INextProvider seems uncomfortably similar to IEnumerator – why would we need a query provider for this? We don’t, hence the “pointless” part, but again – the idea is to examine how building a provider works.

The entry point is NextProviderQueryable which implements IQueryable<T> and uses NextProviderQueryProvider as its Provider and returns aNextProviderEnumerator from its GetEnumerator() call. This means that whenever one of the LINQ methods are called on an instance ofNextProviderQueryable, one of the following happens:

  • If the method is something that creates another queryable out of the existing one (e.g. WhereSelectSelectManyCast, etc.),NextProviderQueryProvider.CreateQuery() is called. That, in turn, creates a new instance of NextProviderQueryable, but with the Expression set to what has been passed in. Thus, every call to CreateQuery ends up creating a new queryable with the Expression property representing the complete call.
  • If the method is something that enumerates a queryable (e.g. ToListToArray, etc. or a foreach loop), the GetEnumerator() method is called and enumeration starts. This means that NextProviderEnumerator takes place. This object is initialized with the current value of Expression as of the time of enumeration, thus it has complete information to parse it, figure out what needs to be done, and then do it using the INextProvider that it is assigned. The class ExpressionParser is used to convert the expression into a series of “nodes” that act on each item in the underlyingINextProvider and do the appropriate thing based on what it is (e.g. if it’s a WhereNode, it will have a predicate that it will run on each item).
  • If the method is something that returns a scalar (e.g. AnyAllFirst, etc.), NextProviderQueryProvider.Execute is called. In our case, we simply pass control to NextProviderEnumerator to enumerate as mentioned in the previous point, and then perform the appropriate action. We do this by getting an IEnumerable<T> that uses NextProviderEnumerator as its enumerator (and that is the NextProviderEnumerable class), and then calling the appropriate IEnumerable version of the IQueryable method that has been called. All of this is handled by the ExpressionExecutor class.

As of now, only the following methods are supported: AllAnyCastCountDistinctElementAtElementAtOrDefaultFirstFirstOrDefaultLast,LastOrDefaultLongCountOfTypeSelectSelectManySingleSingleOrDefaultSkipTake and Where. If you try to use any other methods, you will get an exception. Even within these methods, if you try to use a variant that is not supported, you will get an exception.

Next time, we’ll try our hands at a more real world implementation, i.e. a tiny, tiny ORM.


8 thoughts on “Writing your own LINQ provider, part 3

  1. Pingback: Writing your own LINQ provider, part 2 | Aashish Koirala

  2. Pingback: Writing your own LINQ provider, part 1 | Aashish Koirala

  3. Pingback: Writing your own LINQ provider, part 4 | Aashish Koirala

  4. Adam Bull

    I’m currently considering whether implementing a custom IQueryable LINQ provider to interact with the Shopify API is the right thing to do. Having read through all of your posts on this, I am kinda understanding it. I took a look at the source code for the Expression Parser and was wondering how you would adapt your example provider to interact with the API where the end point follows this convention:


    Where building up the query string portion of the URL would be the job of the ExpressionParser class (e.g. turning LINQ methods into the query string vars above)

    Is this madness?!

      1. Aashish Koirala Post author

        That gives me unreadable obfuscated javascript. On closer look, the javascript is a bunch of “document.write” statements that is writing HTML to the DOM. So looks like the GH gist is malformed?

        [edit] Nevermind, I see the JSON now, let me mull this over and get back to you.

    1. Aashish Koirala Post author

      Looks like my other solution (the one over here: https://github.com/aashishkoirala/snippets/tree/master/src/TinyOrm/AK.Snippets.TinyOrm – this is where I build a very tiny SQL ORM with a LINQ provider) might be more similar to what you will need to build. Instead of generating SQL, you would need to generate what looks like a standard REST GET query.

      Given a class Product, your provider would support IQueryable. The /admin/products.json endpoint allows you to specify fields to filter by (this would inform your Where parser), and also allows you to limit the fields you get back using the ?fields= parameter (this would inform your Select parser).

      This is a very good use case for a LINQ parser. I almost wonder if somebody hasn’t already done a configurable LINQ-to-REST parser that could be adapted.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s