Combining different data sources in jQuery: Fun with the Infragistics Mashup Data Source

Damyan Petev / Thursday, March 8, 2012

Reading our blogs often makes it almost guaranteed that you have seen us mention our jQuery data source here and there or even spot it in code snippets. And while it appears alongside other controls on regular basis, its presence is actually much more impactful than what you may expect from a non-visual component. It’s one of those working their magic behind the scenes type – and that’s not in odd places, but basically with every data-bound control. The igDataScource is in fact the force that feeds data to almost every control in the NetAdvantage for JQuery family. You’d wonder then why the shy-of-attention attitude then? Well, it’s simple – it’s meant to shine while being barely noticeable and saving you all the effort that accessing and transforming data from various sources can take. The component allows for multiple types of data and origin – a range of web services and local data or even when all else fails - the option to feed your data through data-returning function. You will also find quite a few classes that extend it to provide some predefined setting for greater ease of use in the most common cases. Since we put the focus on that it’s probably time to mention that with the latest release the numbers of supported scenarios by multiple implementations got bolstered a bit more, both of them really, with a CPT (Community Technology Preview) of a very interesting ‘type’ of data source – the Mashup Data Source.

Introduction

Now this would be somewhat off-topic, but  the spellchecker insists this is misspelled and I tend to disagree based on this article about Mashup on Wikipedia. And to get back on topic, on that page you can read that the term Mashup can refer to a web page/application “that uses and combines data,[..]” -  I am cutting this quote off here. I believe that is more than enough to give you a decent hint what would a component, that serves as middle layer between data and data-bound controls, would do if it has ‘mashup’ in its name. Pretty straightforward  - the igMashupDataSource is an extension of the base igDataScource and it allows you to combine entries from different sources into one. And since I believe this fits so well I’ll also add that the goal is “to produce enriched results that were not necessarily the original reason for producing the raw source data.” - and I find that thought very pleasing and that is exactly how it should be – this component is not just useful, but also a purveyor of satisfaction and fun – thus, the title above. And this should happen fast and easy and seamless - you still get all the familiar functionality inherited from base class or can also add to already available settings in case you have a running project.

Of course there’s a very minor catch, if you can even call it that – it’s not as simple as pointing to two sources and wait for them to be combined. For that to happen you will have to provide a common key. So while not meant to be combined, still not that random data after all? Well, depends on the case and how look at it – data can be coming from sources that know nothing of each other, yet share keys or you can tweak one to fit or one or all of the sources can be your own – either way you get a single collection consumable by a single control and you get plenty of freedom – I’d put a “fun” label on that, too. Let’s say you have data that is publicly available and therefore you’d imagine it’s two things – probably read-only and while it can be rich, it might still be missing something you need and there’s nothing you can do about that. Or can you? That’s exactly it – you can combine two compatible public sources or enrich what is available with data of your own and those can be with not only different origin, but also with different type.

Note: I’d like to stress one more time this is still a CTP component and therefore it is very possible that it may not function fully and / or some things might be very different in the final version of it.

Getting started

As mentioned before the component is an extension of the base data source class and it should come as no surprise that using it feels very familiar and not that much different. The only additional things you will see is a data source property outside the inherited one and a property indicating whether records that have no data in all sources ( therefore they are partial) should be ignored. The latter is mean to let you get just the final results you want – whether it’s all of records or just a ‘section’ of only those that exist in all sources. The data source is what I’d describe as the ‘master’ one and it takes an array of data sources itself and this is where your data from different origins comes together.

Preparations

Before that there are as always some preparations – adding the required resources and some setup to use the assorted data. Starting off with some resources which this time can be as diverse as you scenario requires – since it is a non-visual component that is built as a JavaScript class it doesn’t rely on jQuery UI scripts and therefore you can only reference the default jQuery  library and the ‘ig.util.js’ where the component is. And even though you can do that and use the Mashup Data Source for anything you see fit, it’s only in rare occasions when you wouldn’t want to present it nicely too. And again how to do that comes with enough freedom(including displaying it in a table using templating as seen in our Ignite UI online samples ), so for this example I will use our jQuery Hierarchical grid. For that reason below in snippets and in the demo project you will see both jQuery and jQuery UI libraries, along with the full Infragistics UI scripts file.

Let’s define two sources for our main data layout – one being a small sample from the Netflix OData catalog and the other one being our very own xml file – the basic idea behind this is to take such online data and add your personal notes to it. We would need to define origin and schema to apply on those:

  1. <!--CSS -->
  2. <link href="../../Content/themes/min/ig/jquery.ui.custom.min.css" rel="stylesheet"
  3.     type="text/css" />
  4. <link href="../../Content/themes/base/ig.ui.min.css" rel="stylesheet" type="text/css" />
  5. <!-- Scripts -->
  6. <script src="../../Scripts/jquery-1.5.1.min.js" type="text/javascript"></script>
  7. <script src="../../Scripts/jquery-ui-1.8.11.min.js" type="text/javascript"></script>
  8. <script src="../../Scripts/ig.ui.min.js" type="text/javascript"></script>
  9. <script type="text/javascript">
  10.     var url, oDataSchema, xmlDoc, xmlSchema,
  11.          mashupSources, dsMashup;
  12.  
  13.     url = "http://odata.netflix.com/Catalog/Titles?$format=json&$callback=?&$top=10";
  14.     oDataSchema = new $.ig.DataSchema("json", { fields: [
  15.             { name: "Id" },
  16.             { name: "Name" },
  17.             { name: "BoxArt" },
  18.             { name: "AverageRating"}],
  19.         searchField: "d"
  20.     });
  21.     xmlDoc = loadXMLDoc("/Content/MovieNotes.xml");
  22.     xmlSchema = new $.ig.DataSchema("xml", {
  23.         fields: [
  24.                     { name: "Id", xpath: "Id" },
  25.                     { name: "Notes", xpath: "Notes" }
  26.                 ],
  27.         searchField: "//Movie"
  28.     });
  29.  
  30.     function loadXMLDoc(dname) {
  31.         if (window.XMLHttpRequest) {
  32.             xhttp = new XMLHttpRequest();
  33.         }
  34.         else {
  35.             xhttp = new ActiveXObject("Microsoft.XMLHTTP");
  36.         }
  37.         xhttp.open("GET", dname, false);
  38.         xhttp.send();
  39.         return xhttp.responseText;
  40.     }
  41.     //continues...

Of course for the XML file you can use jQuery’s get() (short for $.ajax) to perform the HTTP GET request, too. Or that xml can come from just about anywhere as long as it is presented in a form the igDataSource accepts. Defining the schemas provides the way to navigate the responses’ structure and which fields to be taken. As for the contents of the xml file it’s basically movies with ID and Notes nodes and you will find it in the demo project.

Then for a good measure I have an ASP.NET MVC action that returns JSON from a randomly generated values (again IDs and each having a collection of generated Fans with random initial and rating). The idea is that we would not only add a third source, but also it will be the sole provider for data of the child layouts in the hierarchical grid:

  1. [ActionName("getData")]
  2. public JsonResult JsonSource()
  3. {
  4.    var mmData = MoreMovieData.Generate();
  5.    return Json(mmData, JsonRequestBehavior.AllowGet);
  6. }

Note: If you do decide to go with this make sure to check out this topic on the ASP.NET forums for why we have JsonRequestBehavior.AllowGet in there and this blog by Phil Haack on the subject.

All for one

Now that we have three sources ready it is time to create our Mashup Data Source. As said before one of the things it provides is the ‘master’ dataSource which takes an array of igDataSource instances to be combined:

  1. //--- Defining the Mashup Data Source
  2. mashupSources = [
  3.         { type: "remoteUrl", dataSource: url, schema: oDataSchema, responseDataType: "jsonp", primaryKey: "Id" },
  4.         { dataSource: xmlDoc, type: "xml", primaryKey: "Id", schema: xmlSchema },
  5.         { type: "remoteUrl", dataSource: '/Home/getData', primaryKey: "Id" }
  6.     ];
  7. dsMashup = new $.ig.MashupDataSource({ dataSource: mashupSources });
  8. //--- Definition Ends
  9. //continues...

The data source (any of them) assumes that if given a string to that identically named property instead an IQueryable or an array, then that would be an URL that points to the data. So you can safely skip the defining of type being ‘remoteUrl’ for both above. The first one is the Netflix OData as described above with URL and schema, the second is fairly obvious the xml file and the third is the JSON returning action in my Home controller.

That’s it! Now you are free to do with the data what you see fit – call its dataBind() method that does all sorts of stuff for you ( fire bind related events, analiza data, perform requests, transform data and so on) and you can use the callback function to display the data once it’s ready like in the sample linked above OR you can provide it to an UI control to display as we will now do with the igHIerarchicalGrid.

Presenting your data

It’s almost as easy as it gets – you setup the jQuery Hierarchical grid widget like you normally would and use the keys from any source freely, just like that:

  1. <table id="grid"></table>
  1.     $("#grid").igHierarchicalGrid({
  2.         autoGenerateColumns: false,
  3.         autoGenerateLayouts: false,
  4.         primaryKey: "Id",
  5.         defaultChildrenDataProperty: "Fans",
  6.         childrenDataProperty: "Fans",
  7.         columns: [
  8.             //{ headerText: "ID", key: "Id" }, /* optional row if you want to see the ID */
  9.             {headerText: "Movie Name", key: "Name" },
  10.             { headerText: "Image", key: "BoxArt", format: "<img src=\"{0}\" class='tooltip-grid-image'></img>" },
  11.             { headerText: "Average Rating", key: "AverageRating" },
  12.             { headerText: "Notes from XML", key: "Notes" }
  13.         ],
  14.         columnLayouts: [
  15.             {
  16.                 autoGenerateColumns: true,
  17.                 primaryKey: "Id"
  18.             }
  19.         ],
  20.         dataSource: dsMashup,
  21.         width: "100%",
  22.         height: "500px"
  23.     });
  24. });
  25. </script>

As seen in column definitions you shouldn’t feel the difference from using a single source – and that’s the whole point! And just like that set the dataSource of the grid to our Mashup (line 20) and you are done.

Results

And just like that you have a hierarchical grid, data gathered from three different origins – all done fast and easy, keeping to familiar API and methods with a result that is not only very functional and neat but also looking quite well :

jQuery Mashup Data Source consumed by a hierarchical grid

Again the first 3 columns come directly from the Netflix catalogue, the Notes next to them come from an xml file. The child layouts in this structure come from randomly generated collection on the server-side just for the sake of the example and .. for the fun, really.

Conclusion

From this blog you’ve hopefully learned a bit more about the ever so important jQuery Data Source component that is there for every data-bound control to ease your work so much. You’ve also seen what the Mashup version of it can do, how you can combine data sources into a single collection and how you can display them with style and maintain familiar feeling and existing functionality with little to no hassle. The again what you would do with it is really up to your needs and for almost any scenario this component is equipped to handle.

Some useful links: Our Online Documentation, the API references in there and the Online samples and as always you can download the demo project (ASP.NET MVC).