Monday, December 18, 2017

Creating a Custom Virtual Entity Data Provider

One of the cool new features of Dynamics 365 Customer Engagement (or whatever we’re calling it now) v9.0 is Virtual Entities. A quick overview is you weren’t aware, Virtual Entities let you surface external data into the system as if it were actually present in the organization database (naturally with some caveats). This means it could be in forms, views, reports, available from an API, etc. Read-only at the moment, with the promise of create, update, and delete sometime in the future. With that said, this is the foundation for what is to come.

If you aren’t looking to code anything and happen to have some data exposed via an OData v4 endpoint, has a Guid for a primary key, and is not secured or only secured by a static token that can be included in the query string, then you are in good shape. You might actually be able to configure via the Virtual Entity Data Sources UI under Administration.

Of course the problem is that if you do happen to already have a web service for your data, it probably doesn’t meet all those criteria. Enter the Custom Virtual Entity Data Provider. At the time of writing, the first thing you see the linked page is a message saying that it would be better to re-build your existing data provider than to take this approach. Sounds encouraging :) Lets move on.

Again, at the time of writing the documentation: Sample: Generic virtual entity data provider plug-in is in fact incomplete. Mainly it’s missing some key steps in the set up and the information on debugging is TBD. But if you do look at the code, it might seem a little familiar. In the example we’re writing a plug-in for the RetrieveMultiple message and putting our own content in the Output “BusinessEntityCollection”. It doesn’t show Retrieve but again all the really happens is you put your own data in the Output “BusinessEntity”. Nothing we couldn’t do in CRM 2011 but remember this is the foundation of things to come.

The other item to note on the sample is the use of a QueryExpressionVisitor. Stop and think for a moment, if people are going to use Advanced Find to look for your external data, how does the system translate that into something your data source can use? Well it doesn’t, you need to handle that yourself. Microsoft has given us the Microsoft.Sdk.Data namespace to assist with converting a QueryExpression into some other format. Considering how many operators Advanced Find has between all the different data types, it seems like monumental task to try and support everything. So later when we’re defining the attributes for our external data, don’t forget you can mark them as unsearchable so they can’t be used in Advanced Find.

The code for this is located in GitHub so your can take a look at it. I’m not going to spend too much time on that because the real value comes from walking through the setup I think. Also note in my example, the web service I hooked up to didn’t provide Guids for identifiers but instead had some random characters. Since there was only about 9.7k records for the purposes of making this example work with something I found a bit funny, I created some Guids and joined them up with the provided random character ids and stored them in the plug-in. Obviously, the Guid is important of course so when you try and open a record from a view, the system needs to know which record to retrieve.

Chuck Norris jokes

So I chose to bring Chuck Norris jokes in as external data. As it turns out there is an existing web service for that: https://api.chucknorris.io/ I’ll just explain how the code is working and if you want to grab it later and look, feel free. FYI – there might not be the most “politically" correct” content on this site so be warned.

RetrieveMultiple plug-in

The API lets you search on a keyword. To keep my example minimal I extracted the first value from the first condition in the QueryExpression. Once I had that I made a search request to the API with that key word and got some results back. From there I parsed that data and created new Entity records for each result matching the schema I defined. I used a predefined list to match the API id to a Guid – naturally you’d use the Guid returned from your API. Once I had the Entities in an EntityCollection I put them in the Output “BusinessEntityCollection” and that was it. In case someone didn’t enter any search conditions, instead of returning everything I opted to use the API’s “random” feature and just returned a single result.

Retrieve plug-in

The plug-in Target Id will be the Guid you set in RetrieveMultiple. So here I grab that and use it to look up a specific joke’s random character id. As it turns out there was not a service method to retrieve a single record, it expects you’re just going to build a url and open it in the browser I guess. With that being the case I grabbed the HTML content and parsed out the values I needed for my Entity. Once I had that I put it into the Output “BusinessEntity and was done.

Lots of pictures

Here’s the walkthrough on the setup. I’m using v9.0.0.7 of the Plug-in Registration Tool for this which you can get from NuGet.

First off, register your assembly – don’t worry about adding any steps

image

Choose the new Register New Data Provider option

image

We’re creating a new solution here so add the details

image

Data Source Entity “Create New Data Source”

image

Choose your plug-in assembly & set Retrieve/RetrieveMultiple handlers you’ve created

image

Register!

I had a few issues with crashes so watch out.

In D365

Under Administration –> Virtual Entity Data Sources – New

Choose the data provider you created

image
image

Now you should have a new Virtual Entity in your solution

image

Add the new Data Source and assembly to your solution (optional)

At this point you’ll already see a Virtual Entity for the Data Source
image

Create a new Virtual Entity – this is the one for the actual data

Set the Data Source to the one you just created

image

Save

Create any other data fields, views, etc. – the external names aren’t that important I don’t think since the plug-in is doing the mapping from the real source

image

Publish All Customizations

And that’s it

Advanced Find or View

Get random joke – no search criteria
image
Get a joke containing the search word – this is all the plug-in handles in conjunction with the Chuck Norris joke web service
image
image

Final thoughts

Obviously the data is only going to be returned as fast as the web service can provide it If it’s slow, user experience is going to suffer. Getting your provider in Azure would probably be helpful.
I didn’t look at how paging might work, but that’s something that will need to be figured out at some point.

You also noticed that we didn’t explicitly register any steps on our plug-ins since the setup did that for us. I’m still wondering how you’d go about debugging with the Profiler with no steps (remember the TBD earlier?).

Get the code on GitHub:
https://github.com/jlattimer/D365CustomDataProvider