We are now up to part 4 of Dynamics CRM Upgrade: 2011 to 2016! This series discusses my current customer experience with an enterprise system migration of a very complex, xRM heavy On Premise CRM 2011 system to CRM 2016.
A quick recap of parts 1,2 and 3 of this series: In part 1, we looked at general considerations for migrating CRM On Premise 2011 to 2016 around resource planning, new feature review, test and release planning, and final roll out planning. With part 2, we focused on the database portion of the migration, such as the staged and iterative upgrade process, including how this process impacts resources, timing, and overall planning. With part 3, we continued the database portion, reviewing some kinks in the database upgrade process and some ideas on how to streamline the process.
What’s next?
Now that our database has been upgraded and cleaned up, your system should be in a somewhat workable state. You should be able to login and move throughout your system. If your solution consisted of customization only, then your system should be functional outside of some cosmetic forms changes and navigation updates. Since my customer has many custom components, such as custom plug-ins and workflows, multiple types of web resources, and custom Reporting Services reports, we needed additional steps to complete their upgrade.
Reports and integrations
If you have them, your team should review all custom SQL based SSRS reports and any integrations that connect directly to the CRM organizational database. This is because a major change to the underlying schema for CRM entity tables occurs with the CRM 2011 to 2013 upgrade: base and extension base tables are collapsed into a single database table. In pre-2013 CRM releases, an entity generally meant two corresponding database tables in your database. For example, assume a CRM 2011 solution with a Log entity named new_log. This means we have two corresponding database tables named new_logBase and new_logExtensionBase in the organizational database. The new_logBase table contains the common entity properties such as the Primary Field (new_logid), statecode, statuscode, OwnerId, etc. The new_logExtensionBase table contains any custom attributes, such as new_ErrorMessage as an example. In the upgraded CRM 2013 version of our solution, the Log entity becomes a single table named new_logBase, containing both the common and custom fields related to the entity.
Why is this a big deal? For most users, this is usually nothing to worry about. All of this entity information in the database (and much more) is managed by the CRM platform while working with your customizations. But if you have worked with custom reports or integrations via tools like Scribe, this can force a major change. In addition to the database tables, Dynamics CRM creates SQL Server Views referred to as Filtered Views as per the platform naming convention. In the Log entity example, a new Filtered view is created named Filterednew_log. When queried, the view returns all attributes for an entity, filters the results against security rules for the calling user, and returns additional details for related field information, such as the Option Set labels. Here is a quick article on Filtered views in Dynamics CRM: Use SQL and filtered views to retrieve data for reports. Best practices recommend developers use these views because of the applied security and the additional information, but also to protect them against future updates. Unfortunately, I have seen several developers who feel that the view performance is poor, so they write their reports or other queries against the base and extension base tables directly. With the CRM 2013 and beyond upgrade, all of those custom queries and integrations will require update.
Fortunately, when reviewing our reports and integrations, we verified my customer followed best practices and their reports and integrations worked without issue!
A new SDK!
With each updated release of Dynamics CRM, developers receive an update SDK. Here is the latest and greatest online, including download links: Software Development Kit for Microsoft Dynamics 365 (online) and Dynamics 365 (on-premises) This means updated documentation for new features, new code samples and tools, and new assemblies for building out your customized solution. The SDK assemblies enable plug-ins and custom workflow activities development or building custom web applications using CRM service endpoints. Since CRM 2011, it seems that the APIs provided by the SDK assemblies have remained largely the same and there have not been large changes to the platform with things like the plug-in execution engine. You could swap out the SDK assembly references and things compile fine. When prepping for the upgrade, I was doing a bit of legwork and testing against CRM 2015 before the move to CRM 2016, and this seemed to be the case. I was able to swap out the SDK and our my customer’s projects all compiled without issue. Once we decided on the bump to CRM 2016, we ran into a few problems.
Issue: CrmSvcUtil
The CrmSvcUtil would not connect to our instance. If you have not used the CrmSvcUtil, here is a quick reference for the tool: Create early bound entity classes with the code generation tool (CrmSvcUtil.exe). I was confused as I had been using the CrmSvcUtil on another project regularly without issue. When running the utility, i would receive an error saying, “Unable to Login to Dynamics CRM”. After some digging, I found one comment in a forum indicating that it was an bug introduced in the version of the SDK that I had downloaded, v8.0.2, the latest at the time. This was not an “official” post indicating an issue but, sure enough, when I reverted to v8.0.1 of the SDK, the tool works fine! With the Update 1/Service Pack 1, an incremental release of the SDK was put out, from 8.0 to 8.1. So if you run into odd issues like this, it may have nothing to do with your system and a bug in the SDK tools. My customer’s development team made sure to check in this correct version of the tool as part of their build and release process so that we tracked the versions used in the future. Right now, we have not yet installed the service pack, but when we do, our build process will also require an update.
Issue: Namespace changes!
I immediately ran into issues on compile once I swapped out the SDK references in our plug-in projects. The first error was a bit of a surprise as it is very common: unable to find Microsoft.Xrm.Client. This assembly has been completely removed from the SDK and the Client components moved into the Microsoft.Xrm.Sdk.dll assembly. A few more assemblies had been removed. Here are a few that impacted our project:
- Microsoft.Xrm.Client
- Microsoft.Xrm.Client.CodeGeneration
- Microsoft.Xrm.Portal
- Microsoft.Xrm.Portal.Files
This did not create an enormous amount of issues once I started poking around. We did however hit another issue with the changes in the SDK…
Issue: SDK Extensions are gone!
My customer was using the SDK extensions within their project to make development a bit easier against the CrmSvcUtil generated code. If you have not used the extensions, here is a bit of documentation: SDK extensions for Microsoft Dynamics 365. (Notice the updated title for Dynamics 365!) Where this became an issue was again running CrmSvcUtil to generate our proxy classes. The documentation notes the following:
The SDK extensions are supported for Microsoft Dynamics 365 (online & on-premises). The links below refer to the Microsoft Dynamics CRM 2015 SDK documentation on MSDN. There have been no significant change to the SDK extensions since that version.
The portal assemblies and tools have been removed from the v8.0.0 (and future) SDK download package. You can obtain these assemblies and tools by downloading the SDK Extensions NuGet package: https://www.nuget.org/packages/Microsoft.CrmSdk.Extensions/.
I grabbed the extensions using NuGet and tried running them using the CRM 2016 SDK. Unfortunately, even with the additional Nuget package for CoreCRMTools, the CrmSvcUtil throws an error when attempting to add the /codeCustomization flag, a mismatch assembly versions:
Exiting program with exception: Could not load file or assembly 'CrmSvcUtil, Version=7.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
So, it seems that to use the Extensions, you need to run the CRM 2015 version of the CrmSvcUtil. I am really not sure whether this would be an issue or not, so after discussing with my customer’s development team, we decided to simply remove the Extensions from the project.
The impact of this change wasn’t large but involved some coding changes. Our entity objects referenced in the CrmSvcUtil generated code were changed from CrmEntity to Entity, fixed with a bulk update on the code. This required some additional changes downstream with loading extended attribute values for the Entity objects. This means that related attributes are not longer automatically loaded when running our LINQ queries and needed to manually call LoadProperty on the OrganizationServiceContext to populate the data. Check out this link for details on LoadProperty: OrganizationServiceContext.LoadProperty Method (Entity, Relationship). This was a bit of extra work and testing of our code, but mainly, it just required a bit more discipline by the team so that we did not create Null Reference exceptions! I added extension methods for Entity that allows passing one or more properties to be loaded, with an overloaded method to load all available properties that saved us a bit of time but that was about as fancy as we needed to get.
Some additional minor changes were required with the change, such as changing property types:
Microsoft.Xrm.Sdk.Moneyinstead ofSystem.Nullable<decimal>for Money attributesOptionSetinstead ofSystem.Nullable<int>for OptionSet attributes- Nullable Enums instead of Int values for StateCode attributes (such as:
System.Nullable<Entities.new_logState>)
Once we completed these changes, our solutions compiled correctly and we were able to deploy!
Updating the new assemblies
So after getting past the previous issues, we were ready to update our assemblies. My customer’s solution consists of a large number of plug-ins which means many SDK message processing steps. Recreating these if needed is not a big issue, just a bit time consuming. Their solution also includes a large number of custom Workflow activities used by over 50 CRM workflows. Manually updating these using the web based Workflow designer would be an extremely time consuming task! Fortunately, using the CRM Plug-in Registration tool, you can easily update your assemblies in place. I assume most developers are familiar with the tool, but here is a quick refresher: Walkthrough: Register a plug-in using the plug-in registration tool. The assemblies for my customer’s solution are all deployed to the database, so the Plug-in Registration tool only needed to be run once. Within the tool, all you need to do is Update the assembly, very similar to how you register. Select the Assembly from the list and choose Update. Once you choose the new assembly, you will see a list of the plug-in classes it contains. Just select all of the components and confirm to update!
Of course, you can have some issues here. One very common error you might see is the following:
“Error registering plug-ins and/or workflows. Plug-in assembly does not contain the required types or assembly content cannot be updated.”
This means that your new plug-in assembly does not contain one or more plug-ins that are currently registered in your upgraded solution. During our cleanup and updates to the plug-in code, we had removed one or two plug-ins that we were no longer required. So when we attempted the update, we received the above error. The simplest fix here is to just unregister these components using the plug-in registration tool and then attempt to update once again.
.NET assembly load errors also cropped up because of old dependencies. For example, as mentioned earlier, the Microsoft.Xrm.Client.dll is no longer available, so when the CRM platform attempted to load our older plug-in assembly, it could not locate the dependency. Fortunately, this also had a relatively easy work around. To get past this issue, we simply dropped the assembly into the CRMWeb\bin folder when performing the update. Nothing executed with the plug-ins here, we simply allowed the assembly to be loaded by the CRM platform. Just remember to clean it up once the initial migration is complete!
Once we updated the assemblies using the tool, we had functioning plug-ins and workflows. We still had a few kinks to work out during our testing, but our system passed a big hurdle in the upgrade!
Up next
In the next post, we can dive a bit deeper into the user interface updates and the impact on scripting, ribbon customizations, form layout, and overall navigation for the end user. We can also discuss some of the new features that are available since the CRM 2011 release.
As always, comments, questions and corrections are welcome!