Integrate seamlessly Marketo with 500+ Apps with Zapier (Practical Use Cases with FullContact and Google Sheets)

Objectives

This article, also published on the Marketo Developer Blog, explains in details how to integrate Marketo with potentially over 500 Cloud Apps, thanks to Zapier.

For that, we’re going to build from scratch a Zapier connector for Marketo and implement two practical integration use cases:

Use case 1: a unidirectional Leads integration from FullContact Card Reader to Marketo

  • Scan any contact’s business card with the FullContact mobile Card Reader app and get a lead automatically created in Marketo.

Use case 2: a bidirectional Leads integration between Marketo and Google Sheets

  • Add an existing lead to a static list in Marketo and find the lead automatically added to your Google Sheet.
  • Modify any lead in your Google Sheet and find the change echoed back to Marketo.

Requirements prior to get started

Sign-up for a free account with Zapier

Zapier is a Webapp Automation Service that let you easily automate tasks between other online apps without the need for programmers or any IT resources. Check their web site for more info:

Zapier supports today more than 500 apps in many different domains such as Marketing, CRM, CMS, Customer Support, Electronic Signature, Forms, etc …

A single integration between one app and another is called a Zap.

Check Zapier’s zapbook for an exhaustive list of supported web apps: https://zapier.com/zapbook/.

Sign up for a free account here: https://zapier.com/sign-up/, you’ll get access to up to 100 tasks/month, 5 zaps, zaps running every 15 minutes. You can of course get much more by subscribing to Zapier’s paid plans (basic, business, business plus, etc…).

Access to a Marketo Instance as Administrator or with a provided API User account

Our Zapier connector will use the Marketo REST API in order to push Leads data to Marketo. In order to use this API, you’ll need an API User and a Custom Service that you can create yourself if you are administrator of your Marketo instance. If not, then an administrator will need to provide those to you. There is also a Webhook to create, only accessible to a Marketo Administrator.

The following documentation from the Marketo Developer Web Site explains step by step how to create the Marketo API User and the Custom Service: http://developers.marketo.com/documentation/rest/custom-service/.

Once you’re done, you should have the following credentials in order to invoke the Marketo REST API:

  • Client Id,
  • Client Secret,
  • No need to get a Token as it would be only valid for a single hour; The Connector will generate tokens for you automatically.

Another information you need from your Marketo instance is the Munchkin Account Id. You can get it from the Munchkin or the Web Services Admin screens. Its pattern looks like this: 000-XXX-000.

Sign-up for a free account with Google

Google Docs, Sheets, and Slides are productivity apps that let you create different kinds of online documents, work on them in real time with other people, and store them in your Google Drive online. Our use case needs a Google Sheet.

The following link covers the different features of Google Docs and leads to the creation of an account with Google: https://www.google.com/docs/about/.

Sign-up for a free account with FullContact

FullContact keeps you fully connected to the people who matter most by pulling in all your contacts and continuously syncing them with changes to social profiles, photos, email signatures, company information, and more. They offer a mobile business cards reader that can scan cards into 250+ Web Apps, including Zapier.

You can sign-up for a free account here: https://app.fullcontact.com/login or eventually subscribe to a premium paid account with more features and capacity.

The mobile app can be downloaded here:

The FullContact Zaps are documented here: https://zapier.com/zapbook/fullcontact/.

Implementation of the Marketo Connector for Zapier

Create the Marketo App

From the Zapier web interface, go to the Developers Portal.

Click the button and fill out at minima the Title (e.g ‘Marketo’) and the Description. The logo is optional, but nice to have.

Authentication

In this section we declare the different fields used for the Marketo REST API authentication and the authentication settings.

Create first the following fields:

Label Munchkin Account Id Client Id Client Secret
Key munchkin_account_id client_id client_secret
Type Unicode Unicode Unicode
Required Checked Checked Checked

Edit the ‘Authentication Settings’ as in the following screen:

  • Auth Type: Session Auth
  • Auth Mapping:

    {

    “access_token”:”{{access_token}}”

    }

  • Access Token Placement: Token in Querystring

Zapier proposes a few Authentication Types but typically OAuth V2 would not work with Marketo. Currently the Marketo platform does not offer an Oauth2 authentication similar to Facebook where an end user is forwarded to an authentication page. A Marketo custom service has been created giving access to a client id and client secret. We use the client id and client secret to generate an access token via the REST API authentication endpoint. We can then use this access token to make subsequent requests to the REST API. The token expires after an hour and must be generated again to proceed calling the REST API.

We chose authentication Type = ‘Session Auth’ as it allows us to execute a custom authentication script every time our session token is expired. We’ll see in the section ‘Scripting API’ how to implement this mechanism that can only work with this type of authentication.

Triggers

Zapier Triggers are there to bring data into Zapier. We do not need one for our use cases as we will leverage a Marketo Webhook instead. However, we still need to write a dummy Trigger as a mandatory test for our Marketo connector.

We are going to create a Test Trigger calling the Marketo REST API ‘Get Daily Usage’: http://developers.marketo.com/documentation/rest/get-daily-usage.

Click the button to start the wizard and fill-up the following fields (fields not mentioned can be left blank):

Name and Description

  • Name: Test Trigger
  • Key: test_trigger
  • Description: The Test Trigger of the Marketo App
  • Important? Not checked
  • Hide? Checked

Trigger Fields

  • None

Where Data Comes From

  • Data Source: Polling
  • Polling URL:
    https://{{munchkin_account_id}}.mktorest.com/rest/v1/stats/usage.json

Sample Result

  • Leave blank

Click now the button and set our Test Trigger to be the one we’ll use to verify a user’s auth credentials.

Actions

Zapier Actions are there to send data out from Zapier.

We are going to implement the Create_Update Lead Action calling the Marketo REST API ‘Create/Update Leads’: http://developers.marketo.com/documentation/rest/createupdate-leads/. This Action will allow to create a new lead within Marketo, or if the lead already exists, it will update it with the submitted values. We’ll use the field ’email’ for deduplication.

Click the button to start the wizard and fill-up the following fields (fields not mentioned can be left blank):

Name and Description

  • Name: Create_Update Lead
  • Noun: Lead
  • Key: create-update-lead
  • Description: Create a new lead within Marketo, or if the lead already exists update it with the submitted values
  • Important? Checked
  • Hide? Not Checked

Action Fields

Action Fields are the fields users will map data into. Choose them carefully according to your own needs as they will represent all the data you will be able to update in Marketo. There is an option in Zapier to offer to the end user all fields available in Marketo, but that would induce more code and complexity, not required for a disposable connector.

As an example, we selected the following fields:

Label Key Type Required? Parent key Send in POST by default?
Email Address email Unicode Checked input
Partition Name partitionName Unicode Not Checked Checked
First Name firstName Unicode Not Checked input
Last Name lastName Unicode Not Checked input
Phone Number phone Unicode Not Checked input
Notes Lead_Notes__c Textarea Not Checked input
Called called Unicode Not Checked input

Partition Name is mandatory in our case since our Marketo instance has Lead Partitions in service. It could be omitted otherwise. We separated it from the ‘input’ group so the end-user understand this is not a field to synch.

The field ‘Notes’ comes from a synch between Marketo and Salesforce, do not use it if you don’t have it in your Marketo instance.

The filed ‘Called’ has been created in our Marketo instance, do not use it if you don’t have it in your Marketo instance.

Of course, the goal is to let you pick the fields you need from Marketo. It is recommended to start small and add the extra fields later.

Where to Send Data

  • Action Endpoint URL: https://{{munchkin_account_id}}.mktorest.com/rest/v1/leads.json

Sample Result

  • Leave blank

Scripting API

Zapier’s scripting feature allows to manipulate the requests and responses that are exchanged between your app’s API and Zapier. You can modify HTTP requests just before they are sent and can parse responses before Zapier does anything with them. We need it in order to complete our custom ‘Session Auth’ authentication so it works with Marketo.

More info here: https://zapier.com/developer/documentation/v2/scripting/#intro-to-scripting.

Copy the following code and we’ll go through some explanations later on:


var Zap = {
     
    get_session_info: function(bundle) {
  
       console.log('Entering get_session_info method ...');
    
         var access_token,
            access_token_request_payload,
            access_token_response;
 
    
        // Assemble the meta data for our Access Token swap request
         console.log('building Request with client_id=' + bundle.auth_fields.client_id + ', and client_secret=' + bundle.auth_fields.client_secret);
        access_token_request_payload = {
            method: 'POST',
            url: 'https://' + bundle.auth_fields.munchkin_account_id + '.mktorest.com/identity/oauth/token',
            params: {
                'grant_type' : 'client_credentials',
                'client_id' : bundle.auth_fields.client_id,
                'client_secret' : bundle.auth_fields.client_secret
            },
            headers: {
                'Content-Type': 'application/json',  // Could be anything.
                Accept: 'application/json' 
            }
        };
 
        // Fire off the Access Token request.
        access_token_response = z.request(access_token_request_payload);
 
        // Extract the Access Token from returned JSON.
        access_token = JSON.parse(access_token_response.content).access_token;
        console.log('New Access_Token=' + access_token);
   
        // This will be mixed into bundle.auth_fields in future calls.
        //bundle.auth_fields.access_token=access_token;
        return {'access_token': access_token};
    },
  
  
    test_trigger_pre_poll: function(bundle) {
     
         console.log('Entering test_trigger_pre_poll method ...');
         
         bundle.request.params = {
         'access_token':bundle.auth_fields.access_token
         };
         
         return bundle.request;
        
    },
  
 
    test_trigger_post_poll: function(bundle) {
    
        console.log('Entering test_trigger_post_poll method ...');
        
        var data = JSON.parse(bundle.response.content);
        if ((!data.success)&&((data.errors[0].code=="601")||(data.errors[0].code=="600"))){
            console.log('Access Token expired or invalid, requesting new one - data.success=' + data.success + ', data.errors[0].code=' + data.errors[0].code);
            
           throw new InvalidSessionException(); // Calling get_session_info() to regenerate Access Token
        }
 
        return JSON.parse(bundle.response.content);
    },
     
    create_update_lead_pre_write: function(bundle) {
    
       bundle.request.params = {'access_token':bundle.auth_fields.access_token};  
       return bundle.request;
    },
 
    create_update_lead_post_write: function(bundle) {
         
         var data = JSON.parse(bundle.response.content);
         if ((!data.success)&&((data.errors[0].code=="601")||(data.errors[0].code=="600"))){
            console.log('Access Token expired or invalid, requesting new one - data.success=' + data.success + ', data.errors[0].code=' + data.errors[0].code);
            throw new InvalidSessionException(); // Calling get_session_info() to regenerate Access Token
        }
        return JSON.parse(bundle.response.content);
    }
    
};

get_session_info method

  • This method is responsible for generating or regenerating an access token calling the Marketo REST API ‘Authentication’: http://developers.marketo.com/documentation/rest/authentication/.
  • It is called every time any ‘post_poll’ methods are encountering an ‘Access Token Expired’ error. An access token is scheduled to expired every 1 hour so this is expected.
  • Action Endpoint URL:
    https://{{munchkin_account_id}}.mktorest.com/identity/oauth/token .

All pre_poll and pre_write methods

  • We must create a ‘pre-poll’ method on any Trigger we have created, in order to modify the HTTP request just before it is sent, so we can add the Marketo Access Token in its parameters.
  • We must create a ‘pre-write’ method on any Action we have created, for the same reason.

All post_poll and post_write methods

  • We must create a ‘post-poll’ method on any Trigger we have created, in order to parse responses before Zapier does anything with them, and eventually intercept ‘Access Token Expired’ error.
  • We must create a ‘post-write’ method on any Action we have created, for the same reason.
  • If such an error has occurred, we throw an InvalidSessionException that will tell Zapier to replay the authentication and execute again the get_session_info method.

Note that you can access the Bundle logs from the Scripting API from the ‘Quick links’ menu on the top right corner of the screen. This is really useful to debug the scripts.

And now it’s time for the fun part…

Use Case 1: Integration of Marketo with FullContact Card Reader

For this integration we’ll create one single Zap from FullContact to Marketo. With this Zap, you’ll be able to scan business cards with the FullContact Mobile Card Reader and push the leads to Marketo.

Zap FullContact à Marketo

From the Zapier Dashboard click the button ‘Make a new Zap’.

Trigger in Zapier

  • Pick the App FullContact
  • Choose FullContact Trigger ‘New Business Card’
  • Connect to your FullContact account
  • Test the FullContact App

Action in Zapier

  • Pick the App Marketo we just created earlier, it should display in Beta
  • Choose Marketo Action ‘Create_Update Lead’
  • Connect to your Marketo account, filling up the authentication parameters (Munchkin Account Id, Client Id, Client Secret)
  • Map the fields from FullContact to Marketo

  • Fill-up eventually a Partition Name where your new leads should go (only if partitions exist in your Marketo Instance)
  • Test the Marketo App
  • Activate your Zap

Make sure you download the business cards Reader from FullContact and activate the Zapier Integration right from your mobile device.

Use Case 2: Integration of Marketo with Google Sheets

For this integration we’ll create two Zaps. One from Marketo to Google Sheets and another one from Google Sheets to Marketo. With this Zap, you’ll be able to synch up some of your leads or contacts between Marketo and a Google Sheet.

Zap Marketo Webhook à Google Sheets

For the first Zap, we don’t rely on a custom connector for Marketo, but we leverage Marketo’s Webhooks and the ‘Webhooks by Zapier’ Trigger.

From the Zapier Dashboard click the button ‘Make a new Zap’.

Trigger Part 1 in Zapier

  • Pick the ‘Webhooks by Zapier’ Trigger App
  • Check ‘Catch Hook’ that will allow to wait for a POST or GET to a Zapier URL
  • No need to pick off a child key
  • Zapier generated a custom webhook URL
    for you to send requests to, copy it in the clipboard

Webhook in Marketo (steps to be done by an Administrator)

  • Go to Admin à Webhooks
  • Create a new Webhook called ‘Push Lead to Zapier’ and edit the Webhook form :

  • In the template’s field, declare all the Lead’s fields you would like to transfer to Zapier and leverage the Marketo’s tokens. For our Use Cases, we take the same fields we defined for the custom Zapier connector that push Leads to Marketo:
{“firstName”:”{{lead.First Name}}”,”lastName”:”{{lead.Last Name}}”,”email”:”{{lead.Email Address}}”,”phone”:”{{lead.Phone Number}}”,”leadOwner”:”{{lead.Lead Owner First Name}} {{lead.Lead Owner Last Name}}”,”leadOwnerEmail”:”{{lead.Lead Owner Email Address}}”,”leadNotes”:”{{lead.Lead Notes:default=edit me}}”,”called”:”{{lead.Called}}”}
  • Save the form
  • No need for a Response Mapping, so you’re done with the webhook

Test Campaign in Marketo (steps to be done by a Marketer or an Administrator)

  • From the Marketing Activities, create a new Smart Campaign

    For testing purpose we are going to create a campaign that trigger our Webhook each time a lead is changing its status to MQL. Of course you can use the webhook for any other business purpose.

  • Edit the Smart List

  • Call the Webhook in the Flow

  • Schedule the Campaign

  • Make sure each lead can run through the flow every time
  • Activate the Smart Campaign

Trigger Part 2 in Zapier

  • In order to complete the ‘Webhooks by Zapier’ Trigger App, we need to fire the Marketo Smart Campaign once and catch the Webhook in Zapier
  • In our test case, we just need to go to Marketo Lead Database, open a lead and change its status to ‘MQL’

Create the spreadsheet in Google Sheets

  • Create a new spreadsheet
  • Create a Worksheet or use the default one
  • Add a column for each field you want to synch from Marketo (the ones declared in the Marketo webhook)

Action in Zapier

  • Pick the App Google Sheets
  • Check the option ‘Create Spreadsheet Row’
  • Connect to your Google Sheets account
  • Select your Google Sheets spreadsheet
  • Select the Worksheet
  • Map all the fields between the ‘Webhooks by Zapier’ Trigger App and Google Sheets:

  • Test the Google Sheets App
  • Activate your Zap

Zap Google Sheets à Marketo

From the Zapier Dashboard click the button ‘Make a new Zap’.

Trigger in Zapier

  • Pick the ‘Google Sheets’ Trigger App
  • Tick the ‘Updated Spreadsheet Row’ that triggers when a new row is added or modified in a spreadsheet
  • Connect to your Google account
  • Select the Spreadsheet you want to trigger from (should be the same one used in the previous Zap) and the Worksheet
  • Set Trigger Column to ‘any_column’
  • Test the Google Sheets App

Action in Zapier

  • Pick the App Marketo we just created earlier, it should display in Beta
  • Choose Marketo Action ‘Create_Update Lead’
  • Connect to your Marketo account, filling up the authentication parameters (Munchkin Account Id, Client Id, Client Secret)
  • Map the fields from Google Sheets to Marketo
  • Fill-up eventually a Partition Name where your new leads should go (only if partitions exists in your Marketo Instance)
  • Test the Marketo App
  • Activate your Zap

Wrap-Up

Here are some axes of improvement for our Marketo connector for Zapier:

  • Adding other Triggers and Actions related to diverse Marketo objects (Lists, Custom Objects, etc …).
  • Instead of hard coding the fields from Marketo, it’s possible to pull dynamically the fields from Marketo but that would require some technical translation work between Marketo and Zapier.
  • Sharing the connector with development team and eventually make it generally available.

Zapier has deployed since a Premium Marketo adapter making our use cases much easier to implement. In any cases, this article could always be leveraged in order to integrate Marketo with Zapier with a free Zapier plan and also to build extreme use cases that would not be always supported by the premium adapter. Typically, thanks to the Zapier Scripting API, it would make it possible to implement a connector capable of running some coded logic prior to update Marketo with data.

We hope you enjoyed this article and that it will help you to be even more successful with Marketo and Zapier. Thank You!

Integrating Microsoft Dynamics CRM 2011 Online with Java and other non-.Net Clients

Introduction

While the most common style of .Net development for Microsoft Dynamics CRM Online is using the SOAP endpoint with early or late bound types, there are situations where you need to use the Web Services Description Language (WSDL) endpoint directly. This article is intended to provide some understanding how to leverage this light-weight endpoint that can be used from non-.NET languages. It provides some practical examples in Java that could be transposed to any other popular language.

Microsoft Dynamics CRM Online organizations created since July of 2012 are using the Microsoft Office 365 authentication and billing platform. This was a switch from using Microsoft LiveID since the beginning of Microsoft CRM Online in April of 2008. This article is covering the latest Office 365 Authentication scenario that would fit most of the recent CRM online deployments. An article from the Microsoft Developer Network (referenced at the end of this article) is covering the LiveID scenario.

We’ll start to look at the generation of the Microsoft Dynamics CRM Organization web service stub from its WSDL endpoint and then we’ll see how to authenticate a user via the Office 365 platform’s Secure Token Service (STS) and encode the resulting tokens in a security header securing all operations with the Dynamics CRM Organization service. Finally some of the basic Organization service operations will be covered along with some considerations around performances and potential issues.

Prerequisites for the Java Client

You’ll need of course a subscription or trial to Microsoft Dynamics CRM 2011 Online. If you don’t have a Dynamics CRM Org. available, you can test drive it here: http://www.microsoft.com/en-us/dynamics/crm-test-drive.aspx.

Dynamics CRM Organization Web Service Stub

The Microsoft Dynamics CRM Organization web service stub can be generated with the WSDL2Java command, available right from the bin folder of Apache Axis2:

{Path to Axis2 Lib}\axis2-1.6.2\bin> WSDL2java -uri https://{Your CRM Org domain}.crm.dynamics.com/XRMServices/2011/Organization.svc?wsdl -p {Java package for the generated stub source} -s -o {Path for the generated stub source}

If you prefer to use Apache Ant in order to automate the stub generation, the Ant task would look like this in your build.xml file:

<!—Microsoft Dynamics CRM 2011 Organization Stub generation –>
<target name=“gen-organization-stub”>
    <taskdef  name=“axis2-wsdl2java”
               classname=“org.apache.axis2.tool.ant.AntCodegenTask”
               classpathref=“build.classpath”/>
    <axis2-wsdl2java wsdlfilename=https://{Your CRM Org domain}.crm.dynamics.com/XRMServices/2011/Organization.svc?wsdl packageName={Java package for the generated stub source}output={Path for the generated stub source}syncOnly=”true” /></target>

The URI prefix to the Dynamics CRM 2011 Organization Service should look like https://myCRMOrg.crm.dynamics.com/ if your Organization is hosted in North America, https://myCRMOrg.crm4.dynamics.com/ for EMEA and https://myCRMOrg.crm5.dynamics.com/ for APAC.

An example of Java package could be like “com.mycompany.dynamicscrm.integration” and the path for the generated stub source should be right within your Java project source folder.

Note that with the “s/syncOnly” option of the command, the generated stubs will contain exclusively the synchronous invocation methods.

A detailed documentation of the WSDL2Java command can be found here.

The generated source code for the stub should be a set of 9 Java classes:

Generated Organization Service Stub

OrganizationServiceStub.java is the main stub for the Microsoft Dynamics CRM 2011 Organization Service and the other classes are Java exceptions related to the service’s basic operations.

Authentication via the Microsoft Office 365 Secure Token Service (STS)

Typical STS Authentication request

Here is a typical SOAP/HTTPS request to authenticate a user through the Microsoft Office 365 STS with the associated main headers:

<s:Envelope
xmlns:s=http://www.w3.org/2003/05/soap-envelope&#8221;
        xmlns:a=http://www.w3.org/2005/08/addressing&#8221;
        xmlns:u=http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd&#8221;>
    <s:Header>
        <a:Action
s:mustUnderstand=“1”>http://schemas.xmlsoap.org/ws/2005/02/trust/RST/Issue</a:Action>
        <a:MessageID>urn:uuid:{Message ID}</a:MessageID>
        <a:ReplyTo><a:Address>http://www.w3.org/2005/08/addressing/anonymous</a:Address></a:ReplyTo>
        <VsDebuggerCausalityData
xmlns=http://schemas.microsoft.com/vstudio/diagnostics/servicemodelsink&#8221;>
            uIDPo2V68j15KH9PqGf9DWiAfGQAAAAA/Dr1z6qvqUGzr5Yv4aMcdIr9AKDFU7VHn7lpNp0zeXEACQAA</VsDebuggerCausalityData>
        <a:To
s:mustUnderstand=“1”>https://login.microsoftonline.com/RST2.srf</a:To>
        <o:Security
s:mustUnderstand=“1”
                xmlns:o=http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd&#8221;>
            <u:Timestamp
u:Id=“_0”>
                <u:Created>{Request Timestamp}</u:Created>
                <u:Expires>{Request Expiry Timestamp}</u:Expires>
            </u:Timestamp>
            <o:UsernameToken
u:Id={Token ID}>
                <o:Username>{User Name}</o:Username>
                <o:Password
                Type=http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText&#8221;>
                {User Password}</o:Password>
            </o:UsernameToken>
        </o:Security>
    </s:Header>
    <s:Body>
        <t:RequestSecurityToken
xmlns:t=http://schemas.xmlsoap.org/ws/2005/02/trust&#8221;>
            <wsp:AppliesTo
xmlns:wsp=http://schemas.xmlsoap.org/ws/2004/09/policy&#8221;>
                <a:EndpointReference>

                    <a:Address>urn:crmna:dynamics.com</a:Address>

                </a:EndpointReference>

            </wsp:AppliesTo>

                <t:RequestType>http://schemas.xmlsoap.org/ws/2005/02/trust/Issue</t:RequestType>

        </t:RequestSecurityToken>

    </s:Body>

</s:Envelope>

The Parameters
in red
are the dynamic information to define as variables in every authentication request:

  • {Message ID}: unique request Id, e.g. ‘0d457c4e-8b44-4100-8b7e-085ca6303c7f’,
  • {Request Timestamp} : timestamp of the request (now), e.g. ‘2013-08-17T09:32:27.786Z’,
  • {Request Expiry Timestamp}: timestamp when the request should expire (could be now + 5 minutes), e.g. ‘2013-08-17T09:37:27.786Z’,
  • {Token ID}: unique token Id, e.g. ‘uuid-8c6514f1-8cb5-4c6b-8c0f-e476c7fd7a90-1’,
  • {User Name}: User login, e.g. ‘admin@MyCRMorg.onmicrosoft.com’,
  • {User Password}: User password.

A typical successful authentication answer looks like this:

<?xml
version=“1.0”
encoding=“utf-8”
?>
<S:Envelope
xmlns:S=http://www.w3.org/2003/05/soap-envelope&#8221;

        xmlns:wsse=http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd&#8221;

        xmlns:wsu=http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd&#8221;

        xmlns:wsa=http://www.w3.org/2005/08/addressing&#8221;>
    <S:Header>
        <wsa:Action
xmlns:S=http://www.w3.org/2003/05/soap-envelope&#8221;

                xmlns:wsa=http://www.w3.org/2005/08/addressing&#8221;

                xmlns:wsu=http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd&#8221;

                wsu:Id=“Action”
S:mustUnderstand=“1”>http://schemas.xmlsoap.org/ws/2005/02/trust/RSTR/Issue</wsa:Action>
        <wsa:To
    xmlns:S=http://www.w3.org/2003/05/soap-envelope&#8221;

                xmlns:wsa=http://www.w3.org/2005/08/addressing&#8221;

                xmlns:wsu=http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd&#8221;

                wsu:Id=“To”
S:mustUnderstand=“1”>http://schemas.xmlsoap.org/ws/2004/08/addressing/role/anonymous</wsa:To>
        <wsse:Security
S:mustUnderstand=“1”>
            <wsu:Timestamp
xmlns:wsu=http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd&#8221;

                     wsu:Id=“TS”>
                <wsu:Created>2013-08-17T09:33:00Z</wsu:Created>
                <wsu:Expires>2013-08-17T09:38:00Z</wsu:Expires>
            </wsu:Timestamp>
        </wsse:Security>
    </S:Header>
    <S:Body>
        <wst:RequestSecurityTokenResponse
    xmlns:S=http://www.w3.org/2003/05/soap-envelope&#8221;

                    xmlns:wst=http://schemas.xmlsoap.org/ws/2005/02/trust&#8221;

                    xmlns:wsse=http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd&#8221;

                    xmlns:wsu=http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd&#8221;

                    xmlns:saml=“urn:oasis:names:tc:SAML:1.0:assertion”

                    xmlns:wsp=http://schemas.xmlsoap.org/ws/2004/09/policy&#8221;

                    xmlns:psf=http://schemas.microsoft.com/Passport/SoapServices/SOAPFault&#8221;>

            <wst:TokenType>urn:oasis:names:tc:SAML:1.0</wst:TokenType>

            <wsp:AppliesTo
xmlns:wsa=http://www.w3.org/2005/08/addressing&#8221;>

                <wsa:EndpointReference><wsa:Address>urn:crmna:dynamics.com</wsa:Address></wsa:EndpointReference>

            </wsp:AppliesTo>

            <wst:Lifetime>

                <wsu:Created>2013-08-17T09:33:00Z</wsu:Created>

                <wsu:Expires>2013-08-17T17:33:00Z</wsu:Expires>

            </wst:Lifetime>

            <wst:RequestedSecurityToken>

                <EncryptedData
    xmlns=http://www.w3.org/2001/04/xmlenc#&#8221;

                            Id=“Assertion0”

                            Type=http://www.w3.org/2001/04/xmlenc#Element&#8221;>

                    <EncryptionMethod
Algorithm=http://www.w3.org/2001/04/xmlenc#tripledes-cbc&#8221;>

                    </EncryptionMethod>

                    <ds:KeyInfo
xmlns:ds=http://www.w3.org/2000/09/xmldsig#&#8221;>

                        <EncryptedKey>

                            <EncryptionMethod
Algorithm=http://www.w3.org/2001/04/xmlenc#rsa-oaep-mgf1p&#8221;>

                            </EncryptionMethod>

                            <ds:KeyInfo
Id=“keyinfo”>

                                <wsse:SecurityTokenReference>

                                    <wsse:KeyIdentifier
EncodingType=http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary&#8221;

                                                 ValueType=http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509SubjectKeyIdentifier&#8221;>

                                        D3xjUG3HGaQuKyuGdTWuf6547Lo=

                                    </wsse:KeyIdentifier>

                                </wsse:SecurityTokenReference>

                            </ds:KeyInfo>

                            <CipherData>

                                <CipherValue>{Security Token 0}</CipherValue>

                            </CipherData>

                        </EncryptedKey>

                    </ds:KeyInfo>

                    <CipherData>

                        <CipherValue>{Security Token 1}</CipherValue>

                    </CipherData>

                </EncryptedData>

            </wst:RequestedSecurityToken>

            <wst:RequestedAttachedReference>

                <wsse:SecurityTokenReference>

                    <wsse:KeyIdentifier

                        ValueType=http://docs.oasis-open.org/wss/oasis-wss-saml-token-profile-1.0#SAMLAssertionID&#8221;>

                        {Key Identifier}

                    </wsse:KeyIdentifier>

                </wsse:SecurityTokenReference>

            </wst:RequestedAttachedReference>

            <wst:RequestedProofToken>

                <wst:BinarySecret>/NbucQb9wbn4h2sZZDpRzwsF8q88SeyP</wst:BinarySecret>

            </wst:RequestedProofToken>

        </wst:RequestSecurityTokenResponse>

    </S:Body>

</S:Envelope>

We can extract from this request the following credentials: the Security Token 0, the Security Token 1 and the Key Identifier.

A typical failed authentication answer, because of invalid credentials, looks like this:

<?xml
version=“1.0”
encoding=“utf-8”
?>

<S:Envelope
xmlns:S=http://www.w3.org/2003/05/soap-envelope&#8221;

        xmlns:wsse=http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd&#8221;

        xmlns:wsu=http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd&#8221;

        xmlns:wst=http://schemas.xmlsoap.org/ws/2005/02/trust&#8221;

        xmlns:psf=http://schemas.microsoft.com/Passport/SoapServices/SOAPFault&#8221;>

    <S:Body>

        <S:Fault>

            <S:Code>

                <S:Value>S:Sender</S:Value>

                <S:Subcode>

                    <S:Value>wst:FailedAuthentication</S:Value>

                </S:Subcode>

            </S:Code>

            <S:Reason><S:Text
xml:lang=“en-US”>Authentication Failure</S:Text></S:Reason>

            <S:Detail><psf:error>

                <psf:value>0x80048821</psf:value>

                <psf:internalerror>

                    <psf:code>0x80041012</psf:code>

                    <psf:text>The entered and stored passwords do not match.</psf:text>

                </psf:internalerror>

            </psf:error></S:Detail>

        </S:Fault>

    </S:Body>

</S:Envelope>

Implementation in Java

First create a template with the full authentication SOAP request’s envelope and tokenize the dynamic variables (tokens are defined with %S):

// SOAP envelope template for MSDC Online STS authentication (O365 online platform)
static
public
final String MSDC_ONLINE_AUTH_SOAP_ENVELOPE_TEMPLATE =
“<?xml version=\”1.0\” encoding=\”UTF-8\”?>” +
“<s:Envelope xmlns:s=\”http://www.w3.org/2003/05/soap-envelope\” “ +
“xmlns:a=\”http://www.w3.org/2005/08/addressing\” “ +
“xmlns:u=\”http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd\”>” +
” <s:Header>” +
” <a:Action s:mustUnderstand=\”1\”>http://schemas.xmlsoap.org/ws/2005/02/trust/RST/Issue</a:Action>&#8221; +
” <a:MessageID>urn:uuid:%s</a:MessageID>” +
” <a:ReplyTo><a:Address>http://www.w3.org/2005/08/addressing/anonymous</a:Address></a:ReplyTo>&#8221; +
” <VsDebuggerCausalityData xmlns=\”http://schemas.microsoft.com/vstudio/diagnostics/servicemodelsink\”>uIDPo2V68j15KH9PqGf9DWiAf
GQAAAAA/Dr1z6qvqUGzr5Yv4aMcdIr9AKDFU7VHn7lpNp0zeXEACQAA</VsDebuggerCausalityData>” +
” <a:To s:mustUnderstand=\”1\”>https://login.microsoftonline.com/RST2.srf</a:To>&#8221; +
” <o:Security s:mustUnderstand=\”1\” xmlns:o=\”http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd\”>” +
” <u:Timestamp u:Id=\”_0\”>” +
” <u:Created>%s</u:Created>” +
” <u:Expires>%s</u:Expires>” +
” </u:Timestamp>” +
” <o:UsernameToken u:Id=\”%s\”>” +
” <o:Username>%s</o:Username>” +
” <o:Password Type=\”http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText\”>
%s</o:Password>” +
” </o:UsernameToken>” +
” </o:Security>” +
” </s:Header>” +
” <s:Body>” +
” <t:RequestSecurityToken xmlns:t=\”http://schemas.xmlsoap.org/ws/2005/02/trust\”>” +
” <wsp:AppliesTo xmlns:wsp=\”http://schemas.xmlsoap.org/ws/2004/09/policy\”>” +

” <a:EndpointReference>” +

” <a:Address>urn:crmna:dynamics.com</a:Address>” +

” </a:EndpointReference>” +

” </wsp:AppliesTo>” +

” <t:RequestType>http://schemas.xmlsoap.org/ws/2005/02/trust/Issue</t:RequestType>&#8221; +

” </t:RequestSecurityToken>” +

” </s:Body>” +

“</s:Envelope>”;

Generate the dynamic token values and prepare the SOAP request in order to authenticate a user through the Microsoft Online STS:

// Prepare input parameter for CRM Authentication Request
// > Random Message Id
String paramMessageId = UUID.randomUUID().toString();
// > Request Timestamp and +5 minutes validity
TimeZone gmtTZ = TimeZone.getTimeZone(“GMT”);
SimpleDateFormat formatter = new SimpleDateFormat(“yyyy-MM-dd’T’HH:mm:ss”);
formatter.setTimeZone(gmtTZ);
Calendar calendar = Calendar.getInstance(gmtTZ);
Date timestampRequest = calendar.getTime();
calendar.add(Calendar.MINUTE, 5);
Date timestampExpiryRequest = calendar.getTime();
String paramTimestampRequest = formatter.format(timestampRequest);
String paramTimestampExpiryRequest = formatter.format(timestampExpiryRequest);
// > Random Token Id
String paramTokenId = “uuid-“ + UUID.randomUUID().toString() + “-1”;
// Prepare CRM Online authentication SOAP request
String onlineCRMAuthSOAPEnvelope = String.format(
    MSDC_ONLINE_AUTH_SOAP_ENVELOPE_TEMPLATE,
    paramMessageId,
    paramTimestampRequest,
    paramTimestampExpiryRequest,
    paramTokenId,
    userName,
    userPassword);

Now it is time to send the authentication request to the Microsoft Online STS, via SOAP/HTTPS …

// Send CRM Online authentication SOAP request to Microsoft online STS
String onlineCRMAuthResponseXML = postSOAPRequest(
    “https://login.microsoftonline.com/RST2.srf“,
    onlineCRMAuthSOAPEnvelope);

… For that, we can leverage the Apache HTTP Components:

public
static String postSOAPRequest(URI serviceUri, String soapEnvelope) throws ParseException, IOException {
    HttpResponse response = null;
    HttpParams params = new BasicHttpParams();
    params.setParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, 180000); //time out in ms
    HttpClient client = new DefaultHttpClient(params);
HttpPost post = new HttpPost(serviceUri);
StringEntity entity = new StringEntity(soapEnvelope);
post.setHeader(“Content-Type”, “application/soap+xml; charset=UTF-8”);
post.setEntity(entity);
response = client.execute(post);

return EntityUtils.toString(response.getEntity());
}

Finally, parse the SOAP response from the STS and gather 3 credentials: securityToken0, securityToken1
and keyIdentifier.

If the credentials cannot be found, then we assume we are facing a failed authentication answer. In that case, the reason and the detail of the error can be parsed instead:

// Parse the CRM Online authentication SOAP response from STS
// Create a Java DOM XML Parser
DocumentBuilderFactory builderFactory = DocumentBuilderFactory.newInstance();
DocumentBuilder builder = builderFactory.newDocumentBuilder();
// Parse XML with Java DOM XML Parser
Document xmlDocument = builder.parse(new ByteArrayInputStream(onlineCRMAuthResponseXML.getBytes()));
            
// Retrieve security tokens and key identifier from security token response.
XPath xpath = XPathFactory.newInstance().newXPath();
String securityToken0 = readStringValFromXmlDocument(xmlDocument, “//*[local-name()=’CipherValue’]”,xpath);

// If first token is blank, search eventual authentication failure message
if((securityToken0==null)||(securityToken0.isEmpty())){
    String errorReason = readStringValFromXmlDocument(xmlDocument, “//*[local-name()=’Reason’]”,xpath);
    String errorDetail = readStringValFromXmlDocument(xmlDocument, “//*[local-name()=’Detail’]”,
        xpath).substring(20);
            
    if((errorReason!=null)&&(errorReason.equalsIgnoreCase(“Authentication Failure”))){
        logger.debug(“Failed authentication for User ‘” + userName + “‘. Reason is ‘” + errorReason +
                            “‘ and Detail is “ + errorDetail);
        throw
new MSDCAuthenticationException(errorDetail); // Exception to craft according to your needs
    } else {
        logger.debug(“Failed authentication for User ‘” + userName + “‘ but cannot parse the reasons”);
        throw
new MSDCAuthenticationException (“Failed authentication for unexpected reasons);
    }
}
String securityToken1 = readStringValFromXmlDocument(xmlDocument, “(//*[local-name()=’CipherValue’])[2]”,xpath);
String keyIdentifier = readStringValFromXmlDocument(xmlDocument, “//*[local-name()=’KeyIdentifier’]”, xpath);

Implementation for the method readStringValFromXmlDocument: 

public static String readStringValFromXmlDocument(Document xmlDocument, String xpathQueryExpression, XPath xpathInstance) throws XPathExpressionException {return xpathInstance.compile(xpathQueryExpression).evaluate(xmlDocument);
}

Calling Dynamics CRM Organization Service Invocation Methods in Java

Instantiation of the Organization Service Stub

In order to instantiate the Organization Service Stub from the Java classes generated from the WSDL, we must first configure Axis2 by creating a configuration context from the ‘axis2.xml’ file that we placed at the root of the project source code. This file is a copy of the standard ‘axis2.xml’ file that can be found in the ‘conf’ directory of the Axis2 Java library.

We need to also pass, along with the configuration context, the URL of the Dynamics CRM 2011 Online Organization.

// Create OrganizationServiceStub
String fileSeperator = System.getProperty(“file.separator”);
String userDir = System.getProperty(“user.dir”);
String axis2ConfigFilePath = userDir + fileSeperator + “src” + fileSeperator + “axis2.xml”;
                
ConfigurationContext ctx = ConfigurationContextFactory.createConfigurationContextFromFileSystem(userDir, axis2ConfigFilePath);
organizationServiceStub = new OrganizationServiceStub(ctx, https://myCRMOrg.crm.dynamics.com);
                
// Get service client implementation used by this stub.
serviceClient = organizationServiceStub._getServiceClient();

We keep most of the default parameters in the ‘axis2.xml’ file except a custom inflow predefined Phase called ‘MustUnderstandChecker’ that must be declared in this file:

. . .
<!– ================================================= –>
<!– Phases –>
<!– ================================================= –>
<phaseOrder
type=“InFlow”>
. . .            
    <phase
name=“RMPhase”/>
    <!– System predefined phases –>
    <!– After Postdispatch phase module author or service author can add any phase he want –>
     <phase
name=“OperationInPhase”>
    <handler
name=MustUnderstandChecker
    class=
com.mycompany.dynamicscrm.integration.Axis2MustUnderstandChecker>
        <order
phase=“OperationInPhase”/>
    </handler>
    </phase>
    . . .
</phaseOrder>
. . .

Here is an implementation of the Axis2MustUnderstandChecker class that tells Axis2 client to process the security SOAP header block from the message context header:

/***
* Handler for SOAP header.
*
*/
public
final
class Axis2MustUnderstandChecker extends AbstractHandler {

public Axis2MustUnderstandChecker() {
}

/* (non-Javadoc)
* Process the Security SOAP header block from the message context header.
* @see org.apache.axis2.engine.Handler#invoke(org.apache.axis2.context.MessageContext)
*/

public InvocationResponse invoke(MessageContext msgContext)

throws AxisFault {
SOAPHeader header = msgContext.getEnvelope().getHeader();

if (header != null) {
Iterator<?> blocks = header.examineAllHeaderBlocks();

while (blocks.hasNext()) {
SOAPHeaderBlock block = (SOAPHeaderBlock) blocks.next();

if(block != null){     
    if (block.getLocalName().equals(“Security”)) {
        block.setProcessed();
    }
}
}
}

return InvocationResponse.CONTINUE;
}
}

SOAP Requests options and Security SOAP header block

After the instantiation of the Organization Service Stub, some options must be set and a fresh Security SOAP header block must be defined:

try {
    Options scOptions = serviceClient.getOptions();
    scOptions.setMessageId(“urn:uuid:” + UUID.randomUUID().toString());
    EndpointReference endPoint = new EndpointReference(http://www.w3.org/2005/08/addressing/anonymous&#8221;);
    scOptions.setReplyTo(endPoint);
    serviceClient.setOptions(scOptions);
            
    // Add fresh Security SOAP Header block
    serviceClient.addHeader(generateFreshSecuritySoapHeaderBlock(securityHeader));
    serviceClient.engageModule(“addressing”);
            
} catch (AxisFault af) {
            throw
new MSDCIntegrationException(“Unexpected web service error”, af);
}

Here is the detail of the method ‘generateFreshSecuritySoapHeaderBlock’
that generate a brand new security SOAP header block including some fresh timestamps with a validity period (set arbitrarily in our case to 5 minutes) and a security header embedding the credentials from the Microsoft Online STS authentication:

private SOAPHeaderBlock generateFreshSecuritySoapHeaderBlock (String securityHeaderStr) throws XMLStreamException {
        
    SOAPHeaderBlock securitySoapHeaderBlock = null;
    OMFactory omFactory = OMAbstractFactory.getOMFactory();
    OMNamespace securitySecextNS = omFactory.createOMNamespace(http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd&#8221;, “o”);
    OMNamespace securityUtilityNS = omFactory.createOMNamespace(http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd&#8221;, “u”);
    
    // Create fresh Time stamp element for the SOAP header block
    TimeZone gmtTZ = TimeZone.getTimeZone(“GMT”);
    SimpleDateFormat formatter = new SimpleDateFormat(“yyyy-MM-dd’T’HH:mm:ss”);
    formatter.setTimeZone(gmtTZ);
    Calendar calendar = Calendar.getInstance(gmtTZ);
    Date timestampRequest = calendar.getTime();
    calendar.add(Calendar.MINUTE, 5);
    Date timestampExpiryRequest = calendar.getTime();
    String timestampRequestStr = formatter.format(timestampRequest);
    String timestampExpiryRequestStr = formatter.format(timestampExpiryRequest);
    
    OMElement timeStampElement = omFactory.createOMElement(“Timestamp”, securityUtilityNS);
    timeStampElement.addAttribute(“Id”, “_0”, securityUtilityNS);
    OMElement createdElement = omFactory.createOMElement(“Created”, securityUtilityNS);
    OMText createdTime = omFactory.createOMText(timestampRequestStr + “Z”);
    createdElement.addChild(createdTime);
    OMElement expiresElement = omFactory.createOMElement(“Expires”, securityUtilityNS);
    OMText expiresTime = omFactory.createOMText(timestampExpiryRequestStr + “Z”);
    expiresElement.addChild(expiresTime);
    timeStampElement.addChild(createdElement);
    timeStampElement.addChild(expiresElement);

    // Create the Security SOAP header block and add, as a child, Time stamp element

    securitySoapHeaderBlock = OMAbstractFactory.getSOAP12Factory().createSOAPHeaderBlock(“Security”, securitySecextNS);

    securitySoapHeaderBlock.setMustUnderstand(true);

    securitySoapHeaderBlock.addChild(timeStampElement);

    securitySoapHeaderBlock.addChild(AXIOMUtil.stringToOM(omFactory, securityHeaderStr));

    

    return securitySoapHeaderBlock;

}

The method ‘generateFreshSecuritySoapHeaderBlock’ takes the Security Header string as an input parameter and it can be built with the following template:

// Security header template
static
public
final String MSDC_SECURITY_HEADER_TEMPLATE =
“<EncryptedData xmlns=\”http://www.w3.org/2001/04/xmlenc#\” Id=\”Assertion0\” Type=\”http://www.w3.org/2001/04/xmlenc#Element\”>” +
”    <EncryptionMethod Algorithm=\”http://www.w3.org/2001/04/xmlenc#tripledes-cbc\”/>” +
”    <ds:KeyInfo xmlns:ds=\”http://www.w3.org/2000/09/xmldsig#\”>” +
”        <EncryptedKey>” +
”            <EncryptionMethod Algorithm=\”http://www.w3.org/2001/04/xmlenc#rsa-oaep-mgf1p\”/>” +
”            <ds:KeyInfo Id=\”keyinfo\”>” +
”                <wsse:SecurityTokenReference xmlns:wsse=\”http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd\”>” +
”                    <wsse:KeyIdentifier EncodingType=\”http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary\” “ +
”                        ValueType=\”http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-x509-token-profile-1.0#X509SubjectKeyIdentifier\”>%s</wsse:KeyIdentifier>” +
”                </wsse:SecurityTokenReference>” +
”            </ds:KeyInfo>” +
”            <CipherData>” +
”                <CipherValue>%s</CipherValue>” +
”            </CipherData>” +
”        </EncryptedKey>” +
”    </ds:KeyInfo>” +
”    <CipherData>” +
”        <CipherValue>%s</CipherValue>” +
”    </CipherData>” +
“</EncryptedData>”;

Generate the security header string with the following Java code, replacing the 3 tokens in the template by the credentials obtained from the Microsoft Online STS authentication:

// Generate security header
securityHeader = String.format(
    MSDC_SECURITY_HEADER_TEMPLATE,
    keyIdentifier,
    securityToken0,
    securityToken1);

Stub Invocation Methods

Here is an example of a high level ‘Create’ method that create an object in Dynamics CRM 2011 Online with the following parameters:

  • entityLogicalName: the logical name of the entity to create, e.g. ‘account’,
  • attributesHash: an HashMap of attributes (pairs of attribute keys and values), e.g. {“name”, “Test Account”},{“address1_city”, “Paris”}, etc.,
  • organizationService: the Dynamics CRM Organization Service stub instantiated and augmented with the options and the fresh Security SOAP header block.
public String executeCreate(String entityLogicalName, HashMap<String,String> attributesHash, OrganizationServiceStub organizationService) throws MSDCIntegrationException {
        
    // Transfer attributes from HashMap to Stub attribute collection
    OrganizationServiceStub.AttributeCollection attributeCollection = new OrganizationServiceStub.AttributeCollection();
    Iterator<String> it = attributesHash.keySet().iterator();
    while(it.hasNext()){
        String key = it.next();
        OrganizationServiceStub.KeyValuePairOfstringanyType KeyValuePair =
            new OrganizationServiceStub.KeyValuePairOfstringanyType();
         KeyValuePair.setKey(key);
     KeyValuePair.setValue(attributesHash.get(key));
     attributeCollection.addKeyValuePairOfstringanyType(KeyValuePair);
    }
    // Create Entity with attributes
    OrganizationServiceStub.Entity entity = new OrganizationServiceStub.Entity();
entity.setLogicalName(entityLogicalName);
entity.setAttributes(attributeCollection);
OrganizationServiceStub.Create createEntity = new OrganizationServiceStub.Create();
createEntity.setEntity(entity);

// Send Create command to Organization web service
String resultGuid = null;

try {
    OrganizationServiceStub.CreateResponse createResponse;
        createResponse = organizationService.create(createEntity);
        OrganizationServiceStub.Guid createResultGuid = createResponse.getCreateResult();
        resultGuid = createResultGuid.getGuid();    
    } catch (RemoteException
| IOrganizationService_Create_OrganizationServiceFaultFault_FaultMessage e) {
        throw
new MSDCIntegrationException(“Unexpected web service error”,
e);

    }

    logger.debug(“Entity ‘” + entityLogicalName + “‘ created successfully with GUID = “ + resultGuid);


return resultGuid;

}

Here are some of the other public methods you can invoke on the Organization Service stub:

  • Associate: Creates a link between records.
  • Delete: Deletes a record.
  • Disassociate: Deletes a link between records.
  • Execute: Executes a message in the form of a request, and returns a response.
  • Retrieve: Retrieves a record.
  • RetrieveMultiple: Retrieves a collection of records.
  • Update: Updates an existing record.

Performance considerations

Instantiating the Organization Service stub can takes up to 75% of the time of a single transaction, this is why this operation should be minimized at any cost.

It is recommend to keep the same stub instance with one security header for a few consecutive operations within a single stateless transaction. The security SOAP header block has got a timestamp with a validity period and this period should not expire.

If the application is stateful, the stub can be cached. Once instantiated, the security SOAP header block can be regenerated for each further usages. The cost of this operation is minimal.

Before to rebuild a security SOAP header block, the headers of the stub’s service client must be cleared with the command ‘removeHeaders()’:

// Recycling organizationServiceStub
serviceClient = organizationServiceStub._getServiceClient();
// Remove existing headers in order to produce a fresh one
serviceClient.removeHeaders();

Troubleshooting

  • Always make sure that the account used to authenticate on the Dynamics CRM server is valid and that it has the required privileges to execute the desired operations on the Organization service.
  • Undeclared namespace prefix “wsx” Exception at Java runtime when instantiating OrganizationServiceStub:
    • Exception detail: org.apache.neethi.builders.converters.ConverterException: com.ctc.wstx.exc.WstxParsingException: Undeclared namespace prefix “wsx”
    • Probable cause: the generated stub is declaring the “wsx” domain for some tags that are already embedded within some tags from the same domain and the domain is locally declared and not with a global prefix.
    • Proposed solution: edit the generated Organization Service stub “OrganizationServiceStub.java”, search and replace:
      • wsx:MetadataReference by MetadataReference
      • wsx:MetadataSection by MetadataSection
  • The code samples exposed in this article have been tested only with Dynamics CRM 2011 Online (version 5.0.9690.5010, DB 5.0.9690.3417) and not yet with Dynamics CRM 2013. Some changes will need to be implemented once this new version is officially released.

Conclusion

This article has demonstrated a reliable and quite easy to implement way to integrate Dynamics CRM 2011 Online with Java. The different network frames and algorithms have been detailed so they can be adapted with other languages.

The codes samples of this article have been tested in development but in no mean on a production environment. So prior to implement it in production, adequate testing should be considered and this project will be at your own risks.

The next steps would be to push further this approach to Dynamics CRM On Premise:

  • Internet Facing Deployment with ADFS 2.0
  • Windows authentication

References

Source Code on GitHub

You’ll find a full implementation of the approach explained in this article on the following GitHub repository: https://github.com/pdellecase/CRMJavaConnect

MSDN CoE Blog

This article is also published on the Microsoft Dynamics Center of Excellence Blog : http://blogs.msdn.com/b/dynamics-coe/archive/2013/09/21/integrating-microsoft-dynamics-crm-2011-online-with-java-and-other-non-net-clients.aspx

Microsoft Developer Network article: Connect Using Java for Microsoft Dynamics CRM Online

This great article walks you through a basic sample code to integrate Dynamics CRM 2011 Online with Java. The authentication is through LiveID and would not work anymore for recently provisioned Dynamics CRM 2011 Online Organizations.

At the beginning, I was trying hard to generate the Microsoft Dynamics CRM Organization Service stubs with the Java API for XML Web Services (JAX-WS), without any success. This article put me on the right track with the Apache Axis2 library that can parse the Organization Service WSDL without any fatal errors.

Link to the MDN article: http://msdn.microsoft.com/en-us/library/jj602979.aspx

Microsoft Developer Network article: Download the Endpoints Using the Dynamics CRM Developer Resources Page

This article explains how to download the WSDL from the developer resources pages in the Microsoft Dynamics CRM 2011 and Microsoft Dynamics CRM Online Web application. It provides also some details about the Discovery Service and the Organization Service WSDLs.

Link to the MDN article: http://msdn.microsoft.com/en-us/library/gg309401.aspx

Microsoft Developer Network article: IOrganizationService Interface > IOrganizationService Members

This article list the public methods members exposed by the IorganizationService interface, with all their parameters, and is a good starting point to find practical samples of code.

Link to the MDN article: http://msdn.microsoft.com/en-us/library/microsoft.xrm.sdk.iorganizationservice_members.aspx

The Microsoft Dynamics CRM 2011 Software Development Kit (SDK)

The official SDK for Dynamics CRM provided by Microsoft contains many useful .Net samples to get started. I have used some of these samples to monitor, with a tool like Fiddler, the HTTPS traffic between my PC and Microsoft Dynamics CRM in order to gather the authentication request sent to the Office 365 Secure Token Service and the security header from the SOAP requests.

Link to the Microsoft Dynamics CRM 2011 SDK Documentation: http://msdn.microsoft.com/en-us/library/hh547453.aspx

Link to the Microsoft Dynamics CRM 2011 SDK: http://www.microsoft.com/en-us/download/details.aspx?id=24004