Asynchronous Batch Process Pattern: Part 3

October 12, 2012

19 comments

In the previous post I have described a possible implementation of the Asynchronous Batch Process Pattern described here.

In this post, I would like to demonstrate the actual implementation in action, using a simple business requirement.

Business Scenario

To demonstrate the implementation process, the following business requirement will be used:

The Krusty Krab restaurant manages customer service calls using Microsoft Dynamics CRM Online. Each service call is assigned a due date according to the organization SLA.
Squidward Tentacles
, the Customer Service dept. manager, requires that all active Cases SLA will be automatically inspected daily. If the Follow Up By date (SLA due date) occurs today (inspection day), the Case priority is automatically updated to ‘High’, so it is promoted to the beginning of the work queue. He would also like the Customer Service Representative who owns such Case to receive an email notification regarding the impending SLA exception.

Solution Implementation Walkthrough

The following steps demonstrate the solution implementation:

  1. Install the Asynchronous Batch Process Pattern solution

    Import the Asynchronous Batch Process Pattern managed solution available here. Please do not install this solution on your production environment before thoroughly testing it in your test environment.  


  2. Set up an Action Workflow rule for the required business entity          

    Create and activate the Action Workflow rule for the Case entity to execute the required business logic. Make sure the rule is available to run as an on-demand process and with no automatic events. The rule does not include a Wait step, as the Scheduling Workflow rule takes care of that.
    In this case, the Action Workflow rule changes the Case priority to ‘High’ and then send an email notification to the Case owner.
    This sample Workflow rule is included in the Asynchronous Batch Process Pattern solution.

    Action Workflow Definition

  3. Define target business records query

    Open Advanced Find and define a query that will return the business records to which the Action Workflow will be applied. Use the Results button to view the business records that may be affected by the Action Workflow.
    To get the query definition, click the Download FetchXML button and save the file to your file system. Open the saved file in a text editor to copy the FetchXML used in the next step.
    Note that FetchXML query result is limited to 5000 records.
    In my case, I am defining a query that will return all Case records which are in Active state and Follow Up By equals Today

    FetchXML Query definition

  4. Set up a Batch Process record

    Create and save a Batch Process record to define

    • Name – A meaningful name to indicate the batch process purpose

    • Action Workflow – The business logic Workflow rule to be activated on schedule (defined in step 2 above)

    • Activation Frequency – The frequency in which the Action Workflow will be activated. In this case, the batch process is set to run daily.
    • Next Activation – The next date and time the Action Workflow will be activated and operate on the target records. In this case the batch process is set to run on 21:10.

    • Target Records – a FetchXML query defining target business records to which the Action Workflow will be applied (defined in step 3)

    • Status – Set the status to ‘Scheduled’ in order to enable the Batch Process automation

      2

  5. Batch Process in action

    You can see the state of my sample Batch Process record after few days. According to the Batch Process definition, each day at 21:10 the Scheduling Workflow rule runs and changes the Batch Process record status to Active. This triggers the Executing Plugin which query for the target business records using the FetchXML query and apply the Action Workflow rule to the result business records.
    Disregard the sudden date skip on 10/11 which was caused by the Online organization scheduled deactivation by Microsoft.  They were kind enough to reactivate the organization per my request.

    Batch Process In Action

    For each affected Case record, you can view the instances of Action Workflow rules that were applied by the Plugin. The Case priority was changed to ‘High’ and a notification email has been sent to the Case owner

    Case Record Action Workflow

    7

    Case notification email

Implementation Notes

  • There are some tasks to complete in this solution:
    • Diagnostics: update batch process record with last activation result (success/failure), failure reason and trace report to indicate affected business records
    • Input validation: testing for query and Action Workflow match to validate that the FetchXML query returns records of the entity the Action Workflow is design for
  • What about operations such as Share, which are not supported by the Action Workflow native steps? Triggering a Plugin directly via code could have been useful here, but we don’t have this option yet. Currently, the best solution I can think of is adding a custom code to the executing Plugin component, to apply the required business logic operation (like Share) to the target business records.
    This makes the plugin executing  component less generic and strays from the Asynchronous Batch Process Pattern guidelines.
  • As a side effect, this solution solves a different problem: It allows you to apply a manual Workflow rule to multiple business records without the grid page size limit (250 records), since the Action Workflow is applied to all business records that were returned by the FetchXML query.
    Please note, FetchXML query result is limited to 5000 records. One option to overcome this limitation is to convert the FetchXML query to QueryExpression in the Plugin component using the FetchXmlToQueryExpressionRequest class.  
  • The business requirement I used can be also implemented using a ‘recursive‘ Workflow rule like the I have demonstrated here. There are two major problems with this type of implementation method:
    • For each recursive iteration per each business record, a Wait Workflow rule instance is created. In large scales, Wait condition (monitored by the CRMAsyncService) may consume vast system resources, affecting the application over whole performance.
      The  Asynchronous Batch Process Pattern solution monitor only one running Wait Workflow rule instance (per batch process) and therefor consumes less resources.
    • The definition of a recursive workflow rule is complicated and difficult to manage as the Scheduling logic is mixed with the action logic. The Asynchronous Batch Process Pattern separates the scheduling and action to different scopes, simplifying  authoring and management

Conclusion

The Asynchronous Batch Process Pattern is useful in many business scenarios. Even when Custom Activities will be available in Online deployments, this pattern has an advantage of low resources consumption when compared to activating Workflow Wait steps in large scales.
While the Asynchronous Batch Process Pattern can be implemented in various ways, the suggested implementation main purposes are Online support and MSCRM2011 Solution containment.

I would like to hear your suggestions and remarks on improving this implementation concept.

Add comment
facebook linkedin twitter email

Leave a Reply

Your email address will not be published.

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

*

19 comments

  1. John GraceOctober 12, 2012 ב 10:02 PM

    Your solution looks great & is very similar to the Scheduler we have built. Here is a video of it in action & there’s a free community edition.

    Let me know what you think?

    http://www.youtube.com/watch?v=y7qPQUbSK-0&list=UUv9teTG1oXhrpkuNbEKcIsA&index=2&feature=plcp

    Reply
  2. YanivOctober 17, 2012 ב 8:49 AM

    Hi John,
    Thanks.
    This is indeed a similar solution concept, minus the Formula Manager.
    I guess great minds think alike 🙂
    Yaniv

    Reply
  3. John GraceOctober 19, 2012 ב 11:47 AM

    Yeah, you are correct no need for the formula stuff, you could just roll your own logic.

    Looking forward to your next article.

    John

    Reply
  4. Jonas RappOctober 22, 2012 ב 3:51 PM

    Hi Yaniv – this was a very neat solution, with good step-by-step-description and an appropriate level of technical details.
    Especially like the way you handle the scheduling. Will use!
    Keep it up 🙂

    /Jonas

    Reply
  5. Yaniv ArditiOctober 22, 2012 ב 9:30 PM

    Thanks Jonas, your comment is appreciated. Hope you’ll find this solution useful

    Reply
  6. KCMarch 14, 2013 ב 5:05 AM

    Hi, very good post and thanks for sharing. may i know if the CRM Asynchronous Processing Service is stop and restarted. Can the batch process resume to run automatically?

    Reply
  7. lironMarch 17, 2013 ב 6:36 AM

    thanks! beautiful solution!
    easy to use and reuse, even by non-experts, like me 🙂

    Reply
  8. lironMarch 17, 2013 ב 6:36 AM

    thanks! beautiful solution!
    easy to use and reuse, even by non-experts, like me 🙂

    Reply
  9. TimMarch 30, 2015 ב 11:38 PM

    While this is somewhat old now, I’m wondering if this solution will import into CRM 2015 without issue? As a new user, I’m trying to follow this along, and there’s virtually nothing, at this point, for CRM 2015.

    Reply
    1. rdt
      rdtJuly 5, 2015 ב 6:56 PM

      Hi Tim,

      I will publish a version compatible with 2015 soon.
      Stay tuned.

      Reply
  10. Pingback: Asynchronous Batch Process Solution Revisited – part 1 | Yaniv Arditi

  11. Pingback: Asynchronous Batch Process Solution Revisited – part 1 | YanivRDT

  12. Pingback: Asynchronous Batch Process Solution Revisited – part 1 - Microsoft Dynamics CRM Community

  13. Pingback: Asynchronous Batch Process Solution Revisited – part 1 | Dynamics Tips

  14. Pingback: Hosk’s Top CRM Articles of the week – 24th July – Hosk's Dynamic CRM Blog

  15. Pingback: Hosk’s Top CRM Articles of the week – 24th July - Microsoft Dynamics CRM Community

  16. Tim DutcherJuly 29, 2015 ב 2:56 AM

    We had a client who needed to run workflows on a schedule but upwards of 10,000 records on a regular basis.

    Our solution involved an Azure worker role that retrieved scheduling requests from CRM (sent out with a Service Endpoint) and ran the workflows on FetchXml results.

    We used Azure Scheduler to set the schedules (programmatically). When the scheduler job reached its time/interval it sends a message that the worker role picks up and then runs the correct batch workflow job in CRM.

    I also like the internal-to-CRM approaches I’ve seen others come up with. The approach is really dictated by the requirements. In our case, we solved the schedule workflow problem but now also have a bulk update tool that can be used for any number of records.

    It’s interesting why Microsoft hasn’t added this functionality. I’m sure it’s been on a short list for years but keeps getting crossed out for some reason. I suppose that there’s a risk that the feature can be abused. For example, someone could schedule a workflow to run on 1,000 records every minute. That could be bad. 🙂

    Reply
  17. MohanDecember 21, 2015 ב 1:58 PM

    Do we have 2015 version of solution is available?

    Reply