Skip to content

Latest commit

 

History

History
82 lines (57 loc) · 3.82 KB

README.md

File metadata and controls

82 lines (57 loc) · 3.82 KB
Raw

Xrm.DataManager.Framework

Nuget framework for massive Microsoft Dataverse data operations.

🚀 Features

Xrm.DataManager.Framework provides folling features :

  • Batching patterns : Full, Delta, ...
  • Performances : Perf tuning, multi threading, ...
  • Errors handling and retry : API Limits catching and waits
  • Tracing : Supports file output, App Insights and Graylog...
  • Configuration management : Simplified conf ...

Xrm.DataManager.Framework is designed to simplify dev work by reducing core development and focus to functional implementation.

👐 Community contributions

This project is open to community contributions in order to enrich this tooling and helps everyone to bring more capabilities for Microsoft Dataverse data operations.

✈️ Installation and configuration

Installation

Create a new C# console app projet and add Xrm.DataManager.Framework Nuget package.

Configuration

You need to specify following parameters in 'app.config':

Parameter name Default or available values Description
Crm.ConnectionString Specify connection string to Microsoft Dataverse
Job.Names ex: DeleteAccountsJobs, DeleteContactsJob Specify if you want to run multiple jobs (classe names splitted with ',')
Process.Duration.MaxHours Default: 8 Max job duration, to prevent nigh batch to run in prod during the day
Process.Query.RecordLimit Default: 2500 Records count to retrieve for each page (max 5000)
Process.Thread.Number Default: 10 Specify the number of threads to run
LogLevel Verbose = 0,
Information = 1,
ErrorsAndSuccess = 2,
ErrorsOnly = 3		 Tracing level
AppInsights.Instrumentation.Key Guid Application Insights telemetry key
Graylog.Url URL Graylog Gelf endpoint URI

Usage

3 patterns are available:

  • PickAndProcessDataJob : Query X rows from given QueryExpression and process them then loop as long as data is retrieved.
  • FullProcessDataJob : Query all rows from given QueryExpression, load all data in memory and process.
  • InputFileProcessDataJob : Map a csv or txt file to table/columns and process each line.

Theses patterns provide following overridable methods:

  • GetName : Set a display name for job.
  • IsEnabled : Specify if job is runnable or not.
  • OverrideThreadNumber : Specify thread number for current job (For debug only)
  • PreOperation : This method is called before proceeding data, you can use it to put data in cache for example or disable audit or whatever.
  • GetQuery : Specify the query expression to retrieve data.
  • PrepareData : Usefull with FullProcessDataJob to apply data aggregation like groupings.
  • ProcessRecord : Apply custom operation for a single row.
  • PostOperation : This method is called after all data have been processed, you can use it to send notification email or reactivate audit or whatever.