Azure DevOps Integration: Data Validation Report

Two-way data synchronization implies that both systems can be inaccessible for different periods of time and data can get out of sync. With the data validation report, you are able to find the items that are not synced properly and fix the consequences right from the report.

Who can access the report?

Targetproccess Administrators, as they have access to Targetprocess Settings –> Integrations and therefore to ‘Data validation report’ tab in every issue level integration profile.

What errors could be found and fixed?

With the report, you can find three groups of errors or improper sync consequences.

1. Conflicting fields: out-of-sync fields in all pairs of linked issues.
2. Lost and not imported items: issues that match the integration settings and can be imported.
3. Invalid pairs: pairs, where one of the issues does not exist or its ID/Type changed and issues were lost during sync.

Scopes of the report

Scope for detecting fields out of sync

The first option "Identify conflicts among the synchronized items' fields" will scan all shared issues and detect mismatches in fields values between Targetprocess and Azure DevOps:

To bigger integrations additional fitlers are recommended. You can use DSL filters as in the Targetprocess boards and lists and scan conflcits in selected entities only.

For example, it could be ?ModifyDate == '20-Jul-2023' to find if there are mismatches in the data updated on July 20, 2023 in Targetprocess.

Or it could be a WIQL fitler to select items for scanning from Azure DevOps perspective. Remember to use only content of the WHERE clause only:

[System.AreaPath] UNDER 'ADO_Project\ADO_Team'

Conflicting Items Tab

In case a conflicting pair is valid, fields values will be compared and non-matching fields will show the results in ‘Conflicting items’ tab:

Invalid Links Tab

If the pair is not valid – one of the issues does not exist or change its ID/Type and sync is not possible – it will appear in the ‘Invalid links’ tab of the report:

Not Imported Scope

The next two report options are for detecting not imported issues, which are two separate requests to Azure DevOps and Targetprocess.

By default, with no filters added, the report will search across all the projects selected in ‘Projects’ tab of the profile (static scopes). If JavaScript routing is added, then all accessible projects will be scanned despite the projects referenced in JavaScript mapping.

Conflicts and resolutions

1. 'Conflicting fields'

In this release, the report scans all existing pairs of linked issues and compares field values according to field mappings. ‘Conflicting items’ report tab shows all fields where values do not match when they should.

Conflicts in fields mapped with JavaScript

???? Note: Fields mapped with JavaScript code extensions must have a custom comparator added otherwise they will be skipped from the report by default.

2. 'Invalid links'

This report’s tab shows all pairs where one or both issues do not exist – either deleted or their ID or type was changed and cannot proceed to sync.

This category of errors has the following actions:

• 'Refresh conflicts'
• 'Unlink'

3. 'Not imported'

All issues that match the integration profile filters at the moment of the scanning Targetprocess or Azure DevOps

To fix conflicts from the 'Not Imported' apply one of two actions – "Pull to Targetprocess" or "Push to Azure DevOps" to import all selected (filtered) entities to selected tool.

screen-recording-2023-07-03-at-03-55-52

Filters and actions

Filters

All issues filtered out and viewable in the list are considered as a selected scope for any action. Should you need to apply an action to a specific row of the report, filter the specific row out and then apply changes to the only row left in the report.

Action: ‘Refresh conflict’

In case conflicted fields have their values changed, then ‘Refresh conflicts’ action will bring the actual values. If conflict is not actual anymore, it will disappear from the report (mappings removed, issues unlinked) or change its state to ‘Resolved’ (issue imported)

Action: ‘Ignore’ and ‘Unignore conflicts’

This action moves conflict to the ‘Ignored’ state. In this state, conflict will not be processed – not refreshed, not fixed. Before applying any conflict resolution strategy, you need to move conflict back from ‘Ignored’ state to an actual.

‘Unignore conflicts’ triggers conflict refresh in the background. If conflicts were fixed indirectly in one of the systems, then field conflict will move to a valid state – ‘Resolved’, if values match at this time, or back to ‘Conflict’, if mismatch is valid.

Conflict resolution strategies

To fix the conflicts in the mapped fields, you can choose one of three strategies:

1. Conflict resolution strategy ‘Latest change’

Out of two conflicting values, the latest one will be selected and applied to the matching filed in the other system.

???? Note: This strategy works for conflicts where both fields, Targetprocess and Azure DevOps, have last modified dates. If at least one field in the pair does not have history and last updated date, then only ‘Pull to Targetprocess‘ or ‘Push to Azure DevOps’ can be applied.

2. Conflict resolution strategy ‘Push to Azure DevOps’

With this strategy, all values from Targetprocess fields will be sent to Azure DevOps.

In the ‘Not imported’ section, this will import all selected Targetprocess items to Azure DevOps.

3. Conflict resolution strategy ‘Pull to Targetprocess’

This strategy considers Azure DevOps as a source of truth and rewrites Targetprocess fields with values from Azure DevOps.

In the ‘Not imported’ section, this will import all selected Azure DevOps issues items to Targetprocess.

4. Conflict resolution strategy ‘Unlink’

Unlink is required if the entity has been synced and the link is not valid for various reasons (issue deleted or converted and its ID/Type changed and lost). In such cases, before importing an issue again as new it must be unlinked from the previous item.

As soon as the issues are unlinked, they move to the ‘Resolved’ state and no other actions are possible with resolved invalid links for now. The free issue can be imported manually to the connected tool.

Conflict resolution workflow

Before any action is applied, every field’s conflict is verified and its status and fields are updated. This validation can be called manually with the ‘Refresh’ action.

Conflict statuses

Status ‘Conflict’

Fields or issues that do not match.

With items in the ‘Conflict’ state, you can:

• Verify the conflict by applying ‘Refresh’ action
• ‘Ignore’ field and no action will apply to them, even if such issue gets under the filter
• Fix filtered conflicts with one of the resolving strategies – ‘Pull to Targetprocess’, ‘Push to Azure DevOps’, or ‘Latest values’

Status ‘Processing’

Temporary state which indicates that the existence of conflict is being checked. If values are the same, and if an action selected (’Unignore’, ‘Pull to Targetprocess’, ‘Push to Azure DevOps’, or ‘Latest values’) can be applied to selected items.

‘Processing’ state will automatically change

• To ‘Resolved’, if conflict is fixed
• Back to ‘Conflict’ if selected change cannot be applied or conflict still occurs and no further actions were applied
• To ‘Update Sent’ if action is applied and new value is sent to Azure DevOps

Status ‘Resolved’

Final state. No actions will apply to resolved conflicts in current report version.

Status ‘Ignored’

No actions will apply to conflicts in status ‘Ignored’. The ‘Unignore conflicts’ action will automatically call a ‘Refresh’ and actualize the conflict.

Status ‘Update Sent’

This state indicates that updates are in the queue and will be applied in Azure DevOps shortly. You must manually refresh conflicts in order to know if the changes applied successfully. This state will not change automatically to the next, even if values were updated already.