Exercise 5.6 - Analyse and Comment Card Events

Now that the previous exercise has given a basic overview and understanding of the Tracker, you will process the Events that were fired on your new Business Object Card in the latest iteration.

In order to process any Event as a Team User, you need to acquire an understanding of the cause of the Event. Of course, in any migration, the possible causes for any given Event can be endless and it can be quite a job figuring out what is going on in order to identify the root problem. 

No tool - migFx included - can eliminate this complexity. But as you will see in this exercise, migFx does provide you with efficient means of analyzing and tracing Events and provides great assistance in resolving them.

Have a look at the list of Events that were fired on the Card Business Object. 

Now let's process some of the Events one by one.

Event E0011: Customer {0} is missing

First of all, the Event prefix tells you which migration step fired the Event.

In this case - as in almost any case - a good starting point is to use Studio to find out exactly where in the mapping this Event is fired and why. 

As the Event prefix is E, the Event was fired by the Source Engine, so now: 

1. Open the Workshop Source Map in Studio
2. Locate the the Event E0011 in the Project Explorer tree view
3. Right-click the Event and choose Find references in the context menu

The References tool window will now open at the bottom of the Studio UI and show you where this Event is fired.

The list contains only one reference. Following the hierarchy chain from left to right, it is apparent that the Event is fired from the Card Business Object. More specifically if, in the Source Map of the Business Object, the Source Object CardHolder (a child of Source Object Src.DebitCard) is not found.

Armed with this information, you can double-click the reference line to open the Card Business Object, and locate the Not Found Flag on the CardHolder Source Object.

Right there, you have the root cause of the Event: Apparently for some rows in the Src.DebitCard table in the Staging Database, there is no row in the Vw.Customer table corresponding to the CardHolder of the Src.DebitCard.

Go back to the Portal to do a little more analysis to get closer to an explanation. Try to drill down to 1 single Card Business Object instance firing the Event:

1. Click the Business Object name Card of the Event E0011 to drill down a level in the aggregation for the Event
   
2. In the Intermediate list just click the Count on first first row to drill down yet another level 
3. in list of Business Objects click the Item Id of the row displayed
4. Now the details for the Business object instance are displayed in a dialog

One of the Card Business Object instances that fired the Event E0011 is now displayed in detail. Note that the Card belongs to BankId 30. 

Have a look at the Dependencies:

The Card has a dependency on a parent Customer. But this Customer has another Customer Number than the one the Event is complaining about - and it has also been migrated successfully. So that the parent Customer is not the problem

For Event E0011, the Customer number of the missing Customer is displayed in the Event message. 

Let's see if that Customer has been processed in the migration.

1. Mark and copy the Customer number to the clipboard (ctrl-c)
2. Click the Objects menu
3. Select Business Object Customer
4. Open the filter and paste the Customer number into the Search text box
   

The instance of the Customer Business Object is now found and has been actually migrated successfully. 

But looking carefully it does actually belong to BankId 40. This seems to be the problem: The Card belongs to BankId 30 while the Customer belongs to BankId 40. 

You can do the same analysis on other occurrences of Event E0011. They all seem to be caused by the same problem.

Delegate the Event

At this point, there is not much more to be done. It seems to be a problem with dirty data in the Source System. What you can do is delegate this Event to the Partition Users and ask them to look into the problem in the Source System.

Click the Events tab to show the Event list again and then edit the message for Event E0011 to display the Event Editor: 

• Check Delegate and while you're at it, you can assign yourself as Anchor
• Click the Save button in the title bar to delegate the Event (actually, it may already be delegated, since the Portal Operations automatically delegates new Events with Area Source or Target)
• Enter a suitable comment to explain the problem to the Partition users. For instance: These Cards all belong to Bank 30 while their CardHolder Customers belong to Bank 40. This seems to be a problem in the Source System. Please either correct this problem in the Source System or mark this event as Accepted
  

Now return to the Events list. 

The Event is delegated

and the State has changed to Waiting, because the Event has been delegated to the Partition Users and you and your team are now waiting for them to resolve the Event. 

In addition, you can see there are comments on the event.

Event E0012: Name compressed from ({0}) to ({1}), length ({2})

This Event has Impact Advisory and Disposition Retain. So, nothing is being discarded, but still somebody should verify that the resulting names are acceptable, and - if not - edit the names in the Source System for the next iteration.

So again here, the correct action in Tracker is to delegate the Event to the Partition Users and provide guidance in a Comment. Click the Event message to go to the Event Editor.

• Check Delegate to Partition Users and click the Update button to delegate the Event
• Enter a suitable comment to explain the problem to the Partition users. For instance: Please verify that the edited names are acceptable. If not, the correct action is to edit the names in the Source System to avoid this warning. Mark the Event Accepted when done.

Event E0017: Card is blocked. CardInactiveDate set to default ({0})

The Impact is Information and Disposition is Retain, so apparently the Event has no consequences and serves only to tell that something was done. Anyway, let's trace the Event and figure out why it was fired.

Find references in the Studio Source Map yields just 1 reference:

From the reference chain, it is clear that the Event is fired when the value of the Interface Field CardInactiveDate of Business Object Card is calculated. 

• Double-click the reference to go to the Business Object 
• Click the Export tab to see the list of Target Interface fields
• Click the Content cell for Interface Field CardInActiveDate

In the Flag Handling of the value assignment dialog, it is evident that Event E0017 is fired when the Card is blocked. But why is the default date returned? 

Back to the Portal State, Events menu and drill down to the list of single Business Object instances for Event E0017. Open the Event and have a look at the Source data:

While the two first CardStatus children have a CardStatusEndDate, the third one does not. In addition, it has a CardStatus Blocked. 

We are getting close to an explanation. If you look back the exercise 2.5 - Map Business Object Card in the section Map Target Interface Field CardInactiveDate, you will see that this is a known difference between the Source and Target Systems.

So - again - let's just delegate this Event to the Partition Users to ensure they are aware of the Event and accept it for the final sign-off. In the Event List, click the Event Message to go to the Event Editor. 

• Check Delegate to Partition Users and click the Update button to delegate the Event
• Add a suitable Comment, for instance: Please accept this event. If a card is blocked, the Target System requires a date. The Source System contains no such date. It has been agreed with the Business, that in case of a blocked card, we use first bank day after migration as default.
  

Event I0011: Customer ({0}) does not exist or is rejected in migration

The I prefix of the Event signals that it was created by the Target Engine during the Import step. The Impact is Critical and the Disposition is Full Discard. The Event is obviously serious as it causes the entire Card Business Object instance to be discarded.

Let's open the Target Map in Studio and do a Find References on the Event to see where it was created:

The Event is actually created from 4 different locations, but only the last one concerns the Card Business Object, so let's concentrate on that one. Again looking at the reference chain, it seems the Event is fired on the Not found flag on the CardHolder relationship on the Card Business Object.

Double-click the reference to open the Card Business Object, click the Relationships tab and finally select the CardHolder relationship in the list:

Ok, the Event is fired, if no Customer Business Object instance is found via the CardHolder relationship, let's see if we can help figure out why. In the Portal State Event list, drill down to the intermediate list to see some actual Customer Numbers:

For the first row in the Intermediate list, drill down to the Item Detail for the first Item and click on the Dependencies tab:

The Parents list shows that the Card Business Object Instance depends on a Customer and an Account Business Object instance, both of which have been rejected.

Rejected Customer

Click on the ItemId of the rejected Customer to display the Item Detail of the Customer Business Object instance:

So, the Customer is rejected by the Event E0006 which is fired if an AddressLine on the Customer Business Object instance is too long. In a later exercise, we will have a look at this Event. For now, we are satisfied that the cause of Event I0011 on the Card Business Object Instance is that it is depending on a Customer Business Object instance that in turn is rejected by Event E0006.

From the intermediate Event List you can drill down to the other Card Business instance rejected by I0011 and see that this too is caused by the E0006 Event on a Customer Business Object Instance.

The Rejected Account

The Item Detail of both rejected Card Business Object instances reveals that it is not only the dependency on a rejected Customer that is a problem. Both Cards also depend on an Account Business Object instance, that has been rejected. 

You can use the Portal to trace these rejected Accounts to figure out that these Accounts are actually also rejected because they depend on rejected Customers. Furthermore, that again it is again the Event E0006 that is the cause for these Customers to be rejected.

Finishing off

So by now, it is clear that we must expect that resolving Event E0006 will let the Customer Business Object instances succeed and in turn, both the affected Account and Card Business Object instances should succeed, and the Event I0011 will be resolved. 

Let's update the Event to reflect this. From the Event list edit Event I0011 and set it to be derived from E0006 on Customer and also change the state to Active to signal it is being worked on:

Event I0023 - AccountNumber ({0}) is missing or Rejected in migration

This Event is fired when the Account for a Card is missing. If you drill down on this Event, you will quickly figure out that the Account is missing because it cannot find its Customer.

And the Customer is rejected by E0006 as we figured out above.

Looks like fixing E0006 would be really beneficial! We'll look at that in the next exercise.

In fact, this is quite normal. In any kind of data migration, there will always be relatively few Business Object upon which a lot of other stuff depends. In this case, the Customer is fundamental, and any rejection of a Customer is sure to bring other stuff down with it.

Event I0025 - Card Status ({0}) is unknown

The Severity of this Event is Reject Root, so all the Cards firing the Event are rejected by the Target Engine. Not so good. Let's figure out what the problem is.

A drill-down of just one level from the Event List reveals that Card Status Blocked is unknown.

Find references in the Studio Target Map reveal that the Event I0025 is fired if either of the two Flags on Mapping Rule GetCardStatus is raised while calculating the value for the Target Field Tgt.Card_Status.StatusId:

Judging by the texts on the two Flags, they both signal that the CardStatus is unknown, the only difference between the two is whether the card itself is active or expired.

The Specification for the MappingRule GetCardStatus clearly states that the CardStatus is expected to be found in the ValueSet CardStatus - otherwise, one of the 2 Flags is raised depending on the CardInActiveDate received as a parameter to the rule.

Ok, we are getting there. Looking at the static ValueSet CardStatus, the CardStatus Blocked is obviously missing:

This needs some further analysis. Maybe the CardStatus Blocked is just missing in the ValueSet by mistake - or it is missing by design because this CardStatus does not exist in the Target System.

Another consideration is that the Severity Reject Root seems very harsh. After all, the Event is fired when importing the child Business Object CardStatus. Maybe it is not necessary to reject the entire Card just because one of the CardStatus children has an unknown CardStatus? 

To log that we will continue working on this event in a later exercise, let's update the event in the Event Editor:

• Add a comment, for instance: CardStatus Blocked is unknown. Further analysis is needed.
• Set State to Active

What happened here?

Working on these Events should have given you a good sense of how the Portal State and the Studio in combination give a robust foundation for the analysis of a wide range of migration problems. 

In any data migration there will initially be a myriad of problems of daunting variety. Indeed,  our software is a generic solution that is deployed by different installations in any industry vertical only increases this fact exponentially.

Nevertheless, the unique approach of migFx handles this complexity with ease

• The Business Objects enables the Portal to present all data flowing through the migration for a Business Object instance together. The data are not atomized in the waterfall assembly-line approach of the traditional, flow-based ETL approach
  
  
• The Event is a totally standardized method of reporting situations and issues in the migration providing an efficient and precise link between issues occurring in the migration, visible in the Portal, and the mapping in Studio
  
  
• The State, Comments and Delegation of Events in the Portal provides the migration team with a straightforward and efficient tool to keep in control of the many Events as they come and go in the migration iterations. In addition, all communication on the resolution of Events is kept in place, enhancing the level of support the team may provide to Partition Users involved in the migration 

In the next, final section of exercises, you will use these capabilities combined with the extraordinary speed of iteration in migFx to fix some of the Events analyzed in this exercise.