These release notes are organised into three Sections:

• New and Improved Features - which summarise the new and improved functionality provided in this release
• Issues fixed - which identify all issues corrected in this release
• Installation and Refactoring - which includes instructions as to how to apply this release to your installation

New and Improved features

This section includes a summary of the new functionality provided in this release, it is broken down into the three main components of the software:

• Portal
• Studio
• Engines
• Runtime
• Automation

Where available links are included to other articles in the Support Portal that provide more detailed information about the new features and how they can be utilised.  If the change relates to a Ticket in the Support Portal the number of the Ticket is included in the header after the has symbol (#).

Portal

This section includes improvements to the browser based Portal used to manage the application and display the results of the migrations to the Users.

Subscribe to Events and notifications

Users can now subscribe to events and receive notifications for the following changes in Edit event dialog: changing delegation, anchors, tags, comments.

The bell icon for subscribing to notifications appears when an Event is edited from the Event list and can be used similar as in Item Manager to open the Edit event dialog.

As with other subscriptions a comment should be added which will be included in the Notificaton when it is generated.

Even without a subscription, the team users that are set as Anchors to an event will get a notification.

As will the Delegate external users when they are delegated to a particular event.

White-labelling

An Administrator can now change the colour scheme and logos used in the Portal to "Brand" it to their choosing.  The only references left to HOPP are in the mouse over of the logo in the top left corner of the Portal and in the browser tab showing the Portal.

The colour scheme and logos are controlled using two files which are loaded to the Server on which the Portal is running and are described in this document in the Knowledge base on the support Portal: White labelling the HOPP Portal

Project Level configurable Dashboards

The ability to configure a Dashboard for an individual project has been added to the Portal.  This enables a Project Manager to set up a Dashboard with a specific set of Widgets to reflect the interests/objectives of their stakeholders.  This employs the same functionality that is available for the default Portal level Dashboard to add/remove and position any of the available Widgets in the Portal.

Any Project that does not have a specific Dashboard configured will continue to use the default Portal level Dashboard.

The option to create a Dashboard for a Project can be found in the Configuration view of the Project details.

The default Portal level Dashboard will be used to create the Project level Dashboard and this can then be manipulated by the User with the tools available.

Tags column and filter added to key views

A new column has been added to the following views to display any tags attached to the objects in the views:

• Item List
• Events List
• Test Area Tree

A Filter for Tags has also been included.

Item List (if the Item Manager module is installed)

Events List

Test Area Tree (if the Planning module is installed)

Open Items over the Timelines view (if the Item manager module is installed)

In previous Releases when the user clicked on an Item in the State/Timelines view to open it the Portal automatically switched to the State/Item view.  In this Release a right mouse click gives the user the option to Edit Here (i.e. over the Timelines view) or Edit in Items (i.e. switch to the Items view as previously).

Improved Object Search

The Search option in the Filters on the State/Objects view has been improved to more specifically identify Objects with the value requested in the Migration or Source keys.  Previously it was returning sprurious results.

Improved display of missing Source/Migration Keys

The display of Source and Migration keys in the State/Objects view has been changed so that if the value is missing after the Import has been run then the column will be populated with the value NULL, whereas if the Import has not been run an empty string will be displayed as they have not been generated yet.

Studio

This section includes improvements to the Studio desktop application.

New Any data type for parameters

A new data type has been introduced specifically and only for use when defining parameters to 

• Manual Rules
• Manual Bag Methods
• Events

If a rule parameter is of type Any, values of any data type can be passed to the parameter. 

Read more about the Any data type here: The Any data type.

Cast of value assignments

The validation of value assignments in Studio has undergone a major overhaul. In previous versions all value assignments where validated resulting in assignment errors and warnings. 

Especially the assignment warnings were quite annoying because there were in most cases really not any good way to fix them. As a result, in larger maps, the validation risks being heavily spammed with assignments warnings - potentially hiding other, more relevant, warnings from the validation.

In version 2.3 of Studio, there are no more assignment warnings. Studio will continue to emit validation errors everywhere assignments are clearly incompatible (for instance Char to Date). But in most places, there will be no more warnings (for instance Char(10) to Char(5)) at all.

In 2 places, however, Studio has grown more strict. That is when assigning values to Target Fields in the Target Map and Interface Fields in the Source Map. In these 2 cases, any assignment issue will result in a validation error - even truncations and null assignments (that previously just produced a warning):

To clear an assignment validation error you must inform Studio that the assignment is fine by checking the new Cast checkbox for the field in question:

OBS: This has introduced one stricter validation in Studio: Bag methods that return a Boolean can be used as Filter-, Condition-, Entry- or Exit Rules. But only if the return value of the Bag method is a Boolean without nulls allowed. If you have Boolean Bag methods that return a nullable Boolean (Boolean?) and are using these in one of the contexts mentioned above, you must modify the rule to not allow return of null.  

Read more about cast here: Cast of value assignments in Studio

For convenience, version 2.3 includes a refactor command to automatically cast all assignments where required. See below under Installation and Refactoring.

Skip if Null on Validation Rules

For historical reasons, previous versions of Hopp did not call a Validation Rule assigned to an Interface Field in the Target Map if the value of that Interface Field was Null when received from the Source Map.

This is quite counter intuitive and frankly not very useful. In many cases, a Validation Rule is relevant exactly for the purpose of checking whether the incoming Interface Field is Null - but previous versions did not call the Validation Rule in this case.

In version 2.3 a new checkbox is available when assigning a Validation Rule to an Interface Field to specify if the rule should be called if the incoming value is null.

With the new Skip if Null option on Validation Rules both use-cases can be handled gracefully:

1. Only validate the incoming Interface Field value if it is not Null - check Skip if Null
2. Validate if the incoming Interface Field value is Null - do not check Skip if Null

OBS: Please note that the current behaviour of all Target Maps (and their generated engines) will change. Before 2.3, Validation Rules are not called on any Interface Field if its value is Null. Now, with version 2.3, all Validation Rules are called on all Interface Fields regardless of whether their value is Null or not.

For convenience, version 2.3 includes a refactor command to automatically check the Skip if Null on all Validation Rules in order retain current behaviour with version 2.3. See below under Installation and Refactoring. 

XSD validation in Source Engine is decommissioned

The Source Engine is no longer applying a generated XSD in order to validate that the created Interface Xml for a Business Object adheres to the data types and child cardinalities of the interface published from the Target Map.

Instead, this validation is now performed programmatically by the generated Source Engine code. This allows the validation to be more flexible (a requirement expressed in ticket #2367 Exclude discarded child object from XSD validation).

For instance, 

• The target interface specifies a minimum cardinality for a child Business Object to be 1 
• The Source Engine actually produces an occurrence of the child - but an Event causes it to be discarded
• Previously, the XSD validation would generate a System Event for the missing child. This was rather redundant, since the discard was already documented with an Event
• Now, the Source Engine will no longer emit the redundant System Event

Runtime

This section includes improvements to the Server based components of the software and its deployment.

(No specifics in this release)

Automation

This section includes improvements to the PowerShell Automation module. Please refer to the help documentation for each function for futher information (Get-Help (function name)).

Set Baselines

New functions have been added to set baselines:

• Set-HpBaselineData: Set baseline for Source Data
• Set-HpBaselineViews: Set baseline for Views
• Set-HpbaselineValuesets: Set baseline for Valuesets in Source or Target 
• New-HpBaselinePortal: Creates an new Portal baseline and sets it current

Download Valuesets

A new function Get-HpValuesetDownload fo download Valuesets to Excel. The function takes piped input and can be used in connection with the Get-HpValuesetList function.

For instance:

Get-HpValuesetList target -includeStatic | Get-HpValuesetDownload -folder "target path (must exist)"

Issues fixed

The following bugs have been fixed in this release, where there is a Ticket on the Support Portal its number is included after a hash symbol (#):

• PowerShell Automation: 401 Access Denied http status
  Fixed a concurrency problem in the await mechanism that caused 401 Access Denied http status when submitting a long-running schedule containing multiple batches
  
  
• Studio: Static Valuesets with boolean column takes long to load and sometimes freezes Studio #3491
  Fixed a render problem when editing a data static valueset with a large number of checkboxes
  
  
• Runtime: Potential Duplicate Key error when saving baseline for Valuesets, Source Data or Source Views
  Fixed a problem where Save Baseline in the Portal Operations on Valuesets Source Data or Source Views under certain conditions could result in a duplicate key error
  
  
• Load of Translation Valuesets in runtime job should omit rows for deleted partition #2785
  On update or delete of a Partition: Show info list of valuesets (with row counts) that will be updated or deleted.  A Partition cannot be updated to an existing value

• Static Valuesets with boolean column takes long to load and sometimes freezes Studio #3491

A problem where boolean values in a static valueset took too long to load has been corrected

• Loaded Date not shown for tables that are loaded #3726

In some cases the date for a table that has been loaded was not displayed in the Portal

• Exclude discarded child object from XSD validation #2367
  If a child object has been discarded by previous validations do not put it through the XSD validation

• Exception adds up Partition and Track dropdown #3904

An error that caused the incorrect display of totals for an event has been corrected

• Changes to Validation Rule not saved #3951

If a new validation was added where there was already one in place it was not saved correctly, this has been corrected

• Dump in Views when using Reference #4023
  

If you open a view and click on Reference right after, Studio dumps with this error:
NullReferenceException: Object reference not set to an instance of an object.

• Named values in copy of View #4086
  

Removing named values after copying a view throws a validation error.

• Error editing static Valueset data #4177

Create a static valueset and add data in it. Studio crashes when you add/edit value in the grid cells.

• UI: Do not display empty MigFxContextMenu

If IsVisible is false for all Context menu items, the Context Menu should not be rendered at all
Presently, there is a small, empty square of the empty context menu

• Wrong error indicator on Source Data

An error indicator was being displayed incorrectly.

• Incorrect dependency error on Source/Data

Tables that have been loaded are marked as in error incorrectly

• Upload Parameters: Boolean values wrong
  

After exporting to an excel spreadsheet boolean values are not correctly reimported

• Idempotent Portal script complaining about SourceId column missing

An incorrect error from a script in the Portal

• Large Event tooltip not wrapping

Where an event has a long message it is not wrapping correctly when being displayed in a tooltip

• Test Widget does not add up

Where a test has passed for one Partition but not been tested on another it wasn't clear on the Test Widget

• Null values not saved correctly in Translation Valueset

If a null value is entered as Null it is not being correctly saved in the database

• Missing target xml not handled correctly

If the target xml for an object is removed it is not handled correctly when it is accessed

• Itemset:: List not refreshed when creating or deleting an Itemset
  

If an Itemset is created or deleted the list must be refreshed manually

• Events link dropdown: Include Disposition and Impact

If the Disposition and/or Impact of an Event is changed it will appear twice in the dropdown list of events to link so the Disposition and Impact are included in the description

• Ok button not enabled in Value SourceField selection

In one circumstance the OK button is not enabled in Studio

• Skip if null does not work properly

Skip if Null on a field not behaving correctly

• Source metadata is not loaded in same order as it is exported

If Source Map metadata is exported and reimported with no changes it shows a lot of differences incorrectly

• Error in copy view to Clip board to execute in MS Sql

When using constants of type Time and GuId, declare is wrong when copying a view

• Error validating view using Time and Guid as parameters

When using constants of type Time and GuId as parameters, validation fails

• Binary or Char(*) fields are invalid for use as Key column

A Binary or a Char(*) field cannot be used as a Discriminator in an Extraction map

• Deploy error

There seems to be a racing condition when doing deploy from the VS extension. This error happens intermittently

• Key columns not refreshed

In the option State/Objects the Key columns are not displayed correctly after the Business Object is changed

• Behavior in Operations filter

The filters in the Operations section were not remembered when navigating away from an option

• Translation value set cannot load

An error was encountered loading a translation valueset

• Dynamic valueset cannot be loaded

An error was encountered loading a Dynamic valueset

• Error in generated source engine code when using reserved c# term

Using a reserved c# term caused the generated code to be incorrect

• Formatting of dynamic values in Views, Data and Valueset details

Values displayed in the Portal should be shown based on their data type

• Empty static Valueset causes error in Setup job

The setup job fails on a static valueset with no rows. Even though the xsd schema is present and valid, there are no tables created in the DataSet

• View static source valueset gives an error/dumps

Where there are duplicate views in Source and Target Maps an error is generated if they are viewed in the Portal

• NullReferenceException in ValueLiteral XREF

When using Cross ref on StringOps (BagMethod) Studio crashes

• Display name on Translation valueset does not work

When creating a valueset with displayname the name is correctly saved in Sql server when setting up new target engine, but the text is not displayed on Partiton level (All level is ok). Instead No text Found... is shown as the name

• Error if no Reformatter designed

If all reformatters are removed and Add Reformater is selected it throws an error

• Systemevent flag on filterrule cannot compile in VS

If a System Event is used on a Filter Rule it causes an error in Visual Studio

• Audit Result fails if CRLF in audit data

If an Auditor returns a string result containing CRLF, the AuditResult unload fails

• Multiselection only working on displayed rows

Not all available rows are presented in a multi selection dialogue

• Duplicate key on save baseline

An error was detected in the Save Baseline function

• View Optimizing error

An error was detected in the Load View function

Installation and Refactoring

Artefacts can be downloaded from the download page . Additional database refactor scripts are attached to these release notes

Database

• The Sql scripts to run are all attached below
  
  
• Please back up all databases before running refactor scripts
  
  
• Run attached MigrationDb.14.sql script in all Migration databases
  
  
• Run attached MasterDb.08.sql script in Master database on the Master server
  
  
• Run attached MigFx.PortalDb.Setup.sql in the Portal database on the Master server
  
  

Workstations

Studio

• Reinstall Studio from the download page

Studio Maps

Version 2.3 of Studio includes the following refactorings. When running a refactor, it is important to assure that nobody in your team has anything checked out in any maps in Studio. 

Refactor Cast of fields assignments

The new, stricter validation of assignment of values to Target Fields (Target Map) or Interface Fields (Source Map) will result in validation errors if present. These errors can of course be cleared manually by either checking the Cast for a field or by - if possible - changing the data type of the value being assigned.

Optionally, to automatically refactor your Source- and Target Maps, run this refactor command from a command line:

"(Studio Install Path)\MigFx.Studio.Refactor" Cast --DataSource (sql server instance) --InitialCatalog (repository database) --WorkingFolder (temporary working folder) --Commit --CommitMessage "Refactor: Add cast where required" 

Refactor Skip if Null on Validation Rules

To place a Skip if Null check mark on all Validation Rules in your Target Maps and thus retaining the current behaviour of Validation Rules where they are not called if the Interface Field is null, please run this refactor command from a command line:

"(Studio Install Path)\MigFx.Studio.Refactor" SkipIfNull --DataSource (sql server instance) --InitialCatalog (repository database) --WorkingFolder (temporary working folder) --Commit --CommitMessage "Refactor: Skip if Null on Validation Rules"

Visual Studio Engine Projects

• Upgrade to Visual Studio 2022, minimum release 17.10
  
  
• Reinstall the Visual Studio Extension (MigFx.VsExtension.2022.vsix) from the download page
  
  
• If using Nuget
  • Update Hopp.SourceEngine package reference to 2.3.* in all *.SourceEngineGenerated.csproj files
    <PackageReference Include="Hopp.SourceEngine" Version="2.3.*"/>
    
    
  • Update Hopp.TargetEngine package reference to 2.3.* in all *.TargetEngine.csproj files
    <PackageReference Include="Hopp.TargetEngine" Version="2.3.*"/>
    
    
  • Update the Hopp.Generator tool
    dotnet tool update hopp.tools.enginegenerator --global
    
    
• If using local libraries
  • Reinstall MigFx.Libraries from download page
  • Reinstall MigFx.EngineGenerator from download page

Powershell Automation

• Install new Hopp.Automation from download page
• Install and import publicly available module ThreadJobs as described in Automation: Overview and Installation 

Master Server

If installed, remove the TimeLines extension module from the Portal deploy folder:

Portal appsettings.json modifications

Notification templates for user assignment

Note: The new types of user assignment notifications from Events (anchor and delegate) requires a refactoring of the in-app and email templates. The existing user assignment templates from ItemMgr (owner and assignee) become obsolete: AssignedToElementTemplate and OwnerChangedToElementTemplate. Insead of adding 2 new templates for each event user assignment, they have been replaced with a generic template UserSetToElementTemplate that is having a parameter for type of user assignment: anchor, delegate, owner, assignee.

• Delete from AssignedToElementTemplate  and OwnerChangedToElementTemplate from NotificationConfiguration -> MessageTemplates section
• Delete from AssignedToElementTemplate  and OwnerChangedToElementTemplate from NotificationConfiguration -> EmailTemplates section
• Add the following new in-app template to NotificationConfiguration -> MessageTemplates section

"UserSetToElementTemplate": {
    "TitleTemplate": "Project {0} {1} {3} {2}",
    "ContentTemplate": "<p>This object in Hopp {5} <b>{2}</b> by <b>{3}</b>:</p><p>Project <b>{0} {1}</b></p><p>Please follow this link if you wish to review the changes:</p><p><a href=\"{4}\">{1}</a></p>"
},

• Add the following new email template to NotificationConfiguration -> EmailTemplates  section

"UserSetToElementTemplate": {
    "EmailSubject": "Project {0} {1} {3} {2}",
    "EmailBody": "<div style=\"font-family: sans-serif\"><p>{2},</p><br><p>This object in Hopp {5} <b>{2}</b> by <b>{3}</b>:</p><p>Project <b>{0} {1}</b></p><p>Please follow this link if you wish to review the changes:</p><p><a href=\"{4}\">{1}</a></p><br><p>Regards</p><p>hopp</p></div>"
},

Execution Servers

• Upload runtime.zip from download page