ECM 2.0 (Entity Check Module 2.0)

ECM 2.0

Entity Check Module 2.0

Companies face operational failures and mishaps due to Mission Critical Systems exchanging data in productive environments and this data being out of sync.
Migration projects aiming to move mission critical data from old systems to the new systems strive for correctly executed data migrations. Such projects are in need of a tool to regularly check the migration quality and deliver proper reports for it; along the way.
Challenge: Ensure that migration quality is high and the production level data replications occur correctly, in order to avoid issues in rollout as well as in daily operations. In a nutshell, improve data quality and consequently avoid issues due to Entity mismatches between systems.

ECM : Entity Check Module
is designed to reveal and solve data inconsistencies between entity-master system and data-replicating slave system
is developed by bearing in mind the great performance requirements of comparing huge data sets
is a stable, scalable and above all system-agnostic application
can automatically fix the inconsistencies
in a nutshell : checks the inter-system entity consistency, prepares/executes fix actions and generates reports

system-agnostic approach
- can check the entity consistency between any systems, so long as the user can clearly define the entities
easy configurability and short time-to-market
- Allows users to easily construct any entity in ECM configuration and perform DQ checks in a relatively short time, all due to straight forward design principles
high performance
- Thanks to patent-pending algorithms, delivers the check results with incredibly quick times

results stored in the database
- clearly understandable tables showing erroneous records, as well as missing and zombie records
results presented as reports
- out of the box entity reports present a historicized graphical view per entity
- out of the box summary reports present a historicized text view per entity
- missing and zombie records in slave system (these checks can be disabled individually by database configuration)
- existing but erroneous records with mismatching attributes and mismatch percentages

multi-system checks in the same ECM dB
- Users can configure the ECM to check multiple system pairs, in the same ECM configuration dB
several options regarding fix actions
- can create INSERT statements to add missing entries in the slave system
- can create UPDATE statements to correct wrong entries in the slave system
- can call Stored Procedures to add/correct entries in the slave system
- the SQL statements created for fixing purposes can be configured to be stored in files in UNIX file system (to be executed manually by the user as SQL scripts) and/or in the ECM database
- the SQL statements created for fixing purposes can be configured to be automatically executed by ECM allows operations teams to run various types of post-processing scripts if wished
- more fixing options can be easily added, such as calling Web Services

Needs the source and target system to be an Oracle database
- Work-around is already available – by performing an ETL of entities to be checked into a dummy oracle dB to perform the comparison

quality check in data migrations:
- during implementation and test phases, ECM provides valuable feedback about the soundness of the overall migration process and reduces migration related issues to a minimum
quality check in production systems:
- checks in regular intervals present the DQ tendency allowing operations teams to present clear reports to upper management
- helps detect and solve replication issues in respective modules

the data source system is master
the data target system is slave
the slave system is typically the fix system (the system where the fix commands are executed)

CREATE_PK_ON_RESULT_TABLES
- Values: YES/NO. If YES, ECM creates primary keys on the result tables.
- It is useful for working on result tables
DROP_MS_AND_SL_TABLES
- Values: YES/NO. If YES, ECM drops the _MS and _SL tables at the end of the run.
- It may be useful for seeing what exact entries were read from the master and slave system during the run
NOF_RUNS_TO_KEEP_RESULT_TABLES
- ECM keeps the result tables for the specified number of runs and deletes older ones.
- It is used to perform a regular housekeeping by ECM itself.
- The reports are kept but tables are deleted for saving space.

Definition (Input)
- decide which entity is to be checked
- configure the master and slave statements for getting the entity
- configure fix actions to be performed
- specify source system’s database connection details
- specify target system’s database connection details
Execution (Process)
- run the ECM for the intended entity
- specifying “-d” option presents extended debug information in the log files
- run the entity report process to create the report (and to save it in the dB)
- run the summary report process to create the report (and to save it in the dB)
- alternatively, a shell script could be used to run ECM and the reports
Results (Output)
- check the run results in the STATISTICS and FIX_STATISTICS tables in the ECM dB
- check the created tables relative to the run : missing, errored, zombie entries
- check the graphical entity reports (sent via emails) and summary reports
- check the log files of ECM

Preparing/executing fix actions for the correction of errored and missing records
Graphical entity report generation – automatically sent via email to configured recipients and also stored in the database
New configuration options

Trainings are delivered on customer sites for educating the operations personnel on the application
Consulting services for supporting complicated migration and productive data quality check scenarios
Remote support for configuration and troubleshooting

Heal the acute data quality issues between mission-critical production systems
Increase revenue by ensuring the invoices are generated correctly and are sent to correct addresses
Help the operations teams to generate Data Quality reports in an easy way for upper management
ROI is faster than any comparable tool in the market