{"id":10421,"date":"2016-08-24T09:10:12","date_gmt":"2016-08-24T13:10:12","guid":{"rendered":"http:\/\/www.iri.com\/blog\/?p=10421"},"modified":"2017-11-03T14:45:04","modified_gmt":"2017-11-03T18:45:04","slug":"data-consolidation-wizard-for-data-quality","status":"publish","type":"post","link":"https:\/\/beta.iri.com\/blog\/master-data-metadata-management\/master-data-management\/data-consolidation-wizard-for-data-quality\/","title":{"rendered":"Data Consolidation Wizard for Data Quality"},"content":{"rendered":"

Given the amount of data businesses garner daily from human interaction, it is easy to understand how their sources become rife with redundant or erroneous entries. For the sake of data quality and efficiency, master data management (MDM), and improved customer experiences, data architects are interested in a single, standard representation of their data.<\/span><\/p>\n

If \u201cJohn Smith\u201d is entered into a sales database, for example, and later he is entered again as \u201cJon Smith,\u201d there will be two instances of the same person. As this occurs many times with many different customers, the database gets clogged with redundant data, and processes like profiling, querying, and loading are slowed down. Beyond that, customer service representatives and other stakeholders in the business will not have a reliable view of the customer, misidentifying them, sending multiple catalogs to their home, etc.<\/span><\/p>\n

In short, <\/span>data unification<\/span><\/a> is important to the continued health of your data and your business.<\/span><\/p>\n

\"Unified<\/a><\/p>\n

Source: Tata Consultancy Services<\/span><\/i><\/p>\n

IRI Voracity<\/span><\/a> users can access a new Data Consolidation wizard in <\/span>IRI Workbench<\/span><\/a> to identify and correct redundant records in files and databases. The wizard accepts the following data formats as inputs: CSV, Delimited, LDIF, ODBC, or XML.<\/span><\/p>\n

Source Selection<\/b><\/h3>\n

After designating the location of the project on the setup page, the first part of the wizard involves source selection. If your sources are only tables in a connection profile, you can check the appropriate checkbox. <\/span><\/p>\n

\"Data<\/a>
\nSelecting this box \u00a0and clicking\u00a0<\/span>Next <\/b>opens an input page like the one below where you can choose the tables to be included:<\/span><\/p>\n

\"Data<\/a><\/p>\n

If the checkbox is not selected, you can add files or ODBC sources in the same input screen. On this type of input page, you will also need to add the metadata for each source. In this example, there is an XML file and an Oracle table added.<\/span><\/p>\n

\"Data<\/a><\/p>\n

Once you have added the desired sources added, click <\/span>Next<\/b>.<\/span><\/p>\n

Create Output Fields<\/b><\/h3>\n

This page allows a choice of which fields will be part of the output. Drag and drop the desired fields from the Input Field List to the Output Field List. For example, to create a new customer table to have master IDs and names, you could select just the ID and Name fields. To use all the fields for a particular source, select the data source name and drag it to the Output Field List.<\/span><\/p>\n

\"Data<\/a>
\nYou can also rename the output fields here by clicking\u00a0<\/span>Rename<\/b> or double-clicking the parent field and then clicking another item in the tree to save it. Once all the desired fields are in the Output Field List, click <\/span>Next<\/b>.<\/span><\/p>\n

Create Grouping Methods<\/b><\/h3>\n

Here you create and commit grouping methods the wizard will use to determine if your data has redundancy. This page will also display a bar graph displaying the redundancy groups your data contains.<\/span><\/p>\n

    \n
  1. Displays the name of the field in your output file.<\/span><\/li>\n
  2. Select the matching type for redundancy comparison.<\/span><\/li>\n
  3. Activates if a fuzzy comparison option is selected for the matching type and allows you to select the threshold for the fuzzy comparison.<\/span><\/li>\n
  4. Commit the corresponding method.<\/span><\/li>\n
  5. Redundancy grouping graph.<\/span><\/li>\n<\/ol>\n

    \"Data<\/a><\/span><\/p>\n

    Each field created on the previous page will be displayed on this page to allow you to select a grouping method. An example would be:<\/p>\n