SortCL Archives

ETL

Smart, Safe Test Data for DevOps, MLOps and DataOps

by David Friedland

Data flowing through application development, machine learning, and analytic pipelines must address several needs common to all three, including:

Realism, to reflect production data characteristics and application test requirements; Compliance, with business and data privacy rules, plus DB and analytic models; Availability, or security, of the data (depending on your perspective); and, Auditability, for lineage and accountability. Read More

IRI Workbench

Connecting Google BigQuery to IRI Voracity Software

by Kevin Roldos

BigQuery is a managed, serverless data warehouse in the Google Cloud designed to enable scalable analysis over petabytes of data. It is a relational database Platform as a Service (PaaS) which supports ANSI SQL queries. Read More

Data Masking/Protection

Using IRI Workbench with ASN.1-Encoded Data

by Devon Kozenieski

In part 4 of this five-part series regarding newly added ASN.1 support in the SortCL program central to the IRI Voracity data management platform and its component structured data processing products — IRI CoSort, NextForm, FieldShield, and RowGen — we take a look specifically at their common graphical user interface (GUI) for job design, IRI Workbench. Read More

Data Transformation

ASN.1 Integration with SortCL (IRI Voracity)

by Devon Kozenieski

This article presents a more in-depth look at ASN.1 and processing encoded data messages based on ASN.1 schemas with the SortCL program supporting IRI Voracity platform products. Read More

Business Intelligence (BI)

Processing Data in, and for, Excel Spreadsheets

by Devon Kozenieski

In addition to all the other structured data sources that IRI software already supports, it is now possible to read, process and write data from and to XLS and XLSX files in the SortCL program central to:

IRI CoSort, for fast sorting, transformation, and reporting IRI NextForm, for mapping, migration, and replication IRI RowGen, for random selection or generation of realistic test data IRI FieldShield, for sensitive data masking IRI Voracity, for all the above, plus ETL, data cleansing, and wrangling for analytics

and to write the resulting data back to those or other targets, including one or more Excel sheets. Read More

IRI

Frequently Asked CoSort Tuning Questions

by Dave Zsiga

Since the advent of automatic tuning in CoSort V10, it has been easier to work with the many available performance settings in CoSort Resource Control (which we call cosortrc or “RC”) files. Read More

Data Masking/Protection

Using SharePoint Files with IRI Workbench Software

by Devon Kozenieski

This article explains how to connect with, and use data from, SharePoint sites — using the file system via OneDrive — for operations in IRI Workbench-supported data management software. Read More

Data Masking/Protection

The IRI-Windocks Test Database Repository

by Paul Stanton

Abstract: IRI has partnered with Windocks to provide an on-demand, test-ready virtualized repository that delivers masked, subsetted or synthesized databases in seconds for Docker containers, conventional instances, or workstations. Read More

Data Quality (DQ)

Data Quality Rules in IRI Workbench

by Claudia Irvine

IRI Workbench now contains a section of data cleansing, enrichment, and validation rules called Data Quality Rules for use in IRI CoSort data transformation and IRI Voracity ETL, reporting and data wrangling jobs. Read More

Big Data

Feeding Datadog with Voracity Part 3: Collecting and Leveraging…

by Devon Kozenieski

This article is third in a 4-part series on feeding the Datadog cloud analytic platform with different kinds of data from IRI Voracity operations. It focuses on visualizing Voracity-wrangled in Datadog. Read More

Tag: SortCL