{"id":13874,"date":"2020-07-02T10:47:04","date_gmt":"2020-07-02T14:47:04","guid":{"rendered":"http:\/\/www.iri.com\/blog\/?p=13874"},"modified":"2024-01-15T06:45:33","modified_gmt":"2024-01-15T11:45:33","slug":"darkshield-relational-databases","status":"publish","type":"post","link":"https:\/\/beta.iri.com\/blog\/data-protection\/darkshield-relational-databases\/","title":{"rendered":"Using DarkShield with Relational Databases"},"content":{"rendered":"

Editor’s Note:<\/strong> The content in this article has been deprecated with the release of IRI DarkShield Version 5<\/a> in 2024. To see how you can search and mask data in RDBs with the DarkShield GUI now, see this article<\/a>. To see how to do it in the API, see this article<\/a>.<\/p>\n

<\/em><\/p>\n

Introduction<\/b><\/h5>\n

In the previous article<\/a>, we discussed how IRI DarkShield<\/a> finds sensitive information using Data Class Search Matchers and masks it consistently using Data Rules. We also alluded to DarkShield\u2019s internal processing model, which is based on an abstract view of data as a collection of documents. Understanding this model will help you understand how DarkShield handles structured, semi-structured and unstructured data in relational databases in much the same way it does file-based documents in local file systems, network shares, or S3 buckets.<\/p>\n

Note that the functionality described in this article is only supported through the IRI Workbench<\/a> graphical IDE, built on Eclipse. It cannot be replicated through the DarkShield Command Line Interface (CLI) or Remote Procedure Call (RPC) API frameworks at this time.<\/p>\n

DarkShield vs. FieldShield<\/b><\/h5>\n

IRI FieldShield<\/a> has been the primary product for masking data in flat files and relational databases. There is now overlap due to DarkShield\u2019s new ability to classify, reach, and mask the same data as FieldShield. The main difference between the two is that FieldShield deals with data on a column basis, while DarkShield\u2019s approach to searching and masking is more granular.<\/p>\n

Thus in practice, DarkShield can find and mask PII in Large Objects (LOBs), XML, and other free-form text embedded within columns, while FieldShield can only work with First Normal Form (1NF) atomic column data. However, DarkShield jobs can only discover, extract, and mask data, while FieldShield job specifications can be also be used to \u201cmask data on the fly\u201d within IRI Voracity<\/a>-supported ETL, subsetting, migration, replication, cleansing, and reporting scripts.<\/p>\n

Following is a comparison of DB-centric capabilities in DarkShield and FieldShield:<\/p>\n

\"\"<\/a><\/p>\n

Both products share the same Eclipse front-end (IRI Workbench<\/a>), executable back-end (SortCL<\/a>), data classes, and masking functions. That means you can, for example, encrypt in DarkShield and decrypt in FieldShield, for example. The key interface difference is in their job design wizards and serialized configuration files (defining the masking jobs).<\/p>\n

Both DarkShield and Fieldshield are also included in the\u00a0IRI Voracity<\/a> data management platform (along with CellShield EE and RowGen), should you require both and more capabilities<\/a>.\u00a0 DarkShield pricing information is on this<\/a> page.<\/p>\n

Tables are Documents, Cells are Embedded Documents<\/b><\/h5>\n

To understand the extent of DarkShield searching and masking capabilities within relational databases, it\u2019s important to remember the DarkShield mantra that \u201ceverything is a document.\u201d In the context of relational databases, every table is considered a tabular document, which means it can be filtered by column name or index much like you would a CSV or Excel file.<\/p>\n

Moreover, every cell inside of a table is treated as an embedded document, meaning that cells will be parsed according to their detected media type (text, json, pdf, etc.). Embedded PDFs and images in BLOB columns, for example, will be searched and masked as if they were regular files in a file system, but stored inside the table.<\/p>\n

To reinforce this mental model, we will show you how DarkShield handles a table containing free form text and XML data. The same logic that we will outline here can be applied to tables which contain BLOB data, like embedded images and PDFs.<\/p>\n

Data Connectivity and Setup<\/b><\/h5>\n

If you don\u2019t have any connection profiles to the database(s) that DarkShield needs to access, refer to\u00a0these example how-tos<\/a>\u00a0to add them in IRI Workbench; note that only JDBC connections are needed for DarkShield. The rest of this article assumes that you are familiar with Data Classes and Data Rules from our introductory article, which we encourage you to read before continuing now.<\/p>\n

To demonstrate DarkShield RDB search\/mask capabilities, I created a simple Postgres table named \u201cclients\u201d, which contains some structured columns (name, email), as well as an XML column for address data (address_xml), and a free-flowing text field (notes).<\/p>\n

The DDL for this table can be created and executed in Workbench by clicking \"\"<\/a> on SQL scrapbook in the\u00a0\"\"<\/a> Data Source Explorer<\/i> view, selecting the connection profile in the Name <\/i>field (in our case, the connection profile is called DarkShield<\/i>), and running this DDL statement:<\/p>\n

\"\"<\/a><\/p>\n

We can right click inside the scrapbook and select Execute All<\/i> to create two tables, one inside our public <\/i>schema to act as our source table and another inside the target <\/i>schema. Note that if the target table doesn\u2019t exist, DarkShield will try to automatically create a target table DDL from the metadata of the source table.<\/p>\n

If the DDL fails to execute, DarkShield will log an error in the error view. To see what DDL DarkShield will attempt to generate for your source table, navigate to the table in the Data Source Explorer, right click on the table and select Generate DDL<\/i>.<\/p>\n

Populating this table with test data is outside the scope of this article. Please see IRI RowGen<\/a> for more information about our product synthesizing structurally and referentially correct RDB test data from scratch. RowGen also runs from IRI Workbench and is included in Voracity.<\/p>\n

The Dark Data Schema Masking Wizard<\/b><\/h5>\n

Once your connections are established, you can start defining your search and masking criteria. From the\u00a0\"\"<\/a> DarkShield menu<\/i> dropdown options off the IRI Workbench top toolbar, select the New Dark Data Schema Search\/Masking Job <\/i>option.<\/p>\n

Source Selector<\/b><\/h5>\n

After specifying your Workbench project and job names in the setup page, click Next <\/i>to begin filling out the Source Selector <\/i>page. Here, we can select our connection profile or create a new one from scratch, and then select the schema we wish to search for sensitive information:<\/p>\n

\"\"<\/a><\/p>\n

The form also contains several optional fields:<\/p>\n