{"id":10224,"date":"2016-06-02T15:08:37","date_gmt":"2016-06-02T19:08:37","guid":{"rendered":"http:\/\/www.iri.com\/blog\/?p=10224"},"modified":"2017-11-06T10:26:16","modified_gmt":"2017-11-06T15:26:16","slug":"creating-a-voracity-flow-from-the-palette-part-1-of-2","status":"publish","type":"post","link":"https:\/\/beta.iri.com\/blog\/data-protection\/creating-a-voracity-flow-from-the-palette-part-1-of-2\/","title":{"rendered":"Creating a Voracity Flow from the Palette (Part 1 of 2)"},"content":{"rendered":"

This is the first of two articles showing how to\u00a0create and use job flows in the IRI Workbench<\/span> GUI for Voracity<\/span><\/a>. It follows two other series on creating flows automatically using new job wizards.<\/span>1<\/sup><\/a><\/span><\/span><\/span>\u00a0 We will create the same flow built in the second series, but this time directly through the flow diagram editor.<\/span><\/p>\n

With this \u201cdesign from scratch\u201d method, start with a basic flow building block, or \u201cflowlet\u201d, then add to it by dragging components from the <\/span>Palette <\/b>into the flowlet.<\/span><\/p>\n

Remember that flows contain Extract-Transform-Load (ETL) and other data processing steps that are illustrated in flow and transform mapping diagrams in Workbench. They are saved in .flow files for execution from IRI workbench, or serialized as .sh or .bat files for execution from inside or outside the Workbench.<\/span><\/p>\n

The flow we are making here matches data in an Oracle table called PATIENT_RECORD with billing information from the PATIENTBILLING table, then it sends the results to collections. Some of the data in the PATIENT_RECORD table is sensitive. After extracting the data from the table, this phase of the job will mask some of the columns and eliminate others.<\/span><\/p>\n

Begin by creating a project in IRI Workbench called <\/span>ScratchFlow <\/span><\/i>to hold all the files associated with this Flow. Make sure this project is highlighted in the project explorer before continuing.<\/span><\/p>\n

Create the Flowlet<\/b><\/h3>\n

Click <\/span>File<\/b> on the menu bar > <\/span>New<\/b> > <\/span>Flow<\/b>. Type in <\/span>Scratch.flow<\/span><\/i> for the <\/span>File name<\/b>, then click <\/span>Finish<\/b>.<\/span><\/p>\n

Scratch.flow<\/span> is now in your project and is open in the editor. Click on <\/span>Batch: unnamed <\/b>in the editor. \u00a0Look at the <\/span>Properties<\/b> view at the bottom of the screen. If the <\/span>Properties<\/b> view is not open, follow these steps to open it: click <\/span>Window <\/b>on the menu bar > <\/span>Show View<\/b> > <\/span>Properties<\/b>. Currently, <\/span>Batch<\/b> has <\/span>Name: <\/b>unnamed<\/span><\/i>. Change <\/span>unnamed <\/span><\/i>to <\/span>Scratch<\/span><\/i>, Click in the editor, then click <\/span>Save All <\/b>on the navigation bar. In both the editor and the project, you now have <\/span>Batch: Scratch<\/b> under Scratch.flow.<\/span><\/p>\n

In the project, right-click on <\/span>Batch: Scratch <\/b>> <\/span>New Representation \u00a0<\/b>> <\/span>new Flow Diagram. <\/b>Type in <\/span>Scratch Flow Diagram<\/span><\/i> for the<\/span> Representation name <\/b>and click<\/span> OK. <\/b>Click in the gray flowlet, then select <\/span>Base <\/b>in the <\/span>Properties <\/b>view, and change <\/span>Name <\/b>to Scratch. Click in the flowlet again.<\/span><\/p>\n

If <\/span>New Representation<\/b> is not a choice, follow these steps:<\/span><\/p>\n

Right-click on the Project name in the Project Explorer and click <\/span>Viewpoint Selection<\/b>. Put a check in the <\/span>Flow <\/b>box and click <\/span>OK<\/b>.<\/span> Then go back and follow the previous steps.<\/span><\/p>\n

A flowlet is created and displayed in the editor. It consists of a gray flowlet block that contains a green <\/span>Start <\/b>block as shown below.<\/span><\/p>\n

\"IRI<\/a><\/p>\n

We will create two data transformation jobs in the flowlet: PatientInfoProtect.scl and Collections.scl. The second will be created in the next article. They will each be represented in the flow by a <\/span>Transform Mapping Block<\/b>. You can visualize the contents of those blocks in transform mapping diagrams and in <\/span>SortCL<\/span><\/a> scripts once those scripts have been created.<\/span><\/p>\n

Create the First Transform<\/b><\/h3>\n

Drag a <\/span>Transform Mapping Block<\/b> from the palette on the right into the flowlet. The <\/span>Transform Mapping Block<\/b> diagram is the visual representation of a SortCL script where you can follow the \u00a0source-to-target field mappings in addition to any transforms that are performed.<\/span><\/p>\n

Double-click in the <\/span>Transform Mapping Block<\/b> to see a detailed view of the block. For <\/span>Representation name<\/b>, type PatientInfoProtect Mapping Diagram, then click <\/span>OK<\/b>. There are blocks in this diagram, for Input Data, Action, and Output Data.<\/span><\/p>\n

From the Palette on the right, drag an <\/span>Input ODBC<\/b> icon and drop it into the <\/span>Input Data<\/b> block. This will bring up the <\/span>Database Information<\/b> window. Select the <\/span>Data source name(DSN)<\/b> from the drop-down. The Table selection window will then populate with the <\/span>tables available<\/span><\/a> for that DSN. Click the checkbox for NIGHTLY.PATIENT_RECORD, then click <\/span>Finish<\/b>.<\/span><\/p>\n

The <\/span>Input Data<\/b> block now has a white source block with a table named PATIENT_RECORD, which contains a blue block for fields or columns. It also has a yellow <\/span>Section Options<\/b> block containing items that apply to the table or file as a whole.<\/span><\/p>\n

Click in the white source block. Then click <\/span>Arrange Selection<\/b> from the drop-down menu in the upper left corner of the editor.<\/span><\/p>\n

The next <\/span>Transform Mapping Block<\/b> of this flow will perform a join on IDNUMBER. Therefore, we can sort here using that field as the key. From the <\/span>Transform Action<\/b> section of the <\/span>Palette<\/b>, drag the <\/span>Sort<\/b> icon into the <\/span>Action<\/b> block. This brings up the <\/span>Sort <\/b>window.<\/span><\/p>\n

Select IDNUMBER from the <\/span>Input Fields<\/b> box and select <\/span>Add Key<\/b>. IDNUMBER is now displayed in the <\/span>Key Fields<\/b> box. You can repeat the process for as many keys as are necessary for the sort. In this instance, we only need one key. Click <\/span>Finish<\/b>.<\/span><\/p>\n

\"IRI<\/a><\/p>\n

The <\/span>Action<\/b> block is populated with a white <\/span>Sort<\/b> block that contains a gray block with the fields from the input, and an orange <\/span>Action Key<\/b> block containing the key that was just defined.<\/span><\/p>\n

Define the Target<\/b><\/h3>\n

Drag the <\/span>File<\/b> icon from the <\/span>Output<\/b> section of the <\/span>Palette<\/b> into the <\/span>Output Data<\/b> block. The <\/span>Output Selection Type<\/b> dialog will open. Because we need to define the output format for our target, select <\/span>Editor <\/b>to open the <\/span>Target Field Layout <\/b>editor.<\/span><\/p>\n

\"Output<\/a><\/p>\n

The top half of the editor has source definitions, and the bottom half has target definitions. Initially, the target has the name <\/span>Out_File <\/b>and all the input fields have been copied to the target. If you want, you can click the trash icon to delete all the fields in the target, and then drag the desired fields from source to target.<\/span><\/p>\n

Since we want most of the fields, we will, instead, remove the fields that we don\u2019t want. \u00a0We will remove AGE, EMAIL, DRIVING_LICENSE, and SSN. For each one, right-click on the field and select <\/span>Remove<\/b>. With those fields gone, the field positions need to be recalculated by clicking the <\/span>1-2-3<\/b> icon.<\/span><\/p>\n

Mask the Credit Card Numbers<\/b><\/h3>\n

There are two fields for holding credit card numbers. These need to be masked so that only the last 4 digits of the numbers can be viewed. \u00a0Right-click on the field CREDIT_CARD1 > <\/span>Apply Rule<\/b> > <\/span>Create Rule<\/b>.<\/span><\/p>\n

Click <\/span>Masking Function<\/b> under <\/span>Protection Rules<\/b> and verify that the <\/span>Library location<\/b> has our project name. Type CreditCardMask for the <\/span>Rule name,<\/b> and click <\/span>Next<\/b>.<\/span>\f<\/span><\/p>\n

\"New<\/a><\/p>\n

In the<\/span> Masking Function <\/b>window, select <\/span>Define mask<\/b>. Type * for <\/span>Mask <\/b>character, 1 for <\/span>Start position<\/b>, 12 for <\/span>Length<\/b>, and then click <\/span>Add to table<\/b>. The definition is added to the table at the bottom. Click <\/span>Finish<\/b>.<\/span><\/p>\n

\"Masking<\/a><\/p>\n

We have returned to the <\/span>Target Field Layou<\/b>t editor and now want to mask the other credit card field. \u00a0We already created the masking rule for credit cards, so we want to use that rule. Right-click on the field CREDIT_CARD2 > <\/span>Apply Rule<\/b> > <\/span>Browse Rule.<\/b> When the <\/span>Field Rule Library<\/b> window opens, select the rule named CreditCardMask, and then click <\/span>OK<\/b>.<\/span><\/p>\n

\"Field<\/a><\/p>\n

The two credit card fields now have the <\/span>Field <\/b>designation of <\/span>Function<\/b>. Also, the field names have been changed to MASK_CREDIT_CARD1 and MASK_CREDIT_CARD2 because these are now derived fields. Click <\/span>OK<\/b>.<\/span><\/p>\n

\"Target<\/a><\/p>\n

We are now back in the PatientInforProtect Transform Mapping Diagram. Click in the white <\/span>Out_File<\/b> block. In the <\/span>Properties <\/b>view, select <\/span>Base<\/b>, then change <\/span>Name <\/b>to PatientInfoProtected. Now, select the <\/span>. . .<\/b> button next to the <\/span>Data Info<\/b> field to go to the <\/span>FileDataInfo <\/b>dialog. Change the <\/span>Name <\/b>to PatientInfoProtected<\/span> and <\/span>File Name <\/b>to Out_PatientInfoProtected, and then click <\/span>Finish<\/b>. <\/span><\/p>\n

\"FileDataInfo\"<\/a><\/p>\n

The input file with fields are displayed in the <\/span>Input Data<\/b> block. You can follow the fields through the <\/span>Action <\/b>block where, in this case, the sort key is defined. Then, follow the connection arrows that map the fields into the blue field block of the <\/span>Output Data<\/b> block.<\/span><\/p>\n

Notice that not all fields were mapped into the output, and that the two fields where masking was applied have orange connecting arrows, instead of blue. This is because a masking function was applied to the values in the original fields before being accepted to the output.<\/span><\/p>\n

For more details on these major Voracity task blocks, see <\/span>Creating a Voracity Flow Using Existing IRI Scripts (Part 1<\/span><\/i><\/a>).<\/span><\/i><\/p>\n

\"Input-Action-Output<\/a><\/p>\n

Now click in the area outside the blocks. Select the <\/span>Base <\/b>tab<\/span> in the <\/span>Properties <\/b>view. Change <\/span>Name <\/b>and <\/span>IRI Job<\/b> to <\/span>PatientInfoProtect.scl<\/span><\/i>. The first field names the Transform Mapping Block represented in the above diagram, and the second field names the SortCL Job script that will be created from that diagram.<\/span><\/p>\n

Define the Metadata for Data File PatientInfoProtected<\/b><\/h3>\n

Because the output file <\/span>PatientInfoProtected <\/span><\/i>will be used as an input file for the second <\/span>Transform Mapping Block<\/b>, we need to create a DDF (metadata) file specifying the layout for that file. That layout is currently available in the blue field block for the <\/span>Output Data<\/b> file represented in the above diagram. Right-click in the blue block for the fields in the <\/span>Output Data<\/b> block > <\/span>IRI Diagram Actions<\/b> > <\/span>Create DDF<\/b>. Keep the <\/span>File name<\/b> PatientInfoProtected.ddf <\/span><\/i>in the <\/span>DDF Options<\/b> dialog and click <\/span>Finish<\/b>. The DDF file is now in the metadata directory of our project and can be referenced when the second <\/span>Transform Mapping Block<\/b> is created in the next article. <\/span><\/p>\n

\"Export<\/a><\/p>\n

Return to the Scratch Flow Diagram in the editor. Connect the <\/span>Start <\/b>block to the input table for the transform mapping block named PatientInfoProtect.scl by clicking the <\/span>Connection <\/b>icon in the <\/span>Palette<\/b>. Then click the <\/span>Start <\/b>block, followed by the icon for the input table PATIENT_RECORD. In the diagram below, you can see the flow named Scratch that was created.<\/span><\/p>\n

\"Scratch<\/a><\/p>\n

Serialize the Job for Execution<\/b><\/h3>\n

To create the job script PatientInfoProtect.scl and the batch file for this flow, right-click in the editor > <\/span>IRI Diagram Actions<\/b> > <\/span>Export Flow Component<\/b>. Select <\/span>Windows <\/b>from the <\/span>Platform <\/b>drop-down and verify that Scratch.bat is in the <\/span>File name<\/b> field. This is the name of the file that will execute the flow. Click <\/span>Finish<\/b>. The CoSort\/SortCL Job script and the batch file are created and placed in the project. \u00a0<\/span><\/p>\n

Note that flow and batch files can contain multiple components in addition to transform blocks (SortCL scripts), including blocks for <\/span>IRI FACT<\/span><\/a> (fast extract) or bulk DB load tasks, SQL and command line programs, FTP and decision blocks, etc.<\/span><\/p>\n

\"Export<\/a><\/p>\n

Review the Transform Script<\/b><\/h3>\n

After the job script PatientInfoProtect.scl has been generated, it can be displayed in the editor by double-clicking on that .scl file in the <\/span>Project Explorer<\/b>. Red items in the editor \u00a0indicate <\/span>syntax<\/span><\/a> keywords, and blue items indicate strings.<\/span><\/p>\n

In the <\/span>Model <\/b>outline to the right of the editor, you can click an item and see it highlighted in the editor. For example, if I click <\/span>Output Section<\/b> in the outline, the output section of the (SortCL job) script is highlighted:<\/span><\/p>\n

\"Project<\/a><\/p>\n

You can also right-click on items in the outline to edit them, or in the script editor. This will open IRI menus with editing options. Any edits made to the script are not automatically reflected in the flow (you must save the script and drag its file back into the flow), but it\u2019s another way to make changes.<\/span><\/p>\n

Following are the batch script and the job script it can execute. This is done from within IRI Workbench directly, through its task scheduler, any third-party job scheduler, or from the command line:<\/span><\/p>\n

The\u201cScratch.bat\u201d file contains:<\/span><\/p>\n

@echo off\r\n<\/span>sortcl \/SPECIFICATION=PatientInfoProtect.scl<\/span><\/pre>\n
The PatientInfoProtect.scl job we created contains:<\/span><\/p>\n
\/INFILE=\"NIGHTLY.PATIENT_RECORD;DSN=Oracle_qa2;UID=nightly;PWD=N321ghtly;\"\r\n<\/span>     \/ALIAS=NIGHTLY_PATIENT_RECORD\r\n<\/span>     \/PROCESS=ODBC\r\n<\/span>     \/FIELD=(IDNUMBER, TYPE=ASCII, POSITION=1, SEPARATOR=\"\\t\", EXT_FIELD=\"IDNUMBER\")\r\n<\/span>     \/FIELD=(NAME, TYPE=ASCII, POSITION=2, SEPARATOR=\"\\t\", EXT_FIELD=\"NAME\")\r\n<\/span>     \/FIELD=(AGE, TYPE=ASCII, POSITION=3, SEPARATOR=\"\\t\", EXT_FIELD=\"AGE\")\r\n<\/span>     \/FIELD=(EMAIL, TYPE=ASCII, POSITION=4, SEPARATOR=\"\\t\", EXT_FIELD=\"EMAIL\")\r\n<\/span>     \/FIELD=(PHONE, TYPE=ASCII, POSITION=5, SEPARATOR=\"\\t\", EXT_FIELD=\"PHONE\")\r\n<\/span>     \/FIELD=(CREDIT_CARD1, TYPE=ASCII, POSITION=6, SEPARATOR=\"\\t\", \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0EXT_FIELD=\"CREDIT_CARD1\")\r\n<\/span>     \/FIELD=(CREDIT_CARD2, TYPE=ASCII, POSITION=7, SEPARATOR=\"\\t\", \u00a0\u00a0\u00a0EXT_FIELD=\"CREDIT_CARD2\")\r\n<\/span>     \/FIELD=(DRIVING_LICENSE, TYPE=ASCII, POSITION=8, SEPARATOR=\"\\t\", EXT_FIELD=\"DRIVING_LICENSE\")\r\n<\/span>     \/FIELD=(SSN, TYPE=ASCII, POSITION=9, SEPARATOR=\"\\t\", EXT_FIELD=\"SSN\")\r\n<\/span>     \/FIELD=(CITY, TYPE=ASCII, POSITION=10, SEPARATOR=\"\\t\", EXT_FIELD=\"CITY\")\r\n<\/span>     \/FIELD=(STATE, TYPE=ASCII, POSITION=11, SEPARATOR=\"\\t\", EXT_FIELD=\"STATE\")<\/span>\r\n\r\n\/SORT\r\n<\/span>\/KEY=(IDNUMBER, TYPE=ASCII)<\/span>\r\n\r\n\/OUTFILE=PatientInfoProtected\r\n<\/span>     \/PROCESS=RECORD\r\n<\/span>     \/FIELD=(IDNUMBER, TYPE=ASCII, POSITION=1, SEPARATOR=\"\\t\", EXT_FIELD=\"IDNUMBER\")\r\n<\/span>     \/FIELD=(NAME, TYPE=ASCII, POSITION=2, SEPARATOR=\"\\t\", EXT_FIELD=\"NAME\")\r\n<\/span>     \/FIELD=(PHONE, TYPE=ASCII, POSITION=3, SEPARATOR=\"\\t\", EXT_FIELD=\"PHONE\")\r\n<\/span>     \/FIELD=(MASK_CREDIT_CARD1=replace_chars(CREDIT_CARD1, \"*\", 1, 12), TYPE=ASCII, POSITION=4, SEPARATOR=\"\\t\", EXT_FIELD=\"CREDIT_CARD1\")\r\n<\/span>     \/FIELD=(MASK_CREDIT_CARD2=replace_chars(CREDIT_CARD2, \"*\", 1, 12), TYPE=ASCII, POSITION=5, SEPARATOR=\"\\t\", EXT_FIELD=\"CREDIT_CARD2\")\r\n<\/span>     \/FIELD=(CITY, TYPE=ASCII, POSITION=6, SEPARATOR=\"\\t\", EXT_FIELD=\"CITY\")\r\n<\/span>     \/FIELD=(STATE, TYPE=ASCII, POSITION=7, SEPARATOR=\"\\t\", EXT_FIELD=\"STATE\")<\/span><\/pre>\n
In the next article<\/a>, we will add to the flow by defining a second <\/span>Transform Mapping Block<\/b>. The output from the script, represented here by the Transform Mapping Block PatientInfoProtect.scl, will become one of the inputs for the new <\/span>Transform Mapping Block<\/b>.<\/span><\/p>\n
 <\/p>\n","protected":false},"excerpt":{"rendered":"
This is the first of two articles showing how to\u00a0create and use job flows in the IRI Workbench GUI for Voracity. It follows two other series on creating flows automatically using new job wizards.\u00a0 We will create the same flow built in the second series, but this time directly through the flow diagram editor. With<\/p>\n
Read More<\/a><\/div>\n","protected":false},"author":10,"featured_media":10238,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"footnotes":""},"categories":[8,776,91],"tags":[1122,14,561,1068,100,1127,1123,1125,92,1113,830,789,850,967,489,75,1114,1124,1126,941,1128],"class_list":["post-10224","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-data-protection","category-etl","category-iri-workbench","tag-create-job-flow","tag-data-masking","tag-ddf","tag-dsn","tag-etl","tag-field-rule-library","tag-flow-diagram-editor","tag-flowlet","tag-gui","tag-input","tag-iri-fact","tag-iri-voracity","tag-iri-workbench","tag-job-wizard","tag-metadata","tag-odbc","tag-output","tag-palette","tag-target-field-layout","tag-transform-mapping-block","tag-transform-script"],"acf":[],"yoast_head":"\nCreating a Voracity Flow from the Palette (Part 1 of 2) - IRI<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/beta.iri.com\/blog\/data-protection\/creating-a-voracity-flow-from-the-palette-part-1-of-2\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Creating a Voracity Flow from the Palette (Part 1 of 2) - IRI\" \/>\n<meta property=\"og:description\" content=\"This is the first of two articles showing how to\u00a0create and use job flows in the IRI Workbench GUI for Voracity. It follows two other series on creating flows automatically using new job wizards.\u00a0 We will create the same flow built in the second series, but this time directly through the flow diagram editor. WithRead More\" \/>\n<meta property=\"og:url\" content=\"https:\/\/beta.iri.com\/blog\/data-protection\/creating-a-voracity-flow-from-the-palette-part-1-of-2\/\" \/>\n<meta property=\"og:site_name\" content=\"IRI\" \/>\n<meta property=\"article:published_time\" content=\"2016-06-02T19:08:37+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2017-11-06T15:26:16+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2016\/07\/11-Scratch-Flow-Diagram.png\" \/>\n\t<meta property=\"og:image:width\" content=\"897\" \/>\n\t<meta property=\"og:image:height\" content=\"651\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Susan Gegner\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Susan Gegner\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"11 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/beta.iri.com\/blog\/data-protection\/creating-a-voracity-flow-from-the-palette-part-1-of-2\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/data-protection\/creating-a-voracity-flow-from-the-palette-part-1-of-2\/\"},\"author\":{\"name\":\"Susan Gegner\",\"@id\":\"https:\/\/beta.iri.com\/blog\/#\/schema\/person\/87be5da567628ab9396ca81170f36d63\"},\"headline\":\"Creating a Voracity Flow from the Palette (Part 1 of 2)\",\"datePublished\":\"2016-06-02T19:08:37+00:00\",\"dateModified\":\"2017-11-06T15:26:16+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/data-protection\/creating-a-voracity-flow-from-the-palette-part-1-of-2\/\"},\"wordCount\":2039,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/data-protection\/creating-a-voracity-flow-from-the-palette-part-1-of-2\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2016\/07\/11-Scratch-Flow-Diagram.png\",\"keywords\":[\"create job flow\",\"data masking\",\"DDF\",\"DSN\",\"ETL\",\"field rule library\",\"flow diagram editor\",\"flowlet\",\"GUI\",\"input\",\"IRI FACT\",\"IRI Voracity\",\"IRI Workbench\",\"job wizard\",\"metadata\",\"ODBC\",\"output\",\"Palette\",\"target field layout\",\"transform mapping block\",\"transform script\"],\"articleSection\":[\"Data Masking\/Protection\",\"ETL\",\"IRI Workbench\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/beta.iri.com\/blog\/data-protection\/creating-a-voracity-flow-from-the-palette-part-1-of-2\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/beta.iri.com\/blog\/data-protection\/creating-a-voracity-flow-from-the-palette-part-1-of-2\/\",\"url\":\"https:\/\/beta.iri.com\/blog\/data-protection\/creating-a-voracity-flow-from-the-palette-part-1-of-2\/\",\"name\":\"Creating a Voracity Flow from the Palette (Part 1 of 2) - IRI\",\"isPartOf\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/data-protection\/creating-a-voracity-flow-from-the-palette-part-1-of-2\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/data-protection\/creating-a-voracity-flow-from-the-palette-part-1-of-2\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2016\/07\/11-Scratch-Flow-Diagram.png\",\"datePublished\":\"2016-06-02T19:08:37+00:00\",\"dateModified\":\"2017-11-06T15:26:16+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/data-protection\/creating-a-voracity-flow-from-the-palette-part-1-of-2\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/beta.iri.com\/blog\/data-protection\/creating-a-voracity-flow-from-the-palette-part-1-of-2\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/beta.iri.com\/blog\/data-protection\/creating-a-voracity-flow-from-the-palette-part-1-of-2\/#primaryimage\",\"url\":\"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2016\/07\/11-Scratch-Flow-Diagram.png\",\"contentUrl\":\"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2016\/07\/11-Scratch-Flow-Diagram.png\",\"width\":897,\"height\":651,\"caption\":\"Scratch Flow Diagram\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/beta.iri.com\/blog\/data-protection\/creating-a-voracity-flow-from-the-palette-part-1-of-2\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/beta.iri.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Creating a Voracity Flow from the Palette (Part 1 of 2)\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/beta.iri.com\/blog\/#website\",\"url\":\"https:\/\/beta.iri.com\/blog\/\",\"name\":\"IRI\",\"description\":\"Total Data Management Blog\",\"publisher\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/beta.iri.com\/blog\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/beta.iri.com\/blog\/#organization\",\"name\":\"IRI\",\"url\":\"https:\/\/beta.iri.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/beta.iri.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png\",\"contentUrl\":\"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png\",\"width\":750,\"height\":206,\"caption\":\"IRI\"},\"image\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/beta.iri.com\/blog\/#\/schema\/person\/87be5da567628ab9396ca81170f36d63\",\"name\":\"Susan Gegner\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/beta.iri.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/2b1ca5592a65d44483351292cf1ae00a?s=96&d=blank&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/2b1ca5592a65d44483351292cf1ae00a?s=96&d=blank&r=g\",\"caption\":\"Susan Gegner\"},\"url\":\"https:\/\/beta.iri.com\/blog\/author\/susang\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Creating a Voracity Flow from the Palette (Part 1 of 2) - IRI","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/beta.iri.com\/blog\/data-protection\/creating-a-voracity-flow-from-the-palette-part-1-of-2\/","og_locale":"en_US","og_type":"article","og_title":"Creating a Voracity Flow from the Palette (Part 1 of 2) - IRI","og_description":"This is the first of two articles showing how to\u00a0create and use job flows in the IRI Workbench GUI for Voracity. It follows two other series on creating flows automatically using new job wizards.\u00a0 We will create the same flow built in the second series, but this time directly through the flow diagram editor. WithRead More","og_url":"https:\/\/beta.iri.com\/blog\/data-protection\/creating-a-voracity-flow-from-the-palette-part-1-of-2\/","og_site_name":"IRI","article_published_time":"2016-06-02T19:08:37+00:00","article_modified_time":"2017-11-06T15:26:16+00:00","og_image":[{"width":897,"height":651,"url":"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2016\/07\/11-Scratch-Flow-Diagram.png","type":"image\/png"}],"author":"Susan Gegner","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Susan Gegner","Est. reading time":"11 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/beta.iri.com\/blog\/data-protection\/creating-a-voracity-flow-from-the-palette-part-1-of-2\/#article","isPartOf":{"@id":"https:\/\/beta.iri.com\/blog\/data-protection\/creating-a-voracity-flow-from-the-palette-part-1-of-2\/"},"author":{"name":"Susan Gegner","@id":"https:\/\/beta.iri.com\/blog\/#\/schema\/person\/87be5da567628ab9396ca81170f36d63"},"headline":"Creating a Voracity Flow from the Palette (Part 1 of 2)","datePublished":"2016-06-02T19:08:37+00:00","dateModified":"2017-11-06T15:26:16+00:00","mainEntityOfPage":{"@id":"https:\/\/beta.iri.com\/blog\/data-protection\/creating-a-voracity-flow-from-the-palette-part-1-of-2\/"},"wordCount":2039,"commentCount":0,"publisher":{"@id":"https:\/\/beta.iri.com\/blog\/#organization"},"image":{"@id":"https:\/\/beta.iri.com\/blog\/data-protection\/creating-a-voracity-flow-from-the-palette-part-1-of-2\/#primaryimage"},"thumbnailUrl":"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2016\/07\/11-Scratch-Flow-Diagram.png","keywords":["create job flow","data masking","DDF","DSN","ETL","field rule library","flow diagram editor","flowlet","GUI","input","IRI FACT","IRI Voracity","IRI Workbench","job wizard","metadata","ODBC","output","Palette","target field layout","transform mapping block","transform script"],"articleSection":["Data Masking\/Protection","ETL","IRI Workbench"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/beta.iri.com\/blog\/data-protection\/creating-a-voracity-flow-from-the-palette-part-1-of-2\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/beta.iri.com\/blog\/data-protection\/creating-a-voracity-flow-from-the-palette-part-1-of-2\/","url":"https:\/\/beta.iri.com\/blog\/data-protection\/creating-a-voracity-flow-from-the-palette-part-1-of-2\/","name":"Creating a Voracity Flow from the Palette (Part 1 of 2) - IRI","isPartOf":{"@id":"https:\/\/beta.iri.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/beta.iri.com\/blog\/data-protection\/creating-a-voracity-flow-from-the-palette-part-1-of-2\/#primaryimage"},"image":{"@id":"https:\/\/beta.iri.com\/blog\/data-protection\/creating-a-voracity-flow-from-the-palette-part-1-of-2\/#primaryimage"},"thumbnailUrl":"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2016\/07\/11-Scratch-Flow-Diagram.png","datePublished":"2016-06-02T19:08:37+00:00","dateModified":"2017-11-06T15:26:16+00:00","breadcrumb":{"@id":"https:\/\/beta.iri.com\/blog\/data-protection\/creating-a-voracity-flow-from-the-palette-part-1-of-2\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/beta.iri.com\/blog\/data-protection\/creating-a-voracity-flow-from-the-palette-part-1-of-2\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/beta.iri.com\/blog\/data-protection\/creating-a-voracity-flow-from-the-palette-part-1-of-2\/#primaryimage","url":"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2016\/07\/11-Scratch-Flow-Diagram.png","contentUrl":"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2016\/07\/11-Scratch-Flow-Diagram.png","width":897,"height":651,"caption":"Scratch Flow Diagram"},{"@type":"BreadcrumbList","@id":"https:\/\/beta.iri.com\/blog\/data-protection\/creating-a-voracity-flow-from-the-palette-part-1-of-2\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/beta.iri.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Creating a Voracity Flow from the Palette (Part 1 of 2)"}]},{"@type":"WebSite","@id":"https:\/\/beta.iri.com\/blog\/#website","url":"https:\/\/beta.iri.com\/blog\/","name":"IRI","description":"Total Data Management Blog","publisher":{"@id":"https:\/\/beta.iri.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/beta.iri.com\/blog\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/beta.iri.com\/blog\/#organization","name":"IRI","url":"https:\/\/beta.iri.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/beta.iri.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png","contentUrl":"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png","width":750,"height":206,"caption":"IRI"},"image":{"@id":"https:\/\/beta.iri.com\/blog\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/beta.iri.com\/blog\/#\/schema\/person\/87be5da567628ab9396ca81170f36d63","name":"Susan Gegner","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/beta.iri.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/2b1ca5592a65d44483351292cf1ae00a?s=96&d=blank&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/2b1ca5592a65d44483351292cf1ae00a?s=96&d=blank&r=g","caption":"Susan Gegner"},"url":"https:\/\/beta.iri.com\/blog\/author\/susang\/"}]}},"jetpack_featured_media_url":"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2016\/07\/11-Scratch-Flow-Diagram.png","_links":{"self":[{"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/posts\/10224"}],"collection":[{"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/users\/10"}],"replies":[{"embeddable":true,"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/comments?post=10224"}],"version-history":[{"count":8,"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/posts\/10224\/revisions"}],"predecessor-version":[{"id":10326,"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/posts\/10224\/revisions\/10326"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/media\/10238"}],"wp:attachment":[{"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/media?parent=10224"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/categories?post=10224"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/tags?post=10224"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}