{"id":7583,"date":"2015-07-14T08:23:47","date_gmt":"2015-07-14T12:23:47","guid":{"rendered":"http:\/\/www.iri.com\/blog\/?p=7583"},"modified":"2020-05-29T09:32:31","modified_gmt":"2020-05-29T13:32:31","slug":"drawing-values-from-set-files","status":"publish","type":"post","link":"https:\/\/beta.iri.com\/blog\/data-transformation2\/drawing-values-from-set-files\/","title":{"rendered":"Drawing Values from Set Files"},"content":{"rendered":"<p><em>This article looks at sets from an informational processing perspective; what they are; how they are\u00a0constructed; and, distinct ways in which data can be drawn from sets within IRI software <a href=\"https:\/\/www.iri.com\/products\">products<\/a>\u00a0using the <a href=\"https:\/\/www.iri.com\/products\/cosort\/sortcl\">SortCL<\/a> data definition and processing program; i.e., in CoSort and NextForm to replace values, FieldShield for pseudonomization, RowGen for random value selection, and the IRI Voracity platform (which includes them all) for lookup transforms in ETL and data cleansing jobs.<\/em><\/p>\n<p>&nbsp;<\/p>\n<p><strong>Defining Terms<\/strong><\/p>\n<p>A data set is a collection of objects that are grouped together because they are instances of the same kind. In an informational environment, a set could be the shared repository of everything of\u00a0that class or, at least, everything that is meaningful to the enterprise. The objects are not\u00a0necessarily ordered or\u00a0unique. A set file is also valuable because it is reusable and application\u00ad-independent.<\/p>\n<p>In IRI software applications, sets are represented in ASCII files with ASCII character values. Each\u00a0entry in the set, or row, or object, is a line-\u00adfeed terminated record. IRI ships with several set files, has wizards to help you generate them from DB columns or other files, or you can simply supply your own.<\/p>\n<p>An object can also be a composite; for example, name\u00ad-address-\u00adphone number (presumably attributes\u00a0of one person). Composite values are separated by a \u201ctab\u201d character, which can be easily seen on\u00a0paper or a monitor. Every line would then have the same number of composite items; and if an item\u00a0were null, the tab would be present.<\/p>\n<p>An object can be a numeric range, either open or closed. It that object is drawn, the system will\u00a0derive a value in that range and return it to the caller.\u00a0Data values, such as binary data that naturally contain line feeds or tabs, cannot use the lookup\u00a0methods shown here.<\/p>\n<p>The objects can be derived from a database, produced in a production, edited by hand, or drawn from\u00a0other sets. Once built, we will draw objects for further processing. We now address how the same\u00a0sets can be referenced and used for different purposes.<\/p>\n<p><strong><br \/>\nIn Practice<\/strong><\/p>\n<p>The \u201c<a href=\"http:\/\/www.iri.com\/products\/cosort\/sortcl\" target=\"_blank\" rel=\"noopener\">SortCL<\/a>\u201d\u00a0processing language of the IRI Voracity platform and its component products (CoSort, NextForm, FieldShield, RowGen) has syntax for extracting a value from a set and assigning it to a field.<\/p>\n<p>Set values can be used in: Voracity ETL jobs for lookup transforms; in CoSort scripts for data mapping, enrichment\/validation, and\/or reporting; in NextForm file or database table migrations or updates;\u00a0 in FieldShield pseudonymization jobs; and, in RowGen test data synthesis jobs when random selections from real data are\u00a0preferred over artificial value generation.<\/p>\n<p>A basic reference would look like this:<\/p>\n<pre>\/FIELD = (City, SET &lt;select&gt; = State_City.set [indices])<\/pre>\n<p>There are additional parameters regarding the position of City in the record which are well known to\u00a0the above users. Consider just the\u00a0optional [indices] portion:<\/p>\n<ol start=\"1\">\n<li>With no indices, the draw is based on the relative position of the line in the set (e.g.,\u00a0first, last, or random).<\/li>\n<li>With an index or indices, the selection is based on the content of the data in the set (e.g.,\u00a0the entry whose name is Smith).<\/li>\n<\/ol>\n<p><strong><br \/>\nDrawing by\u00a0Position<\/strong><\/p>\n<p>Without regard to content, you can draw a value from the set by selecting a row or line number which corresponds to a record number of the set file.<\/p>\n<p>RowGen (test data generation) applications make random draws. Note the benefit of having sets with duplicate entries, as:<\/p>\n<pre>Male\r\nFemale\r\nMale\r\n<\/pre>\n<p>Over time, random draws will be 2 to 1 Male. No matter the number of different objects, any desired weighting can be achieved to any level of granularity.<\/p>\n<p>Other types of draws make successive draws to capture every object in the set with\/without repeating any entry. The concern now is what to return when there are more accesses than rows.<\/p>\n<p>The &lt;select&gt; option is one of the following:<\/p>\n<p><a href=\"#\" name=\"2a64818e2acdd7b5acd6030f516311ef786bb9e1\"><\/a><a href=\"#\" name=\"0\"><\/a><\/p>\n<table  class=\" table table-hover\" style=\"border-color: #b0acac;\" border=\"1\" cellspacing=\"0\" cellpadding=\"3\">\n<tbody>\n<tr>\n<td colspan=\"1\" rowspan=\"1\">ANY\u00a0(default)<\/td>\n<td colspan=\"1\" rowspan=\"1\">uses a random number [e.g.,\u00a01,20,000] as a line number<\/td>\n<\/tr>\n<tr>\n<td colspan=\"1\" rowspan=\"1\">ALL<\/td>\n<td colspan=\"1\" rowspan=\"1\">begin at line 1; subsequently increment by 1, after last, repeat<\/td>\n<\/tr>\n<tr>\n<td colspan=\"1\" rowspan=\"1\">ONCE<\/td>\n<td colspan=\"1\" rowspan=\"1\">same as ALL but after the last line, return NULL<\/td>\n<\/tr>\n<tr>\n<td colspan=\"1\" rowspan=\"1\">SUFFIX<\/td>\n<td colspan=\"1\" rowspan=\"1\">same as ALL but after the last, restart the list affixing a counter number at the end of the table entry<\/td>\n<\/tr>\n<tr>\n<td colspan=\"1\" rowspan=\"1\">FIRST<\/td>\n<td colspan=\"1\" rowspan=\"1\">select the first line of the set<\/td>\n<\/tr>\n<tr>\n<td colspan=\"1\" rowspan=\"1\">LAST<\/td>\n<td colspan=\"1\" rowspan=\"1\">select the last line of the set<\/td>\n<\/tr>\n<tr>\n<td colspan=\"1\" rowspan=\"1\">ROW[n]<\/td>\n<td colspan=\"1\" rowspan=\"1\">select the n\u2019th composite from an ANY draw<\/td>\n<\/tr>\n<tr>\n<td colspan=\"1\" rowspan=\"1\">PERMUTE<\/td>\n<td colspan=\"1\" rowspan=\"1\">every combination of composite items from a draw<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>&nbsp;<\/p>\n<p><strong>Drawing by Content<\/strong><\/p>\n<p>These are the fields statements where set indices are given. Indices are one or more comma separated string literals or field names. The number of indices is very important.<\/p>\n<p>A reference into a table will have less indices than there are columns in the table. A trivial example of this is a one-column table which will return a random value when given zero search arguments. In this way, a positional draw described above is just a special case of a content dependent draw.<\/p>\n<p>More typically, a table will have two or more columns. Say we had a short two\u00ad-column table of state name and capitol city. If the search argument is the state name; the system would then return its capitol city. Two-\u00adcolumn tables are used widely for pseudonymization and restoration, daily pricing, and anywhere that variable one-\u00adto-\u00adone mapping is required.<\/p>\n<p>To draw a row from a specified row, it is necessary to impose organization. The set is considered as a database table, spreadsheet, or other flat-file organization. Viewed vertically, the composite values are the columns. To efficiently find a row with a specific value, the table needs be ordered; i.e., sorted from left to right in the order in which the indices will be presented. This allows a search to use logarithmic (log2n) algorithm rather than linear.<\/p>\n<p>In this discussion, we will be drawing cities from a set file that contains 20,000 state\u00ad-county-\u00adcity triples representing every city in the United States. Every triplet will be unique.<\/p>\n<p>The s<em>tate_county_city.set<\/em> file contains this subset of tab-delimited values from Pennsylvania:<\/p>\n<pre>Penn\t Adams\t\tAbbottstown\r\nPenn\t Adams\t\tBenderville\r\nPenn\t Adams\t\tBiglerville\r\n...\r\nPenn\t Adams\t\tYorksprings\r\n...\r\nPenn\t Allegheny\tAllison Park\r\nPenn\t Allegheny\tAspenwalll\r\n...\r\nPenn\t Allegheny\tWilmerding\r\n...\r\nPenn\t Wyoming\tWindham\r\nPenn\t York\t\tDallastown\r\nPenn\t York\t\tDelta\r\n...\r\nPenn\t York\t\tYorklyn\r\n<\/pre>\n<p>For a table where n = 20,000, a specific row can be found, on average, in about 14 tries rather than 10,000. Just as with a database search, to find a specific row, the user needs to provide a key by way of value parameter which is then matched to a value in the table.<\/p>\n<p>The search returns that part of the table entry which follows the index. Refer to the Penn entry in the<br \/>\ntable. The statement:<\/p>\n<pre>\/FIELD = (City, SET = State_City.set [\u201cPenn\u201d, \u201cYork\u201d])\r\n<\/pre>\n<p>Since there is some 50 cities in York County so a possible result would be \u201cYorklyn\u201d While:<\/p>\n<pre>\/FIELD = (County_City, SET = State_City.set [\u201cPenn\u201d])\r\n<\/pre>\n<p>would return something like. \u201cAllegheny Aspenwalll\u201d. You could pare off the \u201cAspenwall\u201d portion if desired. And if you want to be specific, you would need to use the &lt;select&gt; option, as in:<\/p>\n<pre>\/FIELD = (County_City, SET FIRST = State_City.set [\u201cPenn\u201d])\r\n<\/pre>\n<p>which would return \u201cAdams Abbottstown\u201d which is the first entry in the Penn&#8217;s.<\/p>\n<p>Or on two indices,<\/p>\n<pre>\/FIELD = (County_City, SET FIRST = State_City.set [\u201cPenn\u201d, York\u201d])\r\n<\/pre>\n<p>which is \u201cDallastown\u201d<\/p>\n<p><strong><br \/>\nConclusion<\/strong><\/p>\n<p>We have shown that sets &#8212; whether used for random access or to provide table searching &#8212; can be used in a wide variety of applications. Independent of access, sets provide a standard reference and give structure to enterprise data.<\/p>\n<p>It should be clear that the number of indices can be extended to any depth. If your application does a lot of searching, you might consider the use of set files because of their simplicity and speed.<\/p>\n<p>There are more options available for searching sets which await my next article. If you are interested in another\u00a0manifestation, see <a href=\"http:\/\/www.iri.com\/solutions\/business-intelligence\/embedded-bi\/slowly-changing-dimensions\" target=\"_blank\" rel=\"noopener\">this<\/a> page on Slowly Changing Dimensions.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>This article looks at sets from an informational processing perspective; what they are; how they are\u00a0constructed; and, distinct ways in which data can be drawn from sets within IRI software products\u00a0using the SortCL data definition and processing program; i.e., in CoSort and NextForm to replace values, FieldShield for pseudonomization, RowGen for random value selection, and<\/p>\n<div><a class=\"btn-filled btn\" href=\"https:\/\/beta.iri.com\/blog\/data-transformation2\/drawing-values-from-set-files\/\" title=\"Drawing Values from Set Files\">Read More<\/a><\/div>\n","protected":false},"author":8,"featured_media":11762,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"footnotes":""},"categories":[8,31,363,1,776,216,29],"tags":[672,328,5,786,787,784,9,788,851,76,49,785,87,68],"class_list":["post-7583","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-data-protection","category-data-migration","category-data-quality","category-data-transformation2","category-etl","category-master-data-metadata-management","category-test-data","tag-ascii","tag-data-management","tag-data-transformation","tag-defining-terms","tag-discrete-value","tag-drawing-values","tag-fieldshield","tag-iri-software","tag-master-data-metadata-management","tag-nextform","tag-rowgen","tag-set-files","tag-slowly-changing-dimensions","tag-sortcl"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v23.3 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Drawing Values from Set Files - IRI<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/beta.iri.com\/blog\/data-transformation2\/drawing-values-from-set-files\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Drawing Values from Set Files - IRI\" \/>\n<meta property=\"og:description\" content=\"This article looks at sets from an informational processing perspective; what they are; how they are\u00a0constructed; and, distinct ways in which data can be drawn from sets within IRI software products\u00a0using the SortCL data definition and processing program; i.e., in CoSort and NextForm to replace values, FieldShield for pseudonomization, RowGen for random value selection, andRead More\" \/>\n<meta property=\"og:url\" content=\"https:\/\/beta.iri.com\/blog\/data-transformation2\/drawing-values-from-set-files\/\" \/>\n<meta property=\"og:site_name\" content=\"IRI\" \/>\n<meta property=\"article:published_time\" content=\"2015-07-14T12:23:47+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2020-05-29T13:32:31+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2015\/07\/pennsylvania-map.gif\" \/>\n\t<meta property=\"og:image:width\" content=\"750\" \/>\n\t<meta property=\"og:image:height\" content=\"393\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/gif\" \/>\n<meta name=\"author\" content=\"Paul Friedland\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Paul Friedland\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"7 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/beta.iri.com\/blog\/data-transformation2\/drawing-values-from-set-files\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/data-transformation2\/drawing-values-from-set-files\/\"},\"author\":{\"name\":\"Paul Friedland\",\"@id\":\"https:\/\/beta.iri.com\/blog\/#\/schema\/person\/1127ccf5631e8b00cb23974bd1882ea9\"},\"headline\":\"Drawing Values from Set Files\",\"datePublished\":\"2015-07-14T12:23:47+00:00\",\"dateModified\":\"2020-05-29T13:32:31+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/data-transformation2\/drawing-values-from-set-files\/\"},\"wordCount\":1261,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/data-transformation2\/drawing-values-from-set-files\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2015\/07\/pennsylvania-map.gif\",\"keywords\":[\"ASCII\",\"data management\",\"data transformation\",\"defining terms\",\"discrete value\",\"drawing values\",\"FieldShield\",\"IRI software\",\"MDM\",\"NextForm\",\"RowGen\",\"set files\",\"slowly changing dimensions\",\"SortCL\"],\"articleSection\":[\"Data Masking\/Protection\",\"Data Migration\",\"Data Quality (DQ&#041;\",\"Data Transformation\",\"ETL\",\"MDM\",\"Test Data\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/beta.iri.com\/blog\/data-transformation2\/drawing-values-from-set-files\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/beta.iri.com\/blog\/data-transformation2\/drawing-values-from-set-files\/\",\"url\":\"https:\/\/beta.iri.com\/blog\/data-transformation2\/drawing-values-from-set-files\/\",\"name\":\"Drawing Values from Set Files - IRI\",\"isPartOf\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/data-transformation2\/drawing-values-from-set-files\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/data-transformation2\/drawing-values-from-set-files\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2015\/07\/pennsylvania-map.gif\",\"datePublished\":\"2015-07-14T12:23:47+00:00\",\"dateModified\":\"2020-05-29T13:32:31+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/data-transformation2\/drawing-values-from-set-files\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/beta.iri.com\/blog\/data-transformation2\/drawing-values-from-set-files\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/beta.iri.com\/blog\/data-transformation2\/drawing-values-from-set-files\/#primaryimage\",\"url\":\"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2015\/07\/pennsylvania-map.gif\",\"contentUrl\":\"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2015\/07\/pennsylvania-map.gif\",\"width\":750,\"height\":393},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/beta.iri.com\/blog\/data-transformation2\/drawing-values-from-set-files\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/beta.iri.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Drawing Values from Set Files\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/beta.iri.com\/blog\/#website\",\"url\":\"https:\/\/beta.iri.com\/blog\/\",\"name\":\"IRI\",\"description\":\"Total Data Management Blog\",\"publisher\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/beta.iri.com\/blog\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/beta.iri.com\/blog\/#organization\",\"name\":\"IRI\",\"url\":\"https:\/\/beta.iri.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/beta.iri.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png\",\"contentUrl\":\"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png\",\"width\":750,\"height\":206,\"caption\":\"IRI\"},\"image\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/beta.iri.com\/blog\/#\/schema\/person\/1127ccf5631e8b00cb23974bd1882ea9\",\"name\":\"Paul Friedland\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/beta.iri.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/5d48eea2eec2c3bcb24db5080f6ab3c5?s=96&d=blank&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/5d48eea2eec2c3bcb24db5080f6ab3c5?s=96&d=blank&r=g\",\"caption\":\"Paul Friedland\"},\"url\":\"https:\/\/beta.iri.com\/blog\/author\/paulf\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Drawing Values from Set Files - IRI","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/beta.iri.com\/blog\/data-transformation2\/drawing-values-from-set-files\/","og_locale":"en_US","og_type":"article","og_title":"Drawing Values from Set Files - IRI","og_description":"This article looks at sets from an informational processing perspective; what they are; how they are\u00a0constructed; and, distinct ways in which data can be drawn from sets within IRI software products\u00a0using the SortCL data definition and processing program; i.e., in CoSort and NextForm to replace values, FieldShield for pseudonomization, RowGen for random value selection, andRead More","og_url":"https:\/\/beta.iri.com\/blog\/data-transformation2\/drawing-values-from-set-files\/","og_site_name":"IRI","article_published_time":"2015-07-14T12:23:47+00:00","article_modified_time":"2020-05-29T13:32:31+00:00","og_image":[{"width":750,"height":393,"url":"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2015\/07\/pennsylvania-map.gif","type":"image\/gif"}],"author":"Paul Friedland","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Paul Friedland","Est. reading time":"7 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/beta.iri.com\/blog\/data-transformation2\/drawing-values-from-set-files\/#article","isPartOf":{"@id":"https:\/\/beta.iri.com\/blog\/data-transformation2\/drawing-values-from-set-files\/"},"author":{"name":"Paul Friedland","@id":"https:\/\/beta.iri.com\/blog\/#\/schema\/person\/1127ccf5631e8b00cb23974bd1882ea9"},"headline":"Drawing Values from Set Files","datePublished":"2015-07-14T12:23:47+00:00","dateModified":"2020-05-29T13:32:31+00:00","mainEntityOfPage":{"@id":"https:\/\/beta.iri.com\/blog\/data-transformation2\/drawing-values-from-set-files\/"},"wordCount":1261,"commentCount":0,"publisher":{"@id":"https:\/\/beta.iri.com\/blog\/#organization"},"image":{"@id":"https:\/\/beta.iri.com\/blog\/data-transformation2\/drawing-values-from-set-files\/#primaryimage"},"thumbnailUrl":"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2015\/07\/pennsylvania-map.gif","keywords":["ASCII","data management","data transformation","defining terms","discrete value","drawing values","FieldShield","IRI software","MDM","NextForm","RowGen","set files","slowly changing dimensions","SortCL"],"articleSection":["Data Masking\/Protection","Data Migration","Data Quality (DQ&#041;","Data Transformation","ETL","MDM","Test Data"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/beta.iri.com\/blog\/data-transformation2\/drawing-values-from-set-files\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/beta.iri.com\/blog\/data-transformation2\/drawing-values-from-set-files\/","url":"https:\/\/beta.iri.com\/blog\/data-transformation2\/drawing-values-from-set-files\/","name":"Drawing Values from Set Files - IRI","isPartOf":{"@id":"https:\/\/beta.iri.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/beta.iri.com\/blog\/data-transformation2\/drawing-values-from-set-files\/#primaryimage"},"image":{"@id":"https:\/\/beta.iri.com\/blog\/data-transformation2\/drawing-values-from-set-files\/#primaryimage"},"thumbnailUrl":"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2015\/07\/pennsylvania-map.gif","datePublished":"2015-07-14T12:23:47+00:00","dateModified":"2020-05-29T13:32:31+00:00","breadcrumb":{"@id":"https:\/\/beta.iri.com\/blog\/data-transformation2\/drawing-values-from-set-files\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/beta.iri.com\/blog\/data-transformation2\/drawing-values-from-set-files\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/beta.iri.com\/blog\/data-transformation2\/drawing-values-from-set-files\/#primaryimage","url":"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2015\/07\/pennsylvania-map.gif","contentUrl":"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2015\/07\/pennsylvania-map.gif","width":750,"height":393},{"@type":"BreadcrumbList","@id":"https:\/\/beta.iri.com\/blog\/data-transformation2\/drawing-values-from-set-files\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/beta.iri.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Drawing Values from Set Files"}]},{"@type":"WebSite","@id":"https:\/\/beta.iri.com\/blog\/#website","url":"https:\/\/beta.iri.com\/blog\/","name":"IRI","description":"Total Data Management Blog","publisher":{"@id":"https:\/\/beta.iri.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/beta.iri.com\/blog\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/beta.iri.com\/blog\/#organization","name":"IRI","url":"https:\/\/beta.iri.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/beta.iri.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png","contentUrl":"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png","width":750,"height":206,"caption":"IRI"},"image":{"@id":"https:\/\/beta.iri.com\/blog\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/beta.iri.com\/blog\/#\/schema\/person\/1127ccf5631e8b00cb23974bd1882ea9","name":"Paul Friedland","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/beta.iri.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/5d48eea2eec2c3bcb24db5080f6ab3c5?s=96&d=blank&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/5d48eea2eec2c3bcb24db5080f6ab3c5?s=96&d=blank&r=g","caption":"Paul Friedland"},"url":"https:\/\/beta.iri.com\/blog\/author\/paulf\/"}]}},"jetpack_featured_media_url":"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2015\/07\/pennsylvania-map.gif","_links":{"self":[{"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/posts\/7583"}],"collection":[{"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/users\/8"}],"replies":[{"embeddable":true,"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/comments?post=7583"}],"version-history":[{"count":31,"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/posts\/7583\/revisions"}],"predecessor-version":[{"id":13732,"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/posts\/7583\/revisions\/13732"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/media\/11762"}],"wp:attachment":[{"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/media?parent=7583"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/categories?post=7583"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/tags?post=7583"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}