{"id":14388,"date":"2021-06-17T17:12:44","date_gmt":"2021-06-17T21:12:44","guid":{"rendered":"http:\/\/www.iri.com\/blog\/?p=14388"},"modified":"2021-06-18T12:11:56","modified_gmt":"2021-06-18T16:11:56","slug":"shifting-dates-preserving-intervals","status":"publish","type":"post","link":"https:\/\/beta.iri.com\/blog\/data-protection\/shifting-dates-preserving-intervals\/","title":{"rendered":"Shifting Dates While Preserving Intervals"},"content":{"rendered":"

One of the data masking requirements for IRI FieldShield<\/a> that we see in PHI anonymization<\/a> use cases involves the blurring of dates at the row level, instead of the column level while providing a configurable option to retain the interval between those dates. This is typically needed in clinical research or test data scenarios which must maintain specific intervals between hospital admittance or discharge, or treatment start, and end, dates.<\/p>\n

Let\u2019s say for example, that a table or file in production contains a \u2018StartDate\u2019<\/i> and \u2018EndDate\u2019<\/i> column. The values of both dates need to be anonymized but shifted by the same constant number of days, so that the duration of the event does not change. For each record, a new random number within the desired range can be used.<\/p>\n

This image illustrates the requirement:<\/p>\n

\"\"<\/a><\/p>\n

with the procedure expected to be as follows:<\/p>\n

    \n
  1. User selects the input data source (table or file)<\/li>\n
  2. User selects the columns which define the beginning and ending of the interval<\/li>\n
  3. User provides the range of the shift factor (e.g. -10 to +10)<\/li>\n
  4. Software chooses random number within the shift<\/li>\n
  5. Each row has the beginning and ending values shifted by the random amount, preserving the interval<\/li>\n<\/ol>\n

    FieldShield scripting handles this with a separately generated random number offset within a given range. One nice feature of this approach is that with our date and time based math functions, the offset can be defined as seconds, minutes, hours, days, weeks, months, or years. The offset can be applied to any time based data type, such as a date, time, or timestamp.<\/p>\n

    A FieldShield job script to perform the row-based shifting and masking of these dates follows:<\/p>\n

    \"\"<\/a><\/p>\n

    In our example, the two dates in each row are shifted by the same random number of days, between plus and minus 10 days.<\/p>\n

    \"\"<\/a><\/p>\n

    Notice the virtual field defined in the virtual input record (\/INREC) section of the script, called RAND_SHIFT. For each row processed, it randomly generates a new date shifting value from ten fewer to ten greater days.<\/p>\n

    In the resulting target file, the new start and stop date values are adjusted by the RAND_SHIFT value in days by the \u201cchange_dt\u201d function to make sure a proper date is used; see this article<\/a>.<\/p>\n

    Here is the output data:<\/p>\n

    \"\"<\/a><\/p>\n

    This example processes a CSV file, but the same principle applies to a database table, or any other structured data source<\/a> supported by the FieldShield data processing engine, SortCL<\/a>. And of course, different masking functions<\/a> could be applied to other fields in the same job script; for example, pseudonymization<\/a> of the names, and\/or blurring<\/a> of the age values.<\/p>\n

    Below is the same job designed in the free graphical IDE for FieldShield, IRI Workbench<\/a>. The screenshot shows the diagram and outline views of the job script, and the tab-delimited output:<\/p>\n

    \"\"<\/a><\/p>\n

    If you have any questions about this article, or need help implementing a static or dynamic masking solution for privacy law compliance, test data management, or breach risk mitigation, please contact your IRI representative<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"

    One of the data masking requirements for IRI FieldShield that we see in PHI anonymization use cases involves the blurring of dates at the row level, instead of the column level while providing a configurable option to retain the interval between those dates. This is typically needed in clinical research or test data scenarios which<\/p>\n

    Read More<\/a><\/div>\n","protected":false},"author":2,"featured_media":14390,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"footnotes":""},"categories":[8,29],"tags":[14,13,520,789,850,1501,88],"class_list":["post-14388","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-data-protection","category-test-data","tag-data-masking","tag-data-protection-2","tag-iri-fieldshield","tag-iri-voracity","tag-iri-workbench","tag-shifting-dates","tag-test-data-2"],"acf":[],"yoast_head":"\nShifting Dates While Preserving Intervals - IRI<\/title>\n<meta name=\"description\" content=\"One of the data masking requirements for IRI FieldShield that we see in PHI anonymization use cases involves the blurring of dates at the row level\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/beta.iri.com\/blog\/data-protection\/shifting-dates-preserving-intervals\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Shifting Dates While Preserving Intervals - IRI\" \/>\n<meta property=\"og:description\" content=\"One of the data masking requirements for IRI FieldShield that we see in PHI anonymization use cases involves the blurring of dates at the row level\" \/>\n<meta property=\"og:url\" content=\"https:\/\/beta.iri.com\/blog\/data-protection\/shifting-dates-preserving-intervals\/\" \/>\n<meta property=\"og:site_name\" content=\"IRI\" \/>\n<meta property=\"article:published_time\" content=\"2021-06-17T21:12:44+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2021-06-18T16:11:56+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2021\/06\/shifting-dates-before-and-masked.png\" \/>\n\t<meta property=\"og:image:width\" content=\"537\" \/>\n\t<meta property=\"og:image:height\" content=\"221\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Don Purnhagen\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Don Purnhagen\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/beta.iri.com\/blog\/data-protection\/shifting-dates-preserving-intervals\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/data-protection\/shifting-dates-preserving-intervals\/\"},\"author\":{\"name\":\"Don Purnhagen\",\"@id\":\"https:\/\/beta.iri.com\/blog\/#\/schema\/person\/5f6325245947c8b8f0a0a76ef81580ec\"},\"headline\":\"Shifting Dates While Preserving Intervals\",\"datePublished\":\"2021-06-17T21:12:44+00:00\",\"dateModified\":\"2021-06-18T16:11:56+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/data-protection\/shifting-dates-preserving-intervals\/\"},\"wordCount\":509,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/data-protection\/shifting-dates-preserving-intervals\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2021\/06\/shifting-dates-before-and-masked.png\",\"keywords\":[\"data masking\",\"data protection\",\"IRI FieldShield\",\"IRI Voracity\",\"IRI Workbench\",\"shifting dates\",\"test data\"],\"articleSection\":[\"Data Masking\/Protection\",\"Test Data\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/beta.iri.com\/blog\/data-protection\/shifting-dates-preserving-intervals\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/beta.iri.com\/blog\/data-protection\/shifting-dates-preserving-intervals\/\",\"url\":\"https:\/\/beta.iri.com\/blog\/data-protection\/shifting-dates-preserving-intervals\/\",\"name\":\"Shifting Dates While Preserving Intervals - IRI\",\"isPartOf\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/data-protection\/shifting-dates-preserving-intervals\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/data-protection\/shifting-dates-preserving-intervals\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2021\/06\/shifting-dates-before-and-masked.png\",\"datePublished\":\"2021-06-17T21:12:44+00:00\",\"dateModified\":\"2021-06-18T16:11:56+00:00\",\"description\":\"One of the data masking requirements for IRI FieldShield that we see in PHI anonymization use cases involves the blurring of dates at the row level\",\"breadcrumb\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/data-protection\/shifting-dates-preserving-intervals\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/beta.iri.com\/blog\/data-protection\/shifting-dates-preserving-intervals\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/beta.iri.com\/blog\/data-protection\/shifting-dates-preserving-intervals\/#primaryimage\",\"url\":\"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2021\/06\/shifting-dates-before-and-masked.png\",\"contentUrl\":\"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2021\/06\/shifting-dates-before-and-masked.png\",\"width\":537,\"height\":221},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/beta.iri.com\/blog\/data-protection\/shifting-dates-preserving-intervals\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/beta.iri.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Shifting Dates While Preserving Intervals\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/beta.iri.com\/blog\/#website\",\"url\":\"https:\/\/beta.iri.com\/blog\/\",\"name\":\"IRI\",\"description\":\"Total Data Management Blog\",\"publisher\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/beta.iri.com\/blog\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/beta.iri.com\/blog\/#organization\",\"name\":\"IRI\",\"url\":\"https:\/\/beta.iri.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/beta.iri.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png\",\"contentUrl\":\"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png\",\"width\":750,\"height\":206,\"caption\":\"IRI\"},\"image\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/beta.iri.com\/blog\/#\/schema\/person\/5f6325245947c8b8f0a0a76ef81580ec\",\"name\":\"Don Purnhagen\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/beta.iri.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/39a152e4ffaf1f20e40d0c6e56a9a91d?s=96&d=blank&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/39a152e4ffaf1f20e40d0c6e56a9a91d?s=96&d=blank&r=g\",\"caption\":\"Don Purnhagen\"},\"url\":\"https:\/\/beta.iri.com\/blog\/author\/donp\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Shifting Dates While Preserving Intervals - IRI","description":"One of the data masking requirements for IRI FieldShield that we see in PHI anonymization use cases involves the blurring of dates at the row level","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/beta.iri.com\/blog\/data-protection\/shifting-dates-preserving-intervals\/","og_locale":"en_US","og_type":"article","og_title":"Shifting Dates While Preserving Intervals - IRI","og_description":"One of the data masking requirements for IRI FieldShield that we see in PHI anonymization use cases involves the blurring of dates at the row level","og_url":"https:\/\/beta.iri.com\/blog\/data-protection\/shifting-dates-preserving-intervals\/","og_site_name":"IRI","article_published_time":"2021-06-17T21:12:44+00:00","article_modified_time":"2021-06-18T16:11:56+00:00","og_image":[{"width":537,"height":221,"url":"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2021\/06\/shifting-dates-before-and-masked.png","type":"image\/png"}],"author":"Don Purnhagen","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Don Purnhagen","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/beta.iri.com\/blog\/data-protection\/shifting-dates-preserving-intervals\/#article","isPartOf":{"@id":"https:\/\/beta.iri.com\/blog\/data-protection\/shifting-dates-preserving-intervals\/"},"author":{"name":"Don Purnhagen","@id":"https:\/\/beta.iri.com\/blog\/#\/schema\/person\/5f6325245947c8b8f0a0a76ef81580ec"},"headline":"Shifting Dates While Preserving Intervals","datePublished":"2021-06-17T21:12:44+00:00","dateModified":"2021-06-18T16:11:56+00:00","mainEntityOfPage":{"@id":"https:\/\/beta.iri.com\/blog\/data-protection\/shifting-dates-preserving-intervals\/"},"wordCount":509,"commentCount":0,"publisher":{"@id":"https:\/\/beta.iri.com\/blog\/#organization"},"image":{"@id":"https:\/\/beta.iri.com\/blog\/data-protection\/shifting-dates-preserving-intervals\/#primaryimage"},"thumbnailUrl":"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2021\/06\/shifting-dates-before-and-masked.png","keywords":["data masking","data protection","IRI FieldShield","IRI Voracity","IRI Workbench","shifting dates","test data"],"articleSection":["Data Masking\/Protection","Test Data"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/beta.iri.com\/blog\/data-protection\/shifting-dates-preserving-intervals\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/beta.iri.com\/blog\/data-protection\/shifting-dates-preserving-intervals\/","url":"https:\/\/beta.iri.com\/blog\/data-protection\/shifting-dates-preserving-intervals\/","name":"Shifting Dates While Preserving Intervals - IRI","isPartOf":{"@id":"https:\/\/beta.iri.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/beta.iri.com\/blog\/data-protection\/shifting-dates-preserving-intervals\/#primaryimage"},"image":{"@id":"https:\/\/beta.iri.com\/blog\/data-protection\/shifting-dates-preserving-intervals\/#primaryimage"},"thumbnailUrl":"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2021\/06\/shifting-dates-before-and-masked.png","datePublished":"2021-06-17T21:12:44+00:00","dateModified":"2021-06-18T16:11:56+00:00","description":"One of the data masking requirements for IRI FieldShield that we see in PHI anonymization use cases involves the blurring of dates at the row level","breadcrumb":{"@id":"https:\/\/beta.iri.com\/blog\/data-protection\/shifting-dates-preserving-intervals\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/beta.iri.com\/blog\/data-protection\/shifting-dates-preserving-intervals\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/beta.iri.com\/blog\/data-protection\/shifting-dates-preserving-intervals\/#primaryimage","url":"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2021\/06\/shifting-dates-before-and-masked.png","contentUrl":"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2021\/06\/shifting-dates-before-and-masked.png","width":537,"height":221},{"@type":"BreadcrumbList","@id":"https:\/\/beta.iri.com\/blog\/data-protection\/shifting-dates-preserving-intervals\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/beta.iri.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Shifting Dates While Preserving Intervals"}]},{"@type":"WebSite","@id":"https:\/\/beta.iri.com\/blog\/#website","url":"https:\/\/beta.iri.com\/blog\/","name":"IRI","description":"Total Data Management Blog","publisher":{"@id":"https:\/\/beta.iri.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/beta.iri.com\/blog\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/beta.iri.com\/blog\/#organization","name":"IRI","url":"https:\/\/beta.iri.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/beta.iri.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png","contentUrl":"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png","width":750,"height":206,"caption":"IRI"},"image":{"@id":"https:\/\/beta.iri.com\/blog\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/beta.iri.com\/blog\/#\/schema\/person\/5f6325245947c8b8f0a0a76ef81580ec","name":"Don Purnhagen","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/beta.iri.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/39a152e4ffaf1f20e40d0c6e56a9a91d?s=96&d=blank&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/39a152e4ffaf1f20e40d0c6e56a9a91d?s=96&d=blank&r=g","caption":"Don Purnhagen"},"url":"https:\/\/beta.iri.com\/blog\/author\/donp\/"}]}},"jetpack_featured_media_url":"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2021\/06\/shifting-dates-before-and-masked.png","_links":{"self":[{"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/posts\/14388"}],"collection":[{"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/comments?post=14388"}],"version-history":[{"count":8,"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/posts\/14388\/revisions"}],"predecessor-version":[{"id":14399,"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/posts\/14388\/revisions\/14399"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/media\/14390"}],"wp:attachment":[{"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/media?parent=14388"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/categories?post=14388"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/tags?post=14388"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}