{"id":4644,"date":"2014-01-16T13:13:45","date_gmt":"2014-01-16T18:13:45","guid":{"rendered":"http:\/\/www.iri.com\/blog\/?p=4644"},"modified":"2017-11-06T08:25:12","modified_gmt":"2017-11-06T13:25:12","slug":"hadoop-alternative","status":"publish","type":"post","link":"https:\/\/beta.iri.com\/blog\/big-data-2\/hadoop-alternative\/","title":{"rendered":"Before or Instead of Hadoop, Try This …"},"content":{"rendered":"

Before Big Data became a buzz word and Gartner hype cycle fodder (never mind falling into the ‘Trough of Disillusionment<\/a>‘), companies like IRI were handling it. Huge flat files — such as telco CDR and web logs, mainframe datasets and VLDB table extracts — have been the inputs to CoSort’s massive data sorting and transformation engine on multi-threaded servers since the 90’s.<\/p>\n

That’s still the case today, and even as we support Hadoop<\/a> in CoSort’s successor Voracity<\/a> platform,\u00a0we pose the question of whether Hadoop is necessarily the best way to process very large data volumes in the first place. That’s especially so when CoSort is also directly addressing\u00a0big data in relational, non-relational, semi-structured and unstructured<\/a> sources. \u00a0And it’s also questionable given that without Hadoop, CoSort’s SortCL<\/a> program can at once:<\/p>\n

    \n
  1. package<\/a>\u00a0(integrate, migrate, and reformat) big data, through the consolidation of multiple transforms<\/li>\n
  2. protect<\/a>\u00a0big data at the field level, through a choice of data masking functions or big test data generation
    \nand,<\/li>\n
  3. provision<\/a>\u00a0big data for DW \/ ODS, federation, and BI targets (with both embedded reporting and hand-offs to analytic and advanced visualization tools)<\/li>\n<\/ol>\n

    Again, all this happens in one Eclipse-supported<\/a> 4GL job script, and in one I\/O pass … within your existing hardware, file system, and database infrastructure:<\/p>\n

    \"2014<\/a>
    2014 IRI Product Stack & Data Flow<\/figcaption><\/figure>\n

    There is still a considerable skills gap<\/a> around Hadoop, MapReduce, NoSQL, and other hardware-driven technologies like in-memory DBs and ELT appliances. They require an “army of geeks” to set up, and cost a small fortune in servers and software to keep running and producing actionable results. Compare this to the power and simplicity of SortCL, the popularity of Eclipse, and the cost of both (hint: 5 figures + free).<\/p>\n

    So know your alternatives and their risks as you assess your informational goals and processing needs. Can you stay in the IT fabric you already have, exploit both internal and external data, and identify a clear business benefit for the cost? You may\u00a0want to consider starting with something that’s more proven, affordable, and non-disruptive before converting\u00a0to Hadoop.<\/p>\n

    See our “When to Use Hadoop<\/a>” article for more information.<\/p>\n

    <\/div>\n","protected":false},"excerpt":{"rendered":"

    Before Big Data became a buzz word and Gartner hype cycle fodder (never mind falling into the ‘Trough of Disillusionment‘), companies like IRI were handling it. Huge flat files — such as telco CDR and web logs, mainframe datasets and VLDB table extracts — have been the inputs to CoSort’s massive data sorting and transformation<\/p>\n

    Read More<\/a><\/div>\n","protected":false},"author":3,"featured_media":11649,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_exactmetrics_skip_tracking":false,"_exactmetrics_sitenote_active":false,"_exactmetrics_sitenote_note":"","_exactmetrics_sitenote_category":0,"footnotes":""},"categories":[108],"tags":[25,44,106,81,353,368,369,68,143],"class_list":["post-4644","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-big-data-2","tag-big-data","tag-cosort","tag-elt","tag-hadoop","tag-hadoop-alternative","tag-mapreduce","tag-nosql","tag-sortcl","tag-unstructured-data"],"acf":[],"yoast_head":"\nBefore or Instead of Hadoop, Try This ... - IRI<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/beta.iri.com\/blog\/big-data-2\/hadoop-alternative\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Before or Instead of Hadoop, Try This ... - IRI\" \/>\n<meta property=\"og:description\" content=\"Before Big Data became a buzz word and Gartner hype cycle fodder (never mind falling into the ‘Trough of Disillusionment‘), companies like IRI were handling it. Huge flat files — such as telco CDR and web logs, mainframe datasets and VLDB table extracts — have been the inputs to CoSort’s massive data sorting and transformationRead More\" \/>\n<meta property=\"og:url\" content=\"https:\/\/beta.iri.com\/blog\/big-data-2\/hadoop-alternative\/\" \/>\n<meta property=\"og:site_name\" content=\"IRI\" \/>\n<meta property=\"article:published_time\" content=\"2014-01-16T18:13:45+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2017-11-06T13:25:12+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2014\/01\/Hadoop-big-data-risk-die-1.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"225\" \/>\n\t<meta property=\"og:image:height\" content=\"222\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"David Friedland\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"David Friedland\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/beta.iri.com\/blog\/big-data-2\/hadoop-alternative\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/big-data-2\/hadoop-alternative\/\"},\"author\":{\"name\":\"David Friedland\",\"@id\":\"https:\/\/beta.iri.com\/blog\/#\/schema\/person\/cdb89f0c0a9c88810b8516d4b140734a\"},\"headline\":\"Before or Instead of Hadoop, Try This …\",\"datePublished\":\"2014-01-16T18:13:45+00:00\",\"dateModified\":\"2017-11-06T13:25:12+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/big-data-2\/hadoop-alternative\/\"},\"wordCount\":375,\"commentCount\":0,\"publisher\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/big-data-2\/hadoop-alternative\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2014\/01\/Hadoop-big-data-risk-die-1.jpg\",\"keywords\":[\"big data\",\"CoSort\",\"ELT\",\"hadoop\",\"hadoop alternative\",\"MapReduce\",\"NoSQL\",\"SortCL\",\"unstructured data\"],\"articleSection\":[\"Big Data\"],\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/beta.iri.com\/blog\/big-data-2\/hadoop-alternative\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/beta.iri.com\/blog\/big-data-2\/hadoop-alternative\/\",\"url\":\"https:\/\/beta.iri.com\/blog\/big-data-2\/hadoop-alternative\/\",\"name\":\"Before or Instead of Hadoop, Try This ... - IRI\",\"isPartOf\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/big-data-2\/hadoop-alternative\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/big-data-2\/hadoop-alternative\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2014\/01\/Hadoop-big-data-risk-die-1.jpg\",\"datePublished\":\"2014-01-16T18:13:45+00:00\",\"dateModified\":\"2017-11-06T13:25:12+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/big-data-2\/hadoop-alternative\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/beta.iri.com\/blog\/big-data-2\/hadoop-alternative\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/beta.iri.com\/blog\/big-data-2\/hadoop-alternative\/#primaryimage\",\"url\":\"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2014\/01\/Hadoop-big-data-risk-die-1.jpg\",\"contentUrl\":\"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2014\/01\/Hadoop-big-data-risk-die-1.jpg\",\"width\":225,\"height\":222,\"caption\":\"hadopp big risk die\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/beta.iri.com\/blog\/big-data-2\/hadoop-alternative\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/beta.iri.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Before or Instead of Hadoop, Try This …\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/beta.iri.com\/blog\/#website\",\"url\":\"https:\/\/beta.iri.com\/blog\/\",\"name\":\"IRI\",\"description\":\"Total Data Management Blog\",\"publisher\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/beta.iri.com\/blog\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/beta.iri.com\/blog\/#organization\",\"name\":\"IRI\",\"url\":\"https:\/\/beta.iri.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/beta.iri.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png\",\"contentUrl\":\"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png\",\"width\":750,\"height\":206,\"caption\":\"IRI\"},\"image\":{\"@id\":\"https:\/\/beta.iri.com\/blog\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/beta.iri.com\/blog\/#\/schema\/person\/cdb89f0c0a9c88810b8516d4b140734a\",\"name\":\"David Friedland\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/beta.iri.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/995ea08bc7d036da625671cb48a636eb?s=96&d=blank&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/995ea08bc7d036da625671cb48a636eb?s=96&d=blank&r=g\",\"caption\":\"David Friedland\"},\"url\":\"https:\/\/beta.iri.com\/blog\/author\/davidf\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Before or Instead of Hadoop, Try This ... - IRI","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/beta.iri.com\/blog\/big-data-2\/hadoop-alternative\/","og_locale":"en_US","og_type":"article","og_title":"Before or Instead of Hadoop, Try This ... - IRI","og_description":"Before Big Data became a buzz word and Gartner hype cycle fodder (never mind falling into the ‘Trough of Disillusionment‘), companies like IRI were handling it. Huge flat files — such as telco CDR and web logs, mainframe datasets and VLDB table extracts — have been the inputs to CoSort’s massive data sorting and transformationRead More","og_url":"https:\/\/beta.iri.com\/blog\/big-data-2\/hadoop-alternative\/","og_site_name":"IRI","article_published_time":"2014-01-16T18:13:45+00:00","article_modified_time":"2017-11-06T13:25:12+00:00","og_image":[{"width":225,"height":222,"url":"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2014\/01\/Hadoop-big-data-risk-die-1.jpg","type":"image\/jpeg"}],"author":"David Friedland","twitter_card":"summary_large_image","twitter_misc":{"Written by":"David Friedland","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/beta.iri.com\/blog\/big-data-2\/hadoop-alternative\/#article","isPartOf":{"@id":"https:\/\/beta.iri.com\/blog\/big-data-2\/hadoop-alternative\/"},"author":{"name":"David Friedland","@id":"https:\/\/beta.iri.com\/blog\/#\/schema\/person\/cdb89f0c0a9c88810b8516d4b140734a"},"headline":"Before or Instead of Hadoop, Try This …","datePublished":"2014-01-16T18:13:45+00:00","dateModified":"2017-11-06T13:25:12+00:00","mainEntityOfPage":{"@id":"https:\/\/beta.iri.com\/blog\/big-data-2\/hadoop-alternative\/"},"wordCount":375,"commentCount":0,"publisher":{"@id":"https:\/\/beta.iri.com\/blog\/#organization"},"image":{"@id":"https:\/\/beta.iri.com\/blog\/big-data-2\/hadoop-alternative\/#primaryimage"},"thumbnailUrl":"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2014\/01\/Hadoop-big-data-risk-die-1.jpg","keywords":["big data","CoSort","ELT","hadoop","hadoop alternative","MapReduce","NoSQL","SortCL","unstructured data"],"articleSection":["Big Data"],"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/beta.iri.com\/blog\/big-data-2\/hadoop-alternative\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/beta.iri.com\/blog\/big-data-2\/hadoop-alternative\/","url":"https:\/\/beta.iri.com\/blog\/big-data-2\/hadoop-alternative\/","name":"Before or Instead of Hadoop, Try This ... - IRI","isPartOf":{"@id":"https:\/\/beta.iri.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/beta.iri.com\/blog\/big-data-2\/hadoop-alternative\/#primaryimage"},"image":{"@id":"https:\/\/beta.iri.com\/blog\/big-data-2\/hadoop-alternative\/#primaryimage"},"thumbnailUrl":"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2014\/01\/Hadoop-big-data-risk-die-1.jpg","datePublished":"2014-01-16T18:13:45+00:00","dateModified":"2017-11-06T13:25:12+00:00","breadcrumb":{"@id":"https:\/\/beta.iri.com\/blog\/big-data-2\/hadoop-alternative\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/beta.iri.com\/blog\/big-data-2\/hadoop-alternative\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/beta.iri.com\/blog\/big-data-2\/hadoop-alternative\/#primaryimage","url":"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2014\/01\/Hadoop-big-data-risk-die-1.jpg","contentUrl":"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2014\/01\/Hadoop-big-data-risk-die-1.jpg","width":225,"height":222,"caption":"hadopp big risk die"},{"@type":"BreadcrumbList","@id":"https:\/\/beta.iri.com\/blog\/big-data-2\/hadoop-alternative\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/beta.iri.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Before or Instead of Hadoop, Try This …"}]},{"@type":"WebSite","@id":"https:\/\/beta.iri.com\/blog\/#website","url":"https:\/\/beta.iri.com\/blog\/","name":"IRI","description":"Total Data Management Blog","publisher":{"@id":"https:\/\/beta.iri.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/beta.iri.com\/blog\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/beta.iri.com\/blog\/#organization","name":"IRI","url":"https:\/\/beta.iri.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/beta.iri.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png","contentUrl":"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2019\/02\/iri-logo-total-data-management-small-1.png","width":750,"height":206,"caption":"IRI"},"image":{"@id":"https:\/\/beta.iri.com\/blog\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/beta.iri.com\/blog\/#\/schema\/person\/cdb89f0c0a9c88810b8516d4b140734a","name":"David Friedland","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/beta.iri.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/995ea08bc7d036da625671cb48a636eb?s=96&d=blank&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/995ea08bc7d036da625671cb48a636eb?s=96&d=blank&r=g","caption":"David Friedland"},"url":"https:\/\/beta.iri.com\/blog\/author\/davidf\/"}]}},"jetpack_featured_media_url":"https:\/\/beta.iri.com\/blog\/wp-content\/uploads\/2014\/01\/Hadoop-big-data-risk-die-1.jpg","_links":{"self":[{"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/posts\/4644"}],"collection":[{"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/comments?post=4644"}],"version-history":[{"count":37,"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/posts\/4644\/revisions"}],"predecessor-version":[{"id":11650,"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/posts\/4644\/revisions\/11650"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/media\/11649"}],"wp:attachment":[{"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/media?parent=4644"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/categories?post=4644"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/beta.iri.com\/blog\/wp-json\/wp\/v2\/tags?post=4644"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}