{"id":15269,"date":"2021-11-30T16:14:20","date_gmt":"2021-11-30T21:14:20","guid":{"rendered":"http:\/\/www.iri.com\/blog\/?p=15269"},"modified":"2021-11-30T16:14:20","modified_gmt":"2021-11-30T21:14:20","slug":"find-mask-pii-in-bigtable-cosmos-and-dynamo","status":"publish","type":"post","link":"https:\/\/beta.iri.com\/blog\/data-protection\/find-mask-pii-in-bigtable-cosmos-and-dynamo\/","title":{"rendered":"Find & Mask PII in BigTable, Cosmos and Dynamo NoSQL DBs"},"content":{"rendered":"

Abstract: <\/span>This article covers the use of the <\/span><\/i>IRI DarkShield API<\/span><\/i><\/a> for automatically locating and de-identifying PII or other sensitive data in the three major cloud provider NoSQL databases — Google BigTable, MS CosmosDB in Azure, and Amazon DynamoDB. Prior articles in this blog cover how DarkShield wizards in IRI Workbench find and mask data in other popular NoSQL DBs, including <\/span><\/i>Cassandra, Elasticsearch and MongoDB<\/span><\/i><\/a>.<\/span>1<\/sup><\/a><\/span><\/span>\u00a0A subsequent article covers CouchDB, Redis and Solr.<\/span><\/i><\/p>\n

What is NoSQL?<\/b><\/h6>\n

NoSQL typically stands for \u201cnot only SQL\u201d although others may say it stands for \u201cnon SQL\u201d. NoSQL was introduced to provide an alternative to relational databases that at the time, were the dominant force in the industry.\u00a0<\/span><\/p>\n

Because NoSQL databases are non-tabular, data is stored differently compared to SQL databases. There are actually various types of NoSQL databases based on their data model. These data models include documents, key-value pairs, wide-column, and graphs.<\/span><\/p>\n

\"\"<\/p>\n

The Strength of NoSQL Databases<\/b><\/h6>\n

According to CloudGuru.com, relational databases have \u201c<\/span>inflexible schemas and notoriously difficult horizontal scaling [which means] they don\u2019t always fit well in a highly scalable and geographically distributed infrastructure stack<\/span><\/i>\u201d<\/span>2<\/sup><\/a><\/span><\/span>.\u00a0<\/span><\/p>\n

In comparison, the flexibility of the NoSQL document-model makes it easier to change data. NoSQL databases are also easier to scale horizontally, and usually the cloud providers handle the operational overhead of managing infrastructure.<\/span><\/p>\n

\"\"<\/p>\n

To know when to choose NoSQL over relational databases there are generally a few factors to consider for decision makers. According to MongoDB the drivers are: \u201c<\/span>fast-paced Agile development, storage of structured and semi-structured data, huge volumes of data, requirements for scale-out architecture, modern application paradigms like microservices and real-time streaming<\/span><\/i>\u201d<\/span>3<\/sup><\/a><\/span><\/span>.<\/span><\/p>\n

NoSQL DB Security Concerns<\/b><\/h6>\n

As with traditional relational (SQL) databases, NoSQL DBs have similar security issues, but also some unique risks. According to the International Journal of Digital Society, NoSQL vulnerabilities include: \u201c<\/span>insufficient or ineffective input validation, errors in the application level permissions handling, weak authentication, insecure communication, illegal access to unencrypted data, etc. are some of the vulnerabilities applicable for NoSQL<\/span><\/i>\u201d<\/span><\/span>4<\/sup><\/a><\/span><\/span>.<\/span><\/p>\n

Like SQL injections, NoSQL injections are also possible when input validation is not handled properly. Because NoSQL databases do not have a common query language, queries are written in the programming language (PHP, JavaScript, Python, etc) of the application connected to the database. This means NoSQL injections can result in commands being executed not only in the database, but also in the application itself.\u00a0<\/span><\/p>\n

\"\"<\/p>\n

There is a long list of endpoint security practices for NoSQL DBs. But even with them, would-be assailants still manage to punch holes in those defenses. Companies must thus evolve to harden the security profile of these collections with another level of protection.\u00a0\u00a0<\/span><\/p>\n

\"\"<\/p>\n

This is where <\/span>IRI DarkShield<\/span><\/a> comes in. As a data-centric, or \u201cstartpoint security\u201d solution, DarkShield masking provides another important layer of data protection atop the end-point measures deployed by cloud database service providers.<\/span><\/p>\n

About IRI DarkShield<\/b><\/h6>\n

IRI DarkShield is a data masking tool for finding and de-identifying sensitive data in semi-structured and unstructured files and databases. DarkShield is one of three core data masking products in the <\/span>IRI Data Protector Suite<\/span><\/a> which leverage graphical data classification, searching, and masking job design models in the <\/span>IRI Workbench<\/span><\/a> IDE, built on Eclipse.<\/span><\/p>\n

\"\"<\/p>\n

As of DarkShield Version 4, however, two powerful Remote Procedure Call (RPC) Application Programming Interface (API) versions are also provided: the \u201cBase\u201d DarkShield API and the DarkShield-Files API. The DarkShield APIs extend the use of DarkShield functionality outside of Workbench and leverage a plugin on top of an IRI Web Services platform named Plankton.<\/span><\/p>\n

\"\"<\/p>\n

To find and protect sensitive data in a wide range of sources, the DarkShield APIs use specified search matchers and masking rules that follow business rules. For more information on creating search matchers and masking rules, please refer to this <\/span>article<\/span><\/a>.<\/span><\/p>\n

The \u201cBase\u201d DarkShield API is used to search and mask unstructured text outside the context of files. Alternatively, the DarkShield-Files API provides the ability to search and mask PII in files.\u00a0<\/span><\/p>\n

With the assistance of the DarkShield-Files API, semi-structured and unstructured data like plain text files, csv\/tsv, word documents, excel, pdf, json, xml, parquet, jpeg, and png images can be searched and masked.<\/span><\/p>\n

AWS DynamoDB, Azure CosmosDB, Google BigTable and the DarkShield API<\/b><\/h6>\n

The companies reigning over cloud services for NoSQL databases are Amazon AWS with DynamoDB, Microsoft\u2019s Azure CosmosDB, and Google\u2019s Cloud BigTable. The focus of this article is on these three well known service providers and how the DarkShield-Files API can be leveraged to search and mask inside their NoSQL databases located in the cloud.<\/span><\/p>\n

\"\"<\/p>\n

For those unfamiliar with connecting and querying NoSQL databases programmatically, not to worry. AWS, Azure, and Google Cloud are not only known for providing high quality service but also provide copious amounts of documentation on how to access their database content using Software Development toolkits (SDK) supported in various programming languages.<\/span><\/p>\n

The DarkShield-File API demos currently uploaded to GitHub are written in the Python language; as such, those projects use client libraries for Python. However, other calling languages, like Java, can be used.<\/span><\/p>\n

These calling programs, or \u201cglue code\u201d to the API, is where these procedures can be defined. See below for the links to the DarkShield-Files API demos:<\/span><\/p>\n