{"id":15373,"date":"2021-12-07T11:55:00","date_gmt":"2021-12-07T16:55:00","guid":{"rendered":"http:\/\/www.iri.com\/blog\/?p=15373"},"modified":"2021-12-07T11:55:00","modified_gmt":"2021-12-07T16:55:00","slug":"finding-and-masking-pii-in-couchbase-redis-and-solr","status":"publish","type":"post","link":"https:\/\/beta.iri.com\/blog\/data-protection\/finding-and-masking-pii-in-couchbase-redis-and-solr\/","title":{"rendered":"Finding and Masking PII in Couchbase, Redis and Solr NoSQL DBs"},"content":{"rendered":"

Abstract: <\/i><\/b>This article discusses the use of the <\/span><\/i>IRI DarkShield-Files API<\/span><\/i><\/a> for finding and masking PII and other sensitive data in Couchbase, Redis, and Solr NoSQL databases. The two previous articles on NoSQL database masking via DarkShield discussed <\/span><\/i>Cassandra, ElasticSearch, MongoDB<\/span><\/i><\/a>, and <\/span><\/i>BigTable, CosmosDB, and DynamoDB<\/span><\/i><\/a>.<\/span><\/i><\/p>\n

About Couchbase<\/b><\/h6>\n

Couchbase is a multi-model NoSQL database. Not to be confused with CouchDB, the Couchbase server was originally named Membase but later changed after the merger of CouchOne and Membase. In contrast to CouchDB, Couchbase is different in several ways including the lack of ability to attach files in documents and the implementation of a SQL-like language, referred to as N1QL (pronounced Nickel).<\/span><\/p>\n

In Couchbase, documents will have an ID (key) and an associated value where the actual application data resides. Couchbase uses Key Value operations to retrieve or mutate values where the key is present.<\/span>
\n<\/span><\/p>\n

\"\"Couchbase UI<\/span><\/i><\/p>\n

Couchbase Security Concerns<\/b><\/h6>\n

Couchbase (like many database service providers) supports features like authentication, authorization, and encryption in transit. Similarly, Couchbase servers are vulnerable to the same kind of attacks as the rest of the database service providers out there. These include injection attacks, cross site request forgery attacks, credential leaks, and code exploits.<\/span><\/p>\n

Furthermore, in regards to any sensitive data stored on Couchbase servers, Couchbase highly advises<\/span> \u201cto secure such data, encrypt all important data and index storage-locations, using transparent data encryption, provided by 3rd party on-disk encryption software-vendors; which denies data-access to anyone who either does not possess an appropriate encryption-key, or is otherwise non-compliant with the configured security policy\u201d.<\/span><\/i><\/span>1<\/sup><\/a><\/span><\/span><\/p>\n

Encrypting stored data is often referred to as encryption at rest and in an event where the server is compromised, the data residing in storage will remain secure. The purpose of DarkShield is to find and protect that data in a granular, consistent way.<\/span><\/p>\n

About Redis<\/b><\/h6>\n

Redis is an open source BSD license, in-memory NoSQL key-value data structure store. An in-memory database is when data is stored in a system\u2019s main memory, usually RAM.\u00a0<\/span><\/p>\n

By placing data on RAM the access to and processing of data is faster than disk I\/O. The disadvantage of in-memory storage is durability because data is stored in volatile memory by default.<\/span><\/p>\n

As a result of its speed, Redis has gained a large amount of popularity. According to Amazon\u00a0<\/span>\u201cBecause of its fast performance, Redis is a popular choice for caching, session management, gaming, leaderboards, real-time analytics, geospatial, ride-hailing, chat\/messaging, media streaming, and pub\/sub apps\u201d.<\/span><\/i><\/span>2<\/sup><\/a><\/span><\/span><\/p>\n

\"\"RedisInsight UI<\/span><\/i><\/p>\n

Redis Security Concerns<\/b><\/h6>\n

Redis has been designed to be used in a trusted environment by trusted clients and as such it is not recommended to <\/span>3<\/sup><\/a><\/span><\/span>\u201c<\/span>expose the Redis instance directly to the internet or, in general, to an environment where untrusted clients can directly access the Redis TCP port or UNIX socket\u201d.\u00a0<\/span><\/p>\n

Also, authentication is not enabled by default but can be turned on in the <\/span>redis.conf<\/span><\/i> file. The <\/span>redis.conf<\/span><\/i> file is also where the password is stored in clear text.\u00a0<\/span><\/p>\n

Redis does not allow for string escaping so under normal circumstances injection attacks should not occur. That said, currently the CONFIG command allows clients to change the working directory and the name of the dump file.\u00a0<\/span><\/p>\n

According to Redis, <\/span>\u201cthis allows clients to write RDB Redis files at random paths, that is a security issue that may easily lead to the ability to compromise the system and\/or run untrusted code as the same user as Redis is running.\u201d<\/span><\/i><\/p>\n

About Solr<\/b><\/h6>\n

Solr is an open-source search engine based on the Apache Lucene library and is written in Java. <\/span>Its features include full text search, hit highlighting, faceted, real-time indexing, dynamic clustering, database integration, and NoSQL features.\u00a0<\/span><\/p>\n

Although Solr is known for its search functionality, it can<\/span> also be used as a NoSQL document based database. Furthermore, Solr can ingest data from various sources such as JSON, XML, PDFs, and CSV files and index into JSON collections.<\/span><\/p>\n

\"\"Solr Admin user interface<\/span><\/i><\/p>\n

Solr Security Concerns<\/b><\/h6>\n

Solr security supports encryption with TLS to provide encryption in transit. As a result, traffic to and from Solr and between Solr nodes has measures that will prevent data leaks. Solr\u2019s framework also supports authentication, authorization, and audit logging of users to identify users and restrict access to resources.<\/span><\/p>\n

Currently, Solr does not support at rest encryption. Because at rest encryption is not supported by Solr, any individual who gains illegal access will be able to view sensitive data.\u00a0<\/span><\/p>\n

In addition, over the past several years many vulnerabilities using exploited code have been published. Those gaps could have ultimately resulted in the <\/span>hijacking of the entire server<\/span><\/a>.<\/span><\/p>\n

About IRI DarkShield<\/b><\/h6>\n

\"\"<\/p>\n

IRI DarkShield<\/span><\/a> is a data discovery and masking package for finding and de-identifying sensitive data in semi-structured and unstructured files and databases, including the NoSQL platforms in this article. DarkShield is one of three core data masking products in the <\/span>IRI Data Protector Suite<\/span><\/a> which leverage graphical data classification, searching, and masking job design models in the <\/span>IRI Workbench<\/span><\/a> IDE, built on Eclipse.<\/span><\/p>\n

Two powerful Remote Procedure Call (RPC) Application Programming Interface (API) versions are also provided: the\u00a0<\/span>\u201cBase\u201d DarkShield API<\/span><\/a> and the <\/span>DarkShield-Files API<\/span><\/a>. The DarkShield APIs extend the use of DarkShield functionality outside of Workbench and leverage a plugin on top of an IRI Web Services platform named Plankton.<\/span><\/p>\n

To find and protect sensitive data in a wide range of sources, the DarkShield APIs use specified search matchers and masking rules that follow business rules. For more information on creating search matchers and masking rules, please refer to this <\/span>article<\/span><\/a>.<\/span><\/p>\n

The \u201cBase\u201d DarkShield API is used to search and mask unstructured text outside the context of files. Alternatively, the DarkShield-Files API provides the ability to search and mask PII in files.\u00a0<\/span><\/p>\n

With the assistance of the DarkShield-Files API, semi-structured and unstructured data like plain text files, CSV\/TSV, HL7\/X12, Word, Excel, PDF, JSON, XML, Parquet, BMP, GIF, JPG, PNG, TIF, and DICOM images can be searched and masked.<\/span><\/p>\n

How the DarkShield-Files API Can Bolster Data Security<\/b><\/h6>\n

\"\"<\/p>\n

Most modern databases support in-transit encryption to protect data as it travels and at rest encryption for data inside the database. The issue with in transit and at rest encryption is that data encrypted by service providers is undone by the time the data is in front of the individual.\u00a0<\/span><\/p>\n

This means as long they have the proper access rights to see the data, neither encryption method will remain in place. This by itself is not an issue, but what happens if there are improper access controls or an employee\u2019s credentials are misused? This is where client-side encryption comes into play.<\/span><\/p>\n

Client-side encryption is a method by which data is encrypted locally before it is placed in a database. If client-side encryption is implemented then data will still be encrypted when viewed by individuals in the event of illegal access to the data.\u00a0<\/span><\/p>\n

The DarkShield-Files API can implement client-side encryption by extracting data stored in the database, search and mask sensitive data, and then place the newly sanitized data back into the database.<\/span><\/p>\n

DarkShield Search and Mask Contexts<\/b><\/h6>\n

The DarkShield-Files API is structured around using search contexts and mask contexts<\/span> to perform search and masking operations on text parsed from the different file formats. The Files API also handles additional matchers, filters, and configuration options for specific file formats by configuring file search contexts and file mask contexts.<\/span><\/p>\n

A search context designates the PII that will be annotated in the files read through matchers. There are a variety of matcher types for search matchers. The DarkShield-File API supports using search matchers based on regular expressions, named entity recognition (NER) models, and \u2018set\u2019 files, which is a text file with a list of entries separated by newlines.<\/span><\/p>\n

For the API to know what to do with PII that has been discovered during search operations, a mask context must be defined. The first part of a mask context contains a list of rules that we want to apply. Each rule has an expression that dictates what masking function will be used.<\/span><\/p>\n

These expressions are also documented in the IRI FieldShield manual and IRI Workbench, and because the functions are compatible with DarkShield, enterprise data integrity can be preserved post-masking regardless of source. The list of available <\/span>masking functions<\/span><\/a> include:<\/span><\/p>\n