What is data masking?

Data masking is a method for protecting personal or sensitive data that creates a version of the data that can’t be identified or reverse-engineered while retaining referential integrity and usability.

• Personally Identifiable Information (PII) such as names, passport, social security, and telephone numbers
• Protected Health Information (PHI) about an individual’s health status, care, or payment data
• Protected financial data, as mandated by the Payment Card Industry Data Security Standard (PCI-DSS) and the US Federal Trade Commission (FTC) acts and safeguards
• Test data, associated with the Software Development Life Cycle (SDLC)
  
  Masked data is generally used in non-production environments – such as software development, data science, and testing – that don’t require the original production data. 
  
  Simply defined, data masking combines the processes and tools for making sensitive data unrecognizable to – yet functional for – authorized users.

The data masking process

The data masking process is an iterative lifecycle that can be broken down into 4 steps corresponding to 4 data masking requirements – discovering the data, defining the masking rules, deploying the masking functionality, and auditing the entire process on an ongoing basis.

With the right data masking process in place, your data teams can:

1. Discover the data and its relationships 
   Your data masking tools should be in sync with a data catalog that collects, analyzes, and visualizes the metadata for all the information that requires masking.
2. Define data masking rules
   You should be able to restrict access to sensitive data with Role-Based Access Controls (RBAC), and then define different masking rules for different types of users.
3. Deploy the appropriate masking functions
   Your tools should have a wide range of built-in functions – e.g., redaction, scrambling, shuffling, and nulling out – but also allow you to customize functionality to your own specifications.
4. Audit and report for compliance
   Generating exportable audit reports – for instance, including schema, table, and column name, as well as type, field, and probability of a match – is an ongoing part of the process.

Over time, many types of data masking have evolved to provide more sophistication, flexibility and security, including:

Static data masking

Non-production environments, such as those used for analytics, testing, training, and development purposes, often source data from production systems. In such cases, sensitive data is protected with static data masking, a one-way transformation ensuring that the masking process cannot be undone. When it comes to testing and analytics, repeatability is a key concept because using the same input data delivers the same results. This requires the masked data values to persist, over time, and through multiple extractions.

For software testing, static data masking is usually employed on a copy of a production database. Advanced masking tools make data look real enough to enable software development and testing, without exposing the original values.

Dynamic data masking

Dynamic data masking is used to protect, obscure, or block access to, sensitive data. While prevalent in production systems, it is also used when testers or data scientists require real data. Dynamic data masking is performed in real time, in response to a data request. When the data is located in multiple source systems, masking consistency is difficult, especially when dealing with disparate environments, and a wide variety of technologies. Dynamic data masking protects sensitive data on demand.

Dynamic data masking automatically streams data from a production environment, to avoid storing the masked data in a separate database. As a rule, it’s used for role-based security for applications – such as handling customer queries, or processing sensitive data, like health records – and in read-only scenarios, so that the masked data doesn’t get written back to the production system.

This technique is frequently used in customer service applications to ensure that support personnel can access the data they need to assist customers while masking sensitive information like credit card numbers or personal identifiers – to maintain privacy and compliance with data protection regulations.

On-the-fly data masking

When analytics or test data is extracted from production systems, staging sites are often used to integrate, cleanse, and transform the data, before masking it. The masked data is then delivered to the analytics or testing environment. This multi-stage process is slow, cumbersome, and risky due to the possible exposure of private data.

On-the-fly data masking is performed on data as it moves from one environment to another, such as from production, to development or test. It’s ideal for enterprises engaging in continuous software development and large-scale data integrations. A subset of the masked data is generally delivered to authorized users upon request, because keeping a backup of all the masked data is inefficient and impractical.

Statistical data masking

Statistical data masking ensures that any masked data retains the same statistical characteristics and patterns as the real-world data it represents – such as the distribution, mean, and standard deviation. When production data is statistically masked, unauthorized users have great difficulty extracting any information of value afterwards.

Test data masking

Software applications require extensive testing before they can be released into production. Test data management tools that provision production data for testing must mask the test data to protect sensitive information. For example, in a legacy modernization program, the modernized software components must undergo continuous testing, making test data masking a key component in the testing process. Masking data with referential context and relational integrity – from production systems to the test environments – is critical.

Unstructured data masking

When it comes to protecting data privacy, regulations do not differentiate between structured and unstructured data. Scanned documents and image files, such as insurance claims, bank checks, and medical records, contain sensitive data stored as images. Many different formats (e.g., pdf, png, csv, email, and Office docs) are used daily by enterprises in their regular interactions with individuals. With the potential for so much sensitive data to be exposed in unstructured files, the need for unstructured data masking is obvious.

Data masking solutions are important to enterprises because they enable them to:

• Achieve compliance with privacy laws, like CPRA, GDPR and HIPAA, by reducing the risk of exposing personal or sensitive data, as one aspect of the total compliance picture.
• Protect data in testing environments from cyber-attacks, while preserving its usability and consistency.
• Reduce the risk of data sharing, e.g., in the case of cloud migrations, or when integrating with third-party apps.

Masking tools are now needed more than ever before, to effectively safeguard sensitive data and to address the following challenges:

Regulatory compliance

Highly regulated industries, like financial services and healthcare, already operate under strict privacy regulations. Besides adhering to regional standards, such as Europe’s GDPR, California’s CPRA, or Brazil’s LGPD, companies in these fields rely on PII data masking to comply with the Payment Card Industry Data Security Standard (PCI DSS), and the Health Insurance Portability and Accountability Act (HIPAA). 

Insider threats

Many employees and third-party contractors access enterprise systems on a regular basis, for example for software testing or analytics purposes. Production systems are particularly vulnerable, because sensitive information is often used in development, testing, and other pre-production environments. With insider threats rising 47% since 2018, according to the Ponemon Institute report, protecting sensitive data costs companies an average of $200,000 per year.

External threats

In 2020, personal data was compromised in 58% of the data breaches, states a Verizon report. The study further indicates that in 72% of the cases, the victims were large enterprises. With the vast volume, variety and velocity of enterprise data, it is no wonder that breaches proliferate. Taking measures to protect sensitive data in non-production environments will significantly reduce the risk, one of many data masking examples.

Data governance

Your data masking tool should be secured with Role-Based Access Control (RBAC). While static data masking obscures a single dataset, dynamic data masking provides more granular controls. With dynamic data masking, permissions can be granted or denied at many different levels. Only those with the appropriate access rights can access the real data. Others will see only the parts that they are allowed to see. You should also be able to apply different masking policies to different users.

Flexibility

Data masking is highly customizable. Data teams can choose which data fields get masked, and how to select and format each substitute value. For example, every Social Security Number (SSN) has the format xxx-xx-xxxx, where “x” is a number from 0 to 9. They can substitute the first five digits with the letter x, or all 9 numbers with other random numbers, according to their needs.

To understand how data masking works, let’s compare it to data encryption and data tokenization.

While data masking is irreversible, encryption and tokenization are both reversible in the sense that the original values can be derived from the obscured data. Here’s a brief explanation of the 3 methods:

Data masking

Data masking tools substitute realistic, but fake, data for the original values, to ensure data privacy. Development, support, data science, business intelligence, testing, and training teams use masked data to make use of a dataset without exposing real data to any risk. 

There are many techniques for masking data, such as data scrambling, data blinding, or data shuffling, which will be explained later in greater detail. The process of permanently removing all PII from sensitive data is also known as data sanitization. There is no algorithm to recover the original values of masked data.

Data encryption

While data encryption is very secure, data teams can’t analyze or work with encrypted data. The more complex the encryption algorithm, the safer the data will be from unauthorized access. In a data masking vs  encryption comparison, encryption is ideal for storing or transferring sensitive data securely.

Data tokenization

Data tokenization, which substitutes a sensitive data element with random characters (tokens), is a reversible process. The tokens can be mapped back to the original data, with the mappings stored in a secure “data vault”.

In a data masking vs tokenization comparison, tokenization supports operations like processing a credit card payment without revealing the credit card number. The real data never leaves the organization and can’t be seen or decrypted by a third-party processor.
 
Data tokenization supports the Payment Card Industry Data Security Standard (PCI-DSS).

Data masking is not reversible, making it more secure, and less costly, than tokenization. It maintains referential context and relational integrity across systems and databases, which is critical in data analysis and software testing .  

Relational integrity retains data validity and consistency, despite undergoing data de-identification. For example, a real credit card number can be replaced by any 16-digit figure. Once masked and validated, the new value will appear consistently across all systems.