- Release Notes
- Get Started
- Clusters
- Cloud Settings
- Table Type
- Query Data Lakes
- Integration
- Query Acceleration
- Data Loading
- Concepts
- Batch load data from Amazon S3
- Batch load data from Azure cloud storage
- Load data from a local file system
- Load data from Confluent Cloud
- Load data from Amazon MSK
- Load data from Amazon Kinesis
- Data Unloading
- Data Backup
- Security
- Console Access Control
- Data Access Control
- Application keys
- Service accounts
- Use SSL connection
- Alarm
- Usage and Billing
- Organizations and Accounts
- Reference
- Amazon Web Services (AWS)
- Microsoft Azure
- SQL Reference
- Keywords
- ALL statements
- User Account Management
- Cluster Management
- ADMIN CANCEL REPAIR
- ADMIN CHECK TABLET
- ADMIN REPAIR
- ADMIN SET CONFIG
- ADMIN SET REPLICA STATUS
- ADMIN SHOW CONFIG
- ADMIN SHOW REPLICA DISTRIBUTION
- ADMIN SHOW REPLICA STATUS
- ALTER RESOURCE GROUP
- ALTER SYSTEM
- CANCEL DECOMMISSION
- CREATE FILE
- CREATE RESOURCE GROUP
- DROP FILE
- DROP RESOURCE GROUP
- EXPLAIN
- INSTALL PLUGIN
- SET
- SHOW BACKENDS
- SHOW BROKER
- SHOW COMPUTE NODES
- SHOW FRONTENDS
- SHOW FULL COLUMNS
- SHOW INDEX
- SHOW PLUGINS
- SHOW PROCESSLIST
- SHOW RESOURCE GROUP
- SHOW TABLE STATUS
- SHOW FILE
- SHOW VARIABLES
- UNINSTALL PLUGIN
- DDL
- ALTER DATABASE
- ALTER MATERIALIZED VIEW
- ALTER TABLE
- ALTER VIEW
- ANALYZE TABLE
- BACKUP
- CANCEL ALTER TABLE
- CANCEL BACKUP
- CANCEL RESTORE
- CREATE ANALYZE
- CREATE DATABASE
- CREATE EXTERNAL CATALOG
- CREATE INDEX
- CREATE MATERIALIZED VIEW
- CREATE REPOSITORY
- CREATE TABLE AS SELECT
- CREATE TABLE LIKE
- CREATE TABLE
- CREATE VIEW
- CREATE FUNCTION
- DROP ANALYZE
- DROP STATS
- DROP CATALOG
- DROP DATABASE
- DROP INDEX
- DROP MATERIALIZED VIEW
- DROP REPOSITORY
- DROP TABLE
- DROP VIEW
- DROP FUNCTION
- KILL ANALYZE
- RECOVER
- REFRESH EXTERNAL TABLE
- RESTORE
- SET CATALOG
- SHOW ANALYZE JOB
- SHOW ANALYZE STATUS
- SHOW META
- SHOW FUNCTION
- TRUNCATE TABLE
- USE
- DML
- ALTER LOAD
- ALTER ROUTINE LOAD
- BROKER LOAD
- CANCEL LOAD
- CANCEL EXPORT
- CANCEL REFRESH MATERIALIZED VIEW
- CREATE ROUTINE LOAD
- DELETE
- EXPORT
- GROUP BY
- INSERT
- PAUSE ROUTINE LOAD
- RESUME ROUTINE LOAD
- REFRESH MATERIALIZED VIEW
- SELECT
- SHOW ALTER
- SHOW ALTER MATERIALIZED VIEW
- SHOW BACKUP
- SHOW CATALOGS
- SHOW CREATE CATALOG
- SHOW CREATE MATERIALIZED VIEW
- SHOW CREATE TABLE
- SHOW CREATE VIEW
- SHOW DATA
- SHOW DATABASES
- SHOW DELETE
- SHOW DYNAMIC PARTITION TABLES
- SHOW EXPORT
- SHOW LOAD
- SHOW MATERIALIZED VIEW
- SHOW PARTITIONS
- SHOW REPOSITORIES
- SHOW RESTORE
- SHOW ROUTINE LOAD
- SHOW ROUTINE LOAD TASK
- SHOW SNAPSHOT
- SHOW TABLES
- SHOW TABLET
- SHOW TRANSACTION
- STOP ROUTINE LOAD
- STREAM LOAD
- SUBMIT TASK
- UPDATE
- Auxiliary Commands
- Data Types
- Keywords
- SQL Functions
- Function list
- Java UDFs
- Window functions
- Lambda expression
- Date Functions
- add_months
- adddate
- convert_tz
- current_date
- current_time
- current_timestamp
- date
- date_add
- date_diff
- date_format
- date_slice
- date_sub, subdate
- date_trunc
- datediff
- day
- dayofweek_iso
- dayname
- dayofmonth
- dayofweek
- dayofyear
- days_add
- days_diff
- days_sub
- from_days
- from_unixtime
- hour
- hours_add
- hours_diff
- hours_sub
- jodatime_format
- last_day
- makedate
- microseconds_add
- microseconds_sub
- minute
- minutes_add
- minutes_diff
- minutes_sub
- month
- monthname
- months_add
- months_diff
- months_sub
- next_day
- now
- previous_day
- quarter
- second
- seconds_add
- seconds_diff
- seconds_sub
- str_to_date
- str_to_jodatime
- str2date
- time_slice
- time_to_sec
- timediff
- timestamp
- timestampadd
- timestampdiff
- to_date
- to_days
- to_iso8601
- to_tera_date
- to_tera_timestamp
- unix_timestamp
- utc_timestamp
- week
- week_iso
- weekofyear
- weeks_add
- weeks_diff
- weeks_sub
- year
- years_add
- years_diff
- years_sub
- Aggregate Functions
- any_value
- approx_count_distinct
- array_agg
- avg
- bitmap
- bitmap_agg
- count
- count_if
- corr
- covar_pop
- covar_samp
- group_concat
- grouping
- grouping_id
- hll_empty
- hll_hash
- hll_raw_agg
- hll_union
- hll_union_agg
- max
- max_by
- min
- min_by
- multi_distinct_sum
- multi_distinct_count
- percentile_approx
- percentile_cont
- percentile_disc
- retention
- stddev
- stddev_samp
- sum
- variance, variance_pop, var_pop
- var_samp
- window_funnel
- Geographic Functions
- String Functions
- append_trailing_char_if_absent
- ascii
- char
- char_length
- character_length
- concat
- concat_ws
- ends_with
- find_in_set
- group_concat
- hex
- hex_decode_binary
- hex_decode_string
- instr
- lcase
- left
- length
- locate
- lower
- lpad
- ltrim
- money_format
- null_or_empty
- parse_url
- repeat
- replace
- reverse
- right
- rpad
- rtrim
- space
- split
- split_part
- substring_index
- starts_with
- strleft
- strright
- str_to_map
- substring
- trim
- ucase
- unhex
- upper
- url_decode
- url_encode
- Pattern Matching Functions
- JSON Functions
- Overview of JSON functions and operators
- JSON operators
- JSON constructor functions
- JSON query and processing functions
- Bit Functions
- Bitmap Functions
- Array Functions
- all_match
- any_match
- array_agg
- array_append
- array_avg
- array_concat
- array_contains
- array_contains_all
- array_cum_sum
- array_difference
- array_distinct
- array_filter
- array_generate
- array_intersect
- array_join
- array_length
- array_map
- array_max
- array_min
- array_position
- array_remove
- array_slice
- array_sort
- array_sortby
- array_sum
- arrays_overlap
- array_to_bitmap
- cardinality
- element_at
- reverse
- unnest
- Map Functions
- Binary Functions
- cast function
- hash function
- Cryptographic Functions
- Math Functions
- Pattern Matching Functions
- Percentile Functions
- Scalar Functions
- Struct Functions
- Table Functions
- Utility Functions
- AUTO_INCREMENT
- Generated columns
- System variables
- System limits
- Information Schema
- Overview
- be_bvars
- be_cloud_native_compactions
- be_compactions
- character_sets
- collations
- column_privileges
- columns
- engines
- events
- global_variables
- key_column_usage
- load_tracking_logs
- loads
- materialized_views
- partitions
- pipe_files
- pipes
- referential_constraints
- routines
- schema_privileges
- schemata
- session_variables
- statistics
- table_constraints
- table_privileges
- tables
- tables_config
- task_runs
- tasks
- triggers
- user_privileges
- views
- System Metadatabase
- API
- Overview
- Actions
- Clusters
- Create and Manage Clusters
- Query Clusters
- Identity and Access Management
- Organization and Account
- Usage and Billing
- Clusters
- Terraform Provider
- Run scripts
Masking policies
This topic describes what a masking policy is, how to create and apply a masking policy, a use case about phone number masking, how to manage masking policies, and the limits when you work with a masking policy.
For the overview of column and row-level security, see Understand column and row-level security.
For the privileges required for each SQL operation, see Manage privileges for policies.
Definition
Column-level security allows you to apply a masking policy to a column within a table or view, thereby masking sensitive data from specific roles.
CelerData can apply a column masking policy to a column to mask data at query runtime based on the conditions you specified in the policy. Column masking does not alter or encrypt the stored data. It only applies masking rules at query runtime. Additionally, it does not modify basic table information, such as data types or column names.
A column masking policy can be added to a table or view either when the table is created or after the table is created.
Create a masking policy
A policy consists of column name, column type, masking conditions, and masking functions.
Syntax:
CREATE MASKING POLICY [ IF NOT EXISTS ] <name> AS
( <arg_name_to_mask> <arg_type_to_mask> [ , <arg_1> <arg_type_1> ... ] )
RETURNS <arg_type_to_mask> ->
<expression_on_arg_name>
[ COMMENT = '<string_literal>' ]
Parameter | Required | Description |
---|---|---|
name | Yes | The name of the policy, which must be unique across the database. Policies can be referenced across databases and catalogs in the format catalog.db.policy . If no catalog is specified, the current catalog is used. |
arg_name_to_mask | Yes | The name of the column to mask.The first pair of arg_name_to_mask arg_type_to_mask must be the column to mask. The columns following that column are conditional columns. |
arg_type_to_mask | Yes | The data type of the column to mask, which must be the same as the data type in the RETURNS clause. |
arg_1 arg_type_1 | No | The name and data type of the conditional columns. You can specify multiple conditional columns when you create a policy, but you can choose to reference only a few of them when you apply a policy. Conditional columns must reside in the same table as the column to mask. |
expression_on_arg_name | Yes | The expression that is used as the masking condition, which can use any conditional function, such as if(),case when(),and ifnull(). |
COMMENT | No | The description of the policy. |
Examples:
Example 1: Create a masking policy which only allows the
sales
role to see plaintext phone numbers. Other roles can only see masked phone numbers.CREATE MASKING POLICY phone_mask AS (phone string) RETURNS string -> CASE WHEN current_role() = 'sales' THEN phone ELSE '***MASKED***' END COMMENT "for test";
Example 2: Create a masking policy with a conditional column
visibility
. This policy allows only theACCOUNTADMIN
role to see email addresses or allows all roles to see only email addresses whosevisibility
ispublic
.CREATE MASKING POLICY email_visibility AS (email varchar, visibility varchar) RETURNS varchar -> CASE WHEN current_role() = 'ACCOUNTADMIN' THEN email WHEN visibility = 'public' THEN email ELSE '***MASKED***' END;
Example 3: Use a subquery in a masking policy. Only the current role can see email addresses whose
visibility
ispublic
.CREATE MASKING POLICY email_visibility1 AS (email varchar) RETURNS varchar -> CASE WHEN EXISTS (SELECT * FROM user3 WHERE visibility = 'public' AND role = current_role()) THEN email ELSE '***MASKED***' END;
Apply a masking policy
After a policy is created, you can apply it to the column you want to mask.
Syntax:
ALTER TABLE <tbl_name> MODIFY COLUMN <col_name>
SET MASKING POLICY <name> [ USING (<col_name>, <cond_col1> , ...)]
Examples:
-- Suppose you have a table sales_info.
CREATE TABLE `sales_info` (
name varchar(50),
phone string,
region varchar(50),
sales int);
-- Apply the masking policy phone_mask to the phone column of the table.
ALTER TABLE `sales_info` MODIFY COLUMN phone SET MASKING POLICY phone_mask;
You can also apply an existing masking policy to a table column using the WITH clause when you create the table.
CREATE TABLE `sales_info` (
name varchar(50),
phone string WITH MASKING POLICY phone_mask USING (phone),
region varchar(50),
sales int);
Use case - Mask phone numbers for different roles
This use case is to create a masking policy which allows the sales
role to view plaintext phone numbers and allows the analyst
role to view only masked phone numbers.
After connecting to your cluster as user admin
, you have the privileges associated with the default roles user_admin
and db_admin
. For data security concerns, you can create another user and assign only the required privileges to this user to test masking policies. This use case creates the following items:
- Database
db_test
- Table
sales_info
- Two roles that have different access to phone numbers
- User
masking_admin
and rolemasking_admin_role
with masking policy-related privileges. - Masking policy
phone_mask
for thephone
column of the table
Create a database
db_test
and switch to this database.CREATE DATABASE db_test; USE db_test;
Create a sales information table
sales_info
and insert data into this table.CREATE TABLE `sales_info` ( name varchar(50), phone string, region varchar(50), sales INT); INSERT INTO `sales_info` VALUES ('lily','886410','asia',11), ('richard','654321','uk',16), ('amber','789165','africa',17);
Create two roles
sales
andanalyst
. Grant the roles the privilege to query data from the table.CREATE ROLE `sales`,`analyst`; GRANT SELECT ON TABLE `sales_info` TO ROLE `sales`; GRANT SELECT ON TABLE `sales_info` TO ROLE `analyst`;
Create a user
masking_admin
, create a rolemasking_admin_role
, grant the required privileges to this role, and assign roles tomasking_admin
.CREATE USER `masking_admin`; CREATE ROLE `masking_admin_role`; -- Grant the privileges to create masking policies in the database. GRANT CREATE MASKING POLICY ON DATABASE db_test TO ROLE masking_admin_role; -- Grant the privileges to apply all masking policies in the database. GRANT ALTER ON TABLE sales_info TO ROLE masking_admin_role; GRANT APPLY ON ALL MASKING POLICIES to ROLE masking_admin_role; -- Assign the previous roles to the user. GRANT `masking_admin_role`,`sales`,`analyst` TO USER `masking_admin`; -- Switch to user masking_admin. EXECUTE AS `masking_admin` WITH NO REVERT; -- Activate role masking_admin_role to perform masking policy-related operations. SET ROLE `masking_admin_role`;
Create a masking policy which uses CASE WHEN to specify conditions. This policy allows only the
sales
role to see plaintext user phone numbers. Other roles can only see masked phone numbers.CREATE MASKING POLICY phone_mask AS (phone string) RETURNS string -> CASE WHEN current_role() = 'sales' THEN phone ELSE '***MASKED***' END;
Apply the policy to column
phone
you want to mask.ALTER TABLE `sales_info` MODIFY COLUMN phone SET MASKING POLICY phone_mask;
Use the two roles
sales
andanalyst
to query data. The results show that only thesales
role can see plaintext user phone number. Theanalyst
role can only see masked phone numbers.SET ROLE `sales`; SELECT * FROM `sales_info`; +---------+--------+--------+-------+ | name | phone | region | sales | +---------+--------+--------+-------+ | amber | 789165 | africa | 17 | | richard | 654321 | uk | 16 | | lily | 886410 | asia | 11 | +---------+--------+--------+-------+ SET ROLE `analyst`; SELECT * FROM `sales_info`; +---------+--------------+--------+-------+ | name | phone | region | sales | +---------+--------------+--------+-------+ | lily | ***MASKED*** | asia | 11 | | richard | ***MASKED*** | uk | 16 | | amber | ***MASKED*** | africa | 17 | +---------+--------------+--------+-------+
Manage masking policies
Unset a masking policy
Unsets a masking policy from a table column.
Syntax:
ALTER TABLE <tbl_name> MODIFY COLUMN <col_name> UNSET MASKING POLICY
Examples:
ALTER TABLE `sales_info` MODIFY COLUMN phone UNSET MASKING POLICY;
Modify a masking Policy
You can only modify the policy body, rename the policy, or update the comment of the policy. The new policy takes effect immediately after being created without requiring you to re-apply it to each table.
Note
- You cannot modify the data type of the masked column or the number and data types of the conditional columns because the policy may have been applied. In this case, modifying this policy may invalidate the masking policy.
- When you modify the policy body, make sure that the names and return type of conditional columns in the new body are the same as those originally specified when you create the Policy.
Syntax:
ALTER MASKING POLICY [ IF EXISTS ] <name> SET BODY -> <expression_on_arg_name>
ALTER MASKING POLICY [ IF EXISTS ] <name> RENAME TO <new_name>
ALTER MASKING POLICY [ IF EXISTS ] <name> SET COMMENT = '<string_literal>'
Examples:
ALTER MASKING POLICY phone_mask RENAME TO mask_phone;
ALTER MASKING POLICY phone_mask SET COMMENT = 'test';
Query all masking policies
Queries all masking policies in the current database.
SHOW MASKING POLICIES;
+------------------+---------+-----------------+----------+
| Name | Type | Catalog | Database |
+------------------+---------+-----------------+----------+
| email_visibility | MASKING | default_catalog | zj_test |
| phone_mask | MASKING | default_catalog | zj_test |
+------------------+---------+-----------------+----------+
Query the CREATE statement of a masking policy
Syntax:
SHOW CREATE MASKING POLICY <name>;
Examples:
SHOW CREATE MASKING POLICY phone_mask\G
*************************** 1. row ***************************
Policy: phone_mask
Create Policy: CREATE MASKING POLICY phone_mask AS (phone varchar(65533)) RETURNS varchar(65533) -> CASE WHEN ((CURRENT_ROLE()) = 'sales') THEN `phone` ELSE '***MASKED***' END COMMENT "for test"
Drop a masking policy
You are not allowed to drop a policy that has been applied to a table. If you want to drop such a policy, revoke it from all tables to which this policy has been applied and then drop this policy.
DROP MASKING POLICY <name>
Limits
- You can apply only one masking policy to one column.
- Conditional columns must reside in the same table as the column to mask.
- If a column has a masking policy applied to it, it cannot be used as a conditional column in another masking policy, nor can it be referenced in a subquery within another masking policy, and vice versa.
- If a table is a base table of a materialized view, no masking policies can be applied to that table.
- If a column in a table has a masking policy applied to it, a materialized view cannot be created based on that column.
See also
Policy creation and application are controlled by privileges such as CREATE, APPLY, ALTER, and DROP. For more information about how to grant these privileges, the privileges required by each command, and privilege management mode, see Manage privileges for policies.