- Release Notes
- Get Started
- Clusters
- Cloud Settings
- Table Type
- Query Data Lakes
- Integration
- Query Acceleration
- Data Loading
- Concepts
- Batch load data from Amazon S3
- Batch load data from Azure cloud storage
- Load data from a local file system
- Load data from Confluent Cloud
- Load data from Amazon MSK
- Load data from Amazon Kinesis
- Data Unloading
- Data Backup
- Security
- Console Access Control
- Data Access Control
- Application keys
- Service accounts
- Use SSL connection
- Alarm
- Usage and Billing
- Organizations and Accounts
- Reference
- Amazon Web Services (AWS)
- Microsoft Azure
- SQL Reference
- Keywords
- ALL statements
- User Account Management
- Cluster Management
- ADMIN CANCEL REPAIR
- ADMIN CHECK TABLET
- ADMIN REPAIR
- ADMIN SET CONFIG
- ADMIN SET REPLICA STATUS
- ADMIN SHOW CONFIG
- ADMIN SHOW REPLICA DISTRIBUTION
- ADMIN SHOW REPLICA STATUS
- ALTER RESOURCE GROUP
- ALTER SYSTEM
- CANCEL DECOMMISSION
- CREATE FILE
- CREATE RESOURCE GROUP
- DROP FILE
- DROP RESOURCE GROUP
- EXPLAIN
- INSTALL PLUGIN
- SET
- SHOW BACKENDS
- SHOW BROKER
- SHOW COMPUTE NODES
- SHOW FRONTENDS
- SHOW FULL COLUMNS
- SHOW INDEX
- SHOW PLUGINS
- SHOW PROCESSLIST
- SHOW RESOURCE GROUP
- SHOW TABLE STATUS
- SHOW FILE
- SHOW VARIABLES
- UNINSTALL PLUGIN
- DDL
- ALTER DATABASE
- ALTER MATERIALIZED VIEW
- ALTER TABLE
- ALTER VIEW
- ANALYZE TABLE
- BACKUP
- CANCEL ALTER TABLE
- CANCEL BACKUP
- CANCEL RESTORE
- CREATE ANALYZE
- CREATE DATABASE
- CREATE EXTERNAL CATALOG
- CREATE INDEX
- CREATE MATERIALIZED VIEW
- CREATE REPOSITORY
- CREATE TABLE AS SELECT
- CREATE TABLE LIKE
- CREATE TABLE
- CREATE VIEW
- CREATE FUNCTION
- DROP ANALYZE
- DROP STATS
- DROP CATALOG
- DROP DATABASE
- DROP INDEX
- DROP MATERIALIZED VIEW
- DROP REPOSITORY
- DROP TABLE
- DROP VIEW
- DROP FUNCTION
- KILL ANALYZE
- RECOVER
- REFRESH EXTERNAL TABLE
- RESTORE
- SET CATALOG
- SHOW ANALYZE JOB
- SHOW ANALYZE STATUS
- SHOW META
- SHOW FUNCTION
- TRUNCATE TABLE
- USE
- DML
- ALTER LOAD
- ALTER ROUTINE LOAD
- BROKER LOAD
- CANCEL LOAD
- CANCEL EXPORT
- CANCEL REFRESH MATERIALIZED VIEW
- CREATE ROUTINE LOAD
- DELETE
- EXPORT
- GROUP BY
- INSERT
- PAUSE ROUTINE LOAD
- RESUME ROUTINE LOAD
- REFRESH MATERIALIZED VIEW
- SELECT
- SHOW ALTER
- SHOW ALTER MATERIALIZED VIEW
- SHOW BACKUP
- SHOW CATALOGS
- SHOW CREATE CATALOG
- SHOW CREATE MATERIALIZED VIEW
- SHOW CREATE TABLE
- SHOW CREATE VIEW
- SHOW DATA
- SHOW DATABASES
- SHOW DELETE
- SHOW DYNAMIC PARTITION TABLES
- SHOW EXPORT
- SHOW LOAD
- SHOW MATERIALIZED VIEW
- SHOW PARTITIONS
- SHOW REPOSITORIES
- SHOW RESTORE
- SHOW ROUTINE LOAD
- SHOW ROUTINE LOAD TASK
- SHOW SNAPSHOT
- SHOW TABLES
- SHOW TABLET
- SHOW TRANSACTION
- STOP ROUTINE LOAD
- STREAM LOAD
- SUBMIT TASK
- UPDATE
- Auxiliary Commands
- Data Types
- Keywords
- SQL Functions
- Function list
- Java UDFs
- Window functions
- Lambda expression
- Date Functions
- add_months
- adddate
- convert_tz
- current_date
- current_time
- current_timestamp
- date
- date_add
- date_diff
- date_format
- date_slice
- date_sub, subdate
- date_trunc
- datediff
- day
- dayofweek_iso
- dayname
- dayofmonth
- dayofweek
- dayofyear
- days_add
- days_diff
- days_sub
- from_days
- from_unixtime
- hour
- hours_add
- hours_diff
- hours_sub
- jodatime_format
- last_day
- makedate
- microseconds_add
- microseconds_sub
- minute
- minutes_add
- minutes_diff
- minutes_sub
- month
- monthname
- months_add
- months_diff
- months_sub
- next_day
- now
- previous_day
- quarter
- second
- seconds_add
- seconds_diff
- seconds_sub
- str_to_date
- str_to_jodatime
- str2date
- time_slice
- time_to_sec
- timediff
- timestamp
- timestampadd
- timestampdiff
- to_date
- to_days
- to_iso8601
- to_tera_date
- to_tera_timestamp
- unix_timestamp
- utc_timestamp
- week
- week_iso
- weekofyear
- weeks_add
- weeks_diff
- weeks_sub
- year
- years_add
- years_diff
- years_sub
- Aggregate Functions
- any_value
- approx_count_distinct
- array_agg
- avg
- bitmap
- bitmap_agg
- count
- count_if
- corr
- covar_pop
- covar_samp
- group_concat
- grouping
- grouping_id
- hll_empty
- hll_hash
- hll_raw_agg
- hll_union
- hll_union_agg
- max
- max_by
- min
- min_by
- multi_distinct_sum
- multi_distinct_count
- percentile_approx
- percentile_cont
- percentile_disc
- retention
- stddev
- stddev_samp
- sum
- variance, variance_pop, var_pop
- var_samp
- window_funnel
- Geographic Functions
- String Functions
- append_trailing_char_if_absent
- ascii
- char
- char_length
- character_length
- concat
- concat_ws
- ends_with
- find_in_set
- group_concat
- hex
- hex_decode_binary
- hex_decode_string
- instr
- lcase
- left
- length
- locate
- lower
- lpad
- ltrim
- money_format
- null_or_empty
- parse_url
- repeat
- replace
- reverse
- right
- rpad
- rtrim
- space
- split
- split_part
- substring_index
- starts_with
- strleft
- strright
- str_to_map
- substring
- trim
- ucase
- unhex
- upper
- url_decode
- url_encode
- Pattern Matching Functions
- JSON Functions
- Overview of JSON functions and operators
- JSON operators
- JSON constructor functions
- JSON query and processing functions
- Bit Functions
- Bitmap Functions
- Array Functions
- all_match
- any_match
- array_agg
- array_append
- array_avg
- array_concat
- array_contains
- array_contains_all
- array_cum_sum
- array_difference
- array_distinct
- array_filter
- array_generate
- array_intersect
- array_join
- array_length
- array_map
- array_max
- array_min
- array_position
- array_remove
- array_slice
- array_sort
- array_sortby
- array_sum
- arrays_overlap
- array_to_bitmap
- cardinality
- element_at
- reverse
- unnest
- Map Functions
- Binary Functions
- cast function
- hash function
- Cryptographic Functions
- Math Functions
- Pattern Matching Functions
- Percentile Functions
- Scalar Functions
- Struct Functions
- Table Functions
- Utility Functions
- AUTO_INCREMENT
- Generated columns
- System variables
- System limits
- Information Schema
- Overview
- be_bvars
- be_cloud_native_compactions
- be_compactions
- character_sets
- collations
- column_privileges
- columns
- engines
- events
- global_variables
- key_column_usage
- load_tracking_logs
- loads
- materialized_views
- partitions
- pipe_files
- pipes
- referential_constraints
- routines
- schema_privileges
- schemata
- session_variables
- statistics
- table_constraints
- table_privileges
- tables
- tables_config
- task_runs
- tasks
- triggers
- user_privileges
- views
- System Metadatabase
- API
- Overview
- Actions
- Clusters
- Create and Manage Clusters
- Query Clusters
- Identity and Access Management
- Organization and Account
- Usage and Billing
- Clusters
- Terraform Provider
- Run scripts
Overview
The CelerData Cloud BYOC API is an HTTPS-based RESTful interface that provides you with programmatic access to manage administrative objects within CelerData Cloud. With this API, you can securely create, manage, and query various resources such as clusters, warehouses (available only to elastic clusters), usage, and billing.
This API has the following features:
- JSON entities. All entities are expressed in JSON.
- HTTPS-only. You can access the API only via HTTPS. This ensures that all the data sent over the network is encrypted with TLS.
- Client Credentials-based authentication in compliance with OAuth 2.0. Before you access the API, you must generate an application key or create a service account, convert its client ID and secret key into a base64 encoded string, and then use the string to create an access token, which grants you authorization to the API.
This guide introduces what you need to do before using the API and walks you through the process of making an API call.
Before you begin
To complete this guide, you need to:
- Have a CelerData cloud account.
- Install curl or a web development tool such as Postman.
- Obtain an access token.
- (Optional) Obtain the cluster ID.
Obtain access token
Step 1: Generate Client ID and Client secret
Currently, CelerData Cloud BYOC supports authentication with application keys and service accounts.
You can choose either to generate an application key or create a service account to acquire the Client ID and Client secret.
Generate an application key
Sign in to the CelerData Cloud BYOC console.
In the left-side navigation pane, choose Application keys.
On the Application keys page, click New secret.
In the dialog box that appears, optionally enter a description, and then click Generate now to generate an application key. Then, copy the Secret and Client ID before you close the dialog box.
NOTE
The Secret can be viewed only when the application key is created. Make sure that you copy and save the Secret before closing the dialog box.
For more information about managing application keys, see Application keys.
Create a service account
Sign in to the CelerData Cloud BYOC console.
In the left-side navigation pane, choose Access Control > Member.
On the Members page, click the Service accounts tab.
On the Service accounts tab, click Create a service account.
In the Generate secret for service account dialog box, enter a name for the service account, and click Generate now.
Copy the Client Secret and Client ID, and click Yes, I have finished it to close the dialog box.
NOTE
The Client Secret can be viewed only when the service account is created. Make sure that you copy and save the Client Secret before closing the dialog box.
On the detail page of the service account, click the Granted roles tab.
On the Granted roles tab, select the role you want to assign from the Assign new role drop-down list, and click Assign.
For more information about managing service accounts, see Service accounts.
Step 2: Generate a base64 encoded string
Use the following command to generate a base64 encoded string:
echo -n <client_id>:<secret> | base64 | tr -d "\n"
Note that you must replace <client_id>
and <secret>
in the preceding command with the client ID and secret key you obtained in the above Step 1. Example:
echo -n 50292c3c-2b95-417f-880d-0c7e8c26eac1:YOklxY24RKgveikknUI48RuHnkPsFit1u0i2TGyp | base64 | tr -d "\n"
A base64 encoded string like below is returned:
NTAyOTJjM2MtMmI5NS00MTdmLTg4MGQtMGM3ZThjMjZlYWMxOllPa2x4WTI0UktndmVpa2tuVUk0OFJ1SG5rUHNGaXQxdTBpMlRHeXA=
The generated base64 encoded string can then be used to make the API call for creating an access token, in which case it is added as an Authorization: Basic base64(<client_id>:<secret>)
header to the HTTP request.
Step 3: Create an access token
Make an API call to create an access token. You must add the base64 encoded <client_id>:<secret>
string generated in the above Step 2 as an Authorization: Basic base64(<client_id>:<secret>)
header to the HTTP request. For details, see Create access token.
The access token created can then be used to make other API calls in which it is added as an Authorization: Bearer <access_token>
header to the HTTP requests.
(Optional) Obtain cluster ID
For some API calls, you will need to provide the ID of the CelerData cluster you want to query. Follow these steps to obtain the cluster ID:
- Sign in to the CelerData Cloud BYOC console.
- In the left-side navigation pane, choose Clusters.
- On the page that appears, click the cluster you want to query.
- On the Overview tab of the cluster details page, navigate to the Cluster properties section, where you can find the Cluster ID.
Make an API call
This section uses curl as an example to describe how to query the CCU usage of a cluster, helping you understand how you can build an API call and check the response.
Build an API call
Suppose that the cluster ID is 82964bc5-a8ce-4236-9fe6-1ad1e0f80e15
and the access token is NWM0MWU1NJGTZME1OC0ZZDNKLTK0MZKTODQXM2EXNGM3MTYX
.
Run the following command in your terminal to query the CCU usage of the cluster within the time range of June 1, 2023 to June 31, 2023 and view the CCU usage details on a daily basis:
curl --request GET \
--url 'https://cloud-api.celerdata.com/api/1.0/usages/82964bc5-a8ce-4236-9fe6-1ad1e0f80e15?start_date=20230701&end_date=20230731&show_detail=true' \
--header 'Authorization: Bearer NWM0MWU1NJGTZME1OC0ZZDNKLTK0MZKTODQXM2EXNGM3MTYX'
Check the response
After you submit the API call, if the status code in the response is 20000
and you see details about the usage of the cluster, your request is successful.
The response body consists of three parts:
code
: the error code. If the error code returned is20000
, the request succeeded. If any other error code is returned, the request failed.data
: the result of the API call. Thedata
object may not be returned for some API actions.message
: the error message. This field is returned only when thecode
returned is not20000
. Do not conclude whether a request succeeded or failed based on the return value ofmessage
.
Here is an example of the response to a successful API call:
{
"code":20000,
"data":{
"cluster_id":"82964bc5-a8ce-4236-9fe6-1ad1e0f80e15",
"cluster_name":"test_privilege_30-wybing",
"total_usage":"87.025277",
"details":[
{
"usage":"18.130654",
"date":20230703
}
]
}
}