Skip to main content

Provision CelerData Cloud BYOC on AWS

Deploy a CelerData Cloud BYOC cluster on AWS step by step:

Read this article before you start a Terraform configuration for your cluster deployment on AWS.

Preparations​

Before using the CelerData Cloud BYOC provider to create infrastructure at the AWS account level for the first time, you must complete the following preparations:

AWS prerequisites​

  1. Have an AWS account with administrative privileges.

  2. Have an AWS S3 bucket.

  3. Have an IAM user, to which the following policy is attached, and create an access key pair for the IAM user:

    {
        "Version": "2012-10-17",
        "Statement": [
            {
                "Effect": "Allow",
                "Action": [
                    "iam:CreateRole",
                    "iam:PutRolePolicy",
                    "iam:GetRole",
                    "iam:ListRolePolicies",
                    "iam:GetRolePolicy",
                    "iam:ListAttachedRolePolicies",
                    "iam:ListInstanceProfilesForRole",
                    "iam:DeleteRolePolicy",
                    "iam:DeleteRole",
                    "iam:CreateInstanceProfile",
                    "iam:GetInstanceProfile",
                    "iam:RemoveRoleFromInstanceProfile",
                    "iam:DeleteInstanceProfile",
                    "iam:AddRoleToInstanceProfile",
                    "iam:PassRole"
                ],
                "Resource": "*"
            }
        ]
    }

CelerData prrerequisites​

Obtain the credentials with which you can authenticate into the CelerData Cloud BYOC platform. For details, see Authentication.

Terraform prerequisites​

  1. Install Terraform in your terminal.

  2. Have a Terraform project. In your terminal, create an empty directory (for example, terraform) and then switch to it. (Each separate set of Terraform configuration files must be in its own directory, which is called a Terraform project.)

Configure providers​

This section assumes that you have completed the preparations.

Create a .tf file (for example, main.tf) in your Terraform project. Then, add the following code snippet to the .tf file:

terraform {
  required_providers {
    celerdatabyoc = {
      source = "CelerData/celerdatabyoc"
      version = "<provider_version>"
    }
    aws = {
      source  = "hashicorp/aws"
      version = "4.26.0"
    }
  }
}

provider "celerdatabyoc" {
  client_id = "<client_id>"
  client_secret = "<client_secret>"
}

provider "aws" {
  region     = "<region_name>"
  access_key = "<your_access_key>"
  secret_key = "<your_secret_key>"
}

locals {
  s3_bucket = "<bucket_name>" 
}

The parameters you need to specify are as follows:

  • provider_version: Enter the CelerData provider version of your choice. We recommend that you select the latest provider version, for example version = "1.0.2". You can view the provider versions offered by CelerData Cloud BYOC from the CelerData Cloud BYOC provider page.
  • client_id and client_secret: Enter the Client ID and Secret of your application key. See "For CelerData."
  • region_name: Enter the ID of the AWS region in which you want your CelerData cluster to run. See Supported cloud platforms and regions.
  • access_key and secret_key: Enter the access key ID ("access key" for short) and secret access key ("secret key" for short) of your access key pair. See "For AWS." For security purposes, you can set the access key and secret key as environment variables.
  • s3_bucket: Enter the name of your S3 bucket. This way, the bucket element is set as a local value, and you can then directly set the argument for the bucket element in your Terraform configuration to local.s3_bucket to save time. See Local Values.

Describe infrastructure​

This section provides a sample infrastructure configuration that automates the deployment of a classic CelerData cluster on AWS to help you understand how you can work with the CelerData Cloud BYOC provider. It assumes that you have completed the preparations and have configured the providers.

To create a classic CelerData cluster, you need to declare the following resources, which represent the infrastructure to be built, in the .tf file (for example, main.tf) in which you have configured the providers.

data "celerdatabyoc_aws_data_credential_assume_policy" "assume_role" {}

resource "celerdatabyoc_aws_data_credential_policy" "role_policy" {
  bucket = local.s3_bucket
}

resource "aws_iam_role" "celerdata_data_cred_role" {
  name               = "<celerdata_data_credential_role_name>"
  assume_role_policy = data.celerdatabyoc_aws_data_credential_assume_policy.assume_role.json
  description        = "<celerdata_data_credential_role_description>"
  inline_policy {
    name   = "<celerdata_data_credential_role_policy_name>"
    policy = celerdatabyoc_aws_data_credential_policy.role_policy.json
  }
}

resource "celerdatabyoc_aws_data_credential" "data_credential" {
  name = "<celerdata_data_credential_name>"
  role_arn = aws_iam_role.celerdata_data_cred_role.arn
  instance_profile_arn = aws_iam_instance_profile.celerdata_data_cred_profile.arn
  bucket_name = local.s3_bucket
  policy_version = celerdatabyoc_aws_data_credential_policy.role_policy.version
}

celerdatabyoc_aws_data_credential_policy​

This resource contains only the following required argument:

  • bucket: (Forces new resource) The name of the AWS S3 bucket for which to generate the JSON policy document and that stores query profiles. Set this argument to local.s3_bucket, as we recommend that you set the bucket element as a local value s3_bucket in your Terraform configuration. See Local Values.

aws_iam_role (celerdata_data_cred_role)​

This resource contains the following required arguments and optional arguments:

Required:

  • assume_role_policy: The policy that grants an entity permission to assume the IAM role referenced in the data credential. Set the value to data.celerdatabyoc_aws_data_credential_assume_policy.assume_role.json.

Optional:

  • name: (Forces new resource) The name of the IAM role referenced in the data credential. Enter a unique name. If omitted, Terraform will assign a random, unique name. See IAM Identifiers for more information.
  • description: The description of the IAM role.
  • inline_policy: The configuration block that defines an exclusive set of IAM inline policies associated with the IAM role. See below. If no blocks are configured, Terraform will not manage any inline policies in this resource. Configuring one empty block (namely, inline_policy {}) will cause Terraform to remove all inline policies added out of band on apply.
    • name: The name of the IAM policy that will be attached to the IAM role referenced in the data credential.
    • policy: The IAM policy that will be attached to the IAM role. Set the value to celerdatabyoc_aws_data_credential_policy.role_policy.json.

celerdatabyoc_aws_data_credential​

This resource contains the following required arguments and optional arguments:

Required:

  • role_arn: (Forces new resource) The ARN of the IAM role referenced in the data credential. Set the value to aws_iam_role.celerdata_data_cred_role.arn.
  • instance_profile_arn: (Forces new resource) The instance profile ARN of the IAM role referenced in the data credential. Set the value to aws_iam_instance_profile.celerdata_data_cred_profile.arn.
  • bucket_name: (Forces new resource) The name of the AWS S3 bucket for which to generate the policy document and that stores query profiles. Set this argument to local.s3_bucket, as we recommend that you set the bucket element as a local value s3_bucket in your Terraform configuration. See Local Values.
  • policy_version: (Forces new resource) The version of the policy. Set the value to celerdatabyoc_aws_data_credential_policy.role_policy.version.

Optional:

  • name: (Forces new resource) The name of the data credential. Enter a unique name.
resource "aws_iam_instance_profile" "celerdata_data_cred_profile" {
  name = "<celerdata_data_credential_profile_name>"
  role = aws_iam_role.celerdata_data_cred_role.name
}

resource "celerdatabyoc_aws_deployment_credential_policy" "role_policy" {
  bucket = local.s3_bucket
  data_role_arn = aws_iam_role.celerdata_data_cred_role.arn
}

resource "celerdatabyoc_aws_deployment_credential_assume_policy" "role_policy" {}

resource "aws_iam_role" "deploy_cred_role" {
  name               = "<celerdata_deployment_credential_role_name>"
  assume_role_policy = celerdatabyoc_aws_deployment_credential_assume_policy.role_policy.json
  description        = "<celerdata_deployment_credential_role_description>"
  inline_policy {
    name   = "<celerdata_deployment_credential_role_policy_name>"
    policy = celerdatabyoc_aws_deployment_credential_policy.role_policy.json
  }
}

resource "celerdatabyoc_aws_deployment_role_credential" "deployment_role_credential" {
  name = "<celerdata_deployment_credential_name>"
  role_arn = aws_iam_role.deploy_cred_role.arn
  external_id = celerdatabyoc_aws_deployment_credential_assume_policy.role_policy.external_id
  policy_version = celerdatabyoc_aws_deployment_credential_policy.role_policy.version
}

aws_iam_instance_profile​

This resource contains only the following optional arguments:

  • name: (Forces new resource) The name of the instance profile. Enter a unique name. If omitted, Terraform will assign a random, unique name. This argument conflicts with name_prefix. The value of this argument can be a string of characters consisting of upper and lowercase alphanumeric characters and these special characters: _, +, =, ,, ., @, -. Spaces are not allowed.
  • role: The name of the IAM role to add to the instance profile. Set the value to aws_iam_role.celerdata_data_cred_role.name.

celerdatabyoc_aws_deployment_credential_policy​

This resource contains only the following required arguments:

  • bucket: The name of the AWS S3 bucket. Set this argument to local.s3_bucket, as we recommend that you set the bucket element as a local value s3_bucket in your Terraform configuration. See Local Values.
  • data_role_arn: (Forces new resource) The ARN of the IAM role referenced in the deployment credential. Set the value to aws_iam_role.celerdata_data_cred_role.arn.

aws_iam_role (deploy_cred_role)​

This resource contains the following required arguments and optional arguments:

Required:

  • assume_role_policy: The policy that grants an entity permission to assume the IAM role referenced in the deployment credential. Set the value to celerdatabyoc_aws_deployment_credential_assume_policy.role_policy.json.

Optional:

  • name: (Forces new resource) The name of the IAM role referenced in the deployment credential. Enter a unique name. If omitted, Terraform will assign a random, unique name. See IAM Identifiers for more information.
  • description: The description of the IAM role.
  • inline_policy: The configuration block that defines an exclusive set of IAM inline policies associated with the IAM role. See below. If no blocks are configured, Terraform will not manage any inline policies in this resource. Configuring one empty block (namely, inline_policy {}) will cause Terraform to remove all inline policies added out of band on apply.
    • name: The name of the IAM policy that will be attached to the IAM role.
    • policy: The IAM policy that will be attached to the IAM role referenced in the deployment credential. Set the value to celerdatabyoc_aws_deployment_credential_policy.role_policy.json.

celerdatabyoc_aws_deployment_role_credential​

This resource contains the following required arguments and optional arguments:

Required:

  • role_arn: (Forces new resource) The ARN of the cross-account IAM role referenced in the deployment credential. Set the value to aws_iam_role.deploy_cred_role.arn.
  • external_id: (Forces new resource) The external ID that is used to create the cross-account IAM role referenced in the deployment credential. Set the value to celerdatabyoc_aws_deployment_credential_assume_policy.role_policy.external_id.
  • policy_version: (Forces new resource) The version of the policy. Set the value to celerdatabyoc_aws_deployment_credential_policy.role_policy.version.

Optional:

  • name: (Forces new resource) The name of the deployment credential. Enter a unique name.
resource "celerdatabyoc_aws_network" "network" {
  name = "<VPC_name>"
  subnet_id = "<subnet_id>"
  security_group_id = "<security_group_id>"
  region = "<AWS_VPC_region>"
  deployment_credential_id = celerdatabyoc_aws_deployment_role_credential.deployment_role_credential.id
  vpc_endpoint_id = "<vpc_endpoint_id>"
}

The celerdatabyoc_aws_network resource contains the following required arguments and optional arguments:

Required:

  • name: (Forces new resource) The name of the AWS VPC hosting the cluster. Enter a unique name.
  • subnet_id: (Forces new resource) The ID of the subnet within the AWS VPC.
  • security_group_id: (Forces new resource) The ID of the security group within the AWS VPC.
  • region: (Forces new resource) The ID of the AWS region to which the AWS VPC belongs. See Supported cloud platforms and regions.
  • deployment_credential_id: (Forces new resource) The ID of the deployment credential. Set the value to celerdatabyoc_aws_deployment_role_credential.deployment_role_credential.id.

Optional:

  • vpc_endpoint_id: (Optional) The ID of your endpoint within your VPC. Set this argument if you need to achieve a more stringent network communication method.
resource "celerdatabyoc_classic_cluster" "demo_cluster" {
  deployment_credential_id = celerdatabyoc_aws_deployment_role_credential.deployment_role_credential.id
  data_credential_id = celerdatabyoc_aws_data_credential.data_credential.id
  network_id = celerdatabyoc_aws_network.network.id
  
  cluster_name = "<cluster_name>"
  fe_instance_type = "<fe_node_instance_type>"
  fe_node_count = 1

  be_instance_type = "<be_node_instance_type>"
  be_node_count = 1
  // optional
  be_volume_config {
    vol_number = <vol_number>
    vol_size = <vol_size>
    iops = <iops>
    throughput = <throughput>
  }
  
  default_admin_password = "<SQL_user_initial_password>"
  expected_cluster_state = "Running"
  resource_tags = {
    celerdata = "<tag_name>"
  }
  csp = "aws"
  region = "<AWS_VPC_region>"
}

Apply configurations​

After you finish configuring the providers and describing the infrastructure objects in your Terraform configuration, follow these steps to apply the configuration in your Terraform project:

  1. Initialize and install the providers defined in the Terraform configuration:

    terraform init

  2. Verify that your Terraform project has been properly configured:

    terraform plan

    If there are any errors, edit the Terraform configuration and re-run the preceding command.

  3. Apply the Terraform configuration:

    terraform apply

When the system returns a "Apply complete!" message, the Terraform configuration has been successfully applied.

note

After you change the provider versions in the Terraform configuration, you must run terraform init -upgrade to initialize the providers and then run terraform apply again to apply the Terraform configuration.

Delete configurations​

You can delete your Terraform configuration if you no longer need it.

Deleting a Terraform configuration means destroying all resources created by the CelerData Cloud BYOC provider.

To delete a Terraform configuration, run the following command in your Terraform project:

terraform destroy

When the system returns a "Destroy complete!" message, the Terraform configuration has been successfully deleted and the cluster created by the Terraform configuration is also released.