Prepare a CockroachDB Cloud BYOC Deployment in Azure

On this page

This page describes how to prepare a cloud service account to host a BYOC deployment of CockroachDB Cloud Advanced in Microsoft Azure.

Prerequisites

• Review the shared responsibility model for BYOC. Make sure you understand and acknowledge the responsibilities you hold for management of your cloud infrastucture and the necessary permissions you must grant to Cockroach Labs.

• Create a CockroachDB Cloud organization if you do not already have one.

• (Optional) Create an API service account to use the Cloud API with your Cloud organization.

• The BYOC deployment option is not available by default and must be requested. Reach out to your account team to express interest in BYOC.

• Review the Plan a CockroachDB Advanced Cluster documentation to plan your cluster sizing and resource allocation.

• Review cloud service regions supported by CockroachDB Cloud Advanced.

Step 1. Create a new Azure subscription

Provision a new Azure subscription with no existing infrastructure, dedicated to your Cockroach Cloud deployment. The account configuration for BYOC requires you to grant Cockroach Labs permissions to access and modify resources in this subscription, so this step is necessary to isolate these permissions from non-Cockroach Cloud resources. This subscription can be reused for multiple CockroachDB clusters.

Step 2. Set up the admin App Registration

When BYOC is enabled for your account, Cockroach Labs dynamically provisions a multi-tenant admin App Registration associated with your CockroachDB Cloud organization and provides you with a URL to grant tenant-wide admin consent to the application. Granting admin consent creates an admin Service Principal in your tenant, which is used by Cockroach Labs support to act on the Kubernetes cluster, running automation that initializes support infrastructure.

Visit this URL with a user account that is authorized to consent on behalf of your organization. Once the Cockroach Labs App Registration has been granted admin consent in the tenant, grant the following set of roles to the admin Service Principal:

• Role Based Access Control Administrator
• Azure Kubernetes Service Cluster User Role
• Azure Kubernetes Service Contributor Role
• Azure Kubernetes Service RBAC Cluster Admin
• Managed Identity Contributor
• Network Contributor
• Storage Account Contributor
• Storage Blob Data Contributor
• Virtual Machine Contributor
• A custom role, Resource Group Manager, with the following permissions:
  • Microsoft.Resources/subscriptions/resourceGroups/read
  • Microsoft.Resources/subscriptions/resourceGroups/write
  • Microsoft.Resources/subscriptions/resourceGroups/delete
  • Microsoft.Resources/subscriptions/resourceGroups/moveResources/action
  • Microsoft.Resources/subscriptions/resourceGroups/validateMoveResources/action
  • Microsoft.Resources/subscriptions/resourcegroups/deployments/read
  • Microsoft.Resources/subscriptions/resourcegroups/deployments/write
  • Microsoft.Resources/subscriptions/resourcegroups/resources/read
  • Microsoft.Resources/subscriptions/resourcegroups/deployments/operations/read
  • Microsoft.Resources/subscriptions/resourcegroups/deployments/operationstatuses/read

The custom Resource Group Manager role is required to create and manage resource groups in the subscription. This role is used instead of requesting the more broad Contributor role.

Step 3. Set up the reader App Registration

In addition to the admin application, Cockroach Labs provisions the CockroachDB Cloud BYOC Reader App Registration. This App Registration is used by Cockroach Labs support for read access to Kubernetes infrastructure.

This reader application also requires admin consent to deploy the reader Service Principal:

1. Log in to the Azure portal as a user with Global Administrator or Privileged Role Administrator permissions.

2. Open the following URL in your browser:

   https://login.microsoftonline.com/adminconsent?client_id=7f6538cb-f687-4411-9bbe-2f96bfbce028
   

   If you have multiple tenants, replace customer-tenant-id in the following URL with the tenant containing your newly-created Azure subscription:

   https://login.microsoftonline.com/<customer-tenant-id>/adminconsent?client_id=7f6538cb-f687-4411-9bbe-2f96bfbce028
   

3. Review the requested permissions and click Accept.

4. Once the CockroachDB Cloud BYOC Reader App Registration has been granted admin consent in the tenant, grant the following set of roles to the reader Service Principal:

   • Reader
   • Azure Kubernetes Service Cluster User
   • Azure Kubernetes Service RBAC Reader

Step 4. Grant persmissions to Entra groups with Azure Lighthouse

Use Azure Lighthouse to enable cross-tenant management that establishes the support infrastructure that allows Cockroach Labs to assist in the event of a support escalation. Permissions are granted least-privilege access and full visibility, allowing you to review and remove access at any time from the Azure portal.

This Azure Lighthouse deployment grants permissions to Cockroach Labs's managed tenant, which has a tenant ID of a4611215-941c-4f86-b53b-348514e57b45, by assigning the following roles to the reader and admin Entra groups within the tenant:

• Reader Entra group:
  • Reader
  • Azure Kubernetes Service Cluster User Role
• Admin Entra group:
  • Azure Kubernetes Service Contributor Role
  • Azure Kubernetes Service Cluster Admin
  • Managed Identity Contributor
  • Network Contributor
  • Storage Account Contributor
  • Virtual Machine Contributor

Follow these steps to enable secure, scoped access for Cockroach Labs to your subscription using Azure Lighthouse:

1. Save the following ARM template to a file named byoc-lighthouse.json:

   {
   "$schema": "https://schema.management.azure.com/schemas/2019-08-01/subscriptionDeploymentTemplate.json#",
   "contentVersion": "1.0.0.0",
   "parameters": {
     "mspOfferName": {
     "type": "string",
     "metadata": {
       "description": "Specify a unique name for your offer"
     },
     "defaultValue": "CockroachDB Cloud BYOC"
     },
     "mspOfferDescription": {
     "type": "string",
     "metadata": {
       "description": "Name of the Managed Service Provider offering"
     },
     "defaultValue": "Template for secure access to customer clusters in CockroachDB Cloud BYOC"
     }
   },
   "variables": {
     "mspRegistrationName": "[guid(parameters('mspOfferName'))]",
     "mspAssignmentName": "[guid(parameters('mspOfferName'))]",
     "managedByTenantId": "a4611215-941c-4f86-b53b-348514e57b45",
     "authorizations": [
     {
       "principalId": "c4139366-960c-431d-afad-29c65fd68087",
       "roleDefinitionId": "acdd72a7-3385-48ef-bd42-f606fba81ae7",
       "principalIdDisplayName": "CockroachDB Cloud BYOC Reader Entra Group"
     },
     {
       "principalId": "c4139366-960c-431d-afad-29c65fd68087",
       "roleDefinitionId": "4abbcc35-e782-43d8-92c5-2d3f1bd2253f",
       "principalIdDisplayName": "CockroachDB Cloud BYOC Reader Entra Group"
     },
     {
       "principalId": "6532a4f2-3fa1-4b10-a4c2-05368c87c89a",
       "roleDefinitionId": "ed7f3fbd-7b88-4dd4-9017-9adb7ce333f8",
       "principalIdDisplayName": "CockroachDB Cloud BYOC Admin Entra Group"
     },
     {
       "principalId": "6532a4f2-3fa1-4b10-a4c2-05368c87c89a",
       "roleDefinitionId": "0ab0b1a8-8aac-4efd-b8c2-3ee1fb270be8",
       "principalIdDisplayName": "CockroachDB Cloud BYOC Admin Entra Group"
     },
     {
       "principalId": "6532a4f2-3fa1-4b10-a4c2-05368c87c89a",
       "roleDefinitionId": "e40ec5ca-96e0-45a2-b4ff-59039f2c2b59",
       "principalIdDisplayName": "CockroachDB Cloud BYOC Admin Entra Group"
     },
     {
       "principalId": "6532a4f2-3fa1-4b10-a4c2-05368c87c89a",
       "roleDefinitionId": "4d97b98b-1d4f-4787-a291-c67834d212e7",
       "principalIdDisplayName": "CockroachDB Cloud BYOC Admin Entra Group"
     },
     {
       "principalId": "6532a4f2-3fa1-4b10-a4c2-05368c87c89a",
       "roleDefinitionId": "17d1049b-9a84-46fb-8f53-869881c3d3ab",
       "principalIdDisplayName": "CockroachDB Cloud BYOC Admin Entra Group"
     },
     {
       "principalId": "6532a4f2-3fa1-4b10-a4c2-05368c87c89a",
       "roleDefinitionId": "9980e02c-c2be-4d73-94e8-173b1dc7cf3c",
       "principalIdDisplayName": "CockroachDB Cloud BYOC Admin Entra Group"
     }
     ]
   },
   "resources": [
     {
     "type": "Microsoft.ManagedServices/registrationDefinitions",
     "apiVersion": "2022-10-01",
     "name": "[variables('mspRegistrationName')]",
     "properties": {
       "registrationDefinitionName": "[parameters('mspOfferName')]",
       "description": "[parameters('mspOfferDescription')]",
       "managedByTenantId": "[variables('managedByTenantId')]",
       "authorizations": "[variables('authorizations')]"
     }
     },
     {
     "type": "Microsoft.ManagedServices/registrationAssignments",
     "apiVersion": "2022-10-01",
     "name": "[variables('mspAssignmentName')]",
     "dependsOn": [
       "[resourceId('Microsoft.ManagedServices/registrationDefinitions/', variables('mspRegistrationName'))]"
     ],
     "properties": {
       "registrationDefinitionId": "[resourceId('Microsoft.ManagedServices/registrationDefinitions/', variables('mspRegistrationName'))]"
     }
     }
   ],
   "outputs": {
     "mspOfferName": {
     "type": "string",
     "value": "[concat('Managed by', ' ', parameters('mspOfferName'))]"
     },
     "authorizations": {
     "type": "array",
     "value": "[variables('authorizations')]"
     }
   }
   }
   

2. Deploy the template at the subscription scope using Azure CLI, Azure PowerShell, or Azure Portal. The following example command uses the Azure CLI:

   az deployment sub create \
     --name cockroach-byoc-lighthouse \
     --location <region> \
     --template-file byoc-lighthouse.json
   

Step 5. Register resource providers

Register the following resource providers in the Azure subscription:

• Microsoft.ContainerService
• Microsoft.ManagedIdentity
• Microsoft.Network
• Microsoft.Quota
• Microsoft.Storage

Step 6. Create the CockroachDB Cloud cluster

In BYOC deployments, CockroachDB clusters can be deployed in the Cloud Console or with the Cloud API.

Create a cluster with the Cloud Console

Follow these steps to create a CockroachDB cluster in the Cloud console:

1. Open the Cloud and select the organization that has been enabled for BYOC.
2. Click Create cluster
3. Under Select a plan, click **Advanced.
4. Under Cloud & Regions, click Bring Your Own Cloud and select Azure.
5. Under Cloud account, click Select your cloud account > Add new cloud account. Enter the tenant ID and subscription ID associated with your Azure subscription.
6. Follow the rest of the Create Cluster steps to configure your cluster's regions, capacity, and features as desired. Read the Plan a CockroachDB Advanced Cluster documentation for more details.

Create a cluster with the Cloud API

Send a POST request to the the /v1/clusters endpoint to create a CockroachDB Cloud Advanced cluster.

The following example request creates a 3-node Advanced cluster in the centralus region, specifying the subscription-id and customer-tenant-id associated with your Azure subscription:

curl --request POST \
  --url https://cockroachlabs.cloud/api/v1/clusters \
  --header "Authorization: Bearer {secret_key}" \
  --json '{
    "name":"byoc-azure-cluster-1",
    "provider": "AZURE",
    "plan": "ADVANCED",
    "spec": {
      "customer_cloud_account": {
        "azure": {
          "subscription_id": "{subscription-id}",
          "tenant_id": "{customer-tenant-id}"
        }
      },
      "dedicated": {
        "hardware": {
          "machine_spec": {
            "num_virtual_cpus": 4
          },
          "storage_gib": 16
        },
        "region_nodes": {
          "centralus": 3
        }
      }
    }
  }'

Next steps

• Connect to your cluster
• Manage your cluster using the Cloud API
• Prepare your deployment for production