Join a DxSqlAg Cluster to an Existing DxEnterprise Cluster

This guide explains a few different approaches for configuring the DxEnterprise containers in a DxSqlAg to join an existing DxEnterprise cluster somewhere else.

By default, a DxSqlAg creates a new, standalone DxEnterprise cluster made up only of its own pods. If you provide a joinTarget, those pods instead join an existing DxEnterprise cluster. This cluster may already include other Kubernetes pods, virtual machines, or bare-metal hosts.

Optionally, the pods can also join an existing Vhost and availability group, meaning that you can stretch your availability group across namespaces, Kubernetes clusters, off-site VMs, and more.

Prerequisites

These prerequisites apply to all four examples below. The examples also call out additional prerequisites.

• An installed and running DxOperator. For information about installing DxOperator, see the Installing DxOperator guide.

Critical Configuration Caveats

Joining a DxSqlAg to an existing DxEnterprise cluster will make persistent changes to the cluster configuration that will not automatically undo if you delete the DxSqlAg. In production, double-check these points before applying the DxSqlAg.

1. The DxSqlAg will always try to reconcile the DxEnterprise cluster to match what you defined in the spec. Some settings have default values:

   • spec.statefulSetSpec.podSpec.dxEnterpriseContainer.vhostName = VHOST1
   • spec.sqlAgConfiguration.availabilityGroupName = AG1

   When targeting an existing cluster:

   • If these resources already exist in the target cluster, the operator will add its pods to that Vhost and AG.
   • If those resources do not exist in the target cluster, the operator will create a new Vhost and AG.

   To avoid accidentally updating or creating the wrong Vhost/AG, explicitly set both values whenever you join an existing cluster.

2. The DxSqlAg DxEnterprise Secret (dxEnterpriseContainer.clusterSecret) must:

   • Have a DX_PASSKEY that matches the target cluster's passkey, and
   • If joinTarget.useNat: true, have a DX_OTPK that matches the target cluster's OTPK.

   Misconfiguring this Secret will cause join failures or prevent the operator from connecting to its pods after joining.

3. Every DxSqlAg pod name generated by the StatefulSet should be unique within the DxEnterprise cluster. No existing DxEnterprise node can share the same name. Reusing node names can cause cluster members to be removed, prevent new nodes from joining, and disrupt applications.

Join Scenarios

Because of the sheer number of configurations possible with this feature, only a handful of them are covered in this guide.

• Locally: When joining a new DxSqlAg into an existing DxSqlAg within the same Kubernetes cluster.
  • Uses two DxSqlAgs as an example, but a minor tweak (see step #3) makes it applicable to VM/bare-metal target clusters.
• External NAT: When joining a new DxSqlAg to an existing DxEnterprise cluster running on VMs or bare-metal.
  • Best suited for when the DxSqlAg pods and the target cluster are unreachable to each other.
• Per-Pod Load Balancers: When joining a new DxSqlAg to an existing DxSqlAg in a different Kubernetes cluster.
• Cluster Load Balancers: When joining a new DxSqlAg to an existing DxSqlAg in a different Kubernetes cluster.
  • This example has unique network requirements. See the section for more information.

Option 1: Join Locally

This example explains how to join a new DxSqlAg to an existing DxSqlAg running in the same Kubernetes cluster.

Prerequisites

• Top-level prerequisites.
• A deployed and running DxSqlAg. This DxSqlAg can be running in any namespace. For more information about creating a new DxSqlAg cluster, see the DxSqlAg Creation Guide.

Steps

1. Create a new namespace for the new DxSqlAg.

   kubectl create namespace test

2. In the new namespace, create secrets for DxEnterprise and SQL Server.

   1. Create a new DxEnterprise secret that contains your DxEnterprise cluster passkey and license key.

      kubectl create secret generic dxe -n test --from-literal=DX_PASSKEY=<pass> --from-literal=DX_LICENSE=<license_key>

   2. Create a new SQL Server secret that contains the SQL Server SA account password.

      kubectl create secret generic mssql -n test --from-literal=MSSQL_SA_PASSWORD=<pass>

3. Apply the DxSqlAg.yaml example below.

   Configuration

   • Ensure all prerequisites are met, including the top-level prerequisites.

   • The spec.statefulSetSpec.podSpec.dxEnterpriseContainer.joinTarget controls which existing DxEnterprise node this DxSqlAg will join.

     This example uses another DxSqlAg as its target, but joinTarget.target can be set to any DxEnterprise node that is reachable from within the DxEnterprise pods (for example, another DxSqlAg in a different namespace, or a DxEnterprise node running on a VM in another environment).

   • For a local join to another DxSqlAg in the same Kubernetes cluster, the joinTarget.target must point to a pod in the target DxSqlAg cluster.

     In the highlighted YAML example below, <target_name> and <target_namespace> must be the headless service name for a pod in the target cluster and the namespace that service/pod is running in, respectively. For example, a target pod service dxsqlag-a-0 in the default namespace would use:

     dxsqlag-a-0.default.svc.cluster.local

     Headless service name

     For a DxSqlAg pod, the name of the headless service is identical to the pod name.

   DxSqlAg.yaml

     apiVersion: dh2i.com/v1
     kind:  DxSqlAg
     metadata:
       name: dxsqlag-b
       namespace: test
     spec:
       sqlAgConfiguration:
         synchronousReplicas: 3
         asynchronousReplicas: 0
         configurationOnlyReplicas: 0       
         availabilityGroupName: AG1
       statefulSetSpec:
         podSpec:
           dxEnterpriseContainer:
             image: "docker.io/dh2i/dxe:latest"
             imagePullPolicy: Always
             acceptEula: true
             clusterSecret: dxe
             vhostName: VHOST1
             joinTarget:
               target: "<target_name>.<target_namespace>.svc.cluster.local"
           mssqlServerContainer:
             image: "mcr.microsoft.com/mssql/server:latest"
             imagePullPolicy: Always
             mssqlSecret: mssql
             acceptEula: true
             mssqlPID: Developer

4. Verify the pods are created.

   kubectl get pods -n test

5. (optional) After giving the cluster a little while to set up, verify the configuration is correct by running dxcli get-ags-detail.

   kubectl exec -it -n test -c dxe dxsqlag-b-0 -- dxcli get-ags-detail "VHOST1" "AG1"

Option 2: Join with NAT Matchmaking

This second example explains how to join a DxSqlAg to an existing VM cluster running outside of Kubernetes using the NAT join features of DxEnterprise.

Prerequisites

• Top-level prerequisites.

• A valid DxEnterprise license with NAT and tunnels enabled. A fully featured Developer Edition is available free for non-production use. To purchase DxEnterprise software for production workloads, please contact us.

• Permissive NATs for at least one - but preferably both - clusters. You can test your NAT type by visiting the DH2i NAT test page.

• A pre-existing DxEnterprise cluster.

  Setting up a Cluster

  The cloud NAT matchmaker provides a mechanism for peers (DxEnterprise cluster nodes) to find each other, and is used to join nodes together and establish connections. This is helpful in situations where it might be very difficult for the nodes to find each other, such as when they are on different networks. More information about setting up a cluster can be found below:

  • VMs and bare-metal: See the Linux or Windows quick start guides.
    • If desired, set up an AG in this cluster using the DxCLI or DxAdmin guides.
  • Kubernetes: See the DxOperator quick start guide or the Dxemssql StatefulSet guide.
  • Docker: See the Docker quick start guide.

Steps

1. If not done already, set a cluster passkey on the existing (VM/bare-metal/Kubernetes/Docker) DxEnterprise cluster.

   dxcli cluster-set-secret-ex <pass>

2. Set a One-Time Passkey on the existing cluster and copy the OTPK.

   dxcli set-otpk

3. Create secrets for DxEnterprise and SQL Server.

   1. Create a new DxEnterprise secret that contains your DxEnterprise cluster passkey, license key, and OTPK.

      kubectl create secret generic dxe --from-literal=DX_PASSKEY=<pass> --from-literal=DX_LICENSE=<license_key> --from-literal=DX_OTPK=<otpk>

   2. Create a new SQL Server secret that contains the SQL Server SA account password.

      kubectl create secret generic mssql --from-literal=MSSQL_SA_PASSWORD=<pass>

4. Apply the DxSqlAg.yaml example below.

   Configuration

   • Ensure all prerequisites are met, including the top-level prerequisites.
   • Set the spec.statefulSetSpec.podSpec.dxEnterpriseContainer.joinTarget values:
     • target: match.dh2i.com
     • useNat: true

   These values tell the DxEnterprise containers to use the DH2i NAT matchmaker to find the other cluster.

   DxSqlAg.yaml

   apiVersion: dh2i.com/v1
   kind:  DxSqlAg
   metadata:
     name: dxsqlag
   spec:
     sqlAgConfiguration:
       synchronousReplicas: 3
       asynchronousReplicas: 0
       configurationOnlyReplicas: 0
       availabilityGroupName: AG1
     statefulSetSpec:
       podSpec:
         dxEnterpriseContainer:
           image: "docker.io/dh2i/dxe:latest"
           imagePullPolicy: Always
           acceptEula: true
           clusterSecret: dxe
           vhostName: VHOST1
           joinTarget:
             target: "match.dh2i.com"
             useNat: true
         mssqlServerContainer:
           image: "mcr.microsoft.com/mssql/server:latest"
           imagePullPolicy: Always
           mssqlSecret: mssql
           acceptEula: true
           mssqlPID: Developer

5. Verify the pods are created.

   kubectl get pods

6. (optional) After giving the cluster a little while to set up, verify that the pods have joined the cluster by running dxcli get-cluster-nodes

   kubectl exec -it -c dxe dxsqlag-0 -- dxcli get-cluster-nodes

7. (optional) Verify the configuration is correct by running dxcli get-ags-detail.

   kubectl exec -it -c dxe dxsqlag-0 -- dxcli get-ags-detail "VHOST1" "AG1"

Option 3: Join Across Kubernetes Clusters with Per-Pod Load Balancers

In this third example, you will create load balancers for each pod in two different Kubernetes clusters, then join the two DxEnterprise clusters together using those load balancers. Use this example when you need per-node IPs or firewall rules.

Prerequisites

• Top-level prerequisites.
• Two Kubernetes clusters with DxOperator installed.

Steps

1. In the first Kubernetes cluster, create secrets for DxEnterprise and SQL Server.

   1. Create a new DxEnterprise secret that contains your DxEnterprise cluster passkey and license key.

      kubectl create secret generic dxe --from-literal=DX_PASSKEY=<pass> --from-literal=DX_LICENSE=<license_key>

   2. Create a new SQL Server secret that contains the SQL Server SA account password.

      kubectl create secret generic mssql --from-literal=MSSQL_SA_PASSWORD=<pass>

2. For each pod in the first Kubernetes cluster, create a load balancer with a static IP and a selector for the pod. These load balancers let the DxEnterprise cluster nodes (pods) find each other across the network. An example configuration with the necessary ports is given below.

   Kubernetes load balancer implementation

   Kubernetes load balancers are implemented by outside software stacks and vendors, so the details of assigning a load balancer a static IP will vary. Cloud providers like Azure, AWS, and Google Cloud have documentation on how to assign static IPs to load balancers within their own infrastructure.

   warning

   Ports mapped to a load balancer within Kubernetes will be accessible to the external network, which in some configurations may be a public network.

   lbs-a.yaml

   apiVersion: v1
   kind: Service
   metadata:
     name: dxsqlag-a-0-clb
     # Static IPs are often assigned through annotations or the spec.loadBalancerIP. 
     # The annotations below are given for example only from Azure documentation.
     #annotations:
     #  service.beta.kubernetes.io/azure-load-balancer-resource-group: <node resource group name>
     #  service.beta.kubernetes.io/azure-pip-name: myAKSIP1
     #  metallb.universe.tf/loadBalancerIPs: <ip_address>
   spec:
     ports:
     - port: 7980
       targetPort: 7980
       protocol: TCP
       name: join-tcp
     - port: 7981
       targetPort: 7981
       protocol: UDP
       name: group-udp
     # Optional external access to SQL Server and DxAdmin
     - port: 1433
       targetPort: 1433
       protocol: TCP
       name: mssql
     - port: 7979
       targetPort: 7979
       protocol: TCP
       name: dxadmin
     selector:
       statefulset.kubernetes.io/pod-name: dxsqlag-a-0
     type: LoadBalancer
   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: dxsqlag-a-1-clb
     #annotations:
     #  service.beta.kubernetes.io/azure-load-balancer-resource-group: <node resource group name>
     #  service.beta.kubernetes.io/azure-pip-name: myAKSIP2
     #  metallb.universe.tf/loadBalancerIPs: <ip_address>
   spec:
     ports:
     - port: 7980
       targetPort: 7980
       protocol: TCP
       name: join-tcp
     - port: 7981
       targetPort: 7981
       protocol: UDP
       name: group-udp
     # Optional external access to SQL Server and DxAdmin
     - port: 1433
       targetPort: 1433
       protocol: TCP
       name: mssql
     - port: 7979
       targetPort: 7979
       protocol: TCP
       name: dxadmin
     selector:
       statefulset.kubernetes.io/pod-name: dxsqlag-a-1
     type: LoadBalancer

3. Create Services on the A-side to map the names of B-side pods to their external IPs. These must be set to the static IPs allocated to the external load balancers in the B-side cluster below

   hostmap-a.yaml

   apiVersion: v1
   kind: Service
   metadata:
     name: dxsqlag-b-0
   spec:
     clusterIP: None
     ports:
     - name: dxcmonitor-tcp
       protocol: TCP
       port: 7980
     - name: dxcmonitor-udp
       protocol: UDP
       port: 7981
   ---
   apiVersion: discovery.k8s.io/v1
   kind: EndpointSlice
   metadata:
     name: dxsqlag-b-0-ip
     labels:
       kubernetes.io/service-name: dxsqlag-b-0
   addressType: IPv4
   ports:
   - name: dxcmonitor-tcp
     port: 7980
   - name: dxcmonitor-udp
     port: 7981
     protocol: UDP
   endpoints:
   - addresses: [ <dxsqlag-b-0_lb_ip> ]
   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: dxsqlag-b-1
   spec:
     clusterIP: None
     ports:
     - name: dxcmonitor-tcp
       protocol: TCP
       port: 7980
     - name: dxcmonitor-udp
       protocol: UDP
       port: 7981
   ---
   apiVersion: discovery.k8s.io/v1
   kind: EndpointSlice
   metadata:
     name: dxsqlag-b-1-ip
     labels:
       kubernetes.io/service-name: dxsqlag-b-1
   addressType: IPv4
   ports:
   - name: dxcmonitor-tcp
     port: 7980
   - name: dxcmonitor-udp
     port: 7981
     protocol: UDP
   endpoints:
   - addresses: [ <dxsqlag-b-1_lb_ip> ]

4. Use the example DxSqlAg YAML below to deploy the pods in the first Kubernetes cluster.

   Configuration

   • Ensure all prerequisites are met, including the top-level prerequisites.
   • The spec.statefulSetSpec.podSpec.dxEnterpriseContainer.joinTarget is not present because this is a new cluster.
   DxSqlAg-a.yaml

   apiVersion: dh2i.com/v1
   kind:  DxSqlAg
   metadata:
     name: dxsqlag-a
   spec:
     sqlAgConfiguration:
       synchronousReplicas: 2
       asynchronousReplicas: 0
       configurationOnlyReplicas: 0   
       availabilityGroupName: AG1
     statefulSetSpec:
       podSpec:
         dxEnterpriseContainer:
           image: "docker.io/dh2i/dxe:latest"
           imagePullPolicy: Always
           acceptEula: true
           clusterSecret: dxe
           vhostName: VHOST1
         mssqlServerContainer:
           image: "mcr.microsoft.com/mssql/server:latest"
           imagePullPolicy: Always
           mssqlSecret: mssql
           acceptEula: true
           mssqlPID: Developer

5. Switch to the second cluster and repeat step #1.

6. For each pod in the second Kubernetes cluster, create a load balancer with a static IP. Example load balancers are given below.

   warning

   Ports mapped to a load balancer within Kubernetes will be accessible to the external network, which in some configurations may be a public network.

   lbs-b.yaml

   apiVersion: v1
   kind: Service
   metadata:
     name: dxsqlag-b-0-clb
     #annotations:
     #  service.beta.kubernetes.io/azure-load-balancer-resource-group: <node resource group name>
     #  service.beta.kubernetes.io/azure-pip-name: myAKSIP3
     #  metallb.universe.tf/loadBalancerIPs: <ip_address>
   spec:
     ports:
     - port: 7980
       targetPort: 7980
       protocol: TCP
       name: join-tcp
     - port: 7981
       targetPort: 7981
       protocol: UDP
       name: group-udp
     # Optional external access to SQL Server and DxAdmin
     - port: 1433
       targetPort: 1433
       protocol: TCP
       name: mssql
     - port: 7979
       targetPort: 7979
       protocol: TCP
       name: dxadmin
     selector:
       statefulset.kubernetes.io/pod-name: dxsqlag-b-0
     type: LoadBalancer
   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: dxsqlag-b-1-clb
     #annotations:
     #  service.beta.kubernetes.io/azure-load-balancer-resource-group: <node resource group name>
     #  service.beta.kubernetes.io/azure-pip-name: myAKSIP4
     #  metallb.universe.tf/loadBalancerIPs: <ip_address>
   spec:
     ports:
     - port: 7980
       targetPort: 7980
       protocol: TCP
       name: join-tcp
     - port: 7981
       targetPort: 7981
       protocol: UDP
       name: group-udp
     # Optional external access to SQL Server and DxAdmin
     - port: 1433
       targetPort: 1433
       protocol: TCP
       name: mssql
     - port: 7979
       targetPort: 7979
       protocol: TCP
       name: dxadmin
     selector:
       statefulset.kubernetes.io/pod-name: dxsqlag-b-1
     type: LoadBalancer

7. Create Services on the B-side to map the names of A-side pods to their external IPs. These must be set to the static IPs allocated to the external load balancers in the A-side cluster above

   hostmap-b.yaml

   apiVersion: v1
   kind: Service
   metadata:
     name: dxsqlag-a-0
   spec:
     clusterIP: None
     ports:
     - name: dxcmonitor-tcp
       protocol: TCP
       port: 7980
     - name: dxcmonitor-udp
       protocol: UDP
       port: 7981
   ---
   apiVersion: discovery.k8s.io/v1
   kind: EndpointSlice
   metadata:
     name: dxsqlag-a-0-ip
     labels:
       kubernetes.io/service-name: dxsqlag-a-0
   addressType: IPv4
   ports:
   - name: dxcmonitor-tcp
     port: 7980
   - name: dxcmonitor-udp
     port: 7981
     protocol: UDP
   endpoints:
   - addresses: [ <dxsqlag-a-0_lb_ip> ]
   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: dxsqlag-a-1
   spec:
     clusterIP: None
     ports:
     - name: dxcmonitor-tcp
       protocol: TCP
       port: 7980
     - name: dxcmonitor-udp
       protocol: UDP
       port: 7981
   ---
   apiVersion: discovery.k8s.io/v1
   kind: EndpointSlice
   metadata:
     name: dxsqlag-a-1-ip
     labels:
       kubernetes.io/service-name: dxsqlag-a-1
   addressType: IPv4
   ports:
   - name: dxcmonitor-tcp
     port: 7980
   - name: dxcmonitor-udp
     port: 7981
     protocol: UDP
   endpoints:
   - addresses: [ <dxsqlag-a-1_lb_ip> ]

8. Use the example DxSqlAg YAML below to deploy the pods in the second Kubernetes cluster.

   Configuration

   • Ensure all prerequisites are met, including the top-level prerequisites.
   • The spec.statefulSetSpec.podSpec.dxEnterpriseContainer.joinTarget.target must be the static IP of one of the load balancers in the first cluster.
   DxSqlAg-b.yaml

   apiVersion: dh2i.com/v1
   kind:  DxSqlAg
   metadata:
     name: dxsqlag-b
   spec:
     sqlAgConfiguration:
       synchronousReplicas: 2
       asynchronousReplicas: 0
       configurationOnlyReplicas: 0
       availabilityGroupName: AG1
     statefulSetSpec:
       podSpec:
         dxEnterpriseContainer:
           image: "docker.io/dh2i/dxe:latest"
           imagePullPolicy: Always
           acceptEula: true
           clusterSecret: dxe
           vhostName: VHOST1
           joinTarget:
             target: <target_lb_ip>
         mssqlServerContainer:
           image: "mcr.microsoft.com/mssql/server:latest"
           imagePullPolicy: Always
           mssqlSecret: mssql
           acceptEula: true
           mssqlPID: Developer

9. (optional) After giving the cluster a little while to set up, verify that the pods have joined the cluster by running dxcli get-cluster-nodes

   kubectl exec -it -c dxe dxsqlag-b-0 -- dxcli get-cluster-nodes

10. (optional) Verify the configuration is correct by running dxcli get-ags-detail.

    kubectl exec -it -c dxe dxsqlag-b-0 -- dxcli get-ags-detail "VHOST1" "AG1"

Option 4: Join Across Kubernetes Clusters with a Shared Load Balancer

In this fourth and final example, you will create a load balancer in two different Kubernetes clusters, then join the two DxEnterprise clusters together using those load balancers.

Requires Permissive NAT Types

This example uses fewer IPs and simpler infrastructure, but both Kubernetes clusters must have permissive NAT types. Without this, joins might fail or DxEnterprise nodes may lose their connections to each other.

See our NAT Types article for more information.

Additional Prerequisites

• Top-level prerequisites.
• Two Kubernetes clusters with DxOperator installed.

Steps

1. In the first Kubernetes cluster, create secrets for DxEnterprise and SQL Server.

   1. Create a new DxEnterprise secret that contains your DxEnterprise cluster passkey and license key.

      kubectl create secret generic dxe --from-literal=DX_PASSKEY=<pass> --from-literal=DX_LICENSE=<license_key>

   2. Create a new SQL Server secret that contains the SQL Server SA account password.

      kubectl create secret generic mssql --from-literal=MSSQL_SA_PASSWORD=<pass>

2. Create a load balancer with a static IP, an externalTrafficPolicy set to local, and a selector for the entire DxSqlAg cluster. The load balancer lets the DxEnterprise cluster nodes (pods) find each other across the network. An example configuration with the necessary ports is given below.

   Kubernetes load balancer implementation

   Kubernetes load balancers are implemented by outside software stacks and vendors, so the details of assigning a load balancer a static IP will vary. Cloud providers like Azure, AWS, and Google Cloud have documentation on how to assign static IPs to load balancers within their own infrastructure.

   lb-a.yaml

   apiVersion: v1
   kind: Service
   metadata:
     name: dxsqlag-a-clb
     # Static IPs are often assigned through annotations or the spec.loadBalancerIP. 
     # The annotations below are given for example only from Azure documentation.
     #annotations:
     #  service.beta.kubernetes.io/azure-load-balancer-resource-group: <node resource group name>
     #  service.beta.kubernetes.io/azure-pip-name: myAKSIP1
     #  metallb.universe.tf/loadBalancerIPs: <ip_address>
   spec:
     ports:
     - port: 7980
       targetPort: 7980
       protocol: TCP
       name: join-tcp
     - port: 7981
       targetPort: 7981
       protocol: UDP
       name: group-udp
     selector:
       dh2i.com/entity-name: dxsqlag-a
     type: LoadBalancer
     externalTrafficPolicy: Local

3. Create Services on the A-side to map the names of B-side pods to the external IP. These must be set to the single static IP allocated to the external load balancer in the B-side cluster below

   hostmap-a.yaml

   apiVersion: v1
   kind: Service
   metadata:
     name: dxsqlag-b-0
   spec:
     clusterIP: None
     ports:
     - name: dxcmonitor-tcp
       protocol: TCP
       port: 7980
     - name: dxcmonitor-udp
       protocol: UDP
       port: 7981
   ---
   apiVersion: discovery.k8s.io/v1
   kind: EndpointSlice
   metadata:
     name: dxsqlag-b-0-ip
     labels:
       kubernetes.io/service-name: dxsqlag-b-0
   addressType: IPv4
   ports:
   - name: dxcmonitor-tcp
     port: 7980
   - name: dxcmonitor-udp
     port: 7981
     protocol: UDP
   endpoints:
   - addresses: [ <dxsqlag-b_lb_ip> ]
   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: dxsqlag-b-1
   spec:
     clusterIP: None
     ports:
     - name: dxcmonitor-tcp
       protocol: TCP
       port: 7980
     - name: dxcmonitor-udp
       protocol: UDP
       port: 7981
   ---
   apiVersion: discovery.k8s.io/v1
   kind: EndpointSlice
   metadata:
     name: dxsqlag-b-1-ip
     labels:
       kubernetes.io/service-name: dxsqlag-b-1
   addressType: IPv4
   ports:
   - name: dxcmonitor-tcp
     port: 7980
   - name: dxcmonitor-udp
     port: 7981
     protocol: UDP
   endpoints:
   - addresses: [ <dxsqlag-b_lb_ip> ]

4. Use the example DxSqlAg YAML below to deploy the pods in the first Kubernetes cluster. Note the following values:

   Configuration

   • Ensure all prerequisites are met, including the top-level prerequisites.
   • The spec.statefulSetSpec.podSpec.dxEnterpriseContainer.joinTarget is not present because this is a new cluster.
   DxSqlAg-a.yaml

   apiVersion: dh2i.com/v1
   kind:  DxSqlAg
   metadata:
     name: dxsqlag-a
   spec:
     sqlAgConfiguration:
       synchronousReplicas: 2
       asynchronousReplicas: 0
       configurationOnlyReplicas: 0   
       availabilityGroupName: AG1
     statefulSetSpec:
       podSpec:
         dxEnterpriseContainer:
           image: "docker.io/dh2i/dxe:latest"
           imagePullPolicy: Always
           acceptEula: true
           clusterSecret: dxe
           vhostName: VHOST1
         mssqlServerContainer:
           image: "mcr.microsoft.com/mssql/server:latest"
           imagePullPolicy: Always
           mssqlSecret: mssql
           acceptEula: true
           mssqlPID: Developer

5. Switch to the second cluster and repeat step #1.

6. Create a load balancer with a static IP, an externalTrafficPolicy set to local, and a selector for the entire DxSqlAg cluster. Example load balancers are given below.

   lb-b.yaml

   apiVersion: v1
   kind: Service
   metadata:
     name: dxsqlag-b-clb
     #annotations:
     #  service.beta.kubernetes.io/azure-load-balancer-resource-group: <node resource group name>
     #  service.beta.kubernetes.io/azure-pip-name: myAKSIP3
     #  metallb.universe.tf/loadBalancerIPs: <ip_address>
   spec:
     ports:
     - port: 7980
       targetPort: 7980
       protocol: TCP
       name: join-tcp
     - port: 7981
       targetPort: 7981
       protocol: UDP
       name: group-udp
     selector:
       dh2i.com/entity-name: dxsqlag-b
     type: LoadBalancer
     externalTrafficPolicy: Local

7. Create Services on the A-side to map the names of B-side pods to the single external IP. These must be set to the static IP allocated to the one external load balancer in the B-side cluster below.

   hostmap-b.yaml

   apiVersion: v1
   kind: Service
   metadata:
     name: dxsqlag-a-0
   spec:
     clusterIP: None
     ports:
     - name: dxcmonitor-tcp
       protocol: TCP
       port: 7980
     - name: dxcmonitor-udp
       protocol: UDP
       port: 7981
   ---
   apiVersion: discovery.k8s.io/v1
   kind: EndpointSlice
   metadata:
     name: dxsqlag-a-0-ip
     labels:
       kubernetes.io/service-name: dxsqlag-a-0
   addressType: IPv4
   ports:
   - name: dxcmonitor-tcp
     port: 7980
   - name: dxcmonitor-udp
     port: 7981
     protocol: UDP
   endpoints:
   - addresses: [ <dxsqlag-a_lb_ip> ]
   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: dxsqlag-a-1
   spec:
     clusterIP: None
     ports:
     - name: dxcmonitor-tcp
       protocol: TCP
       port: 7980
     - name: dxcmonitor-udp
       protocol: UDP
       port: 7981
   ---
   apiVersion: discovery.k8s.io/v1
   kind: EndpointSlice
   metadata:
     name: dxsqlag-a-1-ip
     labels:
       kubernetes.io/service-name: dxsqlag-a-1
   addressType: IPv4
   ports:
   - name: dxcmonitor-tcp
     port: 7980
   - name: dxcmonitor-udp
     port: 7981
     protocol: UDP
   endpoints:
   - addresses: [ <dxsqlag-a_lb_ip> ]

8. Use the example DxSqlAg YAML below to deploy the pods in the second Kubernetes cluster.

   Configuration

   • Ensure all prerequisites are met, including the top-level prerequisites.
   • The spec.statefulSetSpec.podSpec.dxEnterpriseContainer.joinTarget.target must be the static IP the load balancer in the first cluster.
   DxSqlAg-b.yaml

   apiVersion: dh2i.com/v1
   kind:  DxSqlAg
   metadata:
     name: dxsqlag-b
   spec:
     sqlAgConfiguration:
       synchronousReplicas: 2
       asynchronousReplicas: 0
       configurationOnlyReplicas: 0
       availabilityGroupName: AG1
     statefulSetSpec:
       podSpec:
         dxEnterpriseContainer:
           image: "docker.io/dh2i/dxe:latest"
           imagePullPolicy: Always
           acceptEula: true
           clusterSecret: dxe
           vhostName: VHOST1
           joinTarget:
             target: <target_lb_ip>
         mssqlServerContainer:
           image: "mcr.microsoft.com/mssql/server:latest"
           imagePullPolicy: Always
           mssqlSecret: mssql
           acceptEula: true
           mssqlPID: Developer

9. (optional) After giving the cluster a little while to set up, verify that the pods have joined the cluster by running dxcli get-cluster-nodes

   kubectl exec -it -c dxe dxsqlag-b-0 -- dxcli get-cluster-nodes

10. (optional) Verify the configuration is correct by running dxcli get-ags-detail.

    kubectl exec -it -c dxe dxsqlag-b-0 -- dxcli get-ags-detail "VHOST1" "AG1"

Additional Information

• DxEnterprise Admin Guide
• DxEnterprise in Kubernetes Guides
• DxOperator API Reference
• SQL Server Availability Groups on Kubernetes
• Kubernetes Concepts - Services, Load Balancing, and Networking