Join a DxEnterpriseSqlAg to an Existing Cluster

The DxEnterpriseSqlAg custom resource has two cluster join modes: the default mode creates a brand new DxEnterprise cluster using only the pods, while the second mode allows the pods to join a preexisting DxEnterprise cluster running almost anywhere. Additionally, the pods may also join an existing Vhost and availability group, meaning that you can stretch your availability group across namespaces, Kubernetes clusters, off-site VMs, and more.

info

Because of the sheer number of configurations possible with this feature, only a handful of them will be covered in this guide.

Prerequisites

• An installed and running DxOperator. For information about installing DxOperator, see the Installing DxOperator section of the DxOperator guide.

Join Within a Single Kubernetes Cluster

This first example covers joining another DxEnterprise/DxEnterpriseSqlAg cluster already running in Kubernetes.

Additional Prerequisites

• A deployed and running DxEnterpriseSqlAg with a unique name. This DxEnterpriseSqlAg can be running in any namespace. For more information about creating a new DxEnterpriseSqlAg cluster, see the DxOperator Guide.

Steps

1. Create a new namespace.

   kubectl create namespace test

2. Create secrets for the DxEnterprise passkey and license, and the SQL Server SA account in the new namespace.

   Passkeys must match

   The DX_PASSKEY environment variable must match the cluster passkey of the existing cluster so the join can succeed and the operator can manage the cluster nodes.

   kubectl create secret generic dxe -n test --from-literal=DX_PASSKEY=<pass> --from-literal=DX_LICENSE=<license_key>

   kubectl create secret generic mssql -n test --from-literal=MSSQL_SA_PASSWORD=<pass>

3. Use the example DxEnterpriseSqlAg YAML below. Note that the following values should be changed:

   • The spec.template.spec.dxEnterpriseContainer.joinTarget.target needs to point to a pod in the target cluster. The <target_name> and <target_namespace> should be the name of the headless service for a pod in the target cluster and the namespace the service/pod is running in, respectively. For example, a target pod service dxesqlag-0 and default namespace would be dxesqlag-0.default.svc.cluster.local.

     headless service name

     The name of the headless service for a DxEnterpriseSqlAg pod is identical to the pod name.

   • The spec.availabilityGroupName and spec.template.spec.dxEnterpriseContainer.vhostName should match the names of the existing AG and Vhost (respectively) if you want these new pods to join the existing AG.

   • The name of the DxEnterpriseSqlAg should be unique to the Kubernetes cluster since the pod name is only based off the custom resource name, not the namespace.

     DxEnterpriseSqlAg.yaml

       apiVersion: dh2i.com/v1
       kind:  DxEnterpriseSqlAg
       metadata:
         name: test-dxesqlag
         namespace: test
       spec:
         synchronousReplicas: 3
         asynchronousReplicas: 0
         configurationOnlyReplicas: 0       
         availabilityGroupName: AG1
         template:
           spec:
             dxEnterpriseContainer:
               image: "docker.io/dh2i/dxe:latest"
               imagePullPolicy: Always
               acceptEula: true
               clusterSecret: dxe
               vhostName: VHOST1
               joinTarget:
                 target: "<target_name>.<target_namespace>.svc.cluster.local"
             mssqlServerContainer:
               image: "mcr.microsoft.com/mssql/server:latest"
               imagePullPolicy: Always
               mssqlSecret: mssql
               acceptEula: true
               mssqlPID: Developer

4. Verify the pods are created.

   kubectl get pods

5. (optional) After giving the cluster a little while to setup, verify the configuration is correct by running dxcli get-ags-detail.

   kubectl exec -itn test test-dxesqlag-0 -- dxcli get-ags-detail <vhost_name> <ag_name>

Join with NAT Matchmaking

In the second example, you will join the DxEnterpriseSqlAg to an existing VM cluster running outside of Kubernetes using the NAT join features of DxEnterprise.

Additional Prerequisites

• A pre-existing DxEnterprise cluster.

  setting up a cluster

  The cloud NAT matchmaker provides a mechanism for peers (DxEnterprise cluster nodes) to find each other, and is used to join nodes together and establish connections. This is helpful in situations where it might be very difficult for the nodes to find each other, such as when they are on different networks. More information about setting up a cluster can be found below:

  • VMs and bare-metal: See the Linux or Windows quick start guides.
    • If desired, setup an AG in this cluster using the DxCLI or DxAdmin guides.
  • Kubernetes: See the DxOperator quick start guide or the Dxemssql StatefulSet guide.
  • Docker: Dee the Docker quick start guide.

• A valid DxEnterprise license with NAT and tunnels enabled. A fully featured Developer Edition is available free for non-production use. To purchase DxEnterprise software for production workloads, visit the DH2i Store.

• Permissive NATs for at least one - but preferably both - clusters. You can test your NAT type by visiting the DH2i NAT test page.

Steps

1. If not done already, set a cluster passkey on the existing (VM/bare-metal/Kubernetes/Docker) DxEnterprise cluster.

   dxcli cluster-set-secret-ex <pass>

2. Set a One-Time Passkey on the existing cluster and copy the OTPK.

   dxcli set-otpk

3. Create secrets for the DxEnterprise passkey, license, and OTPK, and the SQL Server SA account.

   Passkeys must match

   The DX_PASSKEY and DX_OTPK environment variables below must match the cluster passkey and OTPK of the existing cluster (steps #1 and #2) so the operator can join and manage the cluster nodes running in Kubernetes.

   kubectl create secret generic dxe --from-literal=DX_PASSKEY=<pass> --from-literal=DX_LICENSE=<license_key> --from-literal=DX_OTPK=<otpk>

   kubectl create secret generic mssql --from-literal=MSSQL_SA_PASSWORD=<pass>

4. Use the example DxEnterpriseSqlAg YAML below. Note the following values:

   • The spec.template.spec.dxEnterpriseContainer.joinTarget.target has been changed to match.dh2i.com. This tells the DxEnterprise containers to use the NAT matchmaker to find the other cluster.

   • The spec.availabilityGroupName and spec.template.spec.dxEnterpriseContainer.vhostName should match the names of the existing AG and Vhost (respectively) if you want these new pods to join an existing AG.

     DxEnterpriseSqlAg.yaml

     apiVersion: dh2i.com/v1
     kind:  DxEnterpriseSqlAg
     metadata:
       name: dxesqlag
     spec:
       synchronousReplicas: 3
       asynchronousReplicas: 0
       configurationOnlyReplicas: 0
       availabilityGroupName: AG1
       template:
         spec:
           dxEnterpriseContainer:
             image: "docker.io/dh2i/dxe:latest"
             imagePullPolicy: Always
             acceptEula: true
             clusterSecret: dxe
             vhostName: VHOST1
             joinTarget:
               target: "match.dh2i.com"
               useNat: true
           mssqlServerContainer:
             image: "mcr.microsoft.com/mssql/server:latest"
             imagePullPolicy: Always
             mssqlSecret: mssql
             acceptEula: true
             mssqlPID: Developer

5. Verify the pods are created.

   kubectl get pods

6. (optional) After giving the cluster a little while to setup, verify that the pods have joined the cluster by running dxcli get-cluster-nodes

   kubectl exec -it dxesqlag-0 -- dxcli get-cluster-nodes

7. (optional) Verify the configuration is correct by running dxcli get-ags-detail.

   kubectl exec -it dxesqlag-0 -- dxcli get-ags-detail <vhost_name> <ag_name>

Join Across Kubernetes Clusters using Load Balancers with a Pod Selector

In the third example, you will create load balancers for each pod in two different Kubernetes clusters, then join the two DxEnterprise clusters together using those load balancers.

Additional Prerequisites

• Two Kubernetes clusters with DxOperator installed.

Steps

1. In the first Kubernetes cluster, create secrets for the DxEnterprise passkey and license, and the SQL Server SA account.

   kubectl create secret generic dxe --from-literal=DX_PASSKEY=<pass> --from-literal=DX_LICENSE=<license_key>

   kubectl create secret generic mssql --from-literal=MSSQL_SA_PASSWORD=<pass>

2. For each pod in the first Kubernetes cluster, create a load balancer with a static IP and a selector for the pod. These load balancers let the DxEnterprise cluster nodes (pods) find each other across the network. An example configuration with the necessary ports is given below.

   Kubernetes load balancer implementation

   Kubernetes load balancers are implemented by outside software stacks and vendors, so the details of assigning a load balancer a static IP will vary. Cloud providers like Azure, AWS, and Google Cloud have documentation on how to assign static IPs to load balancers within their own infrastructure.

   warning

   Ports mapped to a load balancer within Kubernetes will be accessible to the external network, which in some configurations may be a public network.

   apiVersion: v1
   kind: Service
   metadata:
     name: dxesqlag-a-0-clb
     # Static IPs are often assigned through annotations or the spec.loadBalancerIP. 
     # The annotation below are given for example only from Azure documentation.
     #annotations:
     #  service.beta.kubernetes.io/azure-load-balancer-resource-group: <node resource group name>
     #  service.beta.kubernetes.io/azure-pip-name: myAKSIP1
   spec:
     ports:
     - port: 7980
       targetPort: 7980
       protocol: TCP
       name: join-tcp
     - port: 7981
       targetPort: 7981
       protocol: UDP
       name: group-udp
     # Optional external access to SQL Server and DxAdmin
     - port: 1433
       targetPort: 1433
       protocol: TCP
       name: mssql
     - port: 7979
       targetPort: 7979
       protocol: TCP
       name: dxadmin
     selector:
       dh2i.com/name: dxesqlag-a-0
     type: LoadBalancer
   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: dxesqlag-a-1-clb
     #annotations:
     #  service.beta.kubernetes.io/azure-load-balancer-resource-group: <node resource group name>
     #  service.beta.kubernetes.io/azure-pip-name: myAKSIP2
   spec:
     ports:
     - port: 7980
       targetPort: 7980
       protocol: TCP
       name: join-tcp
     - port: 7981
       targetPort: 7981
       protocol: UDP
       name: group-udp
     # Optional external access to SQL Server and DxAdmin
     - port: 1433
       targetPort: 1433
       protocol: TCP
       name: mssql
     - port: 7979
       targetPort: 7979
       protocol: TCP
       name: dxadmin
     selector:
       dh2i.com/name: dxesqlag-a-1
     type: LoadBalancer

3. Create Services on the A-side to map the names of B-side pods to their external IPs. These must be set to the static IPs allocated to the external load balancers in the B-side cluster below

   hostmap-a.yaml

   apiVersion: v1
   kind: Service
   metadata:
     name: dxesqlag-b-0
   spec:
     clusterIP: None
     ports:
     - name: dxcmonitor-tcp
       protocol: TCP
       port: 7980
     - name: dxcmonitor-udp
       protocol: UDP
       port: 7981
   ---
   apiVersion: discovery.k8s.io/v1
   kind: EndpointSlice
   metadata:
     name: dxesqlag-b-0-ip
     labels:
       dh2i.com/name: dxesqlag-b-0
   addressType: IPv4
   ports:
   - name: dxcmonitor-tcp
     port: 7980
   - name: dxcmonitor-udp
     port: 7981
     protocol: UDP
   endpoints:
   - addresses: [ <dxesqlag-b-0_lb_ip> ]
   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: dxesqlag-b-1
   spec:
     clusterIP: None
     ports:
     - name: dxcmonitor-tcp
       protocol: TCP
       port: 7980
     - name: dxcmonitor-udp
       protocol: UDP
       port: 7981
   ---
   apiVersion: discovery.k8s.io/v1
   kind: EndpointSlice
   metadata:
     name: dxesqlag-b-1-ip
     labels:
       dh2i.com/name: dxesqlag-b-1
   addressType: IPv4
   ports:
   - name: dxcmonitor-tcp
     port: 7980
   - name: dxcmonitor-udp
     port: 7981
     protocol: UDP
   endpoints:
   - addresses: [ <dxesqlag-b-1_lb_ip> ]

4. Use the example DxEnterpriseSqlAg YAML below to deploy the pods in the first Kubernetes cluster. Note the following values:

   • The spec.template.spec.dxEnterpriseContainer.joinTarget.target is not present because this is a new cluster.

     DxEnterpriseSqlAg-a.yaml

     apiVersion: dh2i.com/v1
     kind:  DxEnterpriseSqlAg
     metadata:
       name: dxesqlag-a
     spec:
       synchronousReplicas: 2
       asynchronousReplicas: 0
       configurationOnlyReplicas: 0   
       availabilityGroupName: AG1
       template:
         spec:
           dxEnterpriseContainer:
             image: "docker.io/dh2i/dxe:latest"
             imagePullPolicy: Always
             acceptEula: true
             clusterSecret: dxe
             vhostName: VHOST1
           mssqlServerContainer:
             image: "mcr.microsoft.com/mssql/server:latest"
             imagePullPolicy: Always
             mssqlSecret: mssql
             acceptEula: true
             mssqlPID: Developer

5. Switch to the second cluster and repeat step #1.

   Passkeys must match

   The DX_PASSKEY must match the cluster passkey of the first cluster so the operator can join and manage the cluster nodes running in Kubernetes.

6. For each pod in the second Kubernetes cluster, create a load balancer with a static IP. Example load balancers are given below.

   warning

   Ports mapped to a load balancer within Kubernetes will be accessible to the external network, which in some configurations may be a public network.

   lbs-b.yaml

   apiVersion: v1
   kind: Service
   metadata:
     name: dxesqlag-b-0-clb
     #annotations:
     #  service.beta.kubernetes.io/azure-load-balancer-resource-group: <node resource group name>
     #  service.beta.kubernetes.io/azure-pip-name: myAKSIP3
     #  metallb.universe.tf/loadBalancerIPs: <ip_address>
   spec:
     ports:
     - port: 7980
       targetPort: 7980
       protocol: TCP
       name: join-tcp
     - port: 7981
       targetPort: 7981
       protocol: UDP
       name: group-udp
     # Optional external access to SQL Server and DxAdmin
     - port: 1433
       targetPort: 1433
       protocol: TCP
       name: mssql
     - port: 7979
       targetPort: 7979
       protocol: TCP
       name: dxadmin
     selector:
       dh2i.com/name: dxesqlag-b-0
     type: LoadBalancer
   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: dxesqlag-b-1-clb
     #annotations:
     #  service.beta.kubernetes.io/azure-load-balancer-resource-group: <node resource group name>
     #  service.beta.kubernetes.io/azure-pip-name: myAKSIP4
     #  metallb.universe.tf/loadBalancerIPs: <ip_address>
   spec:
     ports:
     - port: 7980
       targetPort: 7980
       protocol: TCP
       name: join-tcp
     - port: 7981
       targetPort: 7981
       protocol: UDP
       name: group-udp
     # Optional external access to SQL Server and DxAdmin
     - port: 1433
       targetPort: 1433
       protocol: TCP
       name: mssql
     - port: 7979
       targetPort: 7979
       protocol: TCP
       name: dxadmin
     selector:
       dh2i.com/name: dxesqlag-b-1
     type: LoadBalancer

7. Create Services on the B-side to map the names of A-side pods to their external IPs. These must be set to the static IPs allocated to the external load balancers in the A-side cluster above

   hostmap-b.yaml

   apiVersion: v1
   kind: Service
   metadata:
     name: dxesqlag-a-0
   spec:
     clusterIP: None
     ports:
     - name: dxcmonitor-tcp
       protocol: TCP
       port: 7980
     - name: dxcmonitor-udp
       protocol: UDP
       port: 7981
   ---
   apiVersion: discovery.k8s.io/v1
   kind: EndpointSlice
   metadata:
     name: dxesqlag-a-0-ip
     labels:
       dh2i.com/name: dxesqlag-a-0
   addressType: IPv4
   ports:
   - name: dxcmonitor-tcp
     port: 7980
   - name: dxcmonitor-udp
     port: 7981
     protocol: UDP
   endpoints:
   - addresses: [ <dxesqlag-a-0_lb_ip> ]
   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: dxesqlag-a-1
   spec:
     clusterIP: None
     ports:
     - name: dxcmonitor-tcp
       protocol: TCP
       port: 7980
     - name: dxcmonitor-udp
       protocol: UDP
       port: 7981
   ---
   apiVersion: discovery.k8s.io/v1
   kind: EndpointSlice
   metadata:
     name: dxesqlag-a-1-ip
     labels:
       dh2i.com/name: dxesqlag-a-1
   addressType: IPv4
   ports:
   - name: dxcmonitor-tcp
     port: 7980
   - name: dxcmonitor-udp
     port: 7981
     protocol: UDP
   endpoints:
   - addresses: [ <dxesqlag-a-1_lb_ip> ]

8. Use the example DxEnterpriseSqlAg YAML below to deploy the pods in the second Kubernetes cluster. Note the following values:

   • The spec.template.spec.dxEnterpriseContainer.joinTarget.target must be the static IP of one of the load balancers in the first cluster.

   • The spec.availabilityGroupName and spec.template.spec.dxEnterpriseContainer.vhostName should match the names of the existing AG and Vhost (respectively) if you want these new pods to join an existing AG.

     DxEnterpriseSqlAg-b.yaml

     apiVersion: dh2i.com/v1
     kind:  DxEnterpriseSqlAg
     metadata:
       name: dxesqlag-b
     spec:
       synchronousReplicas: 2
       asynchronousReplicas: 0
       configurationOnlyReplicas: 0
       availabilityGroupName: AG1
       template:
         spec:
           dxEnterpriseContainer:
             image: "docker.io/dh2i/dxe:latest"
             imagePullPolicy: Always
             acceptEula: true
             clusterSecret: dxe
             vhostName: VHOST1
             joinTarget:
               target: <target_lb_ip>
           mssqlServerContainer:
             image: "mcr.microsoft.com/mssql/server:latest"
             imagePullPolicy: Always
             mssqlSecret: mssql
             acceptEula: true
             mssqlPID: Developer

9. (optional) After giving the cluster a little while to setup, verify that the pods have joined the cluster by running dxcli get-cluster-nodes

   kubectl exec -it dxesqlag-0 -- dxcli get-cluster-nodes

10. (optional) Verify the configuration is correct by running dxcli get-ags-detail.

    kubectl exec -it dxesqlag-0 -- dxcli get-ags-detail <vhost_name> <ag_name>

Join Across Kubernetes Clusters using Load Balancers with a Cluster Selector

Additional Prerequisites

• Two Kubernetes clusters with DxOperator installed.

Steps

1. In the first Kubernetes cluster, create secrets for the DxEnterprise passkey and license, and the SQL Server SA account.

   kubectl create secret generic dxe --from-literal=DX_PASSKEY=<pass> --from-literal=DX_LICENSE=<license_key>

   kubectl create secret generic mssql --from-literal=MSSQL_SA_PASSWORD=<pass>

2. Create a load balancer with a static IP, an externalTrafficPolicy set to local, and a selector for the entire DxEnterpriseSqlAg cluster. The load balancer lets the DxEnterprise cluster nodes (pods) find each other across the network. An example configuration with the necessary ports is given below.

   Kubernetes load balancer implementation

   Kubernetes load balancers are implemented by outside software stacks and vendors, so the details of assigning a load balancer a static IP will vary. Cloud providers like Azure, AWS, and Google Cloud have documentation on how to assign static IPs to load balancers within their own infrastructure.

   apiVersion: v1
   kind: Service
   metadata:
     name: dxesqlag-a-clb
     # Static IPs are often assigned through annotations or the spec.loadBalancerIP. 
     # The annotation below are given for example only from Azure documentation.
     #annotations:
     #  service.beta.kubernetes.io/azure-load-balancer-resource-group: <node resource group name>
     #  service.beta.kubernetes.io/azure-pip-name: myAKSIP1
     #  metallb.universe.tf/loadBalancerIPs: <ip_address>
   spec:
     ports:
     - port: 7980
       targetPort: 7980
       protocol: TCP
       name: join-tcp
     - port: 7981
       targetPort: 7981
       protocol: UDP
       name: group-udp
     selector:
       dh2i.com/entity: dxesqlag-a
     type: LoadBalancer
     externalTrafficPolicy: Local

3. Create Services on the A-side to map the names of B-side pods to the external IP. These must be set to the single static IP allocated to the external load balancer in the B-side cluster below

   hostmap-a.yaml

   apiVersion: v1
   kind: Service
   metadata:
     name: dxesqlag-b-0
   spec:
     clusterIP: None
     ports:
     - name: dxcmonitor-tcp
       protocol: TCP
       port: 7980
     - name: dxcmonitor-udp
       protocol: UDP
       port: 7981
   ---
   apiVersion: discovery.k8s.io/v1
   kind: EndpointSlice
   metadata:
     name: dxesqlag-b-0-ip
     labels:
       dh2i.com/name: dxesqlag-b-0
   addressType: IPv4
   ports:
   - name: dxcmonitor-tcp
     port: 7980
   - name: dxcmonitor-udp
     port: 7981
     protocol: UDP
   endpoints:
   - addresses: [ <dxesqlag-b_lb_ip> ]
   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: dxesqlag-b-1
   spec:
     clusterIP: None
     ports:
     - name: dxcmonitor-tcp
       protocol: TCP
       port: 7980
     - name: dxcmonitor-udp
       protocol: UDP
       port: 7981
   ---
   apiVersion: discovery.k8s.io/v1
   kind: EndpointSlice
   metadata:
     name: dxesqlag-b-1-ip
     labels:
       dh2i.com/name: dxesqlag-b-1
   addressType: IPv4
   ports:
   - name: dxcmonitor-tcp
     port: 7980
   - name: dxcmonitor-udp
     port: 7981
     protocol: UDP
   endpoints:
   - addresses: [ <dxesqlag-b_lb_ip> ]

4. Use the example DxEnterpriseSqlAg YAML below to deploy the pods in the first Kubernetes cluster. Note the following values:

   • The spec.template.spec.dxEnterpriseContainer.joinTarget.target is not present because this is a new cluster.

     DxEnterpriseSqlAg-a.yaml

     apiVersion: dh2i.com/v1
     kind:  DxEnterpriseSqlAg
     metadata:
       name: dxesqlag-a
     spec:
       synchronousReplicas: 2
       asynchronousReplicas: 0
       configurationOnlyReplicas: 0   
       availabilityGroupName: AG1
       template:
         spec:
           dxEnterpriseContainer:
             image: "docker.io/dh2i/dxe:latest"
             imagePullPolicy: Always
             acceptEula: true
             clusterSecret: dxe
             vhostName: VHOST1
           mssqlServerContainer:
             image: "mcr.microsoft.com/mssql/server:latest"
             imagePullPolicy: Always
             mssqlSecret: mssql
             acceptEula: true
             mssqlPID: Developer

5. Switch to the second cluster and repeat step #1.

   Passkeys must match

   The DX_PASSKEY must match the cluster passkey of the first cluster so the operator can join and manage the cluster nodes running in Kubernetes.

6. Create a load balancer with a static IP, an externalTrafficPolicy set to local, and a selector for the entire DxEnterpriseSqlAg cluster. Example load balancers are given below.

   lb-b.yaml

   apiVersion: v1
   kind: Service
   metadata:
     name: dxesqlag-b-clb
     #annotations:
     #  service.beta.kubernetes.io/azure-load-balancer-resource-group: <node resource group name>
     #  service.beta.kubernetes.io/azure-pip-name: myAKSIP3
     #  metallb.universe.tf/loadBalancerIPs: <ip_address>
   spec:
     ports:
     - port: 7980
       targetPort: 7980
       protocol: TCP
       name: join-tcp
     - port: 7981
       targetPort: 7981
       protocol: UDP
       name: group-udp
     selector:
       dh2i.com/entity: dxesqlag-b
     type: LoadBalancer
     externalTrafficPolicy: Local

7. Create Services on the A-side to map the names of B-side pods to the single external IP. These must be set to the static IP allocated to the one external load balancer in the B-side cluster below

   hostmap-b.yaml

   apiVersion: v1
   kind: Service
   metadata:
     name: dxesqlag-a-0
   spec:
     clusterIP: None
     ports:
     - name: dxcmonitor-tcp
       protocol: TCP
       port: 7980
     - name: dxcmonitor-udp
       protocol: UDP
       port: 7981
   ---
   apiVersion: discovery.k8s.io/v1
   kind: EndpointSlice
   metadata:
     name: dxesqlag-a-0-ip
     labels:
       dh2i.com/name: dxesqlag-a-0
   addressType: IPv4
   ports:
   - name: dxcmonitor-tcp
     port: 7980
   - name: dxcmonitor-udp
     port: 7981
     protocol: UDP
   endpoints:
   - addresses: [ <dxesqlag-a_lb_ip> ]
   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: dxesqlag-a-1
   spec:
     clusterIP: None
     ports:
     - name: dxcmonitor-tcp
       protocol: TCP
       port: 7980
     - name: dxcmonitor-udp
       protocol: UDP
       port: 7981
   ---
   apiVersion: discovery.k8s.io/v1
   kind: EndpointSlice
   metadata:
     name: dxesqlag-a-1-ip
     labels:
       dh2i.com/name: dxesqlag-a-1
   addressType: IPv4
   ports:
   - name: dxcmonitor-tcp
     port: 7980
   - name: dxcmonitor-udp
     port: 7981
     protocol: UDP
   endpoints:
   - addresses: [ <dxesqlag-a_lb_ip> ]

8. Use the example DxEnterpriseSqlAg YAML below to deploy the pods in the second Kubernetes cluster. Note the following values:

   • The spec.template.spec.dxEnterpriseContainer.joinTarget.target must be the static IP of the load balancer in the first cluster.

   • The spec.availabilityGroupName and spec.template.spec.dxEnterpriseContainer.vhostName should match the names of the existing AG and Vhost (respectively) if you want these new pods to join an existing AG.

     DxEnterpriseSqlAg-b.yaml

     apiVersion: dh2i.com/v1
     kind:  DxEnterpriseSqlAg
     metadata:
       name: dxesqlag-b
     spec:
       synchronousReplicas: 2
       asynchronousReplicas: 0
       configurationOnlyReplicas: 0
       availabilityGroupName: AG1
       template:
         spec:
           dxEnterpriseContainer:
             image: "docker.io/dh2i/dxe:latest"
             imagePullPolicy: Always
             acceptEula: true
             clusterSecret: dxe
             vhostName: VHOST1
             joinTarget:
               target: <target_lb_ip>
           mssqlServerContainer:
             image: "mcr.microsoft.com/mssql/server:latest"
             imagePullPolicy: Always
             mssqlSecret: mssql
             acceptEula: true
             mssqlPID: Developer

9. (optional) After giving the cluster a little while to setup, verify that the pods have joined the cluster by running dxcli get-cluster-nodes

   kubectl exec -it dxesqlag-0 -- dxcli get-cluster-nodes

10. (optional) Verify the configuration is correct by running dxcli get-ags-detail.

    kubectl exec -it dxesqlag-0 -- dxcli get-ags-detail <vhost_name> <ag_name>