Stretch Execution Runbook — manual steps vs the API

A VCF cluster stretch is one API call doing a lot of work, bracketed by manual prep that the API cannot do for you. This page draws that line explicitly, for both the management domain (first) and a workload domain (after the management domain is stretched).

It is the execution half of 03-multi-az-prep.md (design/prep — what stretches vs per-AZ, witness, capacity), and expands 06-deployment-plan.md epics E7 (management domain) / E9 (workload domain) into a linear runbook. Read 03 first — this page assumes the design decisions there (M1-M6, section D) are already made.

The sequence, end to end

inter-AZ fabric  →  commission AZ2 hosts  →  deploy witness  →  stretch (API)
   (manual)            (manual)                (manual)          (one call)

Same order for a workload domain, once the management domain is stretched.

1. Manual — inter-AZ fabric (before any of this)

Confirm against 03-multi-az-prep.md sections A-C: <5 ms RTT + ≥10 Gbps between AZ1/AZ2, witness RTT budget, MTU 9000 end-to-end, per-AZ subnets for ESX Management / vMotion / vSAN / host TEP, stretched L2 only for VM Management. This is a hard gate — resolve it before touching hosts.

2. Manual — build the AZ2 network pool, then commission the hosts

Broadcom’s own prerequisites list these as two separate, ordered items: “Create a network pool for availability zone 2” comes before “Commission vSAN ESA or OSA hosts for availability zone 2.” Build the pool first — the commission call binds each host to a pool by ID, so there is nothing to bind to until it exists.

• Build the AZ2 network pool first. In VCF 9, this is a vSphere Client task, not SDDC Manager’s own UI (deprecated for VCF 9 — see the callout below): Global Inventory Lists → Hosts → Network Pools → Create Network Pool (or POST /v1/network-pools). Same shape as intake H8 / docs/workbook-cell-mapping.md: the vMotion + vSAN VMkernel network definitions for AZ2 — VLAN, MTU, gateway, IP range. This pool is a distinct object from the AZ2 host TEP (overlay) subnet below — vMotion/vSAN and the NSX host overlay are configured and validated separately, don’t treat “network pool” as covering both.
• Image the AZ2 hosts with the supported ESXi ISO. Use VCFHostPreparation to prep and commission quickly.
• Configure the per-AZ management network on each host (IP / VLAN / gateway), DNS, NTP, root credentials.
• Commission them into SDDC Manager, into the AZ2 network pool you just built. They now sit as available/unassigned hosts, ready for the stretch — SDDC Manager does not touch them until step 4.

Separately, this is where the most common failure mode lives — not the network pool above, but the AZ2 host TEP:

Host TEP must be a genuinely distinct per-AZ subnet, not a reused AZ1 VLAN. Per 03-multi-az-prep.md section D, ESX Host Overlay (TEP) is “unique per availability zone” — there is no option to stretch it, even if every other network (VM Management aside) in your design is flat L2 across sites. If you reuse AZ1’s TEP VLAN/subnet for AZ2, the stretch API call in step 4 fails with ipAssignmentType not found for the NSX overlay VDS for the cluster <id> (PUBLIC_INTERNAL_SERVER_ERROR, IllegalStateException) — SDDC Manager/NSX have nothing to register as a second, distinct IP-assignment entry on the cluster’s overlay VDS. Give AZ2 its own TEP VLAN/subnet even when the rest of the fabric is deliberately L2-stretched end to end.

Reserve the AZ2 TEP subnet/range in your IP plan now — but don’t build anything for it in NSX yet. Unlike the network pool above, there is no manual NSX object to create here. The AZ2 host-overlay IP pool (ipAddressPoolsSpec in the step 4 JSON, e.g. a pool named sfo-w01-az2-host-ip-pool01 with its own CIDR/gateway/range) and the transport-node sub-profile that consumes it are both declared inline in the stretch API payload and created by the stretch call itself. Broadcom is explicit: “SDDC Manager automatically creates this sub-TNP during the stretch operation… Do not create the profile manually before submitting the API request.” Pre-building a TEP profile/pool in NSX Manager ahead of the stretch doesn’t save a step — it conflicts with what step 4 creates for you.

If the AZ2 pool object already exists regardless (e.g. left over from a retry after a failed stretch attempt), you don’t have to delete it and start over — step 4’s ipAddressPoolsSpec[] supports referencing an existing pool by name only instead of declaring it inline; see that section for the exact shape.

3. Manual — deploy the witness

• Deploy the vSAN witness appliance (VMware-VirtualSAN-Witness-*.ova) at the third site. A witness serves only one stretched cluster — the management domain and each stretched workload domain each need their own.
• Route it to both AZs’ ESX-management networks (witness traffic rides the ESX Management VMkernel on data hosts — no dedicated witness VLAN).
• Don’t deploy via the ESXi Host Client — it fails at step 1 (Invalid qualifier: ValueMap{"Management", "Secondary"}). Deploy through a vCenter that reaches the witness host, or use ovftool/govc against the host directly.
• Verify the VMkernel gateways after deploy regardless of method — the witness OVA has shipped with wrong management/vSAN gateways that need fixing by hand (DCUI / Host Client) before it can route back to both AZs.

Full detail, RTT budgets, and the routing requirement: 03-multi-az-prep.md section B.

4. API — the stretch call itself

One PATCH /v1/clusters/{id} (or /v1/domains shape for the initial management-domain case) carrying a clusterStretchSpec, validated first via POST /v1/clusters/{id}/validations. This is what SDDC Manager does for you in that single call:

• Builds the fault domains (AZ1 preferred / AZ2 secondary / witness)
• Balances the commissioned AZ2 hosts across the cluster
• Flips the datastore storage policy to site mirroring (~2x capacity)
• Wires in Edge-cluster multi-AZ config, if you set isEdgeClusterConfiguredForMultiAZ: true correctly (required when the cluster already hosts an NSX Edge cluster — get this wrong and the edge-specific AZ configuration is silently skipped)

You supply: the AZ2 network pool (step 2), the commissioned AZ2 hosts (equal count to AZ1), and the witness (step 3). Nothing else — no separate call to build fault domains or flip the storage policy.

Building the JSON by hand

Get the two lookups out of the way first — you need both before you can fill in the spec:

• AZ2 host IDs. GET /v1/hosts with status=UNASSIGNED_USEABLE, copy the id for each commissioned AZ2 host.
• Cluster ID. GET /v1/clusters, copy the id of the cluster you’re stretching.

Trimmed example (2 hosts; add one hostSpecs entry per AZ2 host, equal count to AZ1) — field names and shape verbatim from Broadcom’s SDDC Manager API Explorer walkthrough. The shape is identical for a management-domain and a workload-domain stretch; there’s no per-domain-type field to flip (see the isDefault note below the JSON — an earlier revision of this doc claimed one existed and that claim didn’t hold up):

{
  "clusterStretchSpec": {
    "deployWithoutLicenseKeys": true,
    "hostSpecs": [
      {
        "id": "<AZ2 host 1 ID>",
        "hostName": "sfo02-m01-r01-esx01.sfo.rainpole.io",
        "hostNetworkSpec": {
          "networkProfileName": "sfo02-m01-r01-network-profile01",
          "vmNics": [
            { "id": "vmnic0", "vdsName": "sfo-m01-cl01-vds01", "uplink": "uplink1" },
            { "id": "vmnic1", "vdsName": "sfo-m01-cl01-vds01", "uplink": "uplink2" }
          ]
        }
      },
      {
        "id": "<AZ2 host 2 ID>",
        "hostName": "sfo02-m01-r01-esx02.sfo.rainpole.io",
        "hostNetworkSpec": {
          "networkProfileName": "sfo02-m01-r01-network-profile01",
          "vmNics": [
            { "id": "vmnic0", "vdsName": "sfo-m01-cl01-vds01", "uplink": "uplink1" },
            { "id": "vmnic1", "vdsName": "sfo-m01-cl01-vds01", "uplink": "uplink2" }
          ]
        }
      }
    ],
    "networkSpec": {
      "networkProfiles": [
        {
          "name": "sfo02-m01-r01-network-profile01",
          "nsxtHostSwitchConfigs": [
            {
              "ipAddressPoolName": "sfo02-m01-r01-ip-pool01-host",
              "uplinkProfileName": "sfo02-m01-r01-uplink-profile01",
              "vdsName": "sfo-m01-cl01-vds01",
              "vdsUplinkToNsxUplink": [
                { "nsxUplinkName": "uplink1", "vdsUplinkName": "uplink1" },
                { "nsxUplinkName": "uplink2", "vdsUplinkName": "uplink2" }
              ]
            }
          ]
        }
      ],
      "nsxClusterSpec": {
        "ipAddressPoolsSpec": [
          {
            "name": "sfo02-m01-r01-ip-pool01-host",
            "subnets": [
              {
                "cidr": "10.12.14.0/24",
                "gateway": "10.12.14.1",
                "ipAddressPoolRanges": [
                  { "start": "10.12.14.101", "end": "10.12.14.132" }
                ]
              }
            ]
          }
        ],
        "uplinkProfiles": [
          {
            "name": "sfo02-m01-r01-uplink-profile01",
            "transportVlan": 1214,
            "teamings": [
              {
                "name": "DEFAULT",
                "policy": "LOADBALANCE_SRCID",
                "standByUplinks": [],
                "activeUplinks": ["uplink1", "uplink2"]
              }
            ]
          }
        ]
      }
    },
    "isEdgeClusterConfiguredForMultiAZ": true,
    "witnessSpec": {
      "fqdn": "sfo-m01-cl01-vsw01.sfo.rainpole.io",
      "vsanCidr": "10.21.10.0/24",
      "vsanIp": "10.21.10.218"
    },
    "witnessTrafficSharedWithVsanTraffic": false
  }
}

What each block is doing, and why it trips people up:

• hostSpecs[] — one entry per AZ2 host. id/hostname come from the two lookups above. vmNics must mirror the exact vmnic-to-vDS mapping the AZ1 hosts on this cluster already use — same uplink names, same count — or the host fails to join the cluster’s vDS.
• networkSpec.networkProfiles[] — this is the sub-TNP from the TEP callout above. name is a label you choose; don’t pre-create it in NSX — SDDC Manager creates it from this spec during the PATCH. Not shown: isDefault. An earlier revision of this doc had you set isDefault: true/false here depending on management vs. workload domain, sourced from “Broadcom’s SDDC Manager API Explorer walkthrough.” That was wrong. The stretch call’s network profile type (StretchClusterNetworkProfile) has no isDefault property at all — confirmed against the real VCF OpenAPI spec across 9.0.0.0, 9.1.0.0, and 9.1.1.0 (isDefault only exists on the separate NetworkProfile schema used by cluster/domain creation, a different endpoint). Lab-verified 2026-09-17 against Holodeck (9.1.1): isDefault set to true, false, omitted, and even an intentionally-wrong type (a string) all produced byte-for-byte identical /validations responses — if the field were real and validated, the wrong-type payload should have tripped a schema-layer error and didn’t. Leave it out; there’s nothing to set here for either domain type.
• networkProfiles[].nsxtHostSwitchConfigs[] — the wiring: it binds the network profile to the vDS (vdsName) and points it at the pool and uplink profile defined below by name reference (ipAddressPoolName/uplinkProfileName), not by inline value. Get a name mismatch between here and nsxClusterSpec and the profile silently has no pool to assign from.
• networkSpec.nsxClusterSpec.ipAddressPoolsSpec[] — the actual AZ2 TEP subnet: cidr / gateway / ipAddressPoolRanges. This subnet must be genuinely distinct from AZ1’s — see the TEP callout above; reusing AZ1’s address range here is what produces the ipAssignmentType not found for the NSX overlay VDS failure. There’s a second, API-level meaning of “reuse” that doesn’t contradict this: if the AZ2 pool object already exists in NSX (e.g. from an earlier attempt), you can reference it by name instead of re-declaring it inline — set only name, leave subnets out (null/omitted), and skip cidr/gateway/ipAddressPoolRanges entirely. That’s a shorthand for pointing at an already-created AZ2 pool, not license to point AZ2 at AZ1’s pool. Lab-verified against VCF 9.1.1 (Holodeck, /v1/clusters/{id}/validations, 2026-09-17): a payload using this by-name-only shape passed schema validation. The Clusters API docs — Stretch Cluster use case describe the same mechanism and match this behavior, but that page is pinned to the VCF 5.2.x line (latest = 5.2.4 as of 2026-09-17, developer.broadcom.com has no published 9.x Clusters API reference) — cite it as corroborating precedent, not as a VCF-9-confirmed source.
• nsxClusterSpec.uplinkProfiles[] — transportVlan is the AZ2 host-overlay VLAN ID; teamings[].activeUplinks must match how AZ1’s uplink profile is already teamed (same policy, same uplink count) so the two AZs’ hosts behave identically on the shared vDS. This is also where the SECONDARY_AZ_OVERLAY_VLANID cannot be blank validation error traces back to: there’s a top-level, deprecated clusterStretchSpec.secondaryAzOverlayVlanId (integer) field from an older API shape, confirmed still present but "deprecated": true in the real VCF 9.1.1 OpenAPI spec (SDDC Manager API Reference Guide, ClusterStretchSpec schema) — “the secondary AZ overlay vlan id should be mentioned in the uplinkProfile field instead.” SDDC Manager needs an overlay VLAN from one of the two places; omit networkSpec entirely with nothing set in the deprecated field and that’s the blank-VLAN error. Use transportVlan here, not the deprecated field.
• isEdgeClusterConfiguredForMultiAZ — true only if this cluster already hosts an NSX Edge cluster (per the precondition list above); leave false otherwise.
• witnessSpec — fqdn / vsanCidr / vsanIp of the witness appliance deployed and gateway-fixed in step 3.
• deployWithoutLicenseKeys — defers license-key validation for whatever the call would otherwise check licensing on, same as the domain-creation spec’s field of the same name. true lets the stretch proceed on evaluation/unlicensed capacity and enforces licensing later; false makes the call fail up front if a required key isn’t already assigned. Leave it true (as shown) unless you specifically want the stretch to hard-fail on a missing license.
• witnessTrafficSharedWithVsanTraffic — the vSAN witness traffic separation toggle: true tags witness traffic onto the same VMkernel interface as vSAN data traffic; false routes it over a separate interface instead. This runbook’s own step 3 puts witness traffic on the ESX Management VMkernel, not the vSAN VMkernel — so false (as shown) is the value that matches the routing this doc has you build. Only flip it to true if you’ve instead made the vSAN network itself routable to the witness site and want witness traffic riding that interface.

Optionally build, validate, and submit the spec with VCFJsonSpecCreators’s New-VCFvSANStretchSpec.ps1, which assembles the JSON, calls /validations, and PATCHes it — setting isEdgeClusterConfiguredForMultiAZ for you. That script still prompts for and sets networkProfiles[].isDefault (management vs. workload domain) — per the finding above, this field doesn’t exist on the stretch endpoint’s schema and had no effect in lab testing, so treat that prompt as a no-op until the script is updated to match.

Preconditions that hard-fail this call

Check these before submitting — none of them are visible from the JSON shape alone:

• The cluster must not share a vSAN storage policy with another cluster
• No DPU-backed hosts
• No L3-different subnets within a single AZ (the per-AZ split in step 2 is AZ1-vs-AZ2, not host-vs-host inside one AZ)
• vSphere Supervisor must not be enabled on the cluster — enable Supervisor only after the stretch, not before

Reading the response

Verified against a live VCF 9.1 SDDC Manager (03-multi-az-prep.md / the JSON spec creators repo):

5. Acceptance

• SDDC Manager reports the cluster stretched
• vSAN healthy and storage-policy compliant (site mirroring)
• Isolating one AZ keeps VMs running on the surviving site

Workload domain: same pattern, one precondition

A stretched workload domain follows steps 1-5 identically, with its own AZ2 hosts and its own witness (never shared with the management witness — the shared-witness feature is 2-node-cluster only, not stretched). The one hard gate: the management domain must already be stretched (E7) before any workload-domain cluster can be stretched. Edge Overlay + Uplinks are stretched only under NSX Centralized connectivity; under Distributed they’re per-AZ, same as the management-domain case.

Full story breakdown: 06-deployment-plan.md E9, stories 9.1-9.6.