NSX Upgrade Failed but Still Showing as In Progress in VCF
The NSX upgrade was successful for the Edge clusters but failed when attempting to upgrade the transport node.
The failure occurred during the NSX Transport Node Precheck Stage, just a few minutes into the process. However, in VCF, the upgrade task continues to appear as “In Progress.”
No specific error is visible in VCF since the task status has not changed. I attempted to restart services and reboot the VCF appliance, but the issue persists.
Upon logging into the NSX Manager, the upgrade status showed as paused.
Resolution:
After completing the upgrade, update the VCF database to reflect the new NSX version, ensuring consistency between NSX and VCF.
Perform a manual upgrade of both the Transport Node and the NSX Manager directly from the NSX Manager interface.
Post-Upgrade Steps
Once the NSX upgrade is completed:
- Power off the VCF appliance.
- Take a cold snapshot of the VCF.
- Power on the VCF appliance after the snapshot is taken.
Next, log in to the VCF interface and retrieve the task ID of the failed task that is still marked as running.
SSH to VCF and become root
psql -U postgres -h localhost -d lcm
select * from upgrade_activity_log where upgrade_id=’ TASK ID’;
select * from upgrade_element where upgrade_id=’UPGRADEID’ and upgrade_status=’COMPLETED_WITH_FAILURE’;
Select * from upgrade_element where upgrade_status=’INPROGRESS’;
update upgrade_element set upgrade_staatus=’COMPLETED_WITH_SUCCESS’ where upgrade_status=’INPROGRESS’;
Switch to Platform
\c Platform
select id,version,cluster_fqdn from nsxt;
Then
update nsxt set version=”4.2.0.0-24304122′ where id=’XXXXXXXX’;
Run this command again to make sure the record has been updated
select id,version,cluster_fqdn from nsxt;
Restart SDDC manager services
/opt/vmware/vcf/operationsmanager/scripts/cli/sddcmanager_restart_services.sh