§ Wiki · Wiki entry

Troubleshooting Node Deployment Errors

Diagnostic guide for the common failure modes encountered while installing IC-OS on a node machine — registration errors, blank consoles, missing drives, network failures, and how to get a shell in SetupOS.

·May 4, 2026·8 min read

This entry collects the error patterns most commonly seen when installing IC-OS on a node machine. Read it in full before posting a support request in the Node Provider Matrix channel — the troubleshooting sequence below resolves the vast majority of issues.

If you need Dell to service the machine, see the iDRAC Access and TSR Logs entry for how to retrieve a Dell TSR log and reset the iDRAC password.

General troubleshooting steps

These steps have been refined over hundreds of deployments. Complete all of them before messaging in the Matrix channel.

  1. Re-attempt the verification step from your deployment runbook, exactly as written:
  2. Confirm you are using the latest IC-OS release from the Internet Computer Dashboard Releases. When in doubt, redownload the latest release and retry.
  3. Confirm you are following the runbook that matches your hardware generation and onboarding path:
  4. Reread every step in your runbook. The instructions are precise and they evolve over time.
  5. Verify the networking values written to config.ini against the data center's actual configuration.
  6. Reread the Node Provider Networking Guide, especially the "What NOT to do" section.
  7. Restart every piece of equipment between the node and the internet — switch, router, firewall.
  8. Restart the deployment from the beginning and try to reproduce the error.
  9. Try a different node machine that is not currently in a subnet. If the failure reproduces on multiple machines, that points away from a single-machine hardware fault.

Support request information requirements

If the steps above did not resolve your issue, post in the IC Node Provider Matrix channel with all of the following — requests missing this information will be sent back:

  • Confirmation that you completed every general troubleshooting step above.
  • Screenshots of the failure and any relevant logs.
  • The deployment stage at which the failure occurs.
  • Which runbook you are following: Gen-2 (no HSM) or Gen-1 (with HSM).
  • Whether this is your first IC-OS installation overall.
  • Whether this is your first IC-OS installation in this data center.
  • Whether this is your first IC-OS installation with this Node Operator key.
  • Whether the issue reproduces.
  • Hardware details: generation (Gen-1, Gen-2) and server brand.
  • The data center you are deploying out of.
  • Whether you are onboarding with an IPv4 address.
  • Any other relevant networking details.

Unhealthy nodes

The node installed and registered successfully, but the dashboard shows a status that is not "Awaiting Subnet" or "Active in Subnet".

Refer to the Node Provider Troubleshooting guide for the unhealthy-node diagnostic flow.

Node registration failures

The node installed cleanly. HostOS came up. But the node never appears on the dashboard and you have already worked through the general troubleshooting steps.

Node allowance for this node operator is exhausted

The node allowance on your operator record (set during node provider onboarding) is full. Check the dashboard for nodes registered under your principal — if old nodes are still listed that should not be, see Removing a Node From the Registry.

Cannot remove a node that is a member of a subnet

You are attempting to redeploy a node that is currently assigned to a subnet. The node must be removed from the subnet first. Note: if you are performing a firmware upgrade only, you do not need to redeploy.

Node reward type is required

The node_reward_type field in config.ini is unset or invalid. See Node Deployment config.ini to look up the correct value for your operator record, then redeploy.

cannot set up guest memory 'pc.ram': Cannot allocate memory

Full message:

QEMU unexpectedly closed the monitor (vm='guestos'): ... qemu-system-x86_64: cannot set up guest memory 'pc.ram': Cannot allocate memory

HostOS cannot allocate the required RAM to the GuestOS VM. This is almost always a hardware fault. Run memtest and confirm every DIMM is seated correctly.

Blank console after install

After SetupOS finishes, the machine reboots and HostOS comes up. After a few minutes you should see a HostOS console log. This log alone does not signify a successful onboarding — wait at least 10 minutes for the Join request successful! line.

If after 10 minutes nothing further has been logged, leave the node running and post in the Matrix channel with a screenshot of the console plus all the support request information requirements above.

IC-OS installation failures

Blank console

If the SetupOS console is blank, wait 5 minutes. Some nodes are slow to start logging and this is not necessarily a failure.

If the console is still blank after 5 minutes, return to the general troubleshooting steps.

Missing drives

--------------------------------------------------------------------------------
                      INTERNET COMPUTER - SETUP - FAILED
--------------------------------------------------------------------------------



       Please consult the wiki guide: Troubleshooting Node Deployment Errors.



--------------------------------------------------------------------------------
                                    ERROR
--------------------------------------------------------------------------------


Not enough drives found. Are all drives correctly installed?


--------------------------------------------------------------------------------
                                    ERROR
--------------------------------------------------------------------------------

A variant of this error reads Aggregate Disk size does not meet requirements.

Cause. The installer cannot detect every required drive. Even if all drives appear physically present, one or more may be poorly seated or non-functional.

Resolution. Reseat every drive. Check drive indicator LEDs to identify any that are not powering up. Install the required number of drives if any are missing.

Invalid CPU configuration

--------------------------------------------------------------------------------
                      INTERNET COMPUTER - SETUP - FAILED
--------------------------------------------------------------------------------


       Please consult the wiki guide: Troubleshooting Node Deployment Errors.

--------------------------------------------------------------------------------
                                    ERROR
--------------------------------------------------------------------------------

Number of threads (16/32) does NOT meet system requirements.

--------------------------------------------------------------------------------
                                    ERROR
--------------------------------------------------------------------------------

Cause. The CPUs are not configured correctly in the BIOS — usually disabled cores or disabled SMT.

Resolution. Reset the BIOS to factory defaults, then reapply the UEFI configuration for your hardware vendor:

Unable to reach internet

--------------------------------------------------------------------------------
                      INTERNET COMPUTER - SETUP - FAILED
--------------------------------------------------------------------------------



       Please consult the wiki guide: Troubleshooting Node Deployment Errors.



--------------------------------------------------------------------------------
                                    ERROR
--------------------------------------------------------------------------------


 Unable to ping IPv6 gateway.


--------------------------------------------------------------------------------
                                    ERROR
--------------------------------------------------------------------------------

Cause. The node cannot communicate with the network — either a misconfigured config.ini, or a problem somewhere between the node and the internet.

Resolution. Capture any output that appears before the error. The installer prints the user-defined and system network settings:

* Printing user defined network settings...
 IPv6 Prefix : XXX
 IPv6 Subnet : XXX
 IPv6 Gateway: XXX

* Printing system's network settings...
 IPv6 Prefix : XXX
 IPv6 Subnet : XXX
 IPv6 Gateway: XXX

* Printing IPv6 addresses...
 SetupOS: XXX
 HostOS : XXX
 GuestOS: XXX

Compare these against the values you expect. If they don't match, correct config.ini and try again.

If they do match, the network path between the node and the wider internet is the problem — likely a firewall. Reboot every device between the node and the internet even if everything appears to be working. Rebooting an apparently healthy firewall has resolved this class of issue many times.

Unable to setup PV

--------------------------------------------------------------------------------
                      INTERNET COMPUTER - SETUP - FAILED
--------------------------------------------------------------------------------



       Please consult the wiki guide: Troubleshooting Node Deployment Errors.



--------------------------------------------------------------------------------
                                    ERROR
--------------------------------------------------------------------------------


 Unable to setup PV on drive '/dev/nvme8n1'.


--------------------------------------------------------------------------------
                                    ERROR
--------------------------------------------------------------------------------

Cause. The node sees the drive but cannot write to it. This usually indicates a hardware fault on that specific drive.

Resolution. Remove and reinstall every drive. Independently verify that each drive is functioning before retrying the installation.

Warning: Gen-2 hardware detected but no Node Operator Private Key found

This is a warning, not an error. The installer is reminding you that Gen-2 hardware should be onboarded via the Gen-2 deployment guide (no HSM), not the legacy HSM path.

  • If you are redeploying a node that was originally onboarded with an HSM, you may continue — wait 5 minutes for the installation to resume.
  • If you are onboarding a new Gen-2 machine, abort and switch to the Gen-2 deployment guide.

Getting a shell in SetupOS

During the IC-OS installation phase, you can drop to a console shell to diagnose problems directly:

  1. Press Enter at any time during installation, or at the error page.
  2. Continue pressing Enter until you see a login prompt.
  3. Log in as root with an empty password.
  4. You now have root access for diagnostics.

[!NOTE] The SetupOS shell is only available during the IC-OS installation phase. After the machine reboots into HostOS, this troubleshooting console is no longer accessible. To inspect a HostOS-stage failure, boot the machine from a live USB distribution such as Ubuntu Live USB and troubleshoot from there.

Related