Installing Proxmox 9.0 with iSCSI Boot

If you're setting up a diskless Proxmox 9.0 installation with iSCSI boot, you might encounter a critical bug that prevents your system from booting properly. This post details the installation process for Proxmox 9.0 with iSCSI boot and includes an important workaround for the open-iscsi bug present in version 2.1.11-1.

The Problem

After upgrading from Proxmox 8.0 to Proxmox 9.0 or during a fresh installation, your system may fail during boot with these error messages in initramfs:

iscsistart: version 2.1.11
iscsistart: Could not make /var/lib/iscsi 2
iscsistart: exiting due to idbm configuration error

This occurs because the open-iscsi package (version 2.1.11-1) included with Proxmox 9.0 has a bug where the /var/lib/iscsi directory is missing in the initramfs. This issue is documented in Debian bug #1103644.

Prerequisites

Before starting, ensure you have:

  • An iSCSI target properly configured
  • A network card supporting iSCSI boot
  • Proxmox 9.0 installation ISO
  • Network connectivity to your iSCSI target
  • Internet connectivity during installation (for updating packages) or access to a fixed version of open-iscsi package

Installation Process

Step 1: Boot the Proxmox Install ISO and Access Command Line

  1. Boot from the Proxmox 9.0 installation ISO
  2. Select "Advanced Options"
  3. Choose one of the Debug modes
  4. At the first prompt, hit Ctrl+D to get to a second prompt with your network driver loaded

Step 2: Set Up iSCSI Networking

# Find your network card name
ip a

# Bring up the interface
ip link set INTERFACE_NAME up  # Replace INTERFACE_NAME with your actual interface

# Set an appropriate IP address
ip addr add YOUR_IP_ADDRESS/SUBNET dev INTERFACE_NAME  # Example: 10.1.60.5/24

# Verify network configuration
ip a

# Test connectivity to your iSCSI target
ping YOUR_ISCSI_TARGET_IP

Step 3: Install iSCSI Packages

# Install required packages (versions may vary)
dpkg -i /cdrom/debian/proxmox/packages/libisns0t64_0.101-1+b1_adm64.deb
dpkg -i /cdrom/debian/proxmox/packages/libopeniscsiusr_2.1.11-1_adm64.deb
dpkg -i /cdrom/debian/proxmox/packages/open-iscsi_2.1.11-1_adm64.deb
# Don't worry about errors at this point

Step 4: Add the LUN

# Create necessary directories
mkdir /run/lock
mkdir /run/lock/iscsi

# Set your initiator name
echo "InitiatorName=iqn.YOUR_INITIATOR_NAME" > /etc/iscsi/initiatorname.iscsi  # Example: iqn.1991-05.com:your-server-name

# Start the iSCSI service
service iscsid start

# Discover available targets
iscsiadm --mode discovery --op update --type sendtargets --portal YOUR_ISCSI_TARGET_IP

# Add all nodes
iscsiadm --mode node -l all

Step 5: Install Proxmox

  1. Return to the installer by pressing Ctrl+D
  2. Proceed with normal Proxmox installation
  3. When asked about rebooting, choose "No"

Step 6: Fix the open-iscsi Bug (Critical Step)

When you reach the command prompt after installation:

# Mount the new file systems
mount /dev/pve/root /mnt
mount -o bind /dev /mnt/dev
mount -o bind /proc /mnt/proc
mount -o bind /sys /mnt/sys
chroot /mnt /bin/bash

# configure network/internet connectivity
# Bring up the management interface
ip link set MGMT_INTERFACE_NAME up  # Replace with your management interface name

# Set an appropriate IP address for management
ip addr add MGMT_IP_ADDRESS/SUBNET dev MGMT_INTERFACE_NAME  # Example: 192.168.1.100/24

# Add default route for internet access
ip route add default via MGMT_GATEWAY_IP

# If using VLANs for management, configure it
ip link add link MGMT_INTERFACE name MGMT_INTERFACE.VLAN_ID type vlan id VLAN_ID
ip link set MGMT_INTERFACE.VLAN_ID up
ip addr add MGMT_VLAN_IP/SUBNET dev MGMT_INTERFACE.VLAN_ID
ip route add default via MGMT_VLAN_GATEWAY

# Test internet connectivity 
ping 8.8.8.8

# Add nameserver for DNS resolution
echo "nameserver 8.8.8.8" > /etc/resolv.conf  # can use your DNS server
echo "nameserver 1.1.1.1" >> /etc/resolv.conf

# Update open-iscsi to a newer version before configuring
# This is the critical step to avoid the boot failure in Proxmox 9.0
apt update
apt install open-iscsi

# Configure iSCSI auto boot
echo "ISCSI_AUTO=true" > /etc/iscsi/iscsi.initramfs

# Update bootloader and initramfs
update-grub && update-initramfs -u -k all

Step 7: Complete and Reboot

Exit the chroot environment with Ctrl+D (twice), and your system should reboot into Proxmox 9.0 properly.

Manual Recovery (If Needed)

If you've already upgraded to Proxmox 9.0 and encounter the boot failure, you can manually recover from the initramfs prompt:

# Create the missing directory
mkdir -p /var/lib/iscsi

# Restart the iSCSI boot process
iscsistart -b

# Once the iSCSI connection is established, continue the boot process
exit  # or Ctrl+D

Root Cause and Long-term Fix

The issue is due to a missing directory in the initramfs environment. The open-iscsi package version 2.1.11-1 fails to create the /var/lib directory before trying to create /var/lib/iscsi, resulting in the boot failure.

A fixed version of the package (2.1.11-1+deb13u1) includes a patch that creates this directory. The patch is a simple one-liner:

+ mkdir -p /var/lib
  iscsistart -b

Conclusion

By following these steps, you can successfully install or upgrade to Proxmox 9.0 with iSCSI boot. The critical step is installing an updated version of the open-iscsi package before updating the initramfs, which prevents the boot failure caused by the missing directory.

For those interested in diskless setups, iSCSI boot provides a great solution for Proxmox deployments, eliminating the need for local storage while maintaining excellent performance.