iSCSI and Proxmox

[BeefaloFan]

In general I have setup targets for openVZ and targets for KVM machines because they potentially have different read write characteristics that can be adjusted via the target setup on the iSCSI server (in our case openfiler).

 

So for one 1TB openfiler storage array I have 3 LUNs - 2 are used for openVZ (1 lun for the master proxmox server and the other for the cluster proxmox server), and then 3rd lun is used for KVM's and this lun is automatically shared between the 2 proxmox servers.

 

Why 2 LUNs for openVZ and not a shared LUN? Because openVZ cannot share the same lun as it uses it directly and creates and writes files like any locally mounted filesystem.

 

Here is a wiki for setting up openVZ for use in an iSCSI environment to allow for easy migration
http://pve.proxmox.com/wiki/OpenVZ_on_ISCSI_howto" target="_blank">http://pve.proxmox.com/wiki/OpenVZ_on_ISCSI_howto

 

For KVM on openfiler there is an issue with version 2.3 of openfiler that is not very well documented - so you might have problems getting that working, unless you do this on the openfiler:

 

i. Comment these lines out in /etc/rc.sysinit. Lines 333 to 337.

Code:# if [ -x /sbin/lvm.static ]; then# if /sbin/lvm.static vgscan --mknodes --ignorelockingfailure > /dev/null 2>&1 ; then# action $"Setting up Logical Volume Management:" /sbin/lvm.static vgchange -a y --ignorelockingfailure# fi# fi

 

ii. Turn off aoe (ATA over Ethernet) service from autostart by running:

chkconfig aoe off

 

 

What this resolves is a problem with openfiler activating the logical volumes that are created by proxmox when setting up a logical volume group for KVM virtual machines. For some reason in openfiler 2.3 it has a problem with activated logical volumes within other activated logical volumes and locking files. So (and you can test this for yourself), once you have a new KVM virtual machine setup - shut the vm down and shutdown proxmox and reboot your openfiler server and you will see that the iscsi target is no longer available and your kvm vm will fail to start once you start the proxmox server back up.

 

Things to consider - KVM migration allows live migration with almost ZERO downtime, however performance for linux is not so good

openVZ setup as per the above wiki - allows for iSCSI and migration - however migration is offline so there is a small outage - look at the end of the wiki for more info

 

You may also want to reference http://pve.proxmox.com/wiki/Storage_Model#LVM_Groups_with_Network_Backing" target="_blank">http://pve.proxmox.com/wiki/Storage_Model#LVM_Groups_with_Network_Backing