Hi All, I'd like some advice with selecting the best settings for Xen. Currently I have 2 servers, each with 8 CPU Xeon cores, 12Gb memory and 1TB of Raid 5 storage. I had installed CentOS6 on both of these servers, and they were only used for a very trivial task. Having closely monitored the CPUs, I find that CPU0 is used on average around 40~60% and CPUs1~7 are barely used at all at less than 5%. I decided its time to make better use of my resourses, and I installed Xen. I followed the instructions from http://www.howtoforge.com/virtualiz...aravirtualization-and-hardware-virtualization And it seems that I have a successful installation. My boot/grub parameters are set up with kernel /xen.gz dom0_mem=1024M cpufreq=xen dom0_max_vcpus=1 dom0_vcpus_pin I installed the first PV DomU (snl3) it seemed fine. I gave it 3 CPUs and 4Gb of memory, and CentOS6.4 I installed the 2nd PVDomU (snl4) it also seemed fine, I gave that 2CPUs and 3Gb of memory, CentOS6.4 In both cases I created and started these DomUs with xl create (not using xm or virsh) I then installed the applications on each DomU, and this is where the problems began. Originally I ran a script on the dedicated server, it took just over 4 seconds. On the DomU snl3, its now taking over a minute to run, which is not good, because the script is executed by cron every minute. The script that I ran on DomU snl4, well, it just didn't finish, where as previously on dedicated server it took just over 2 mins. Upon checking xl vcpu-list, I see that Dom0 is using VCPU0=CPU0, and DomU(snl3) is using VCPU0=CPU0 , VCPU1=CPU2, VCPU2=6 and DomU(snl4) VCPU0=0 and VCPU1=4, there appeared to be no consistency, IE not choosing the next available unused CPU, however, refreshing xl cpu-list then showed it all changed again. I checked the CPU utilization of Dom0 and the DomUs, and in all cases, CPU0=100% utilized. (I was also very surprised that Dom0 had only 1 CPU, as I expected to see 8 CPUS from the SNMP stats, but I think I misunderstand how the hardware interacts with CentOS and Xen and the Dom0) I decided to make a couple of new CPU pools, "Pool-snl3", and "Pool-snl4". I removed CPUS 3~7 from Pool-0 and added 3,4,5 to Pool-snl3 and 6,7 to Pool-snl4. I assigned DomU (snl3) to pool-snl3 and then pinned the CPUs to 3,4,5 I assigned DomU (snl4) to pool-snl4 and then pinned the CPUs to 6,7 I ran the script on snl3 again, and this time it completed in under a minute, but only just, but still there were many times when it took longer than a minute. I checked the CPU utilization of the Dom0 and DomUs again and still they were at 100%. I tried to use taskset to get the scripts to run on a specific VCPU, but the results I got led me to believe that taskset was trying to run the task on the actual CPU specified rather than the VCPU. At this point, I decided to abandon what I had done and restart from new. Reading further made me believe that Dom0 controls how the DomUs are given access to the hardware CPU and memory resources, etc. I re-installed CentOS6.4 on my server, and re-installed Xen again. I changed the boot/grub kernel /xen.gz dom0_mem=12G cpufreq=xen dom0_max_vcpus=8 dom0_vcpus_pin It failed to start I changed it to kernel /xen.gz dom0_mem=11G cpufreq=xen dom0_max_vcpus=8 dom0_vcpus_pin it started, but then I could not create and DomUs I changed it to kernel /xen.gz dom0_mem=8G cpufreq=xen dom0_max_vcpus=6 dom0_vcpus_pin I have now created DomU(snl3), installed CentOS6.4, with 2 VCPUs (from Pool-0) and 2G memory, and the script how takes 10 seconds to execute. But, SNMP shows that VCPU0 is averaging at 97% while VCPU1 is at 9%. I've also checked Dom0 with SNMP and all CPU0 is under 4%, while CPUs1~5 are all under 1% xl vcpu-list shows that Dom0 VCPU0=0, V1=1, V2=3, V3=3, V4=4, V5=5, and DomU(snl3) is VCPU0=7, V1=5 (but again, refreshing xl vcpu-list shows that this changes, except for Dom0 that doesnt change) So now my head is quite messed up. I cant see the relationship between how the DomU is allocated a VCPU, sometimes it comes from a free CPU not allocated to the Dom0, and sometimes its allocated a CPU that is pinned to the Dom0. The bottom line is this, I have 8 CPU Cores, and 12G memory. Should I be allocating all my CPUs and memory to Dom0, thus to allow Dom0 to allocate its known resources to my DomUs ? Or should I stick with letting the Dom0 have 1G memory and 1 CPU, and allocate resources from what is left over to each of my DomUs ? Also, If I wish to pin a CPU to a DomU and give it exclusive, unscheduled and unshared access, how can I do this ? Please can you advise me, as I'm starting to lose sleep over this now. Ian
