[solved] kvm-qemu ubuntu 16.04 vm going to pause state

Discussion in 'Technical' started by Tomislav Aurednik, Apr 19, 2017.

  1. Hi guys I have a wierd problem. I'm running CentOS 7.2 host server with qemu-kvm-1.5.3-126.el7_3.5.x86_64. On it I'm running a vm with Ubuntu 16.04.2 LTS. After a time running the vm crashes to pause state and you can only force-off to shut it down. After that it won't work anymore. It crashes to pause state every time I try to boot it up.
    First I found on google that it is a disk space issue, that the problem is when the host has less space than needed for the vm disk, but on my ssd disk is 20GB more space than the vm-disk needs and on my hdd its 6TB more than the vm needs and the vm hass still a lot of free space. So this cannot be the problem.

    The qemu log shows

    main_channel_link: add main channel client
    main_channel_handle_parsed: net test: latency 0.338000 ms, bitrate 10138613861 bps (9668.935643 Mbps)
    ((null):36002): Spice-Warning **: red_channel.c:542:red_channel_client_send_ping: getsockopt failed, Operation not supported
    inputs_connect: inputs channel client create
    red_dispatcher_set_cursor_peer:
    ((null):36002): Spice-Warning **: red_channel.c:542:red_channel_client_send_ping: getsockopt failed, Operation not supported
    ((null):36002): Spice-Warning **: red_channel.c:542:red_channel_client_send_ping: getsockopt failed, Operation not supported
    ((null):36002): Spice-Warning **: red_channel.c:542:red_channel_client_send_ping: getsockopt failed, Operation not supported
    ((null):36002): Spice-Warning **: red_channel.c:542:red_channel_client_send_ping: getsockopt failed, Operation not supported
    ((null):36002): Spice-Warning **: red_channel.c:542:red_channel_client_send_ping: getsockopt failed, Operation not supported
    ((null):36002): Spice-Warning **: red_channel.c:542:red_channel_client_send_ping: getsockopt failed, Operation not supported
    ((null):36002): Spice-Warning **: red_channel.c:542:red_channel_client_send_ping: getsockopt failed, Operation not supported
    ((null):36002): Spice-Warning **: red_channel.c:542:red_channel_client_send_ping: getsockopt failed, Operation not supported
    ((null):36002): Spice-Warning **: red_channel.c:542:red_channel_client_send_ping: getsockopt failed, Operation not supported
    ((null):36002): Spice-Warning **: red_channel.c:542:red_channel_client_send_ping: getsockopt failed, Operation not supported
    ((null):36002): Spice-Warning **: red_channel.c:542:red_channel_client_send_ping: getsockopt failed, Operation not supported
    ((null):36002): Spice-Warning **: red_channel.c:542:red_channel_client_send_ping: getsockopt failed, Operation not supported
    ((null):36002): Spice-Warning **: red_channel.c:542:red_channel_client_send_ping: getsockopt failed, Operation not supported
    ((null):36002): Spice-Warning **: red_channel.c:542:red_channel_client_send_ping: getsockopt failed, Operation not supported
    ((null):36002): Spice-Warning **: red_channel.c:542:red_channel_client_send_ping: getsockopt failed, Operation not supported
    ((null):36002): Spice-Warning **: red_channel.c:542:red_channel_client_send_ping: getsockopt failed, Operation not supported
    ((null):36002): Spice-Warning **: red_channel.c:542:red_channel_client_send_ping: getsockopt failed, Operation not supported
    ((null):36002): Spice-Warning **: red_channel.c:542:red_channel_client_send_ping: getsockopt failed, Operation not supported
    ((null):36002): Spice-Warning **: red_channel.c:542:red_channel_client_send_ping: getsockopt failed, Operation not supported
    ((null):36002): Spice-Warning **: red_channel.c:542:red_channel_client_send_ping: getsockopt failed, Operation not supported
    ((null):36002): Spice-Warning **: red_channel.c:542:red_channel_client_send_ping: getsockopt failed, Operation not supported
    ((null):36002): Spice-Warning **: red_channel.c:542:red_channel_client_send_ping: getsockopt failed, Operation not supported
    ((null):36002): Spice-Warning **: red_channel.c:542:red_channel_client_send_ping: getsockopt failed, Operation not supported
    ((null):36002): Spice-Warning **: red_channel.c:542:red_channel_client_send_ping: getsockopt failed, Operation not supported
    ((null):36002): Spice-Warning **: red_channel.c:542:red_channel_client_send_ping: getsockopt failed, Operation not supported
    ((null):36002): Spice-Warning **: red_channel.c:542:red_channel_client_send_ping: getsockopt failed, Operation not supported
    ((null):36002): Spice-Warning **: red_channel.c:542:red_channel_client_send_ping: getsockopt failed, Operation not supported
    ((null):36002): Spice-Warning **: red_channel.c:542:red_channel_client_send_ping: getsockopt failed, Operation not supported
    ((null):36002): Spice-Warning **: red_channel.c:542:red_channel_client_send_ping: getsockopt failed, Operation not supported
    red_channel_client_disconnect: rcc=0x7fcd8cf68000 (channel=0x7fcd8b0b2000 type=3 id=0)
    red_channel_client_disconnect: rcc=0x7fcd8ceac000 (channel=0x7fcd8bb84600 type=2 id=0)
    red_channel_client_disconnect: rcc=0x7fcd8d272000 (channel=0x7fcd8b0524e0 type=9 id=0)
    red_channel_client_disconnect: rcc=0x7fcd8cf6d000 (channel=0x7fcd8b052680 type=9 id=1)
    red_channel_client_disconnect: rcc=0x7fcd8c67a000 (channel=0x7fcd8b8d7180 type=4 id=0)
    red_channel_client_disconnect: rcc=0x7fcd8c35d000 (channel=0x7fcd8b0aa000 type=1 id=0)
    main_channel_client_on_disconnect: rcc=0x7fcd8c35d000
    red_client_destroy: destroy client 0x7fcd8b038f00 with #channels=4
    red_dispatcher_disconnect_display_peer:
    red_dispatcher_disconnect_cursor_peer:
    qemu: terminating on signal 15 from pid 1825
    2017-04-18 07:03:39.475+0000: shutting down
    2017-04-18 07:09:08.160+0000: starting up libvirt version: 2.0.0, package: 10.el7_3.5 (CentOS BuildSystem <http://bugs.centos.org>, 2017-03-03-02:09:45, c1bm.rdu2.centos.org), qemu version: 1.5.3 (
    qemu-kvm-1.5.3-126.el7_3.5), hostname: server.mydomain.si

    Googling this error found me only unsolved threads and no fix. Does anyone know how to fix this?

    I think it happend when I dd if=/dev/zero a 2 TB partition. After restor from snapshot it happend wen I synced this partiton with the primary server over drbd. So it probablly some 2TB disk issue.
     
    Last edited: Apr 19, 2017
  2. It is a disk space issue, at least I think. The snapshots that you create with wirt manager are saved in the qcow2 files and they really blow up. 30GB data with 4 snapshots blow the qcow2 file to nearly 200GB! No wonder that it froze.
    I deleted the snapshots but didn't try the operations that resulted in the crash to pause state.
    Need to figure out first how to do external snapshots. Tryed this tutorial https://www.virtkick.com/docs/how-to-perform-a-live-backup-on-your-kvm-virtual-machines.html
    It all works except the important part of mearging the backup image with the original disk image using blockcommit.
    I can't run this commands on my CentOS 7.2 no pkg_install or check_pkg.
    pkg_install "http://resources.ovirt.org/pub/yum-repo/ovirt-release40.rpm"
    check_pkg ovirt-release40
    yum --disablerepo="*" --enablerepo="ovirt-4.0" update -y

    Any advice on backing up kvm-qemu vms?
     
  3. The thing that coused the problem was disk space. The internal snapshots use space in the disk image file so it can get bigger than the space intended for it. Plus drbd on initial sync fills the secondarys nodes disk image with empty space. So the disk image becomes full of empty space. But you can shrink it with no problem with qemu-img convert.
     

Share This Page