From patchwork Fri Nov 4 20:20:44 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Roy Keene X-Patchwork-Id: 80921 Delivered-To: patch@linaro.org Received: by 10.140.97.165 with SMTP id m34csp163544qge; Fri, 4 Nov 2016 13:29:39 -0700 (PDT) X-Received: by 10.28.229.132 with SMTP id c126mr170058wmh.110.1478291379151; Fri, 04 Nov 2016 13:29:39 -0700 (PDT) Return-Path: Received: from mx4-phx2.redhat.com (mx4-phx2.redhat.com. [209.132.183.25]) by mx.google.com with ESMTPS id i21si7149787wmf.35.2016.11.04.13.29.38 (version=TLS1 cipher=AES128-SHA bits=128/128); Fri, 04 Nov 2016 13:29:39 -0700 (PDT) Received-SPF: pass (google.com: domain of libvir-list-bounces@redhat.com designates 209.132.183.25 as permitted sender) client-ip=209.132.183.25; Authentication-Results: mx.google.com; spf=pass (google.com: domain of libvir-list-bounces@redhat.com designates 209.132.183.25 as permitted sender) smtp.mailfrom=libvir-list-bounces@redhat.com Received: from lists01.pubmisc.prod.ext.phx2.redhat.com (lists01.pubmisc.prod.ext.phx2.redhat.com [10.5.19.33]) by mx4-phx2.redhat.com (8.13.8/8.13.8) with ESMTP id uA4KKnEl031039; Fri, 4 Nov 2016 16:20:49 -0400 Received: from int-mx10.intmail.prod.int.phx2.redhat.com (int-mx10.intmail.prod.int.phx2.redhat.com [10.5.11.23]) by lists01.pubmisc.prod.ext.phx2.redhat.com (8.13.8/8.13.8) with ESMTP id uA4KKlH7009937 for ; Fri, 4 Nov 2016 16:20:47 -0400 Received: from mx1.redhat.com (ext-mx02.extmail.prod.ext.phx2.redhat.com [10.5.110.26]) by int-mx10.intmail.prod.int.phx2.redhat.com (8.14.4/8.14.4) with ESMTP id uA4KKlJA009015 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Fri, 4 Nov 2016 16:20:47 -0400 Received: from smtp78.ord1c.emailsrvr.com (smtp78.ord1c.emailsrvr.com [108.166.43.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5FEFF8F29A for ; Fri, 4 Nov 2016 20:20:46 +0000 (UTC) Received: from smtp10.relay.ord1c.emailsrvr.com (localhost [127.0.0.1]) by smtp10.relay.ord1c.emailsrvr.com (SMTP Server) with ESMTP id DE05E40845 for ; Fri, 4 Nov 2016 16:20:45 -0400 (EDT) X-SMTPDoctor-Processed: csmtpprox beta Received: from smtp10.relay.ord1c.emailsrvr.com (localhost [127.0.0.1]) by smtp10.relay.ord1c.emailsrvr.com (SMTP Server) with ESMTP id DB27D4082F for ; Fri, 4 Nov 2016 16:20:45 -0400 (EDT) X-Auth-ID: rkeene@knightpoint.com Received: by smtp10.relay.ord1c.emailsrvr.com (Authenticated sender: rkeene-AT-knightpoint.com) with ESMTPSA id 7408340845 for ; Fri, 4 Nov 2016 16:20:45 -0400 (EDT) X-Sender-Id: rkeene@knightpoint.com Received: from [10.193.13.51] (99-91-65-205.lightspeed.nworla.sbcglobal.net [99.91.65.205]) (using TLSv1.2 with cipher DHE-RSA-AES128-SHA) by 0.0.0.0:587 (trex/5.7.7); Fri, 04 Nov 2016 16:20:45 -0400 To: libvir-list@redhat.com From: Roy Keene Message-ID: <6f39beb8-cd59-deb3-bd0a-753bf813185e@knightpoint.com> Date: Fri, 4 Nov 2016 15:20:44 -0500 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.4.0 MIME-Version: 1.0 X-Greylist: Sender passed SPF test, Sender IP whitelisted by DNSRBL, ACL 191 matched, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Fri, 04 Nov 2016 20:20:46 +0000 (UTC) X-Greylist: inspected by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.26]); Fri, 04 Nov 2016 20:20:46 +0000 (UTC) for IP:'108.166.43.78' DOMAIN:'smtp78.ord1c.emailsrvr.com' HELO:'smtp78.ord1c.emailsrvr.com' FROM:'rkeene@knightpoint.com' RCPT:'' X-RedHat-Spam-Score: 0.371 (BAYES_50, DCC_REPUT_00_12, HTML_MESSAGE, RCVD_IN_DNSWL_NONE, RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL) 108.166.43.78 smtp78.ord1c.emailsrvr.com 108.166.43.78 smtp78.ord1c.emailsrvr.com X-Scanned-By: MIMEDefang 2.68 on 10.5.11.23 X-Scanned-By: MIMEDefang 2.78 on 10.5.110.26 X-loop: libvir-list@redhat.com Subject: [libvirt] [PATCH] Allow saving QEMU libvirt state to a pipe X-BeenThere: libvir-list@redhat.com X-Mailman-Version: 2.1.12 Precedence: junk List-Id: Development discussions about the libvirt library & tools List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: libvir-list-bounces@redhat.com Errors-To: libvir-list-bounces@redhat.com All, Currently the "virsh save" command opens a file to save a domain's XML and memory state, does the save, and then re-opens the file to simulate seeking to the beginning to update the header to indicate that the file is complete. For pipes this is not possible, attempting to re-open the pipe will not connect you to the same consumer. Seeking is also not possible on a pipe. Attached is a patch that detects if the file being written to is not seekable, and if so writes the completed header initially, then writes the normal data, and avoids re-opening the file. This is useful to me for saving a VM state directly to Ceph RBD images without having an intermediate file. I've tested this patch successfully (against v1.3.5) for both saving and restoring a domain: # *( fifo="$(mktemp -u)"; mkfifo "${fifo}" && virsh save one-0 "${fifo}" & cat "${fifo}" | rbd import - rbd/test1234 & wait; rm -f "${fifo}" )* Domain one-0 saved to /tmp/tmp.HK4hChiQqB # # *( fifo="$(mktemp -u)"; mkfifo "${fifo}" && rbd export rbd/test1234 - > "${fifo}" & virsh restore "${fifo}" & wait; rm -f "${fifo}" ) * Exporting image: 100% complete...done. Domain restored from /tmp/tmp.0YaUZ5Y2yT # *virsh list* Id Name State ---------------------------------------------------- 11 one-0 running # -- Roy Keene Knight Point Systems, LLC Service-Disabled Veteran-Owned Business 1775 Wiehle Avenue Suite 101 | Reston, VA 20190 c: 813-956-3808 f: 571-266-3106 www.knightpoint.com DHS EAGLE II Prime Contractor: FC1 SDVOSB Track GSA Schedule 70 SDVOSB: GS-35F-0646S GSA MOBIS Schedule: GS-10F-0404Y ISO 20000 / ISO 27001 Notice: This e-mail message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, copy, use, disclosure, or distribution is STRICTLY prohibited. If you are not the intended recipient, please contact the sender by reply e-mail and destroy all copies of the original message. -- libvir-list mailing list libvir-list@redhat.com https://www.redhat.com/mailman/listinfo/libvir-list diff --no-dereference -uNr libvirt-1.3.5.orig/src/qemu/qemu_driver.c libvirt-1.3.5-savepipe/src/qemu/qemu_driver.c --- libvirt-1.3.5.orig/src/qemu/qemu_driver.c 2016-05-28 11:47:33.000000000 -0500 +++ libvirt-1.3.5-savepipe/src/qemu/qemu_driver.c 2016-11-04 10:54:15.160805261 -0500 @@ -3080,6 +3080,7 @@ virQEMUSaveHeader header; bool bypassSecurityDriver = false; bool needUnlink = false; + bool canReopen = true; int ret = -1; int fd = -1; int directFlag = 0; @@ -3087,7 +3088,6 @@ unsigned int wrapperFlags = VIR_FILE_WRAPPER_NON_BLOCKING; memset(&header, 0, sizeof(header)); - memcpy(header.magic, QEMU_SAVE_PARTIAL, sizeof(header.magic)); header.version = QEMU_SAVE_VERSION; header.was_running = was_running ? 1 : 0; header.compressed = compressed; @@ -3109,12 +3109,32 @@ if (fd < 0) goto cleanup; + /* + * Determine if this file can be re-opened + * + * Right now we just try to seek into it, if that fails then that means we + * are probably not dealing with a regular file here and we probably + * cannot reopen it. + */ + if (lseek(fd, 0, SEEK_SET) == ((off_t) -1)) { + canReopen = false; + } + if (virSecurityManagerSetImageFDLabel(driver->securityManager, vm->def, fd) < 0) goto cleanup; if (!(wrapperFd = virFileWrapperFdNew(&fd, path, wrapperFlags))) goto cleanup; + /* Set the header magic */ + if (canReopen) { + /* We will update the magic after the saving completes successfully */ + memcpy(header.magic, QEMU_SAVE_PARTIAL, sizeof(header.magic)); + } else { + /* If we cannot re-open the output, include the final magic here */ + memcpy(header.magic, QEMU_SAVE_MAGIC, sizeof(header.magic)); + } + /* Write header to file, followed by XML */ if (qemuDomainSaveHeader(fd, path, domXML, &header) < 0) goto cleanup; @@ -3124,28 +3144,30 @@ asyncJob) < 0) goto cleanup; - /* Touch up file header to mark image complete. */ + if (canReopen) { + /* Touch up file header to mark image complete. */ - /* Reopen the file to touch up the header, since we aren't set - * up to seek backwards on wrapperFd. The reopened fd will - * trigger a single page of file system cache pollution, but - * that's acceptable. */ - if (VIR_CLOSE(fd) < 0) { - virReportSystemError(errno, _("unable to close %s"), path); - goto cleanup; - } + /* Reopen the file to touch up the header, since we aren't set + * up to seek backwards on wrapperFd. The reopened fd will + * trigger a single page of file system cache pollution, but + * that's acceptable. */ + if (VIR_CLOSE(fd) < 0) { + virReportSystemError(errno, _("unable to close %s"), path); + goto cleanup; + } - if (virFileWrapperFdClose(wrapperFd) < 0) - goto cleanup; + if (virFileWrapperFdClose(wrapperFd) < 0) + goto cleanup; - if ((fd = qemuOpenFile(driver, vm, path, O_WRONLY, NULL, NULL)) < 0) - goto cleanup; + if ((fd = qemuOpenFile(driver, vm, path, O_WRONLY, NULL, NULL)) < 0) + goto cleanup; - memcpy(header.magic, QEMU_SAVE_MAGIC, sizeof(header.magic)); + memcpy(header.magic, QEMU_SAVE_MAGIC, sizeof(header.magic)); - if (safewrite(fd, &header, sizeof(header)) != sizeof(header)) { - virReportSystemError(errno, _("unable to write %s"), path); - goto cleanup; + if (safewrite(fd, &header, sizeof(header)) != sizeof(header)) { + virReportSystemError(errno, _("unable to write %s"), path); + goto cleanup; + } } if (VIR_CLOSE(fd) < 0) {