531 lines
20 KiB
Plaintext
531 lines
20 KiB
Plaintext
% QQ2CLONE(1) qq2clone 0.1
|
|
% Jesse Gardner
|
|
% February 2021
|
|
|
|
# NAME
|
|
qq2clone - Create and manage QEMU/KVM VMs using template machines and qcow2
|
|
images with backing files
|
|
|
|
# SYNOPSIS
|
|
**qq2clone** [*OPTION*]... *COMMAND* [*ARG*]...
|
|
|
|
# DESCRIPTION
|
|
**qq2clone** is a tool working on top of virsh that makes creating clones
|
|
of template QEMU/KVM machines simple. By using the copy on write feature
|
|
for which qcow2 is named, clones of an existing virtual machine can be made
|
|
without inadvertently altering the original image (with caveats - read the
|
|
**LIMITATIONS** section if you aren't already familiar with how copy on
|
|
write works). **qq2clone** makes creating and managing these clones simple
|
|
and efficient.
|
|
|
|
**qq2clone** supports creating numerous clones of a template and
|
|
performing batch operations on them - including the execution of arbitrary
|
|
commands with exec. This simplifies workflows involving large numbers of
|
|
virtual machines, or the frequent creation/destruction of virtual machines.
|
|
|
|
In addition to virsh, basic linux utilities and QEMU/KVM, qq2clone
|
|
requires:
|
|
|
|
Bash 4.0+
|
|
sqlite3
|
|
qemu-img
|
|
libvirt tools:
|
|
virt-clone
|
|
virt-xml
|
|
virt-xml-validate
|
|
xmllint (from libxml2)
|
|
|
|
If you want to easily establish graphical connections to your virtual
|
|
machines, you should have virt-viewer and/or spicy installed and configure
|
|
your templates to use Spice graphics. This is not strictly necessary, and
|
|
with the use of **qq2clone** **exec** and a small script of your own
|
|
you can automate connecting to Spice/VNC clients of your choice without
|
|
too much hassle
|
|
|
|
# OPTIONS
|
|
|
|
Not every option has an effect in the context of every command. Specifying
|
|
an option that has no effect in the context of the command being invoked
|
|
will not produce an error, it simply will not do anything
|
|
|
|
Options are parsed left to right, and right-hand options override
|
|
left-hand options. The only exception is for \-Q/\-\-quieter, which *must*
|
|
be the first option listed to work properly.
|
|
|
|
\-c, \-\-connection [*URI*]
|
|
: Specify a non-default connection URI: sets the value of
|
|
LIBVIRT_DEFAULT_URI
|
|
|
|
\-C, \-\-copy\-disks
|
|
: When importing or copying a template, make a (full) copy of images
|
|
|
|
\-f, \-\-no\-spice
|
|
: Do not attempt to connect to a virtual machine's Spice graphics.
|
|
Overrides USE_SPICE setting in configuration
|
|
|
|
\-g, \-\-use\-spice
|
|
: Attempt to connect to a virtual machine's spice graphics. Overrides
|
|
SPICE setting in configuration
|
|
|
|
\-h, \-\-help
|
|
: Print basic help information and exit
|
|
|
|
\-n, \-\-no-run
|
|
: After making a clone of a template, do not run it. Overrides NORUN
|
|
setting in configuration
|
|
|
|
\-q, \-\-quiet
|
|
: Suppress most non-error output. Overrides QUIET setting in
|
|
configuration. Also suppresses various prompts for user choices, either
|
|
exiting with an error or making a safe default choice depending on the
|
|
command. Recommended only once familiar with the behavior of **qq2clone**
|
|
|
|
\-Q, \-\-quieter
|
|
: This option is (currently) required to appear immediately following the
|
|
invocation of **qq2clone**. Suppresses all output, error message or
|
|
otherwise, except when running interactive commands or
|
|
commands that require output to be useful. The commands for which output
|
|
is not entirely supressed are: config list, config info, list,
|
|
list-templates, exec, edit, modify-template edit, and check. Other
|
|
commands will receive only an exit code as output. This option is intended
|
|
for calling qq2clone from a script.
|
|
|
|
\-r, \-\-run
|
|
: Run a clone when creating it. Overrides NORUN setting in configuration
|
|
|
|
\-s, \-\-storage [*LOCATION*]
|
|
: When creating a clone, place new disk image file(s) at location
|
|
specified by [*LOCATION*]. [*LOCATION*] may be one of an absolute
|
|
filepath, or the name of a libvirt directory type storage pool. Also
|
|
defines where state files will be saved when using **save** command.
|
|
Overrides STORAGE option in configuration
|
|
|
|
\-S, \-\-spicy
|
|
: Use spicy rather than virt-viewer when connecting to the spice graphics
|
|
of a clone. Overrides SPICY setting in configuration
|
|
|
|
\-t, \-\-template [*NAME*]
|
|
: Use template of given name as context when executing a clone command
|
|
(see TYPES OF COMMAND section above). Overrides TEMPLATE option in
|
|
configuration
|
|
|
|
\-v, \-\-verbose
|
|
: Enable all output. Overrides QUIET setting in configuration
|
|
|
|
\-V, \-\-virt\-viewer
|
|
: Use virt-viewer rather than spicy when connecting to the spice graphics
|
|
of a clone. Overrides SPICY setting in configuration
|
|
|
|
# TYPES OF COMMAND
|
|
|
|
There are two main classes of commands: commands that operate directly on
|
|
templates, and commands that create or operate on clones of templates. In
|
|
order to make it less likely that the user may unintentionally invoke a
|
|
command of one class when they intended to invoke one of the other, they
|
|
use a different syntax. Commands that operate on templates use the syntax:
|
|
|
|
|
|
**qq2clone** **command** [*template-name*] [*ARG*] ...
|
|
|
|
while commands that operate on clones use the syntax:
|
|
|
|
**qq2clone** \-\-template [*template-name*] **command** [*ARG*] ...
|
|
|
|
Notice that commands operating on clones work within the context of a
|
|
template defined by the option \-\-template/\-t. Conversely, commands
|
|
operating on templates specify the template as an argument to the
|
|
command. There can also be a default template defined by the
|
|
TEMPLATE option in the configuration file, allowing the \-\-template
|
|
option to be omitted for commands that operate on clones. Commands
|
|
operating on templates do not respect this default - the template must
|
|
always be explicitly defined, further reducing the likelihood of
|
|
accidentally modifying or deleting a template.
|
|
|
|
# TEMPLATE COMMMANDS
|
|
|
|
**copy-template** [*CURRENT-NAME*] [*NEW-NAME*]
|
|
: Copy the XML of template *CURRENT-NAME* to a new template with
|
|
*NEW-NAME*. The new template will not receive a copy of the old template's
|
|
storage devices - it will point to the same locations
|
|
|
|
**delete-template** [*NAME*]
|
|
: Delete the template *NAME*. This operation will succeed only if there
|
|
are currently no clones of the template
|
|
|
|
**import-template** [*LIBVIRT-DOMAIN*] [*NAME*], **import-template**
|
|
[*XML-LOCATION*] [*NAME*]
|
|
: Import a new template from either an existing libvirt domain, or a fully
|
|
qualified filepath to a libvirt domain XML file on disk. If argument
|
|
*NAME* is ommited, qq2clone will assume you want to use the machine's name
|
|
as described in the XML file as the template name
|
|
|
|
**list-templates**
|
|
: List the names of all existing templates
|
|
|
|
**modify-template** [*NAME*] **sub-command** [*ARG*] ...
|
|
: Templates can be modified in various ways by invoking
|
|
**modify-template**. Each subcommand is described below
|
|
|
|
**modify-template** [*NAME*] **commit-image**
|
|
: After an image has been created and modified as desired using
|
|
**modify-template** [*NAME*] **prepare-image**, **commit-image** is used
|
|
to alter a template's underlying storage device by commiting any changes
|
|
made using prepare-image. See the commit command described in **man**
|
|
**qemu-img** for more information on how this works
|
|
|
|
**modify-template** [*NAME*] **destroy-image**
|
|
: Invoke virsh destroy on a running image created/run through
|
|
**modify-template** [*NAME*] **prepare-image**. This is generally not
|
|
wise, as it is equivalent to unplugging a physical machine and could cause
|
|
corruption to the image that will later be commited as a permanent change
|
|
to the template's image
|
|
|
|
**modify-template** [*NAME*] **discard-image**
|
|
: Delete an image produced by **modify-template** [*NAME*]
|
|
**prepare-image** without commiting any changes
|
|
|
|
**modify-template** [*NAME*] **edit**
|
|
: Edit the XML document defining a template
|
|
|
|
**modify-template** [*NAME*] **rename** [*NEW-NAME*]
|
|
: Change the name of a template, and all of its clones
|
|
|
|
**modify-template** [*NAME*] **prepare-image**
|
|
: Create and/or run a clone that acts as a staging area for changes to
|
|
the `template's` actual image. For instance, you could update the
|
|
`template's` software by running **modify-template** [*NAME*]
|
|
**prepare-image**, updating the clone produced by this command,
|
|
shutting it down, and then running **modify-template** [*NAME*]
|
|
**commit-image**. This serves a twofold purpose - to prevent incidental
|
|
damage to an underlying image by providing a safe buffer to work in, and
|
|
to allow modifications to be safely prepared for an underlying image even
|
|
while that image has existing clones.
|
|
|
|
# CLONE COMMANDS
|
|
|
|
A description of the argument *SET* is described in the **SETS** section
|
|
below
|
|
|
|
**clone** [*NUMBER*]
|
|
: Invoke without any argument to produce a single clone. Supply a number
|
|
as an argument to specify the number of clones to create
|
|
|
|
**connect** [*SET*]
|
|
: Start any machine in *SET* that `isn't` already running. If any machine
|
|
in *SET* has spice graphics and spicy or virt-viewer is installed, use one
|
|
or the other (chosen by command-line option or configuration) to connect
|
|
to the graphical console
|
|
|
|
**destroy** [*SET*]
|
|
: Invoke virsh destroy on any running machine in *SET* (in other words, if
|
|
the domain is running forcibly turn it off)
|
|
|
|
**edit** [*NUMBER*]
|
|
: Edit the XML file of the clone with given number
|
|
|
|
**exec** [*SET*] [*command-string*]
|
|
: For every machine in *SET*, sequentially, execute the contents of the
|
|
command string in an environment where the following variables are defined
|
|
per clone: `"$uuid"`, `"$name"`, `"$disks"` (a newline delimited string
|
|
containing the machine's qcow2 disk device filepaths). This is done using
|
|
bash's eval command, so be sure to put any instances of these variables in
|
|
single quotes (double quotes inside the single quotes is best practice) or
|
|
they will not be set properly. If any instance of exec has a non-zero
|
|
return value, execution stops.
|
|
|
|
**list** [*ARG*]
|
|
: Without arguments, list all clones of the current template and their
|
|
state. With argument "all", provide list including all clones of every
|
|
template. With argument "xml", produce an XML document with information
|
|
about every template, their clones, and their state. The XML option
|
|
is not complete - its format is at this point defined only implicitly, by
|
|
the output of this command.
|
|
|
|
**resume** [*SET*]
|
|
: Resume any suspended machines in *SET*
|
|
|
|
**rm** [*SET*]
|
|
: Destroy every domain in *SET* (if running), undefine them and delete
|
|
their storage volumes
|
|
|
|
**rm-wipe** [*SET*]
|
|
: Destroy every domain in *SET* (if running), undefine them and wipe their
|
|
storage volumes using virsh
|
|
|
|
**rm-shred** [*SET*]
|
|
: Destroy every domain in *SET* (if running), undefine them and shred
|
|
their storage volumes
|
|
|
|
**save** [*SET*]
|
|
: Save execution state of every running domain in *SET* to file
|
|
|
|
**save-rm** [*SET*]
|
|
: Delete the state file associated with every machine in *SET*
|
|
|
|
**start** [*SET*]
|
|
: Start every machine in *SET* that is currently not running. For saved
|
|
domains, their state will be restored
|
|
|
|
**suspend** [*SET*]
|
|
: Suspend execution of every machine in *SET*
|
|
|
|
# OTHER COMMANDS
|
|
|
|
**check** [*TEMPLATE-NAME*]
|
|
: As described in the limitations section, there are ways that qq2clone
|
|
can lose track of a clone. If this happens, it will remain in qq2clone's
|
|
database, its ID number will remain reserved, and its image files may not
|
|
be deleted and take up space doing nothing. The **check** command
|
|
finds and fixes occurences of this problem. The *TEMPLATE-NAME* argument is
|
|
optional, and restricts the check to that template and its clones.
|
|
Otherwise, all templates are checked
|
|
|
|
**config** list, **config** info [*OPTION*], **config** edit [*OPTION*]
|
|
: List all configuration options and their current value, get info about a
|
|
particular option, or edit one
|
|
|
|
# SETS
|
|
|
|
*SET* is listed as an argument to many commands. *SET* simply describes a
|
|
set of virtual machines - clones of a given template. *SET* is a comma
|
|
delimited list with no whitespace. *SET* can be an individual machine or
|
|
several individual machines designated by number:
|
|
|
|
1 (Machine 1)
|
|
3,7 (Machines 3 and 7)
|
|
|
|
Machine numbers can be shown with **qq2clone** **list**. Ranges and
|
|
omitted values are supported as well:
|
|
|
|
1,2-5,^3 (Machines 1 and 2-5 excluding 3)
|
|
1-10,^3-7 (Machines 1-10 excluding 3-7)
|
|
|
|
Lastly, groups of machines can be addressed by their state:
|
|
|
|
all (All machines)
|
|
all,^running (All machines that aren't running)
|
|
^running,1-10 (Machines 1-10 except those that are running)
|
|
|
|
The possible states of a virtual machine are based on the states listed in
|
|
**man virsh**, with some modifications. States in qq2clone are:
|
|
|
|
all
|
|
crashed
|
|
idle
|
|
in-shutdown
|
|
off
|
|
paused
|
|
pmsuspended
|
|
running
|
|
saved
|
|
|
|
Specifying machines that do not exist will not cause an error: i.e.,
|
|
1-10 is a valid set even if only machines 3-7 exist. A set will only cause
|
|
an error if it is malformed, includes zero existing machines, contains no
|
|
machines that the command being invoked may act upon, or includes numbers
|
|
less than 1.
|
|
|
|
# CONFIG
|
|
|
|
There is no need to refer to the manual to understand configuration
|
|
options. Use "**qq2clone** config list" to see all options and their
|
|
current values, and "**qq2clone** config info [*OPTION*]" to get
|
|
information about a particular option. However, here is the same
|
|
information provided by **qq2clone** info for each option
|
|
|
|
TEMPLATE
|
|
|
|
> This template will be used for commands like clone, rm, destroy when
|
|
option \-\-template/\-t is not specified
|
|
>
|
|
> Default value: `'0'`
|
|
|
|
TEMPLATE_DIR
|
|
|
|
> This is the where template XML files will be kept
|
|
>
|
|
> Default value: `'${HOME}/storage-qq2clone/templates'`
|
|
|
|
QUIET
|
|
|
|
> If set to 1, most non-error output will be suppressed
|
|
>
|
|
> Default value: `'0'`
|
|
|
|
USE_SPICE
|
|
|
|
> If set to 1, attempt to connect
|
|
> to the spice graphics of a virtual machine by default when cloning it,
|
|
if it is configured to use spice graphics. qq2clone can do this using the
|
|
programs spicy and virt-viewer. If either is installed on your system
|
|
during the first run, the default value is `'1'` (enabled). Otherwise, the
|
|
default value is `'0'`
|
|
|
|
S_TIMEOUT
|
|
|
|
> Wait this many seconds before timing out when trying to connect to a
|
|
virtual `machine's` spice graphics.
|
|
>
|
|
> Default value: `'10'`
|
|
|
|
STORAGE
|
|
|
|
> The default location to store clone images when creating them. Changing
|
|
this location is fine, but it is a good idea to ensure that whatever
|
|
location you do choose is only used by qq2clone
|
|
>
|
|
> Default value: `'${HOME}/storage-qq2clone/qq2clone-pool'`
|
|
|
|
# EXAMPLES
|
|
|
|
**qq2clone** \-\-template Debian \-\-run \-\-virt-viewer clone
|
|
: Make a clone of Debian, run it, and connect to its spice graphics using
|
|
virt\-viewer. All of these options could have instead been defined in the
|
|
configuration, so that the entire command would be: **qq2clone** clone
|
|
|
|
**qq2clone** \-\-template Debian exec 3 'virsh console "$uuid"'
|
|
: Use virsh to connect to the serial console of template Debian's clone
|
|
with number 3 (as shown in **qq2clone** list)
|
|
|
|
**qq2clone** **modify-template** Debian *prepare-image*
|
|
: Create a clone of Debian that can be used as a staging area for
|
|
permanent changes to the backing template storage device
|
|
|
|
**qq2clone** **modify-template** Debian **commit-image**
|
|
: Commit changes to the image Debian staged with the previous command
|
|
|
|
**qq2clone** **copy-template** Debian Debian_2
|
|
: Copy the XML of template Debian, creating a new template with the same
|
|
backing storage device that you can edit as you please
|
|
|
|
# LIMITATIONS
|
|
|
|
The largest limitation of **qq2clone** is that it cannot protect your
|
|
template images from the actions of other software. If nothing else
|
|
touches a template's storage volumes, qq2clone can safely handle them
|
|
(barring unknown bugs or bad luck during a commit-image). However,
|
|
if something else alters the image upon which a template is based, its
|
|
existing clones may be corrupted and future clones may behave differently
|
|
than expected. It is the user's responsibility to understand this aspect
|
|
of copy on write and carefully manage template images. Future updates to
|
|
qq2clone may add features that give some additional protections, but this
|
|
risk is inherent to copy on write.
|
|
|
|
Libvirt has permissions errors when a storage pool is in a "hidden"
|
|
directory with a name beginning with "." and qcow2 files with backing files
|
|
are involved. This may be due to apparmor, or it may be an issue with
|
|
libvirt. It is unknown how widespread this issue is, but it is the reason
|
|
that the default directory storage-qq2clone does not start with '.'
|
|
|
|
If the UUID of a clone is changed, qq2clone will no longer be able to
|
|
track it and will not be able to perform commands on it anymore. This will
|
|
be addressed in the future using custom metadata in the libvirt domain
|
|
XML. If the user undefines a domain, this will obviously cause it to
|
|
disappear from qq2clone's perspective when it is turned off, creating a
|
|
discrepancy in its database. This can be fixed with **qq2clone** **check**.
|
|
|
|
qq2clone can only produce clones by making qcow2 image files. The backing
|
|
file need not be qcow2, but the images produced by qq2clone always will
|
|
be. This is unlikely to ever change - levaraging the features of qcow2 is
|
|
the entire purpose of qq2clone. If it does change, qq2clone will need a
|
|
new name.
|
|
|
|
qq2clone does not support creating images in pool types other than
|
|
directories, and attempting to use a machine as a template when it has
|
|
storage volumes in a non-directory pool is likely to fail or have
|
|
unexpected results. Support for some other pool types may be added in the
|
|
future.
|
|
|
|
qq2clone currently cannot copy storage volumes when importing a template
|
|
(it just references the originals), or when copying a template. This will
|
|
change in the future, and qq2clone will also be able to handle more complex
|
|
relationships between templates, clones and their images
|
|
|
|
# FILES
|
|
|
|
~/.config/qq2clone
|
|
|
|
: This document simply contains a string defining the location at which
|
|
qq2clone will store files, including the database containing the rest of
|
|
it configuration options. Currently, qq2clone cannot run without ${HOME}
|
|
being defined unless a few lines are altered to refer to a new location
|
|
|
|
~/storage-qq2clone
|
|
|
|
: Directory where qq2clone stores all files and binary executables. Can be
|
|
changed by modifying ~/.config/qq2clone. This directory is not
|
|
named "qq2clone" because (at least on Ubuntu 20.04) default Bash completion
|
|
scripts will see a file starting with "qq2clone" as well as a command in
|
|
PATH of that name, and fail to insert a space after "qq2clone"
|
|
when in the home directory. It does not start with a '.'
|
|
for the reasons described in the **LIMITATIONS** section above
|
|
|
|
~/storage-qq2clone/qq2clone.db
|
|
: sqlite3 database containing the configuration information for qq2clone,
|
|
as well as data about templates and clones
|
|
|
|
~/storage-qq2clone/qq2clone-pool
|
|
|
|
: Storage pool used for clone images, if the
|
|
\-\-storage option is not used when creating or saving a clone and the
|
|
option STORAGE is not changed in the configuration file
|
|
|
|
~/qq2clone/templates
|
|
|
|
: Directory in which template XML files are stored. These can be edited
|
|
manually, but it is more advisable to use **qq2clone** **modify-template**
|
|
[*template-name*] edit
|
|
|
|
# BUGS
|
|
|
|
As described in the options section, the implementation of the
|
|
\-\-quieter/\-Q option needs some work. Its current behavior is the
|
|
easiest functional approach without complicating the options parser, but it
|
|
will eventually be modified and become better behaved. In addition to the
|
|
previously described problem, very early error messages will not be
|
|
suppressed. Most likely, the solution is to implement a better options
|
|
parser and make it the first thing to run when executing qq2clone. However,
|
|
the impact of this bug is minimal and other improvements are likely to
|
|
come before this bug fix.
|
|
|
|
If you find any worse bugs, and I'm sure I missed some, please let me know
|
|
and I will fix them as time allows. Contact me at:
|
|
jgardner7289@protonmail.com
|
|
|
|
# EXIT VALUES
|
|
|
|
**10**
|
|
: No permission to access file or file doesn't exist
|
|
|
|
**11**
|
|
: Required software dependencies are not met (see description for a list),
|
|
or are cannot be found in PATH
|
|
|
|
**12**
|
|
: Invalid command line argument specified, or command specifies an invalid
|
|
action
|
|
|
|
**13**
|
|
: Problem with a template - i.e., specified template does not exist, or
|
|
import-template failed because template of specified name already exists
|
|
|
|
**14**
|
|
: Invocation of an external command failed
|
|
|
|
**15**
|
|
: Problem with a libvirt XML file
|
|
|
|
**16**
|
|
: Attempted action with a libvirt tool resulted in failure
|
|
|
|
**17**
|
|
: Could not establish graphical spice connection to machine before timeout
|
|
expired
|
|
|
|
**18**
|
|
: A file is of the wrong type or does not exist
|
|
|
|
**19**
|
|
: Unexpected error - a bug in qq2clone, or a highly unexpected failure of
|
|
some command
|