[v6,00/32] Add subcluster allocation to qcow2

Hi,

here's the new version of the patches to add subcluster allocation
support to qcow2.

Please refer to the cover letter of the first version for a full
description of the patches:

   https://lists.gnu.org/archive/html/qemu-block/2019-10/msg00983.html

The big change here is that the code does not need to iterate over
every subcluster anymore. There's now a new function called
qcow2_get_subcluster_range_type() that checks a group of subclusters
in one go using cto and ctz operations.

Berto

v6:
- Patch 07: Fix indentation issues in the documentation [Eric]
- Patch 14: Use half-open intervals for the subcluster macros.
            New method to detected invalid subclusters.
- Patch 15: New patch
- Patch 20: Use a faster method to check the status of a groups of
            subclusters without having to iterate over each one of
            them.
- Patch 21: Optimize count_contiguous_subclusters() by removing the
            iteration over every subcluster.
            Earlier detection of invalid subclusters.
            Minor documentation updates [Eric]
- Patch 25: Update after the macro changes in patch 14
- Patch 28: Update after the macro changes in patch 14
- Patch 29: Change patch order so this goes before patch 30
- Patch 30: Fix rebase conflicts now that compression_type has been
            merged.
            Disable toggling extended_l2 using amend [Eric]
- Patch 32: Many new tests and refactoring

v5: https://lists.gnu.org/archive/html/qemu-block/2020-05/msg00251.html
v4: https://lists.gnu.org/archive/html/qemu-block/2020-03/msg00966.html
v3: https://lists.gnu.org/archive/html/qemu-block/2019-12/msg00587.html
v2: https://lists.gnu.org/archive/html/qemu-block/2019-10/msg01642.html
v1: https://lists.gnu.org/archive/html/qemu-block/2019-10/msg00983.html

Output of git backport-diff against v5:

Key:
[----] : patches are identical
[####] : number of functional differences between upstream/downstream patch
[down] : patch is downstream-only
The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively

001/32:[----] [--] 'qcow2: Make Qcow2AioTask store the full host offset'
002/32:[----] [--] 'qcow2: Convert qcow2_get_cluster_offset() into qcow2_get_host_offset()'
003/32:[----] [--] 'qcow2: Add calculate_l2_meta()'
004/32:[----] [--] 'qcow2: Split cluster_needs_cow() out of count_cow_clusters()'
005/32:[----] [--] 'qcow2: Process QCOW2_CLUSTER_ZERO_ALLOC clusters in handle_copied()'
006/32:[----] [--] 'qcow2: Add get_l2_entry() and set_l2_entry()'
007/32:[0006] [FC] 'qcow2: Document the Extended L2 Entries feature'
008/32:[----] [--] 'qcow2: Add dummy has_subclusters() function'
009/32:[----] [--] 'qcow2: Add subcluster-related fields to BDRVQcow2State'
010/32:[----] [--] 'qcow2: Add offset_to_sc_index()'
011/32:[----] [--] 'qcow2: Add offset_into_subcluster() and size_to_subclusters()'
012/32:[----] [--] 'qcow2: Add l2_entry_size()'
013/32:[----] [--] 'qcow2: Update get/set_l2_entry() and add get/set_l2_bitmap()'
014/32:[0039] [FC] 'qcow2: Add QCow2SubclusterType and qcow2_get_subcluster_type()'
015/32:[down] 'qcow2: Add qcow2_get_subcluster_range_type()'
016/32:[----] [--] 'qcow2: Add qcow2_cluster_is_allocated()'
017/32:[----] [--] 'qcow2: Add cluster type parameter to qcow2_get_host_offset()'
018/32:[----] [--] 'qcow2: Replace QCOW2_CLUSTER_* with QCOW2_SUBCLUSTER_*'
019/32:[----] [--] 'qcow2: Handle QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC'
020/32:[0111] [FC] 'qcow2: Add subcluster support to calculate_l2_meta()'
021/32:[0091] [FC] 'qcow2: Add subcluster support to qcow2_get_host_offset()'
022/32:[----] [--] 'qcow2: Add subcluster support to zero_in_l2_slice()'
023/32:[----] [--] 'qcow2: Add subcluster support to discard_in_l2_slice()'
024/32:[----] [--] 'qcow2: Add subcluster support to check_refcounts_l2()'
025/32:[0004] [FC] 'qcow2: Update L2 bitmap in qcow2_alloc_cluster_link_l2()'
026/32:[----] [--] 'qcow2: Clear the L2 bitmap when allocating a compressed cluster'
027/32:[----] [--] 'qcow2: Add subcluster support to handle_alloc_space()'
028/32:[0004] [FC] 'qcow2: Add subcluster support to qcow2_co_pwrite_zeroes()'
029/32:[0001] [FC] 'qcow2: Add subcluster support to qcow2_measure()'
030/32:[0549] [FC] 'qcow2: Add the 'extended_l2' option and the QCOW2_INCOMPAT_EXTL2 bit'
031/32:[----] [--] 'qcow2: Assert that expand_zero_clusters_in_l1() does not support subclusters'
032/32:[0775] [FC] 'iotests: Add tests for qcow2 images with extended L2 entries'

Alberto Garcia (32):
  qcow2: Make Qcow2AioTask store the full host offset
  qcow2: Convert qcow2_get_cluster_offset() into qcow2_get_host_offset()
  qcow2: Add calculate_l2_meta()
  qcow2: Split cluster_needs_cow() out of count_cow_clusters()
  qcow2: Process QCOW2_CLUSTER_ZERO_ALLOC clusters in handle_copied()
  qcow2: Add get_l2_entry() and set_l2_entry()
  qcow2: Document the Extended L2 Entries feature
  qcow2: Add dummy has_subclusters() function
  qcow2: Add subcluster-related fields to BDRVQcow2State
  qcow2: Add offset_to_sc_index()
  qcow2: Add offset_into_subcluster() and size_to_subclusters()
  qcow2: Add l2_entry_size()
  qcow2: Update get/set_l2_entry() and add get/set_l2_bitmap()
  qcow2: Add QCow2SubclusterType and qcow2_get_subcluster_type()
  qcow2: Add qcow2_get_subcluster_range_type()
  qcow2: Add qcow2_cluster_is_allocated()
  qcow2: Add cluster type parameter to qcow2_get_host_offset()
  qcow2: Replace QCOW2_CLUSTER_* with QCOW2_SUBCLUSTER_*
  qcow2: Handle QCOW2_SUBCLUSTER_UNALLOCATED_ALLOC
  qcow2: Add subcluster support to calculate_l2_meta()
  qcow2: Add subcluster support to qcow2_get_host_offset()
  qcow2: Add subcluster support to zero_in_l2_slice()
  qcow2: Add subcluster support to discard_in_l2_slice()
  qcow2: Add subcluster support to check_refcounts_l2()
  qcow2: Update L2 bitmap in qcow2_alloc_cluster_link_l2()
  qcow2: Clear the L2 bitmap when allocating a compressed cluster
  qcow2: Add subcluster support to handle_alloc_space()
  qcow2: Add subcluster support to qcow2_co_pwrite_zeroes()
  qcow2: Add subcluster support to qcow2_measure()
  qcow2: Add the 'extended_l2' option and the QCOW2_INCOMPAT_EXTL2 bit
  qcow2: Assert that expand_zero_clusters_in_l1() does not support
    subclusters
  iotests: Add tests for qcow2 images with extended L2 entries

 docs/interop/qcow2.txt           |  68 ++-
 docs/qcow2-cache.txt             |  19 +-
 qapi/block-core.json             |   7 +
 block/qcow2.h                    | 203 ++++++-
 include/block/block_int.h        |   1 +
 block/qcow2-cluster.c            | 911 +++++++++++++++++++++----------
 block/qcow2-refcount.c           |  38 +-
 block/qcow2.c                    | 291 ++++++----
 block/trace-events               |   2 +-
 tests/qemu-iotests/031.out       |   8 +-
 tests/qemu-iotests/036.out       |   4 +-
 tests/qemu-iotests/049.out       | 102 ++--
 tests/qemu-iotests/060.out       |   1 +
 tests/qemu-iotests/061           |   6 +
 tests/qemu-iotests/061.out       |  25 +-
 tests/qemu-iotests/065           |  12 +-
 tests/qemu-iotests/082.out       |  48 +-
 tests/qemu-iotests/085.out       |  38 +-
 tests/qemu-iotests/144.out       |   4 +-
 tests/qemu-iotests/182.out       |   2 +-
 tests/qemu-iotests/185.out       |   8 +-
 tests/qemu-iotests/198.out       |   2 +
 tests/qemu-iotests/206.out       |   4 +
 tests/qemu-iotests/242.out       |   5 +
 tests/qemu-iotests/255.out       |   8 +-
 tests/qemu-iotests/271           | 705 ++++++++++++++++++++++++
 tests/qemu-iotests/271.out       | 603 ++++++++++++++++++++
 tests/qemu-iotests/274.out       |  49 +-
 tests/qemu-iotests/280.out       |   2 +-
 tests/qemu-iotests/common.filter |   1 +
 tests/qemu-iotests/group         |   1 +
 31 files changed, 2616 insertions(+), 562 deletions(-)
 create mode 100755 tests/qemu-iotests/271
 create mode 100644 tests/qemu-iotests/271.out