[v5,00/15] Hardware wrapped key support for qcom ice and ufs

The fifth iteration of patches that add support to Qualcomm ICE (Inline Crypto Engine) for hardware wrapped keys using Qualcomm Hardware Key Manager (HWKM)

They patches do the following:
- Address comments from previous versions (https://lore.kernel.org/all/20240127232436.2632187-1-quic_gaurkash@quicinc.com/)
- Tested on top of Eric's latest fscrypt and block set: https://lore.kernel.org/all/20231104211259.17448-1-ebiggers@kernel.org/
- Rebased and tested on top of Linaro's SHMBridge patches: (https://lore.kernel.org/all/20240527-shm-bridge-v10-0-ce7afaa58d3a@linaro.org/)

Explanation and use of hardware-wrapped-keys can be found here:
Documentation/block/inline-encryption.rst

Testing: 
Test platform: SM8650 MTP

The changes were tested by mounting initramfs and running the fscryptctl
tool (Ref: https://github.com/ebiggers/fscryptctl/tree/wip-wrapped-keys) to
generate and prepare keys, as well as to set policies on folders, which
consequently invokes disk encryption flows through UFS.

Tested both standard and wrapped keys (Removing qcom,ice-use-hwkm from dtsi will support using standard keys)

Steps to test:

The following configs were enabled:
CONFIG_BLK_INLINE_ENCRYPTION=y
CONFIG_QCOM_INLINE_CRYPTO_ENGINE=m
CONFIG_FS_ENCRYPTION_INLINE_CRYPT=y
CONFIG_SCSI_UFS_CRYPTO=y

Flash boot image to shell and run the following commands

Creating and preparing keys
- mkfs.ext4 -F -O encrypt,stable_inodes /dev/disk/by-partlabel/userdata
- mount /dev/disk/by-partlabel/userdata -o inlinecrypt /mnt
- ./fscryptctl generate_hw_wrapped_key /dev/disk/by-partlabel/userdata > /mnt/key.longterm  OR dd if=/dev/zero bs=32 count=1 | tr '\0' 'X' \ | fscryptctl import_hw_wrapped_key $BLOCKDEV > /mnt/key.longterm
- ./fscryptctl prepare_hw_wrapped_key /dev/disk/by-partlabel/userdata < /mnt/key.longterm > /tmp/key.ephemeral
- ./fscryptctl add_key --hw-wrapped-key < /tmp/key.ephemeral /mnt

Create a folder and associate created keys with the folder
- rm -rf /mnt/dir
- mkdir /mnt/dir
- ./fscryptctl set_policy --hw-wrapped-key --iv-ino-lblk-64 "$keyid" /mnt/dir
- dmesg > /mnt/dir/test.txt
- sync

- Reboot
- mount /dev/disk/by-partlabel/userdata -o inlinecrypt /mnt
- ls /mnt/dir (You should see an encrypted file)
- ./fscryptctl prepare_hw_wrapped_key /dev/disk/by-partlabel/userdata < /mnt/key.longterm > /tmp/key.ephemeral
- 

- cat /mnt/dir/test.txt

NOTE: Evicting a key with HWKM is not supported in the current SCM call for HWKM v2 chipsets, TZ already supports a different call for this.
Changes will be added separately for these after further internal discussions. But this should not stop merging the existing patches.

Merge Strategy:

This is an open-ended question to the community and the respective component maintainers.
The changes have the following components.

- SHMBridge patches (Bartosz Golaszewski)
- Fscrypt and block patches (From Eric Biggers)
- Qualcomm SCM (This patchset)
- Qualcomm ICE (This patchset)
- UFS Core ((This patchset))
- Qualcomm UFS Host (This patchset)

It would be ideal if one maintainer can take in all the changes together since working with many immutable branches shared with each other might get tricky.

Gaurav Kashyap (15):
  ice, ufs, mmc: use blk_crypto_key for program_key
  qcom_scm: scm call for deriving a software secret
  qcom_scm: scm call for create, prepare and import keys
  soc: qcom: ice: add hwkm support in ice
  soc: qcom: ice: support for hardware wrapped keys
  soc: qcom: ice: support for generate, import and prepare key
  ufs: core: support wrapped keys in ufs core
  ufs: core: add support to derive software secret
  ufs: core: add support for generate, import and prepare keys
  ufs: host: wrapped keys support in ufs qcom
  ufs: host: implement derive sw secret vop in ufs qcom
  ufs: host: support for generate, import and prepare key
  dt-bindings: crypto: ice: document the hwkm property
  arm64: dts: qcom: sm8650: add hwkm support to ufs ice
  arm64: dts: qcom: sm8550: add hwkm support to ufs ice

 .../crypto/qcom,inline-crypto-engine.yaml     |  10 +
 arch/arm64/boot/dts/qcom/sm8550.dtsi          |   5 +-
 arch/arm64/boot/dts/qcom/sm8650.dtsi          |   4 +-
 drivers/firmware/qcom/qcom_scm.c              | 240 ++++++++++++
 drivers/firmware/qcom/qcom_scm.h              |   4 +
 drivers/mmc/host/cqhci-crypto.c               |   7 +-
 drivers/mmc/host/cqhci.h                      |   2 +
 drivers/mmc/host/sdhci-msm.c                  |   6 +-
 drivers/soc/qcom/ice.c                        | 351 +++++++++++++++++-
 drivers/ufs/core/ufshcd-crypto.c              |  87 ++++-
 drivers/ufs/host/ufs-qcom.c                   |  61 ++-
 include/linux/firmware/qcom/qcom_scm.h        |   7 +
 include/soc/qcom/ice.h                        |  18 +-
 include/ufs/ufshcd.h                          |  22 ++
 14 files changed, 785 insertions(+), 39 deletions(-)

On Sun, Jun 16, 2024 at 05:50:59PM GMT, Gaurav Kashyap wrote:
> Qualcomm's ICE (Inline Crypto Engine) contains a proprietary
> key management hardware called Hardware Key Manager (HWKM).
> This patch integrates HWKM support in ICE when it is
> available. HWKM primarily provides hardware wrapped key support
> where the ICE (storage) keys are not available in software and
> protected in hardware.
> 
> When HWKM software support is not fully available (from Trustzone),
> there can be a scenario where the ICE hardware supports HWKM, but
> it cannot be used for wrapped keys. In this case, standard keys have
> to be used without using HWKM. Hence, providing a toggle controlled
> by a devicetree entry to use HWKM or not.
> 
> Tested-by: Neil Armstrong <neil.armstrong@linaro.org>
> Signed-off-by: Gaurav Kashyap <quic_gaurkash@quicinc.com>
> ---
>  drivers/soc/qcom/ice.c | 153 +++++++++++++++++++++++++++++++++++++++--
>  include/soc/qcom/ice.h |   1 +
>  2 files changed, 150 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/soc/qcom/ice.c b/drivers/soc/qcom/ice.c
> index 6f941d32fffb..d5e74cf2946b 100644
> --- a/drivers/soc/qcom/ice.c
> +++ b/drivers/soc/qcom/ice.c
> @@ -26,6 +26,40 @@
>  #define QCOM_ICE_REG_FUSE_SETTING		0x0010
>  #define QCOM_ICE_REG_BIST_STATUS		0x0070
>  #define QCOM_ICE_REG_ADVANCED_CONTROL		0x1000
> +#define QCOM_ICE_REG_CONTROL			0x0
> +/* QCOM ICE HWKM registers */
> +#define QCOM_ICE_REG_HWKM_TZ_KM_CTL			0x1000
> +#define QCOM_ICE_REG_HWKM_TZ_KM_STATUS			0x1004
> +#define QCOM_ICE_REG_HWKM_BANK0_BANKN_IRQ_STATUS	0x2008
> +#define QCOM_ICE_REG_HWKM_BANK0_BBAC_0			0x5000
> +#define QCOM_ICE_REG_HWKM_BANK0_BBAC_1			0x5004
> +#define QCOM_ICE_REG_HWKM_BANK0_BBAC_2			0x5008
> +#define QCOM_ICE_REG_HWKM_BANK0_BBAC_3			0x500C
> +#define QCOM_ICE_REG_HWKM_BANK0_BBAC_4			0x5010
> +
> +/* QCOM ICE HWKM reg vals */
> +#define QCOM_ICE_HWKM_BIST_DONE_V1		BIT(16)
> +#define QCOM_ICE_HWKM_BIST_DONE_V2		BIT(9)
> +#define QCOM_ICE_HWKM_BIST_DONE(ver)		QCOM_ICE_HWKM_BIST_DONE_V##ver
> +
> +#define QCOM_ICE_HWKM_CRYPTO_BIST_DONE_V1		BIT(14)
> +#define QCOM_ICE_HWKM_CRYPTO_BIST_DONE_V2		BIT(7)
> +#define QCOM_ICE_HWKM_CRYPTO_BIST_DONE(v)		QCOM_ICE_HWKM_CRYPTO_BIST_DONE_V##v
> +
> +#define QCOM_ICE_HWKM_BOOT_CMD_LIST1_DONE		BIT(2)
> +#define QCOM_ICE_HWKM_BOOT_CMD_LIST0_DONE		BIT(1)
> +#define QCOM_ICE_HWKM_KT_CLEAR_DONE			BIT(0)
> +
> +#define QCOM_ICE_HWKM_BIST_VAL(v)	(QCOM_ICE_HWKM_BIST_DONE(v) |		\
> +					QCOM_ICE_HWKM_CRYPTO_BIST_DONE(v) |	\
> +					QCOM_ICE_HWKM_BOOT_CMD_LIST1_DONE |	\
> +					QCOM_ICE_HWKM_BOOT_CMD_LIST0_DONE |	\
> +					QCOM_ICE_HWKM_KT_CLEAR_DONE)
> +
> +#define QCOM_ICE_HWKM_V1_STANDARD_MODE_VAL	(BIT(0) | BIT(1) | BIT(2))
> +#define QCOM_ICE_HWKM_V2_STANDARD_MODE_MASK	GENMASK(31, 1)
> +#define QCOM_ICE_HWKM_DISABLE_CRC_CHECKS_VAL	(BIT(1) | BIT(2))
> +#define QCOM_ICE_HWKM_RSP_FIFO_CLEAR_VAL	BIT(3)
>  
>  /* BIST ("built-in self-test") status flags */
>  #define QCOM_ICE_BIST_STATUS_MASK		GENMASK(31, 28)
> @@ -34,6 +68,9 @@
>  #define QCOM_ICE_FORCE_HW_KEY0_SETTING_MASK	0x2
>  #define QCOM_ICE_FORCE_HW_KEY1_SETTING_MASK	0x4
>  
> +#define QCOM_ICE_HWKM_REG_OFFSET	0x8000
> +#define HWKM_OFFSET(reg)		((reg) + QCOM_ICE_HWKM_REG_OFFSET)
> +
>  #define qcom_ice_writel(engine, val, reg)	\
>  	writel((val), (engine)->base + (reg))
>  
> @@ -46,6 +83,9 @@ struct qcom_ice {
>  	struct device_link *link;
>  
>  	struct clk *core_clk;
> +	u8 hwkm_version;
> +	bool use_hwkm;
> +	bool hwkm_init_complete;
>  };
>  
>  static bool qcom_ice_check_supported(struct qcom_ice *ice)
> @@ -63,8 +103,21 @@ static bool qcom_ice_check_supported(struct qcom_ice *ice)
>  		return false;
>  	}
>  
> -	dev_info(dev, "Found QC Inline Crypto Engine (ICE) v%d.%d.%d\n",
> -		 major, minor, step);
> +	if (major >= 4 || (major == 3 && minor == 2 && step >= 1))
> +		ice->hwkm_version = 2;
> +	else if (major == 3 && minor == 2)
> +		ice->hwkm_version = 1;
> +	else
> +		ice->hwkm_version = 0;
> +
> +	if (ice->hwkm_version == 0)
> +		ice->use_hwkm = false;
> +
> +	dev_info(dev, "Found QC Inline Crypto Engine (ICE) v%d.%d.%d, HWKM v%d\n",
> +		 major, minor, step, ice->hwkm_version);
> +
> +	if (!ice->use_hwkm)
> +		dev_info(dev, "QC ICE HWKM (Hardware Key Manager) not used/supported");
>  
>  	/* If fuses are blown, ICE might not work in the standard way. */
>  	regval = qcom_ice_readl(ice, QCOM_ICE_REG_FUSE_SETTING);
> @@ -113,27 +166,106 @@ static void qcom_ice_optimization_enable(struct qcom_ice *ice)
>   * fails, so we needn't do it in software too, and (c) properly testing
>   * storage encryption requires testing the full storage stack anyway,
>   * and not relying on hardware-level self-tests.
> + *
> + * However, we still care about if HWKM BIST failed (when supported) as
> + * important functionality would fail later, so disable hwkm on failure.
>   */
>  static int qcom_ice_wait_bist_status(struct qcom_ice *ice)
>  {
>  	u32 regval;
> +	u32 bist_done_val;
>  	int err;
>  
>  	err = readl_poll_timeout(ice->base + QCOM_ICE_REG_BIST_STATUS,
>  				 regval, !(regval & QCOM_ICE_BIST_STATUS_MASK),
>  				 50, 5000);
> -	if (err)
> +	if (err) {
>  		dev_err(ice->dev, "Timed out waiting for ICE self-test to complete\n");
> +		return err;
> +	}
>  
> +	if (ice->use_hwkm) {
> +		bist_done_val = ice->hwkm_version == 1 ?
> +				QCOM_ICE_HWKM_BIST_VAL(1) :
> +				QCOM_ICE_HWKM_BIST_VAL(2);
> +		if (qcom_ice_readl(ice,
> +				   HWKM_OFFSET(QCOM_ICE_REG_HWKM_TZ_KM_STATUS)) !=
> +				   bist_done_val) {
> +			dev_err(ice->dev, "HWKM BIST error\n");
> +			ice->use_hwkm = false;
> +			err = -ENODEV;
> +		}
> +	}
>  	return err;
>  }
>  
> +static void qcom_ice_enable_standard_mode(struct qcom_ice *ice)
> +{
> +	u32 val = 0;
> +
> +	/*
> +	 * When ICE is in standard (hwkm) mode, it supports HW wrapped
> +	 * keys, and when it is in legacy mode, it only supports standard
> +	 * (non HW wrapped) keys.

I can't say this is very logical.

standard mode => HW wrapped keys
legacy mode => standard keys

Consider changing the terms.

> +	 *
> +	 * Put ICE in standard mode, ICE defaults to legacy mode.
> +	 * Legacy mode - ICE HWKM slave not supported.
> +	 * Standard mode - ICE HWKM slave supported.

s/slave/some other term/

Is it possible to use both kind of keys when working on standard mode?
If not, it should be the user who selects what type of keys to be used.
Enforcing this via DT is not a way to go.

> +	 *
> +	 * Depending on the version of HWKM, it is controlled by different
> +	 * registers in ICE.
> +	 */
> +	if (ice->hwkm_version >= 2) {
> +		val = qcom_ice_readl(ice, QCOM_ICE_REG_CONTROL);
> +		val = val & QCOM_ICE_HWKM_V2_STANDARD_MODE_MASK;
> +		qcom_ice_writel(ice, val, QCOM_ICE_REG_CONTROL);
> +	} else {
> +		qcom_ice_writel(ice, QCOM_ICE_HWKM_V1_STANDARD_MODE_VAL,
> +				HWKM_OFFSET(QCOM_ICE_REG_HWKM_TZ_KM_CTL));
> +	}
> +}
> +
> +static void qcom_ice_hwkm_init(struct qcom_ice *ice)
> +{
> +	/* Disable CRC checks. This HWKM feature is not used. */
> +	qcom_ice_writel(ice, QCOM_ICE_HWKM_DISABLE_CRC_CHECKS_VAL,
> +			HWKM_OFFSET(QCOM_ICE_REG_HWKM_TZ_KM_CTL));
> +
> +	/*
> +	 * Give register bank of the HWKM slave access to read and modify
> +	 * the keyslots in ICE HWKM slave. Without this, trustzone will not
> +	 * be able to program keys into ICE.
> +	 */
> +	qcom_ice_writel(ice, GENMASK(31, 0), HWKM_OFFSET(QCOM_ICE_REG_HWKM_BANK0_BBAC_0));
> +	qcom_ice_writel(ice, GENMASK(31, 0), HWKM_OFFSET(QCOM_ICE_REG_HWKM_BANK0_BBAC_1));
> +	qcom_ice_writel(ice, GENMASK(31, 0), HWKM_OFFSET(QCOM_ICE_REG_HWKM_BANK0_BBAC_2));
> +	qcom_ice_writel(ice, GENMASK(31, 0), HWKM_OFFSET(QCOM_ICE_REG_HWKM_BANK0_BBAC_3));
> +	qcom_ice_writel(ice, GENMASK(31, 0), HWKM_OFFSET(QCOM_ICE_REG_HWKM_BANK0_BBAC_4));
> +
> +	/* Clear HWKM response FIFO before doing anything */
> +	qcom_ice_writel(ice, QCOM_ICE_HWKM_RSP_FIFO_CLEAR_VAL,
> +			HWKM_OFFSET(QCOM_ICE_REG_HWKM_BANK0_BANKN_IRQ_STATUS));
> +	ice->hwkm_init_complete = true;
> +}
> +
>  int qcom_ice_enable(struct qcom_ice *ice)
>  {
> +	int err;
> +
>  	qcom_ice_low_power_mode_enable(ice);
>  	qcom_ice_optimization_enable(ice);
>  
> -	return qcom_ice_wait_bist_status(ice);
> +	if (ice->use_hwkm)
> +		qcom_ice_enable_standard_mode(ice);
> +
> +	err = qcom_ice_wait_bist_status(ice);
> +	if (err)
> +		return err;
> +
> +	if (ice->use_hwkm)
> +		qcom_ice_hwkm_init(ice);
> +
> +	return err;
>  }
>  EXPORT_SYMBOL_GPL(qcom_ice_enable);
>  
> @@ -149,6 +281,10 @@ int qcom_ice_resume(struct qcom_ice *ice)
>  		return err;
>  	}
>  
> +	if (ice->use_hwkm) {
> +		qcom_ice_enable_standard_mode(ice);
> +		qcom_ice_hwkm_init(ice);
> +	}
>  	return qcom_ice_wait_bist_status(ice);
>  }
>  EXPORT_SYMBOL_GPL(qcom_ice_resume);
> @@ -156,6 +292,7 @@ EXPORT_SYMBOL_GPL(qcom_ice_resume);
>  int qcom_ice_suspend(struct qcom_ice *ice)
>  {
>  	clk_disable_unprepare(ice->core_clk);
> +	ice->hwkm_init_complete = false;
>  
>  	return 0;
>  }
> @@ -205,6 +342,12 @@ int qcom_ice_evict_key(struct qcom_ice *ice, int slot)
>  }
>  EXPORT_SYMBOL_GPL(qcom_ice_evict_key);
>  

Documentation?

> +bool qcom_ice_hwkm_supported(struct qcom_ice *ice)
> +{
> +	return ice->use_hwkm;

I see that use_hwkm can change during runtime. Will it have an impact on
a driver that calls this first?

> +}
> +EXPORT_SYMBOL_GPL(qcom_ice_hwkm_supported);
> +
>  static struct qcom_ice *qcom_ice_create(struct device *dev,
>  					void __iomem *base)
>  {
> @@ -239,6 +382,8 @@ static struct qcom_ice *qcom_ice_create(struct device *dev,
>  		engine->core_clk = devm_clk_get_enabled(dev, NULL);
>  	if (IS_ERR(engine->core_clk))
>  		return ERR_CAST(engine->core_clk);
> +	engine->use_hwkm = of_property_read_bool(dev->of_node,
> +						 "qcom,ice-use-hwkm");

DT bindings should come before driver changes.

>  
>  	if (!qcom_ice_check_supported(engine))
>  		return ERR_PTR(-EOPNOTSUPP);
> diff --git a/include/soc/qcom/ice.h b/include/soc/qcom/ice.h
> index 9dd835dba2a7..1f52e82e3e1c 100644
> --- a/include/soc/qcom/ice.h
> +++ b/include/soc/qcom/ice.h
> @@ -34,5 +34,6 @@ int qcom_ice_program_key(struct qcom_ice *ice,
>  			 const struct blk_crypto_key *bkey,
>  			 u8 data_unit_size, int slot);
>  int qcom_ice_evict_key(struct qcom_ice *ice, int slot);
> +bool qcom_ice_hwkm_supported(struct qcom_ice *ice);
>  struct qcom_ice *of_qcom_ice_get(struct device *dev);
>  #endif /* __QCOM_ICE_H__ */
> -- 
> 2.43.0
>

Hi,

On 17/06/2024 02:51, Gaurav Kashyap wrote:
> The Inline Crypto Engine (ICE) for UFS/EMMC supports the
> Hardware Key Manager (HWKM) to securely manage storage
> keys. Enable using this hardware on sm8650.
> 
> This requires two changes:
> 1. Register size increase: HWKM is an additional piece of hardware
>     sitting alongside ICE, and extends the old ICE's register space.
> 2. Explicitly tell the ICE driver to use HWKM with ICE so that
>     wrapped keys are used in sm8650.
> 
> Reviewed-by: Om Prakash Singh <quic_omprsing@quicinc.com>
> Tested-by: Neil Armstrong <neil.armstrong@linaro.org>
> Signed-off-by: Gaurav Kashyap <quic_gaurkash@quicinc.com>
> ---
>   arch/arm64/boot/dts/qcom/sm8650.dtsi | 4 +++-
>   1 file changed, 3 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/arm64/boot/dts/qcom/sm8650.dtsi b/arch/arm64/boot/dts/qcom/sm8650.dtsi
> index bb0b3c48ee4b..a34c4b7ccbac 100644
> --- a/arch/arm64/boot/dts/qcom/sm8650.dtsi
> +++ b/arch/arm64/boot/dts/qcom/sm8650.dtsi
> @@ -2593,9 +2593,11 @@ &mc_virt SLAVE_EBI1 QCOM_ICC_TAG_ALWAYS>,
>   		ice: crypto@1d88000 {
>   			compatible = "qcom,sm8650-inline-crypto-engine",
>   				     "qcom,inline-crypto-engine";
> -			reg = <0 0x01d88000 0 0x8000>;
> +			reg = <0 0x01d88000 0 0x10000>;
>   
>   			clocks = <&gcc GCC_UFS_PHY_ICE_CORE_CLK>;
> +
> +			qcom,ice-use-hwkm;
>   		};
>   
>   		tcsr_mutex: hwlock@1f40000 {

Please split this (and next) in two patches:
- one extending the register size + Fixes tag so it can backported to stable kernels
- one adding qcom,ice-use-hwkm (if bindings maintainers agrees with this property)

And please send sm8550 before sm8650...

Thanks,
Neil

On 6/17/24 02:51, Gaurav Kashyap wrote:
> Block crypto allows storage controllers like UFS to
> register an op derive a software secret from wrapped
> keys added to the kernel.
> 
> Wrapped keys in most cases will have vendor specific
> implementations, which means this op would need to have
> a corresponding UFS variant op.
> This change adds hooks in UFS core to support this variant
> ops and tie them to the blk crypto op.
> 
> Tested-by: Neil Armstrong <neil.armstrong@linaro.org>
> Reviewed-by: Om Prakash Singh <quic_omprsing@quicinc.com>
> Signed-off-by: Gaurav Kashyap <quic_gaurkash@quicinc.com>
> ---
>   drivers/ufs/core/ufshcd-crypto.c | 15 +++++++++++++++
>   include/ufs/ufshcd.h             |  4 ++++
>   2 files changed, 19 insertions(+)
> 
> diff --git a/drivers/ufs/core/ufshcd-crypto.c b/drivers/ufs/core/ufshcd-crypto.c
> index 399b55d67b3b..c14800eac1ff 100644
> --- a/drivers/ufs/core/ufshcd-crypto.c
> +++ b/drivers/ufs/core/ufshcd-crypto.c
> @@ -119,6 +119,20 @@ static int ufshcd_crypto_keyslot_evict(struct blk_crypto_profile *profile,
>   	return ufshcd_clear_keyslot(hba, slot);
>   }
>   
> +static int ufshcd_crypto_derive_sw_secret(struct blk_crypto_profile *profile,
> +					  const u8 wkey[], size_t wkey_size,
> +					  u8 sw_secret[BLK_CRYPTO_SW_SECRET_SIZE])
> +{
> +	struct ufs_hba *hba =
> +		container_of(profile, struct ufs_hba, crypto_profile);
> +
> +	if (hba->vops && hba->vops->derive_sw_secret)
> +		return  hba->vops->derive_sw_secret(hba, wkey, wkey_size,

Double space

Konrad

On 17/06/2024 02:50, Gaurav Kashyap wrote:
> Qualcomm's ICE (Inline Crypto Engine) contains a proprietary
> key management hardware called Hardware Key Manager (HWKM).
> This patch integrates HWKM support in ICE when it is
> available. HWKM primarily provides hardware wrapped key support
> where the ICE (storage) keys are not available in software and
> protected in hardware.
> 
> When HWKM software support is not fully available (from Trustzone),
> there can be a scenario where the ICE hardware supports HWKM, but
> it cannot be used for wrapped keys. In this case, standard keys have
> to be used without using HWKM. Hence, providing a toggle controlled
> by a devicetree entry to use HWKM or not.
> 
> Tested-by: Neil Armstrong <neil.armstrong@linaro.org>
> Signed-off-by: Gaurav Kashyap <quic_gaurkash@quicinc.com>
> ---
>   drivers/soc/qcom/ice.c | 153 +++++++++++++++++++++++++++++++++++++++--
>   include/soc/qcom/ice.h |   1 +
>   2 files changed, 150 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/soc/qcom/ice.c b/drivers/soc/qcom/ice.c
> index 6f941d32fffb..d5e74cf2946b 100644
> --- a/drivers/soc/qcom/ice.c
> +++ b/drivers/soc/qcom/ice.c
> @@ -26,6 +26,40 @@

<snip>

> +
>   static struct qcom_ice *qcom_ice_create(struct device *dev,
>   					void __iomem *base)
>   {
> @@ -239,6 +382,8 @@ static struct qcom_ice *qcom_ice_create(struct device *dev,
>   		engine->core_clk = devm_clk_get_enabled(dev, NULL);
>   	if (IS_ERR(engine->core_clk))
>   		return ERR_CAST(engine->core_clk);
> +	engine->use_hwkm = of_property_read_bool(dev->of_node,
> +						 "qcom,ice-use-hwkm");

Please drop this property and instead add an scm function calling:

__qcom_scm_is_call_available(QCOM_SCM_SVC_ES, QCOM_SCM_ES_DERIVE_SW_SECRET)

like

bool qcom_scm_derive_sw_secret_available(void)
{
	if (!__qcom_scm_is_call_available(__scm->dev, QCOM_SCM_SVC_ES,
					  QCOM_SCM_ES_DERIVE_SW_SECRET))
		return false;

	return true;
}

You may perhaps only call qcom_scm_derive_sw_secret_available() for some
ICE versions.

Neil

>   
>   	if (!qcom_ice_check_supported(engine))
>   		return ERR_PTR(-EOPNOTSUPP);
> diff --git a/include/soc/qcom/ice.h b/include/soc/qcom/ice.h
> index 9dd835dba2a7..1f52e82e3e1c 100644
> --- a/include/soc/qcom/ice.h
> +++ b/include/soc/qcom/ice.h
> @@ -34,5 +34,6 @@ int qcom_ice_program_key(struct qcom_ice *ice,
>   			 const struct blk_crypto_key *bkey,
>   			 u8 data_unit_size, int slot);
>   int qcom_ice_evict_key(struct qcom_ice *ice, int slot);
> +bool qcom_ice_hwkm_supported(struct qcom_ice *ice);
>   struct qcom_ice *of_qcom_ice_get(struct device *dev);
>   #endif /* __QCOM_ICE_H__ */

Hello Dmitry,

On 06/17/2024 12:55 AM PDT, Dmitry Baryshkov wrote:
> On Sun, Jun 16, 2024 at 05:50:59PM GMT, Gaurav Kashyap wrote:
> > Qualcomm's ICE (Inline Crypto Engine) contains a proprietary key
> > management hardware called Hardware Key Manager (HWKM).
> > This patch integrates HWKM support in ICE when it is available. HWKM
> > primarily provides hardware wrapped key support where the ICE
> > (storage) keys are not available in software and protected in
> > hardware.
> >
> > When HWKM software support is not fully available (from Trustzone),
> > there can be a scenario where the ICE hardware supports HWKM, but it
> > cannot be used for wrapped keys. In this case, standard keys have to
> > be used without using HWKM. Hence, providing a toggle controlled by a
> > devicetree entry to use HWKM or not.
> >
> > Tested-by: Neil Armstrong <neil.armstrong@linaro.org>
> > Signed-off-by: Gaurav Kashyap <quic_gaurkash@quicinc.com>
> > ---
> >  drivers/soc/qcom/ice.c | 153
> +++++++++++++++++++++++++++++++++++++++--
> >  include/soc/qcom/ice.h |   1 +
> >  2 files changed, 150 insertions(+), 4 deletions(-)
> >
> > diff --git a/drivers/soc/qcom/ice.c b/drivers/soc/qcom/ice.c index
> > 6f941d32fffb..d5e74cf2946b 100644
> > --- a/drivers/soc/qcom/ice.c
> > +++ b/drivers/soc/qcom/ice.c
> > @@ -26,6 +26,40 @@
> >  #define QCOM_ICE_REG_FUSE_SETTING            0x0010
> >  #define QCOM_ICE_REG_BIST_STATUS             0x0070
> >  #define QCOM_ICE_REG_ADVANCED_CONTROL                0x1000
> > +#define QCOM_ICE_REG_CONTROL                 0x0
> > +/* QCOM ICE HWKM registers */
> > +#define QCOM_ICE_REG_HWKM_TZ_KM_CTL                  0x1000
> > +#define QCOM_ICE_REG_HWKM_TZ_KM_STATUS                       0x1004
> > +#define QCOM_ICE_REG_HWKM_BANK0_BANKN_IRQ_STATUS     0x2008
> > +#define QCOM_ICE_REG_HWKM_BANK0_BBAC_0                       0x5000
> > +#define QCOM_ICE_REG_HWKM_BANK0_BBAC_1                       0x5004
> > +#define QCOM_ICE_REG_HWKM_BANK0_BBAC_2                       0x5008
> > +#define QCOM_ICE_REG_HWKM_BANK0_BBAC_3                       0x500C
> > +#define QCOM_ICE_REG_HWKM_BANK0_BBAC_4                       0x5010
> > +
> > +/* QCOM ICE HWKM reg vals */
> > +#define QCOM_ICE_HWKM_BIST_DONE_V1           BIT(16)
> > +#define QCOM_ICE_HWKM_BIST_DONE_V2           BIT(9)
> > +#define QCOM_ICE_HWKM_BIST_DONE(ver)
> QCOM_ICE_HWKM_BIST_DONE_V##ver
> > +
> > +#define QCOM_ICE_HWKM_CRYPTO_BIST_DONE_V1            BIT(14)
> > +#define QCOM_ICE_HWKM_CRYPTO_BIST_DONE_V2            BIT(7)
> > +#define QCOM_ICE_HWKM_CRYPTO_BIST_DONE(v)
> QCOM_ICE_HWKM_CRYPTO_BIST_DONE_V##v
> > +
> > +#define QCOM_ICE_HWKM_BOOT_CMD_LIST1_DONE            BIT(2)
> > +#define QCOM_ICE_HWKM_BOOT_CMD_LIST0_DONE            BIT(1)
> > +#define QCOM_ICE_HWKM_KT_CLEAR_DONE                  BIT(0)
> > +
> > +#define QCOM_ICE_HWKM_BIST_VAL(v)
> (QCOM_ICE_HWKM_BIST_DONE(v) |           \
> > +                                     QCOM_ICE_HWKM_CRYPTO_BIST_DONE(v) |     \
> > +                                     QCOM_ICE_HWKM_BOOT_CMD_LIST1_DONE |     \
> > +                                     QCOM_ICE_HWKM_BOOT_CMD_LIST0_DONE |     \
> > +                                     QCOM_ICE_HWKM_KT_CLEAR_DONE)
> > +
> > +#define QCOM_ICE_HWKM_V1_STANDARD_MODE_VAL   (BIT(0) | BIT(1)
> | BIT(2))
> > +#define QCOM_ICE_HWKM_V2_STANDARD_MODE_MASK
> GENMASK(31, 1) #define
> > +QCOM_ICE_HWKM_DISABLE_CRC_CHECKS_VAL (BIT(1) | BIT(2))
> > +#define QCOM_ICE_HWKM_RSP_FIFO_CLEAR_VAL     BIT(3)
> >
> >  /* BIST ("built-in self-test") status flags */
> >  #define QCOM_ICE_BIST_STATUS_MASK            GENMASK(31, 28)
> > @@ -34,6 +68,9 @@
> >  #define QCOM_ICE_FORCE_HW_KEY0_SETTING_MASK  0x2  #define
> > QCOM_ICE_FORCE_HW_KEY1_SETTING_MASK  0x4
> >
> > +#define QCOM_ICE_HWKM_REG_OFFSET     0x8000
> > +#define HWKM_OFFSET(reg)             ((reg) +
> QCOM_ICE_HWKM_REG_OFFSET)
> > +
> >  #define qcom_ice_writel(engine, val, reg)    \
> >       writel((val), (engine)->base + (reg))
> >
> > @@ -46,6 +83,9 @@ struct qcom_ice {
> >       struct device_link *link;
> >
> >       struct clk *core_clk;
> > +     u8 hwkm_version;
> > +     bool use_hwkm;
> > +     bool hwkm_init_complete;
> >  };
> >
> >  static bool qcom_ice_check_supported(struct qcom_ice *ice) @@ -63,8
> > +103,21 @@ static bool qcom_ice_check_supported(struct qcom_ice *ice)
> >               return false;
> >       }
> >
> > -     dev_info(dev, "Found QC Inline Crypto Engine (ICE) v%d.%d.%d\n",
> > -              major, minor, step);
> > +     if (major >= 4 || (major == 3 && minor == 2 && step >= 1))
> > +             ice->hwkm_version = 2;
> > +     else if (major == 3 && minor == 2)
> > +             ice->hwkm_version = 1;
> > +     else
> > +             ice->hwkm_version = 0;
> > +
> > +     if (ice->hwkm_version == 0)
> > +             ice->use_hwkm = false;
> > +
> > +     dev_info(dev, "Found QC Inline Crypto Engine (ICE) v%d.%d.%d,
> HWKM v%d\n",
> > +              major, minor, step, ice->hwkm_version);
> > +
> > +     if (!ice->use_hwkm)
> > +             dev_info(dev, "QC ICE HWKM (Hardware Key Manager) not
> > + used/supported");
> >
> >       /* If fuses are blown, ICE might not work in the standard way. */
> >       regval = qcom_ice_readl(ice, QCOM_ICE_REG_FUSE_SETTING); @@
> > -113,27 +166,106 @@ static void qcom_ice_optimization_enable(struct
> qcom_ice *ice)
> >   * fails, so we needn't do it in software too, and (c) properly testing
> >   * storage encryption requires testing the full storage stack anyway,
> >   * and not relying on hardware-level self-tests.
> > + *
> > + * However, we still care about if HWKM BIST failed (when supported)
> > + as
> > + * important functionality would fail later, so disable hwkm on failure.
> >   */
> >  static int qcom_ice_wait_bist_status(struct qcom_ice *ice)  {
> >       u32 regval;
> > +     u32 bist_done_val;
> >       int err;
> >
> >       err = readl_poll_timeout(ice->base + QCOM_ICE_REG_BIST_STATUS,
> >                                regval, !(regval & QCOM_ICE_BIST_STATUS_MASK),
> >                                50, 5000);
> > -     if (err)
> > +     if (err) {
> >               dev_err(ice->dev, "Timed out waiting for ICE self-test
> > to complete\n");
> > +             return err;
> > +     }
> >
> > +     if (ice->use_hwkm) {
> > +             bist_done_val = ice->hwkm_version == 1 ?
> > +                             QCOM_ICE_HWKM_BIST_VAL(1) :
> > +                             QCOM_ICE_HWKM_BIST_VAL(2);
> > +             if (qcom_ice_readl(ice,
> > +
> HWKM_OFFSET(QCOM_ICE_REG_HWKM_TZ_KM_STATUS)) !=
> > +                                bist_done_val) {
> > +                     dev_err(ice->dev, "HWKM BIST error\n");
> > +                     ice->use_hwkm = false;
> > +                     err = -ENODEV;
> > +             }
> > +     }
> >       return err;
> >  }
> >
> > +static void qcom_ice_enable_standard_mode(struct qcom_ice *ice) {
> > +     u32 val = 0;
> > +
> > +     /*
> > +      * When ICE is in standard (hwkm) mode, it supports HW wrapped
> > +      * keys, and when it is in legacy mode, it only supports standard
> > +      * (non HW wrapped) keys.
> 
> I can't say this is very logical.
> 
> standard mode => HW wrapped keys
> legacy mode => standard keys
> 
> Consider changing the terms.
> 

Ack, will make this clearer

> > +      *
> > +      * Put ICE in standard mode, ICE defaults to legacy mode.
> > +      * Legacy mode - ICE HWKM slave not supported.
> > +      * Standard mode - ICE HWKM slave supported.
> 
> s/slave/some other term/
> 
Ack - will address this.

> Is it possible to use both kind of keys when working on standard mode?
> If not, it should be the user who selects what type of keys to be used.
> Enforcing this via DT is not a way to go.
> 

Unfortunately, that support is not there yet. When you say user, do you mean to have it as a filesystem
mount option?

The way the UFS/EMMC crypto layer is designed currently is that, this information
is needed when the modules are loaded.
https://lore.kernel.org/all/20231104211259.17448-2-ebiggers@kernel.org/#Z31drivers:ufs:core:ufshcd-crypto.c

I am thinking of a way now to do this with DT, but without having a new vendor property.
Is it acceptable to use the addressable range as the deciding factor? Say use legacy mode of ICE
when the addressable size is 0x8000 and use HWKM mode of ICE when the addressable size is
0x10000.

> > +      *
> > +      * Depending on the version of HWKM, it is controlled by different
> > +      * registers in ICE.
> > +      */
> > +     if (ice->hwkm_version >= 2) {
> > +             val = qcom_ice_readl(ice, QCOM_ICE_REG_CONTROL);
> > +             val = val & QCOM_ICE_HWKM_V2_STANDARD_MODE_MASK;
> > +             qcom_ice_writel(ice, val, QCOM_ICE_REG_CONTROL);
> > +     } else {
> > +             qcom_ice_writel(ice,
> QCOM_ICE_HWKM_V1_STANDARD_MODE_VAL,
> > +                             HWKM_OFFSET(QCOM_ICE_REG_HWKM_TZ_KM_CTL));
> > +     }
> > +}
> > +
> > +static void qcom_ice_hwkm_init(struct qcom_ice *ice) {
> > +     /* Disable CRC checks. This HWKM feature is not used. */
> > +     qcom_ice_writel(ice, QCOM_ICE_HWKM_DISABLE_CRC_CHECKS_VAL,
> > +                     HWKM_OFFSET(QCOM_ICE_REG_HWKM_TZ_KM_CTL));
> > +
> > +     /*
> > +      * Give register bank of the HWKM slave access to read and modify
> > +      * the keyslots in ICE HWKM slave. Without this, trustzone will not
> > +      * be able to program keys into ICE.
> > +      */
> > +     qcom_ice_writel(ice, GENMASK(31, 0),
> HWKM_OFFSET(QCOM_ICE_REG_HWKM_BANK0_BBAC_0));
> > +     qcom_ice_writel(ice, GENMASK(31, 0),
> HWKM_OFFSET(QCOM_ICE_REG_HWKM_BANK0_BBAC_1));
> > +     qcom_ice_writel(ice, GENMASK(31, 0),
> HWKM_OFFSET(QCOM_ICE_REG_HWKM_BANK0_BBAC_2));
> > +     qcom_ice_writel(ice, GENMASK(31, 0),
> HWKM_OFFSET(QCOM_ICE_REG_HWKM_BANK0_BBAC_3));
> > +     qcom_ice_writel(ice, GENMASK(31, 0),
> > + HWKM_OFFSET(QCOM_ICE_REG_HWKM_BANK0_BBAC_4));
> > +
> > +     /* Clear HWKM response FIFO before doing anything */
> > +     qcom_ice_writel(ice, QCOM_ICE_HWKM_RSP_FIFO_CLEAR_VAL,
> > +
> HWKM_OFFSET(QCOM_ICE_REG_HWKM_BANK0_BANKN_IRQ_STATUS));
> > +     ice->hwkm_init_complete = true;
> > +}
> > +
> >  int qcom_ice_enable(struct qcom_ice *ice)  {
> > +     int err;
> > +
> >       qcom_ice_low_power_mode_enable(ice);
> >       qcom_ice_optimization_enable(ice);
> >
> > -     return qcom_ice_wait_bist_status(ice);
> > +     if (ice->use_hwkm)
> > +             qcom_ice_enable_standard_mode(ice);
> > +
> > +     err = qcom_ice_wait_bist_status(ice);
> > +     if (err)
> > +             return err;
> > +
> > +     if (ice->use_hwkm)
> > +             qcom_ice_hwkm_init(ice);
> > +
> > +     return err;
> >  }
> >  EXPORT_SYMBOL_GPL(qcom_ice_enable);
> >
> > @@ -149,6 +281,10 @@ int qcom_ice_resume(struct qcom_ice *ice)
> >               return err;
> >       }
> >
> > +     if (ice->use_hwkm) {
> > +             qcom_ice_enable_standard_mode(ice);
> > +             qcom_ice_hwkm_init(ice);
> > +     }
> >       return qcom_ice_wait_bist_status(ice);  }
> > EXPORT_SYMBOL_GPL(qcom_ice_resume);
> > @@ -156,6 +292,7 @@ EXPORT_SYMBOL_GPL(qcom_ice_resume);
> >  int qcom_ice_suspend(struct qcom_ice *ice)  {
> >       clk_disable_unprepare(ice->core_clk);
> > +     ice->hwkm_init_complete = false;
> >
> >       return 0;
> >  }
> > @@ -205,6 +342,12 @@ int qcom_ice_evict_key(struct qcom_ice *ice, int
> > slot)  }  EXPORT_SYMBOL_GPL(qcom_ice_evict_key);
> >
> 
> Documentation?
> 
> > +bool qcom_ice_hwkm_supported(struct qcom_ice *ice)
> > +{
> > +     return ice->use_hwkm;
> 
> I see that use_hwkm can change during runtime. Will it have an impact on
> a driver that calls this first?
> 
> > +}
> > +EXPORT_SYMBOL_GPL(qcom_ice_hwkm_supported);
> > +
> >  static struct qcom_ice *qcom_ice_create(struct device *dev,
> >                                       void __iomem *base)
> >  {
> > @@ -239,6 +382,8 @@ static struct qcom_ice *qcom_ice_create(struct
> device *dev,
> >               engine->core_clk = devm_clk_get_enabled(dev, NULL);
> >       if (IS_ERR(engine->core_clk))
> >               return ERR_CAST(engine->core_clk);
> > +     engine->use_hwkm = of_property_read_bool(dev->of_node,
> > +                                              "qcom,ice-use-hwkm");
> 
> DT bindings should come before driver changes.
> 
> >
> >       if (!qcom_ice_check_supported(engine))
> >               return ERR_PTR(-EOPNOTSUPP);
> > diff --git a/include/soc/qcom/ice.h b/include/soc/qcom/ice.h
> > index 9dd835dba2a7..1f52e82e3e1c 100644
> > --- a/include/soc/qcom/ice.h
> > +++ b/include/soc/qcom/ice.h
> > @@ -34,5 +34,6 @@ int qcom_ice_program_key(struct qcom_ice *ice,
> >                        const struct blk_crypto_key *bkey,
> >                        u8 data_unit_size, int slot);
> >  int qcom_ice_evict_key(struct qcom_ice *ice, int slot);
> > +bool qcom_ice_hwkm_supported(struct qcom_ice *ice);
> >  struct qcom_ice *of_qcom_ice_get(struct device *dev);
> >  #endif /* __QCOM_ICE_H__ */
> > --
> > 2.43.0
> >
> 
> --
> With best wishes
> Dmitry

Regards,
Gaurav

Hello Neil,

On 06/18/2024 12:14 AM PDT, Neil Armstrong wrote:
> On 17/06/2024 02:50, Gaurav Kashyap wrote:
> > Qualcomm's ICE (Inline Crypto Engine) contains a proprietary key
> > management hardware called Hardware Key Manager (HWKM).
> > This patch integrates HWKM support in ICE when it is available. HWKM
> > primarily provides hardware wrapped key support where the ICE
> > (storage) keys are not available in software and protected in
> > hardware.
> >
> > When HWKM software support is not fully available (from Trustzone),
> > there can be a scenario where the ICE hardware supports HWKM, but it
> > cannot be used for wrapped keys. In this case, standard keys have to
> > be used without using HWKM. Hence, providing a toggle controlled by a
> > devicetree entry to use HWKM or not.
> >
> > Tested-by: Neil Armstrong <neil.armstrong@linaro.org>
> > Signed-off-by: Gaurav Kashyap <quic_gaurkash@quicinc.com>
> > ---
> >   drivers/soc/qcom/ice.c | 153
> +++++++++++++++++++++++++++++++++++++++--
> >   include/soc/qcom/ice.h |   1 +
> >   2 files changed, 150 insertions(+), 4 deletions(-)
> >
> > diff --git a/drivers/soc/qcom/ice.c b/drivers/soc/qcom/ice.c index
> > 6f941d32fffb..d5e74cf2946b 100644
> > --- a/drivers/soc/qcom/ice.c
> > +++ b/drivers/soc/qcom/ice.c
> > @@ -26,6 +26,40 @@
> 
> <snip>
> 
> > +
> >   static struct qcom_ice *qcom_ice_create(struct device *dev,
> >                                       void __iomem *base)
> >   {
> > @@ -239,6 +382,8 @@ static struct qcom_ice *qcom_ice_create(struct
> device *dev,
> >               engine->core_clk = devm_clk_get_enabled(dev, NULL);
> >       if (IS_ERR(engine->core_clk))
> >               return ERR_CAST(engine->core_clk);
> > +     engine->use_hwkm = of_property_read_bool(dev->of_node,
> > +                                              "qcom,ice-use-hwkm");
> 
> Please drop this property and instead add an scm function calling:
> 
> __qcom_scm_is_call_available(QCOM_SCM_SVC_ES,
> QCOM_SCM_ES_DERIVE_SW_SECRET)
> 
> like
> 
> bool qcom_scm_derive_sw_secret_available(void)
> {
>         if (!__qcom_scm_is_call_available(__scm->dev, QCOM_SCM_SVC_ES,
>                                           QCOM_SCM_ES_DERIVE_SW_SECRET))
>                 return false;
> 
>         return true;
> }
> 
> You may perhaps only call qcom_scm_derive_sw_secret_available() for
> some ICE versions.
> 
> Neil

The issue here is that for the same ICE version, based on the chipset,
there might be different configurations.

Is it acceptable to use the addressable size from DTSI instead?
Meaning, if it 0x8000, it would take the legacy route, and only when it has been
updated to 0x10000, we would use HWKM and wrapped keys.

> 
> >
> >       if (!qcom_ice_check_supported(engine))
> >               return ERR_PTR(-EOPNOTSUPP); diff --git
> > a/include/soc/qcom/ice.h b/include/soc/qcom/ice.h index
> > 9dd835dba2a7..1f52e82e3e1c 100644
> > --- a/include/soc/qcom/ice.h
> > +++ b/include/soc/qcom/ice.h
> > @@ -34,5 +34,6 @@ int qcom_ice_program_key(struct qcom_ice *ice,
> >                        const struct blk_crypto_key *bkey,
> >                        u8 data_unit_size, int slot);
> >   int qcom_ice_evict_key(struct qcom_ice *ice, int slot);
> > +bool qcom_ice_hwkm_supported(struct qcom_ice *ice);
> >   struct qcom_ice *of_qcom_ice_get(struct device *dev);
> >   #endif /* __QCOM_ICE_H__ */

Regards,
Gaurav

On Wed, 19 Jun 2024 at 01:07, Gaurav Kashyap (QUIC)
<quic_gaurkash@quicinc.com> wrote:
>
> Hello Dmitry,
>
> On 06/17/2024 12:55 AM PDT, Dmitry Baryshkov wrote:
> > On Sun, Jun 16, 2024 at 05:50:59PM GMT, Gaurav Kashyap wrote:
> > > Qualcomm's ICE (Inline Crypto Engine) contains a proprietary key
> > > management hardware called Hardware Key Manager (HWKM).
> > > This patch integrates HWKM support in ICE when it is available. HWKM
> > > primarily provides hardware wrapped key support where the ICE
> > > (storage) keys are not available in software and protected in
> > > hardware.
> > >
> > > When HWKM software support is not fully available (from Trustzone),
> > > there can be a scenario where the ICE hardware supports HWKM, but it
> > > cannot be used for wrapped keys. In this case, standard keys have to
> > > be used without using HWKM. Hence, providing a toggle controlled by a
> > > devicetree entry to use HWKM or not.
> > >
> > > Tested-by: Neil Armstrong <neil.armstrong@linaro.org>
> > > Signed-off-by: Gaurav Kashyap <quic_gaurkash@quicinc.com>
> > > ---
> > >  drivers/soc/qcom/ice.c | 153
> > +++++++++++++++++++++++++++++++++++++++--
> > >  include/soc/qcom/ice.h |   1 +
> > >  2 files changed, 150 insertions(+), 4 deletions(-)
> > >
> > > diff --git a/drivers/soc/qcom/ice.c b/drivers/soc/qcom/ice.c index
> > > 6f941d32fffb..d5e74cf2946b 100644
> > > --- a/drivers/soc/qcom/ice.c
> > > +++ b/drivers/soc/qcom/ice.c
> > > @@ -26,6 +26,40 @@
> > >  #define QCOM_ICE_REG_FUSE_SETTING            0x0010
> > >  #define QCOM_ICE_REG_BIST_STATUS             0x0070
> > >  #define QCOM_ICE_REG_ADVANCED_CONTROL                0x1000
> > > +#define QCOM_ICE_REG_CONTROL                 0x0
> > > +/* QCOM ICE HWKM registers */
> > > +#define QCOM_ICE_REG_HWKM_TZ_KM_CTL                  0x1000
> > > +#define QCOM_ICE_REG_HWKM_TZ_KM_STATUS                       0x1004
> > > +#define QCOM_ICE_REG_HWKM_BANK0_BANKN_IRQ_STATUS     0x2008
> > > +#define QCOM_ICE_REG_HWKM_BANK0_BBAC_0                       0x5000
> > > +#define QCOM_ICE_REG_HWKM_BANK0_BBAC_1                       0x5004
> > > +#define QCOM_ICE_REG_HWKM_BANK0_BBAC_2                       0x5008
> > > +#define QCOM_ICE_REG_HWKM_BANK0_BBAC_3                       0x500C
> > > +#define QCOM_ICE_REG_HWKM_BANK0_BBAC_4                       0x5010
> > > +
> > > +/* QCOM ICE HWKM reg vals */
> > > +#define QCOM_ICE_HWKM_BIST_DONE_V1           BIT(16)
> > > +#define QCOM_ICE_HWKM_BIST_DONE_V2           BIT(9)
> > > +#define QCOM_ICE_HWKM_BIST_DONE(ver)
> > QCOM_ICE_HWKM_BIST_DONE_V##ver
> > > +
> > > +#define QCOM_ICE_HWKM_CRYPTO_BIST_DONE_V1            BIT(14)
> > > +#define QCOM_ICE_HWKM_CRYPTO_BIST_DONE_V2            BIT(7)
> > > +#define QCOM_ICE_HWKM_CRYPTO_BIST_DONE(v)
> > QCOM_ICE_HWKM_CRYPTO_BIST_DONE_V##v
> > > +
> > > +#define QCOM_ICE_HWKM_BOOT_CMD_LIST1_DONE            BIT(2)
> > > +#define QCOM_ICE_HWKM_BOOT_CMD_LIST0_DONE            BIT(1)
> > > +#define QCOM_ICE_HWKM_KT_CLEAR_DONE                  BIT(0)
> > > +
> > > +#define QCOM_ICE_HWKM_BIST_VAL(v)
> > (QCOM_ICE_HWKM_BIST_DONE(v) |           \
> > > +                                     QCOM_ICE_HWKM_CRYPTO_BIST_DONE(v) |     \
> > > +                                     QCOM_ICE_HWKM_BOOT_CMD_LIST1_DONE |     \
> > > +                                     QCOM_ICE_HWKM_BOOT_CMD_LIST0_DONE |     \
> > > +                                     QCOM_ICE_HWKM_KT_CLEAR_DONE)
> > > +
> > > +#define QCOM_ICE_HWKM_V1_STANDARD_MODE_VAL   (BIT(0) | BIT(1)
> > | BIT(2))
> > > +#define QCOM_ICE_HWKM_V2_STANDARD_MODE_MASK
> > GENMASK(31, 1) #define
> > > +QCOM_ICE_HWKM_DISABLE_CRC_CHECKS_VAL (BIT(1) | BIT(2))
> > > +#define QCOM_ICE_HWKM_RSP_FIFO_CLEAR_VAL     BIT(3)
> > >
> > >  /* BIST ("built-in self-test") status flags */
> > >  #define QCOM_ICE_BIST_STATUS_MASK            GENMASK(31, 28)
> > > @@ -34,6 +68,9 @@
> > >  #define QCOM_ICE_FORCE_HW_KEY0_SETTING_MASK  0x2  #define
> > > QCOM_ICE_FORCE_HW_KEY1_SETTING_MASK  0x4
> > >
> > > +#define QCOM_ICE_HWKM_REG_OFFSET     0x8000
> > > +#define HWKM_OFFSET(reg)             ((reg) +
> > QCOM_ICE_HWKM_REG_OFFSET)
> > > +
> > >  #define qcom_ice_writel(engine, val, reg)    \
> > >       writel((val), (engine)->base + (reg))
> > >
> > > @@ -46,6 +83,9 @@ struct qcom_ice {
> > >       struct device_link *link;
> > >
> > >       struct clk *core_clk;
> > > +     u8 hwkm_version;
> > > +     bool use_hwkm;
> > > +     bool hwkm_init_complete;
> > >  };
> > >
> > >  static bool qcom_ice_check_supported(struct qcom_ice *ice) @@ -63,8
> > > +103,21 @@ static bool qcom_ice_check_supported(struct qcom_ice *ice)
> > >               return false;
> > >       }
> > >
> > > -     dev_info(dev, "Found QC Inline Crypto Engine (ICE) v%d.%d.%d\n",
> > > -              major, minor, step);
> > > +     if (major >= 4 || (major == 3 && minor == 2 && step >= 1))
> > > +             ice->hwkm_version = 2;
> > > +     else if (major == 3 && minor == 2)
> > > +             ice->hwkm_version = 1;
> > > +     else
> > > +             ice->hwkm_version = 0;
> > > +
> > > +     if (ice->hwkm_version == 0)
> > > +             ice->use_hwkm = false;
> > > +
> > > +     dev_info(dev, "Found QC Inline Crypto Engine (ICE) v%d.%d.%d,
> > HWKM v%d\n",
> > > +              major, minor, step, ice->hwkm_version);
> > > +
> > > +     if (!ice->use_hwkm)
> > > +             dev_info(dev, "QC ICE HWKM (Hardware Key Manager) not
> > > + used/supported");
> > >
> > >       /* If fuses are blown, ICE might not work in the standard way. */
> > >       regval = qcom_ice_readl(ice, QCOM_ICE_REG_FUSE_SETTING); @@
> > > -113,27 +166,106 @@ static void qcom_ice_optimization_enable(struct
> > qcom_ice *ice)
> > >   * fails, so we needn't do it in software too, and (c) properly testing
> > >   * storage encryption requires testing the full storage stack anyway,
> > >   * and not relying on hardware-level self-tests.
> > > + *
> > > + * However, we still care about if HWKM BIST failed (when supported)
> > > + as
> > > + * important functionality would fail later, so disable hwkm on failure.
> > >   */
> > >  static int qcom_ice_wait_bist_status(struct qcom_ice *ice)  {
> > >       u32 regval;
> > > +     u32 bist_done_val;
> > >       int err;
> > >
> > >       err = readl_poll_timeout(ice->base + QCOM_ICE_REG_BIST_STATUS,
> > >                                regval, !(regval & QCOM_ICE_BIST_STATUS_MASK),
> > >                                50, 5000);
> > > -     if (err)
> > > +     if (err) {
> > >               dev_err(ice->dev, "Timed out waiting for ICE self-test
> > > to complete\n");
> > > +             return err;
> > > +     }
> > >
> > > +     if (ice->use_hwkm) {
> > > +             bist_done_val = ice->hwkm_version == 1 ?
> > > +                             QCOM_ICE_HWKM_BIST_VAL(1) :
> > > +                             QCOM_ICE_HWKM_BIST_VAL(2);
> > > +             if (qcom_ice_readl(ice,
> > > +
> > HWKM_OFFSET(QCOM_ICE_REG_HWKM_TZ_KM_STATUS)) !=
> > > +                                bist_done_val) {
> > > +                     dev_err(ice->dev, "HWKM BIST error\n");
> > > +                     ice->use_hwkm = false;
> > > +                     err = -ENODEV;
> > > +             }
> > > +     }
> > >       return err;
> > >  }
> > >
> > > +static void qcom_ice_enable_standard_mode(struct qcom_ice *ice) {
> > > +     u32 val = 0;
> > > +
> > > +     /*
> > > +      * When ICE is in standard (hwkm) mode, it supports HW wrapped
> > > +      * keys, and when it is in legacy mode, it only supports standard
> > > +      * (non HW wrapped) keys.
> >
> > I can't say this is very logical.
> >
> > standard mode => HW wrapped keys
> > legacy mode => standard keys
> >
> > Consider changing the terms.
> >
>
> Ack, will make this clearer
>
> > > +      *
> > > +      * Put ICE in standard mode, ICE defaults to legacy mode.
> > > +      * Legacy mode - ICE HWKM slave not supported.
> > > +      * Standard mode - ICE HWKM slave supported.
> >
> > s/slave/some other term/
> >
> Ack - will address this.
>
> > Is it possible to use both kind of keys when working on standard mode?
> > If not, it should be the user who selects what type of keys to be used.
> > Enforcing this via DT is not a way to go.
> >
>
> Unfortunately, that support is not there yet. When you say user, do you mean to have it as a filesystem
> mount option?

During cryptsetup time. When running e.g. cryptsetup I, as a user,
would like to be able to use either a hardware-wrapped key or a
standard key.

> The way the UFS/EMMC crypto layer is designed currently is that, this information
> is needed when the modules are loaded.
>
> https://lore.kernel.org/all/20231104211259.17448-2-ebiggers@kernel.org/#Z31drivers:ufs:core:ufshcd-crypto.c

I see that the driver lists capabilities here. E.g. that it supports
HW-wrapped keys. But the line doesn't specify that standard keys are
not supported.

Also, I'd have expected that hw-wrapped keys are handled using trusted
keys mechanism (see security/keys/trusted-keys/). Could you please
point out why that's not the case?

> I am thinking of a way now to do this with DT, but without having a new vendor property.
> Is it acceptable to use the addressable range as the deciding factor? Say use legacy mode of ICE
> when the addressable size is 0x8000 and use HWKM mode of ICE when the addressable size is
> 0x10000.

Definitely, this is a NAK. It's a very unobvious hack. You have been
asked to use compatible strings to detect whether HW keys are
supported or not.

On 19/06/2024 00:08, Gaurav Kashyap (QUIC) wrote:
>>
>> You may perhaps only call qcom_scm_derive_sw_secret_available() for
>> some ICE versions.
>>
>> Neil
> 
> The issue here is that for the same ICE version, based on the chipset,
> there might be different configurations.

That's not what your DTS said. To remind: your DTS said that all SM8550
and all SM8650 have it. Choice is obvious then: it's deducible from
compatible.

I still do not understand why your call cannot return you correct
"configuration".

> 
> Is it acceptable to use the addressable size from DTSI instead?
> Meaning, if it 0x8000, it would take the legacy route, and only when it has been
> updated to 0x10000, we would use HWKM and wrapped keys.

No.

Best regards,
Krzysztof

Le 19/06/2024 à 00:08, Gaurav Kashyap (QUIC) a écrit :
> Hello Neil,
> 
> On 06/18/2024 12:14 AM PDT, Neil Armstrong wrote:
>> On 17/06/2024 02:50, Gaurav Kashyap wrote:
>>> Qualcomm's ICE (Inline Crypto Engine) contains a proprietary key
>>> management hardware called Hardware Key Manager (HWKM).
>>> This patch integrates HWKM support in ICE when it is available. HWKM
>>> primarily provides hardware wrapped key support where the ICE
>>> (storage) keys are not available in software and protected in
>>> hardware.
>>>
>>> When HWKM software support is not fully available (from Trustzone),
>>> there can be a scenario where the ICE hardware supports HWKM, but it
>>> cannot be used for wrapped keys. In this case, standard keys have to
>>> be used without using HWKM. Hence, providing a toggle controlled by a
>>> devicetree entry to use HWKM or not.
>>>
>>> Tested-by: Neil Armstrong <neil.armstrong@linaro.org>
>>> Signed-off-by: Gaurav Kashyap <quic_gaurkash@quicinc.com>
>>> ---
>>>    drivers/soc/qcom/ice.c | 153
>> +++++++++++++++++++++++++++++++++++++++--
>>>    include/soc/qcom/ice.h |   1 +
>>>    2 files changed, 150 insertions(+), 4 deletions(-)
>>>
>>> diff --git a/drivers/soc/qcom/ice.c b/drivers/soc/qcom/ice.c index
>>> 6f941d32fffb..d5e74cf2946b 100644
>>> --- a/drivers/soc/qcom/ice.c
>>> +++ b/drivers/soc/qcom/ice.c
>>> @@ -26,6 +26,40 @@
>>
>> <snip>
>>
>>> +
>>>    static struct qcom_ice *qcom_ice_create(struct device *dev,
>>>                                        void __iomem *base)
>>>    {
>>> @@ -239,6 +382,8 @@ static struct qcom_ice *qcom_ice_create(struct
>> device *dev,
>>>                engine->core_clk = devm_clk_get_enabled(dev, NULL);
>>>        if (IS_ERR(engine->core_clk))
>>>                return ERR_CAST(engine->core_clk);
>>> +     engine->use_hwkm = of_property_read_bool(dev->of_node,
>>> +                                              "qcom,ice-use-hwkm");
>>
>> Please drop this property and instead add an scm function calling:
>>
>> __qcom_scm_is_call_available(QCOM_SCM_SVC_ES,
>> QCOM_SCM_ES_DERIVE_SW_SECRET)
>>
>> like
>>
>> bool qcom_scm_derive_sw_secret_available(void)
>> {
>>          if (!__qcom_scm_is_call_available(__scm->dev, QCOM_SCM_SVC_ES,
>>                                            QCOM_SCM_ES_DERIVE_SW_SECRET))
>>                  return false;
>>
>>          return true;
>> }
>>
>> You may perhaps only call qcom_scm_derive_sw_secret_available() for
>> some ICE versions.
>>
>> Neil
> 
> The issue here is that for the same ICE version, based on the chipset,
> there might be different configurations.

So use a combination of a list of compatible strings + qcom_scm_derive_sw_secret_available()
to enable hwkm.

Neil

> 
> Is it acceptable to use the addressable size from DTSI instead?
> Meaning, if it 0x8000, it would take the legacy route, and only when it has been
> updated to 0x10000, we would use HWKM and wrapped keys.
> 
>>
>>>
>>>        if (!qcom_ice_check_supported(engine))
>>>                return ERR_PTR(-EOPNOTSUPP); diff --git
>>> a/include/soc/qcom/ice.h b/include/soc/qcom/ice.h index
>>> 9dd835dba2a7..1f52e82e3e1c 100644
>>> --- a/include/soc/qcom/ice.h
>>> +++ b/include/soc/qcom/ice.h
>>> @@ -34,5 +34,6 @@ int qcom_ice_program_key(struct qcom_ice *ice,
>>>                         const struct blk_crypto_key *bkey,
>>>                         u8 data_unit_size, int slot);
>>>    int qcom_ice_evict_key(struct qcom_ice *ice, int slot);
>>> +bool qcom_ice_hwkm_supported(struct qcom_ice *ice);
>>>    struct qcom_ice *of_qcom_ice_get(struct device *dev);
>>>    #endif /* __QCOM_ICE_H__ */
> 
> Regards,
> Gaurav

Hello Krzysztof

On 06/18/2024 11:17 PM PDT, Krzysztof Kozlowski wrote:
> On 19/06/2024 00:08, Gaurav Kashyap (QUIC) wrote:
> >>
> >> You may perhaps only call qcom_scm_derive_sw_secret_available() for
> >> some ICE versions.
> >>
> >> Neil
> >
> > The issue here is that for the same ICE version, based on the chipset,
> > there might be different configurations.
> 
> That's not what your DTS said. To remind: your DTS said that all SM8550 and
> all SM8650 have it. Choice is obvious then: it's deducible from compatible.
> 
> I still do not understand why your call cannot return you correct
> "configuration".
> 

ICE version and chipsets are disjoint, meaning for the same ICE HW present in SM8650 vs SMxxxx target,
SM8650 will have necessary TZ support, but SM8xxxx may not, that is the reason I was trying to indicate all SM8550 and
SM8650 have the necessary TZ support. There might have been a miscommunication there.

However , availability of QCOM_SCM_ES_GENERATE_ICE_KEY will directly translate to having the necessary firmware support.
So, I will pursue going that route and upload another set of patches to remove the DT property.

> >
> > Is it acceptable to use the addressable size from DTSI instead?
> > Meaning, if it 0x8000, it would take the legacy route, and only when
> > it has been updated to 0x10000, we would use HWKM and wrapped keys.
> 
> No.
>

Ack

> Best regards,
> Krzysztof

Regards,
Gaurav

On 06/19/2024 12:12 AM PDT, Neil Armstrong wrote:
> Le 19/06/2024 à 00:08, Gaurav Kashyap (QUIC) a écrit :
> > Hello Neil,
> >
> > On 06/18/2024 12:14 AM PDT, Neil Armstrong wrote:
> >> On 17/06/2024 02:50, Gaurav Kashyap wrote:
> >>> Qualcomm's ICE (Inline Crypto Engine) contains a proprietary key
> >>> management hardware called Hardware Key Manager (HWKM).
> >>> This patch integrates HWKM support in ICE when it is available. HWKM
> >>> primarily provides hardware wrapped key support where the ICE
> >>> (storage) keys are not available in software and protected in
> >>> hardware.
> >>>
> >>> When HWKM software support is not fully available (from Trustzone),
> >>> there can be a scenario where the ICE hardware supports HWKM, but it
> >>> cannot be used for wrapped keys. In this case, standard keys have to
> >>> be used without using HWKM. Hence, providing a toggle controlled by
> >>> a devicetree entry to use HWKM or not.
> >>>
> >>> Tested-by: Neil Armstrong <neil.armstrong@linaro.org>
> >>> Signed-off-by: Gaurav Kashyap <quic_gaurkash@quicinc.com>
> >>> ---
> >>>    drivers/soc/qcom/ice.c | 153
> >> +++++++++++++++++++++++++++++++++++++++--
> >>>    include/soc/qcom/ice.h |   1 +
> >>>    2 files changed, 150 insertions(+), 4 deletions(-)
> >>>
> >>> diff --git a/drivers/soc/qcom/ice.c b/drivers/soc/qcom/ice.c index
> >>> 6f941d32fffb..d5e74cf2946b 100644
> >>> --- a/drivers/soc/qcom/ice.c
> >>> +++ b/drivers/soc/qcom/ice.c
> >>> @@ -26,6 +26,40 @@
> >>
> >> <snip>
> >>
> >>> +
> >>>    static struct qcom_ice *qcom_ice_create(struct device *dev,
> >>>                                        void __iomem *base)
> >>>    {
> >>> @@ -239,6 +382,8 @@ static struct qcom_ice *qcom_ice_create(struct
> >> device *dev,
> >>>                engine->core_clk = devm_clk_get_enabled(dev, NULL);
> >>>        if (IS_ERR(engine->core_clk))
> >>>                return ERR_CAST(engine->core_clk);
> >>> +     engine->use_hwkm = of_property_read_bool(dev->of_node,
> >>> +                                              "qcom,ice-use-hwkm");
> >>
> >> Please drop this property and instead add an scm function calling:
> >>
> >> __qcom_scm_is_call_available(QCOM_SCM_SVC_ES,
> >> QCOM_SCM_ES_DERIVE_SW_SECRET)
> >>
> >> like
> >>
> >> bool qcom_scm_derive_sw_secret_available(void)
> >> {
> >>          if (!__qcom_scm_is_call_available(__scm->dev,
> QCOM_SCM_SVC_ES,
> >>                                            QCOM_SCM_ES_DERIVE_SW_SECRET))
> >>                  return false;
> >>
> >>          return true;
> >> }
> >>
> >> You may perhaps only call qcom_scm_derive_sw_secret_available() for
> >> some ICE versions.
> >>
> >> Neil
> >
> > The issue here is that for the same ICE version, based on the chipset,
> > there might be different configurations.
> 
> So use a combination of a list of compatible strings +
> qcom_scm_derive_sw_secret_available()
> to enable hwkm.
> 
> Neil
>

Okay, that makes sense to me, I will try that.
In fact, looking for only QCOM_SCM_ES_GENERATE_ICE_KEY instead of SW_SECRET works better.
And would work even without compatible strings.
As availability of QCOM_SCM_ES_GENERATE_ICE_KEY will correlate with TZ/firmware having the necessary support.

> >
> > Is it acceptable to use the addressable size from DTSI instead?
> > Meaning, if it 0x8000, it would take the legacy route, and only when
> > it has been updated to 0x10000, we would use HWKM and wrapped keys.
> >
> >>
> >>>
> >>>        if (!qcom_ice_check_supported(engine))
> >>>                return ERR_PTR(-EOPNOTSUPP); diff --git
> >>> a/include/soc/qcom/ice.h b/include/soc/qcom/ice.h index
> >>> 9dd835dba2a7..1f52e82e3e1c 100644
> >>> --- a/include/soc/qcom/ice.h
> >>> +++ b/include/soc/qcom/ice.h
> >>> @@ -34,5 +34,6 @@ int qcom_ice_program_key(struct qcom_ice *ice,
> >>>                         const struct blk_crypto_key *bkey,
> >>>                         u8 data_unit_size, int slot);
> >>>    int qcom_ice_evict_key(struct qcom_ice *ice, int slot);
> >>> +bool qcom_ice_hwkm_supported(struct qcom_ice *ice);
> >>>    struct qcom_ice *of_qcom_ice_get(struct device *dev);
> >>>    #endif /* __QCOM_ICE_H__ */
> >
> > Regards,
> > Gaurav

Regards,
Gaurav

Hello Dmitry

On 06/18/2024 3:17 PM PDT, Dmitry Baryshkov wrote:
> On Wed, 19 Jun 2024 at 01:07, Gaurav Kashyap (QUIC)
> <quic_gaurkash@quicinc.com> wrote:
> >
> > Hello Dmitry,
> >
> > On 06/17/2024 12:55 AM PDT, Dmitry Baryshkov wrote:
> > > On Sun, Jun 16, 2024 at 05:50:59PM GMT, Gaurav Kashyap wrote:
> > > > Qualcomm's ICE (Inline Crypto Engine) contains a proprietary key
> > > > management hardware called Hardware Key Manager (HWKM).
> > > > This patch integrates HWKM support in ICE when it is available.
> > > > HWKM primarily provides hardware wrapped key support where the
> ICE
> > > > (storage) keys are not available in software and protected in
> > > > hardware.
> > > >
> > > > When HWKM software support is not fully available (from
> > > > Trustzone), there can be a scenario where the ICE hardware
> > > > supports HWKM, but it cannot be used for wrapped keys. In this
> > > > case, standard keys have to be used without using HWKM. Hence,
> > > > providing a toggle controlled by a devicetree entry to use HWKM or not.
> > > >
> > > > Tested-by: Neil Armstrong <neil.armstrong@linaro.org>
> > > > Signed-off-by: Gaurav Kashyap <quic_gaurkash@quicinc.com>
> > > > ---
> > > >  drivers/soc/qcom/ice.c | 153
> > > +++++++++++++++++++++++++++++++++++++++--
> > > >  include/soc/qcom/ice.h |   1 +
> > > >  2 files changed, 150 insertions(+), 4 deletions(-)
> > > >
> > > > diff --git a/drivers/soc/qcom/ice.c b/drivers/soc/qcom/ice.c index
> > > > 6f941d32fffb..d5e74cf2946b 100644
> > > > --- a/drivers/soc/qcom/ice.c
> > > > +++ b/drivers/soc/qcom/ice.c
> > > > @@ -26,6 +26,40 @@
> > > >  #define QCOM_ICE_REG_FUSE_SETTING            0x0010
> > > >  #define QCOM_ICE_REG_BIST_STATUS             0x0070
> > > >  #define QCOM_ICE_REG_ADVANCED_CONTROL                0x1000
> > > > +#define QCOM_ICE_REG_CONTROL                 0x0
> > > > +/* QCOM ICE HWKM registers */
> > > > +#define QCOM_ICE_REG_HWKM_TZ_KM_CTL                  0x1000
> > > > +#define QCOM_ICE_REG_HWKM_TZ_KM_STATUS                       0x1004
> > > > +#define QCOM_ICE_REG_HWKM_BANK0_BANKN_IRQ_STATUS
> 0x2008
> > > > +#define QCOM_ICE_REG_HWKM_BANK0_BBAC_0                       0x5000
> > > > +#define QCOM_ICE_REG_HWKM_BANK0_BBAC_1                       0x5004
> > > > +#define QCOM_ICE_REG_HWKM_BANK0_BBAC_2                       0x5008
> > > > +#define QCOM_ICE_REG_HWKM_BANK0_BBAC_3                       0x500C
> > > > +#define QCOM_ICE_REG_HWKM_BANK0_BBAC_4                       0x5010
> > > > +
> > > > +/* QCOM ICE HWKM reg vals */
> > > > +#define QCOM_ICE_HWKM_BIST_DONE_V1           BIT(16)
> > > > +#define QCOM_ICE_HWKM_BIST_DONE_V2           BIT(9)
> > > > +#define QCOM_ICE_HWKM_BIST_DONE(ver)
> > > QCOM_ICE_HWKM_BIST_DONE_V##ver
> > > > +
> > > > +#define QCOM_ICE_HWKM_CRYPTO_BIST_DONE_V1            BIT(14)
> > > > +#define QCOM_ICE_HWKM_CRYPTO_BIST_DONE_V2            BIT(7)
> > > > +#define QCOM_ICE_HWKM_CRYPTO_BIST_DONE(v)
> > > QCOM_ICE_HWKM_CRYPTO_BIST_DONE_V##v
> > > > +
> > > > +#define QCOM_ICE_HWKM_BOOT_CMD_LIST1_DONE            BIT(2)
> > > > +#define QCOM_ICE_HWKM_BOOT_CMD_LIST0_DONE            BIT(1)
> > > > +#define QCOM_ICE_HWKM_KT_CLEAR_DONE                  BIT(0)
> > > > +
> > > > +#define QCOM_ICE_HWKM_BIST_VAL(v)
> > > (QCOM_ICE_HWKM_BIST_DONE(v) |           \
> > > > +                                     QCOM_ICE_HWKM_CRYPTO_BIST_DONE(v) |     \
> > > > +                                     QCOM_ICE_HWKM_BOOT_CMD_LIST1_DONE |     \
> > > > +                                     QCOM_ICE_HWKM_BOOT_CMD_LIST0_DONE |     \
> > > > +                                     QCOM_ICE_HWKM_KT_CLEAR_DONE)
> > > > +
> > > > +#define QCOM_ICE_HWKM_V1_STANDARD_MODE_VAL   (BIT(0) |
> BIT(1)
> > > | BIT(2))
> > > > +#define QCOM_ICE_HWKM_V2_STANDARD_MODE_MASK
> > > GENMASK(31, 1) #define
> > > > +QCOM_ICE_HWKM_DISABLE_CRC_CHECKS_VAL (BIT(1) | BIT(2))
> > > > +#define QCOM_ICE_HWKM_RSP_FIFO_CLEAR_VAL     BIT(3)
> > > >
> > > >  /* BIST ("built-in self-test") status flags */
> > > >  #define QCOM_ICE_BIST_STATUS_MASK            GENMASK(31, 28)
> > > > @@ -34,6 +68,9 @@
> > > >  #define QCOM_ICE_FORCE_HW_KEY0_SETTING_MASK  0x2  #define
> > > > QCOM_ICE_FORCE_HW_KEY1_SETTING_MASK  0x4
> > > >
> > > > +#define QCOM_ICE_HWKM_REG_OFFSET     0x8000
> > > > +#define HWKM_OFFSET(reg)             ((reg) +
> > > QCOM_ICE_HWKM_REG_OFFSET)
> > > > +
> > > >  #define qcom_ice_writel(engine, val, reg)    \
> > > >       writel((val), (engine)->base + (reg))
> > > >
> > > > @@ -46,6 +83,9 @@ struct qcom_ice {
> > > >       struct device_link *link;
> > > >
> > > >       struct clk *core_clk;
> > > > +     u8 hwkm_version;
> > > > +     bool use_hwkm;
> > > > +     bool hwkm_init_complete;
> > > >  };
> > > >
> > > >  static bool qcom_ice_check_supported(struct qcom_ice *ice) @@
> > > > -63,8
> > > > +103,21 @@ static bool qcom_ice_check_supported(struct qcom_ice
> > > > +*ice)
> > > >               return false;
> > > >       }
> > > >
> > > > -     dev_info(dev, "Found QC Inline Crypto Engine (ICE) v%d.%d.%d\n",
> > > > -              major, minor, step);
> > > > +     if (major >= 4 || (major == 3 && minor == 2 && step >= 1))
> > > > +             ice->hwkm_version = 2;
> > > > +     else if (major == 3 && minor == 2)
> > > > +             ice->hwkm_version = 1;
> > > > +     else
> > > > +             ice->hwkm_version = 0;
> > > > +
> > > > +     if (ice->hwkm_version == 0)
> > > > +             ice->use_hwkm = false;
> > > > +
> > > > +     dev_info(dev, "Found QC Inline Crypto Engine (ICE)
> > > > + v%d.%d.%d,
> > > HWKM v%d\n",
> > > > +              major, minor, step, ice->hwkm_version);
> > > > +
> > > > +     if (!ice->use_hwkm)
> > > > +             dev_info(dev, "QC ICE HWKM (Hardware Key Manager)
> > > > + not used/supported");
> > > >
> > > >       /* If fuses are blown, ICE might not work in the standard way. */
> > > >       regval = qcom_ice_readl(ice, QCOM_ICE_REG_FUSE_SETTING); @@
> > > > -113,27 +166,106 @@ static void
> > > > qcom_ice_optimization_enable(struct
> > > qcom_ice *ice)
> > > >   * fails, so we needn't do it in software too, and (c) properly testing
> > > >   * storage encryption requires testing the full storage stack anyway,
> > > >   * and not relying on hardware-level self-tests.
> > > > + *
> > > > + * However, we still care about if HWKM BIST failed (when
> > > > + supported) as
> > > > + * important functionality would fail later, so disable hwkm on failure.
> > > >   */
> > > >  static int qcom_ice_wait_bist_status(struct qcom_ice *ice)  {
> > > >       u32 regval;
> > > > +     u32 bist_done_val;
> > > >       int err;
> > > >
> > > >       err = readl_poll_timeout(ice->base +
> QCOM_ICE_REG_BIST_STATUS,
> > > >                                regval, !(regval & QCOM_ICE_BIST_STATUS_MASK),
> > > >                                50, 5000);
> > > > -     if (err)
> > > > +     if (err) {
> > > >               dev_err(ice->dev, "Timed out waiting for ICE
> > > > self-test to complete\n");
> > > > +             return err;
> > > > +     }
> > > >
> > > > +     if (ice->use_hwkm) {
> > > > +             bist_done_val = ice->hwkm_version == 1 ?
> > > > +                             QCOM_ICE_HWKM_BIST_VAL(1) :
> > > > +                             QCOM_ICE_HWKM_BIST_VAL(2);
> > > > +             if (qcom_ice_readl(ice,
> > > > +
> > > HWKM_OFFSET(QCOM_ICE_REG_HWKM_TZ_KM_STATUS)) !=
> > > > +                                bist_done_val) {
> > > > +                     dev_err(ice->dev, "HWKM BIST error\n");
> > > > +                     ice->use_hwkm = false;
> > > > +                     err = -ENODEV;
> > > > +             }
> > > > +     }
> > > >       return err;
> > > >  }
> > > >
> > > > +static void qcom_ice_enable_standard_mode(struct qcom_ice *ice) {
> > > > +     u32 val = 0;
> > > > +
> > > > +     /*
> > > > +      * When ICE is in standard (hwkm) mode, it supports HW wrapped
> > > > +      * keys, and when it is in legacy mode, it only supports standard
> > > > +      * (non HW wrapped) keys.
> > >
> > > I can't say this is very logical.
> > >
> > > standard mode => HW wrapped keys
> > > legacy mode => standard keys
> > >
> > > Consider changing the terms.
> > >
> >
> > Ack, will make this clearer
> >
> > > > +      *
> > > > +      * Put ICE in standard mode, ICE defaults to legacy mode.
> > > > +      * Legacy mode - ICE HWKM slave not supported.
> > > > +      * Standard mode - ICE HWKM slave supported.
> > >
> > > s/slave/some other term/
> > >
> > Ack - will address this.
> >
> > > Is it possible to use both kind of keys when working on standard mode?
> > > If not, it should be the user who selects what type of keys to be used.
> > > Enforcing this via DT is not a way to go.
> > >
> >
> > Unfortunately, that support is not there yet. When you say user, do
> > you mean to have it as a filesystem mount option?
> 
> During cryptsetup time. When running e.g. cryptsetup I, as a user, would like
> to be able to use either a hardware-wrapped key or a standard key.
> 

What we are looking for with these patches is for per-file/folder encryption using fscrypt policies.
Cryptsetup to my understanding supports only full-disk , and does not support FBE (File-Based)

Hence the idea here is that we mount an unencrypted device (with the inlinecrypt option that indicates inline encryption is supported)
And specify policies (links to keys) for different folders.

> > The way the UFS/EMMC crypto layer is designed currently is that, this
> > information is needed when the modules are loaded.
> >
> > https://lore.kernel.org/all/20231104211259.17448-2-ebiggers@kernel.org
> > /#Z31drivers:ufs:core:ufshcd-crypto.c
> 
> I see that the driver lists capabilities here. E.g. that it supports HW-wrapped
> keys. But the line doesn't specify that standard keys are not supported.
> 

Those are capabilities that are read from the storage controller. However, wrapped keys
Are not a standard in the ICE JEDEC specification, and in most cases, is a value add coming
from the SoC.

QCOM SOC and firmware currently does not support both kinds of keys in the HWKM mode.
That is something we are internally working on, but not available yet.

> Also, I'd have expected that hw-wrapped keys are handled using trusted
> keys mechanism (see security/keys/trusted-keys/). Could you please point
> out why that's not the case?
> 

I will evaluate this.
But my initial response is that we currently cannot communicate to our TPM directly from HLOS, but
goes through QTEE, and I don't think our qtee currently interfaces with the open source tee
driver. The interface is through QCOM SCM driver.

> > I am thinking of a way now to do this with DT, but without having a new
> vendor property.
> > Is it acceptable to use the addressable range as the deciding factor?
> > Say use legacy mode of ICE when the addressable size is 0x8000 and use
> > HWKM mode of ICE when the addressable size is 0x10000.
> 
> Definitely, this is a NAK. It's a very unobvious hack. You have been asked to
> use compatible strings to detect whether HW keys are supported or not.
> 
> --
> With best wishes
> Dmitry

Regards,
Gaurav

On 20/06/2024 00:02, Gaurav Kashyap (QUIC) wrote:
> Hello Krzysztof
> 
> On 06/18/2024 11:17 PM PDT, Krzysztof Kozlowski wrote:
>> On 19/06/2024 00:08, Gaurav Kashyap (QUIC) wrote:
>>>>
>>>> You may perhaps only call qcom_scm_derive_sw_secret_available() for
>>>> some ICE versions.
>>>>
>>>> Neil
>>>
>>> The issue here is that for the same ICE version, based on the chipset,
>>> there might be different configurations.
>>
>> That's not what your DTS said. To remind: your DTS said that all SM8550 and
>> all SM8650 have it. Choice is obvious then: it's deducible from compatible.
>>
>> I still do not understand why your call cannot return you correct
>> "configuration".
>>
> 
> ICE version and chipsets are disjoint, meaning for the same ICE HW present in SM8650 vs SMxxxx target,
> SM8650 will have necessary TZ support, but SM8xxxx may not, that is the reason I was trying to indicate all SM8550 and
> SM8650 have the necessary TZ support. There might have been a miscommunication there.

No. Read your DTS again.



Best regards,
Krzysztof

On Thu, 20 Jun 2024 at 01:30, Gaurav Kashyap (QUIC)
<quic_gaurkash@quicinc.com> wrote:
>
> Hello Dmitry
>
> On 06/18/2024 3:17 PM PDT, Dmitry Baryshkov wrote:
> > On Wed, 19 Jun 2024 at 01:07, Gaurav Kashyap (QUIC)
> > <quic_gaurkash@quicinc.com> wrote:
> > >
> > > Hello Dmitry,
> > >
> > > On 06/17/2024 12:55 AM PDT, Dmitry Baryshkov wrote:
> > > > On Sun, Jun 16, 2024 at 05:50:59PM GMT, Gaurav Kashyap wrote:
> > > > > Qualcomm's ICE (Inline Crypto Engine) contains a proprietary key
> > > > > management hardware called Hardware Key Manager (HWKM).
> > > > > This patch integrates HWKM support in ICE when it is available.
> > > > > HWKM primarily provides hardware wrapped key support where the
> > ICE
> > > > > (storage) keys are not available in software and protected in
> > > > > hardware.
> > > > >
> > > > > When HWKM software support is not fully available (from
> > > > > Trustzone), there can be a scenario where the ICE hardware
> > > > > supports HWKM, but it cannot be used for wrapped keys. In this
> > > > > case, standard keys have to be used without using HWKM. Hence,
> > > > > providing a toggle controlled by a devicetree entry to use HWKM or not.
> > > > >
> > > > > Tested-by: Neil Armstrong <neil.armstrong@linaro.org>
> > > > > Signed-off-by: Gaurav Kashyap <quic_gaurkash@quicinc.com>
> > > > > ---
> > > > >  drivers/soc/qcom/ice.c | 153
> > > > +++++++++++++++++++++++++++++++++++++++--
> > > > >  include/soc/qcom/ice.h |   1 +
> > > > >  2 files changed, 150 insertions(+), 4 deletions(-)
> > > > >
> > > > > diff --git a/drivers/soc/qcom/ice.c b/drivers/soc/qcom/ice.c index
> > > > > 6f941d32fffb..d5e74cf2946b 100644
> > > > > --- a/drivers/soc/qcom/ice.c
> > > > > +++ b/drivers/soc/qcom/ice.c
> > > > > @@ -26,6 +26,40 @@
> > > > >  #define QCOM_ICE_REG_FUSE_SETTING            0x0010
> > > > >  #define QCOM_ICE_REG_BIST_STATUS             0x0070
> > > > >  #define QCOM_ICE_REG_ADVANCED_CONTROL                0x1000
> > > > > +#define QCOM_ICE_REG_CONTROL                 0x0
> > > > > +/* QCOM ICE HWKM registers */
> > > > > +#define QCOM_ICE_REG_HWKM_TZ_KM_CTL                  0x1000
> > > > > +#define QCOM_ICE_REG_HWKM_TZ_KM_STATUS                       0x1004
> > > > > +#define QCOM_ICE_REG_HWKM_BANK0_BANKN_IRQ_STATUS
> > 0x2008
> > > > > +#define QCOM_ICE_REG_HWKM_BANK0_BBAC_0                       0x5000
> > > > > +#define QCOM_ICE_REG_HWKM_BANK0_BBAC_1                       0x5004
> > > > > +#define QCOM_ICE_REG_HWKM_BANK0_BBAC_2                       0x5008
> > > > > +#define QCOM_ICE_REG_HWKM_BANK0_BBAC_3                       0x500C
> > > > > +#define QCOM_ICE_REG_HWKM_BANK0_BBAC_4                       0x5010
> > > > > +
> > > > > +/* QCOM ICE HWKM reg vals */
> > > > > +#define QCOM_ICE_HWKM_BIST_DONE_V1           BIT(16)
> > > > > +#define QCOM_ICE_HWKM_BIST_DONE_V2           BIT(9)
> > > > > +#define QCOM_ICE_HWKM_BIST_DONE(ver)
> > > > QCOM_ICE_HWKM_BIST_DONE_V##ver
> > > > > +
> > > > > +#define QCOM_ICE_HWKM_CRYPTO_BIST_DONE_V1            BIT(14)
> > > > > +#define QCOM_ICE_HWKM_CRYPTO_BIST_DONE_V2            BIT(7)
> > > > > +#define QCOM_ICE_HWKM_CRYPTO_BIST_DONE(v)
> > > > QCOM_ICE_HWKM_CRYPTO_BIST_DONE_V##v
> > > > > +
> > > > > +#define QCOM_ICE_HWKM_BOOT_CMD_LIST1_DONE            BIT(2)
> > > > > +#define QCOM_ICE_HWKM_BOOT_CMD_LIST0_DONE            BIT(1)
> > > > > +#define QCOM_ICE_HWKM_KT_CLEAR_DONE                  BIT(0)
> > > > > +
> > > > > +#define QCOM_ICE_HWKM_BIST_VAL(v)
> > > > (QCOM_ICE_HWKM_BIST_DONE(v) |           \
> > > > > +                                     QCOM_ICE_HWKM_CRYPTO_BIST_DONE(v) |     \
> > > > > +                                     QCOM_ICE_HWKM_BOOT_CMD_LIST1_DONE |     \
> > > > > +                                     QCOM_ICE_HWKM_BOOT_CMD_LIST0_DONE |     \
> > > > > +                                     QCOM_ICE_HWKM_KT_CLEAR_DONE)
> > > > > +
> > > > > +#define QCOM_ICE_HWKM_V1_STANDARD_MODE_VAL   (BIT(0) |
> > BIT(1)
> > > > | BIT(2))
> > > > > +#define QCOM_ICE_HWKM_V2_STANDARD_MODE_MASK
> > > > GENMASK(31, 1) #define
> > > > > +QCOM_ICE_HWKM_DISABLE_CRC_CHECKS_VAL (BIT(1) | BIT(2))
> > > > > +#define QCOM_ICE_HWKM_RSP_FIFO_CLEAR_VAL     BIT(3)
> > > > >
> > > > >  /* BIST ("built-in self-test") status flags */
> > > > >  #define QCOM_ICE_BIST_STATUS_MASK            GENMASK(31, 28)
> > > > > @@ -34,6 +68,9 @@
> > > > >  #define QCOM_ICE_FORCE_HW_KEY0_SETTING_MASK  0x2  #define
> > > > > QCOM_ICE_FORCE_HW_KEY1_SETTING_MASK  0x4
> > > > >
> > > > > +#define QCOM_ICE_HWKM_REG_OFFSET     0x8000
> > > > > +#define HWKM_OFFSET(reg)             ((reg) +
> > > > QCOM_ICE_HWKM_REG_OFFSET)
> > > > > +
> > > > >  #define qcom_ice_writel(engine, val, reg)    \
> > > > >       writel((val), (engine)->base + (reg))
> > > > >
> > > > > @@ -46,6 +83,9 @@ struct qcom_ice {
> > > > >       struct device_link *link;
> > > > >
> > > > >       struct clk *core_clk;
> > > > > +     u8 hwkm_version;
> > > > > +     bool use_hwkm;
> > > > > +     bool hwkm_init_complete;
> > > > >  };
> > > > >
> > > > >  static bool qcom_ice_check_supported(struct qcom_ice *ice) @@
> > > > > -63,8
> > > > > +103,21 @@ static bool qcom_ice_check_supported(struct qcom_ice
> > > > > +*ice)
> > > > >               return false;
> > > > >       }
> > > > >
> > > > > -     dev_info(dev, "Found QC Inline Crypto Engine (ICE) v%d.%d.%d\n",
> > > > > -              major, minor, step);
> > > > > +     if (major >= 4 || (major == 3 && minor == 2 && step >= 1))
> > > > > +             ice->hwkm_version = 2;
> > > > > +     else if (major == 3 && minor == 2)
> > > > > +             ice->hwkm_version = 1;
> > > > > +     else
> > > > > +             ice->hwkm_version = 0;
> > > > > +
> > > > > +     if (ice->hwkm_version == 0)
> > > > > +             ice->use_hwkm = false;
> > > > > +
> > > > > +     dev_info(dev, "Found QC Inline Crypto Engine (ICE)
> > > > > + v%d.%d.%d,
> > > > HWKM v%d\n",
> > > > > +              major, minor, step, ice->hwkm_version);
> > > > > +
> > > > > +     if (!ice->use_hwkm)
> > > > > +             dev_info(dev, "QC ICE HWKM (Hardware Key Manager)
> > > > > + not used/supported");
> > > > >
> > > > >       /* If fuses are blown, ICE might not work in the standard way. */
> > > > >       regval = qcom_ice_readl(ice, QCOM_ICE_REG_FUSE_SETTING); @@
> > > > > -113,27 +166,106 @@ static void
> > > > > qcom_ice_optimization_enable(struct
> > > > qcom_ice *ice)
> > > > >   * fails, so we needn't do it in software too, and (c) properly testing
> > > > >   * storage encryption requires testing the full storage stack anyway,
> > > > >   * and not relying on hardware-level self-tests.
> > > > > + *
> > > > > + * However, we still care about if HWKM BIST failed (when
> > > > > + supported) as
> > > > > + * important functionality would fail later, so disable hwkm on failure.
> > > > >   */
> > > > >  static int qcom_ice_wait_bist_status(struct qcom_ice *ice)  {
> > > > >       u32 regval;
> > > > > +     u32 bist_done_val;
> > > > >       int err;
> > > > >
> > > > >       err = readl_poll_timeout(ice->base +
> > QCOM_ICE_REG_BIST_STATUS,
> > > > >                                regval, !(regval & QCOM_ICE_BIST_STATUS_MASK),
> > > > >                                50, 5000);
> > > > > -     if (err)
> > > > > +     if (err) {
> > > > >               dev_err(ice->dev, "Timed out waiting for ICE
> > > > > self-test to complete\n");
> > > > > +             return err;
> > > > > +     }
> > > > >
> > > > > +     if (ice->use_hwkm) {
> > > > > +             bist_done_val = ice->hwkm_version == 1 ?
> > > > > +                             QCOM_ICE_HWKM_BIST_VAL(1) :
> > > > > +                             QCOM_ICE_HWKM_BIST_VAL(2);
> > > > > +             if (qcom_ice_readl(ice,
> > > > > +
> > > > HWKM_OFFSET(QCOM_ICE_REG_HWKM_TZ_KM_STATUS)) !=
> > > > > +                                bist_done_val) {
> > > > > +                     dev_err(ice->dev, "HWKM BIST error\n");
> > > > > +                     ice->use_hwkm = false;
> > > > > +                     err = -ENODEV;
> > > > > +             }
> > > > > +     }
> > > > >       return err;
> > > > >  }
> > > > >
> > > > > +static void qcom_ice_enable_standard_mode(struct qcom_ice *ice) {
> > > > > +     u32 val = 0;
> > > > > +
> > > > > +     /*
> > > > > +      * When ICE is in standard (hwkm) mode, it supports HW wrapped
> > > > > +      * keys, and when it is in legacy mode, it only supports standard
> > > > > +      * (non HW wrapped) keys.
> > > >
> > > > I can't say this is very logical.
> > > >
> > > > standard mode => HW wrapped keys
> > > > legacy mode => standard keys
> > > >
> > > > Consider changing the terms.
> > > >
> > >
> > > Ack, will make this clearer
> > >
> > > > > +      *
> > > > > +      * Put ICE in standard mode, ICE defaults to legacy mode.
> > > > > +      * Legacy mode - ICE HWKM slave not supported.
> > > > > +      * Standard mode - ICE HWKM slave supported.
> > > >
> > > > s/slave/some other term/
> > > >
> > > Ack - will address this.
> > >
> > > > Is it possible to use both kind of keys when working on standard mode?
> > > > If not, it should be the user who selects what type of keys to be used.
> > > > Enforcing this via DT is not a way to go.
> > > >
> > >
> > > Unfortunately, that support is not there yet. When you say user, do
> > > you mean to have it as a filesystem mount option?
> >
> > During cryptsetup time. When running e.g. cryptsetup I, as a user, would like
> > to be able to use either a hardware-wrapped key or a standard key.
> >
>
> What we are looking for with these patches is for per-file/folder encryption using fscrypt policies.
> Cryptsetup to my understanding supports only full-disk , and does not support FBE (File-Based)

I must admit, I mostly used dm-crypt beforehand, so I had to look at
fscrypt now. Some of my previous comments might not be fully
applicable.

> Hence the idea here is that we mount an unencrypted device (with the inlinecrypt option that indicates inline encryption is supported)
> And specify policies (links to keys) for different folders.
>
> > > The way the UFS/EMMC crypto layer is designed currently is that, this
> > > information is needed when the modules are loaded.
> > >
> > > https://lore.kernel.org/all/20231104211259.17448-2-ebiggers@kernel.org
> > > /#Z31drivers:ufs:core:ufshcd-crypto.c
> >
> > I see that the driver lists capabilities here. E.g. that it supports HW-wrapped
> > keys. But the line doesn't specify that standard keys are not supported.
> >
>
> Those are capabilities that are read from the storage controller. However, wrapped keys
> Are not a standard in the ICE JEDEC specification, and in most cases, is a value add coming
> from the SoC.
>
> QCOM SOC and firmware currently does not support both kinds of keys in the HWKM mode.
> That is something we are internally working on, but not available yet.

I'd say this is a significant obstacle, at least from my point of
view. I understand that the default might be to use hw-wrapped keys,
but it should be possible for the user to select non-HW keys if the
ability to recover the data is considered to be important. Note, I'm
really pointing to the user here, not to the system integrator. So
using DT property or specifying kernel arguments to switch between
these modes is not really an option.

But I'd really love to hear some feedback from linux-security and/or
linux-fscrypt here.

In my humble opinion the user should be able to specify that the key
is wrapped using the hardware KMK. Then if the hardware has already
started using the other kind of keys, it should be able to respond
with -EINVAL / whatever else. Then the user can evict previously
programmed key and program a desired one.

> > Also, I'd have expected that hw-wrapped keys are handled using trusted
> > keys mechanism (see security/keys/trusted-keys/). Could you please point
> > out why that's not the case?
> >
>
> I will evaluate this.
> But my initial response is that we currently cannot communicate to our TPM directly from HLOS, but
> goes through QTEE, and I don't think our qtee currently interfaces with the open source tee
> driver. The interface is through QCOM SCM driver.

Note, this is just an API interface, see how it is implemented for the
CAAM hardware.

>
> > > I am thinking of a way now to do this with DT, but without having a new
> > vendor property.
> > > Is it acceptable to use the addressable range as the deciding factor?
> > > Say use legacy mode of ICE when the addressable size is 0x8000 and use
> > > HWKM mode of ICE when the addressable size is 0x10000.
> >
> > Definitely, this is a NAK. It's a very unobvious hack. You have been asked to
> > use compatible strings to detect whether HW keys are supported or not.

On Thu, Jun 20, 2024 at 02:57:40PM +0300, Dmitry Baryshkov wrote:
> > > >
> > > > > Is it possible to use both kind of keys when working on standard mode?
> > > > > If not, it should be the user who selects what type of keys to be used.
> > > > > Enforcing this via DT is not a way to go.
> > > > >
> > > >
> > > > Unfortunately, that support is not there yet. When you say user, do
> > > > you mean to have it as a filesystem mount option?
> > >
> > > During cryptsetup time. When running e.g. cryptsetup I, as a user, would like
> > > to be able to use either a hardware-wrapped key or a standard key.
> > >
> >
> > What we are looking for with these patches is for per-file/folder encryption using fscrypt policies.
> > Cryptsetup to my understanding supports only full-disk , and does not support FBE (File-Based)
> 
> I must admit, I mostly used dm-crypt beforehand, so I had to look at
> fscrypt now. Some of my previous comments might not be fully
> applicable.
> 
> > Hence the idea here is that we mount an unencrypted device (with the inlinecrypt option that indicates inline encryption is supported)
> > And specify policies (links to keys) for different folders.
> >
> > > > The way the UFS/EMMC crypto layer is designed currently is that, this
> > > > information is needed when the modules are loaded.
> > > >
> > > > https://lore.kernel.org/all/20231104211259.17448-2-ebiggers@kernel.org
> > > > /#Z31drivers:ufs:core:ufshcd-crypto.c
> > >
> > > I see that the driver lists capabilities here. E.g. that it supports HW-wrapped
> > > keys. But the line doesn't specify that standard keys are not supported.
> > >
> >
> > Those are capabilities that are read from the storage controller. However, wrapped keys
> > Are not a standard in the ICE JEDEC specification, and in most cases, is a value add coming
> > from the SoC.
> >
> > QCOM SOC and firmware currently does not support both kinds of keys in the HWKM mode.
> > That is something we are internally working on, but not available yet.
> 
> I'd say this is a significant obstacle, at least from my point of
> view. I understand that the default might be to use hw-wrapped keys,
> but it should be possible for the user to select non-HW keys if the
> ability to recover the data is considered to be important. Note, I'm
> really pointing to the user here, not to the system integrator. So
> using DT property or specifying kernel arguments to switch between
> these modes is not really an option.
> 
> But I'd really love to hear some feedback from linux-security and/or
> linux-fscrypt here.
> 
> In my humble opinion the user should be able to specify that the key
> is wrapped using the hardware KMK. Then if the hardware has already
> started using the other kind of keys, it should be able to respond
> with -EINVAL / whatever else. Then the user can evict previously
> programmed key and program a desired one.
> 
> > > Also, I'd have expected that hw-wrapped keys are handled using trusted
> > > keys mechanism (see security/keys/trusted-keys/). Could you please point
> > > out why that's not the case?
> > >
> >
> > I will evaluate this.
> > But my initial response is that we currently cannot communicate to our TPM directly from HLOS, but
> > goes through QTEE, and I don't think our qtee currently interfaces with the open source tee
> > driver. The interface is through QCOM SCM driver.
> 
> Note, this is just an API interface, see how it is implemented for the
> CAAM hardware.
> 

The problem is that this patchset was sent out without the patches that add the
block and filesystem-level framework for hardware-wrapped inline encryption
keys, which it depends on.  So it's lacking context.  The proposed framework can
be found at
https://lore.kernel.org/linux-block/20231104211259.17448-1-ebiggers@kernel.org/T/#u

As for why "trusted keys" aren't used, they just aren't helpful here.  "Trusted
keys" are based around a model where the kernel can request that keys be sealed
and unsealed using a trust source, and the kernel gets access to the raw
unsealed keys.  Hardware-wrapped inline encryption keys use a different model
where the kernel never gets access to the raw keys.  They also have the concept
of ephemeral wrapping which does not exist in "trusted keys".  And they need to
be properly integrated with the inline encryption framework in the block layer.

- Eric

On Fri, 21 Jun 2024 at 07:47, Eric Biggers <ebiggers@kernel.org> wrote:
>
> On Thu, Jun 20, 2024 at 02:57:40PM +0300, Dmitry Baryshkov wrote:
> > > > >
> > > > > > Is it possible to use both kind of keys when working on standard mode?
> > > > > > If not, it should be the user who selects what type of keys to be used.
> > > > > > Enforcing this via DT is not a way to go.
> > > > > >
> > > > >
> > > > > Unfortunately, that support is not there yet. When you say user, do
> > > > > you mean to have it as a filesystem mount option?
> > > >
> > > > During cryptsetup time. When running e.g. cryptsetup I, as a user, would like
> > > > to be able to use either a hardware-wrapped key or a standard key.
> > > >
> > >
> > > What we are looking for with these patches is for per-file/folder encryption using fscrypt policies.
> > > Cryptsetup to my understanding supports only full-disk , and does not support FBE (File-Based)
> >
> > I must admit, I mostly used dm-crypt beforehand, so I had to look at
> > fscrypt now. Some of my previous comments might not be fully
> > applicable.
> >
> > > Hence the idea here is that we mount an unencrypted device (with the inlinecrypt option that indicates inline encryption is supported)
> > > And specify policies (links to keys) for different folders.
> > >
> > > > > The way the UFS/EMMC crypto layer is designed currently is that, this
> > > > > information is needed when the modules are loaded.
> > > > >
> > > > > https://lore.kernel.org/all/20231104211259.17448-2-ebiggers@kernel.org
> > > > > /#Z31drivers:ufs:core:ufshcd-crypto.c
> > > >
> > > > I see that the driver lists capabilities here. E.g. that it supports HW-wrapped
> > > > keys. But the line doesn't specify that standard keys are not supported.
> > > >
> > >
> > > Those are capabilities that are read from the storage controller. However, wrapped keys
> > > Are not a standard in the ICE JEDEC specification, and in most cases, is a value add coming
> > > from the SoC.
> > >
> > > QCOM SOC and firmware currently does not support both kinds of keys in the HWKM mode.
> > > That is something we are internally working on, but not available yet.
> >
> > I'd say this is a significant obstacle, at least from my point of
> > view. I understand that the default might be to use hw-wrapped keys,
> > but it should be possible for the user to select non-HW keys if the
> > ability to recover the data is considered to be important. Note, I'm
> > really pointing to the user here, not to the system integrator. So
> > using DT property or specifying kernel arguments to switch between
> > these modes is not really an option.
> >
> > But I'd really love to hear some feedback from linux-security and/or
> > linux-fscrypt here.
> >
> > In my humble opinion the user should be able to specify that the key
> > is wrapped using the hardware KMK. Then if the hardware has already
> > started using the other kind of keys, it should be able to respond
> > with -EINVAL / whatever else. Then the user can evict previously
> > programmed key and program a desired one.
> >
> > > > Also, I'd have expected that hw-wrapped keys are handled using trusted
> > > > keys mechanism (see security/keys/trusted-keys/). Could you please point
> > > > out why that's not the case?
> > > >
> > >
> > > I will evaluate this.
> > > But my initial response is that we currently cannot communicate to our TPM directly from HLOS, but
> > > goes through QTEE, and I don't think our qtee currently interfaces with the open source tee
> > > driver. The interface is through QCOM SCM driver.
> >
> > Note, this is just an API interface, see how it is implemented for the
> > CAAM hardware.
> >
>
> The problem is that this patchset was sent out without the patches that add the
> block and filesystem-level framework for hardware-wrapped inline encryption
> keys, which it depends on.  So it's lacking context.  The proposed framework can
> be found at
> https://lore.kernel.org/linux-block/20231104211259.17448-1-ebiggers@kernel.org/T/#u

Thank you. I have quickly skimmed through the patches, but I didn't
review them thoroughly. Maybe the patchset already implements the
interfaces that I'm thinking about. In such a case please excuse me. I
will give it a more thorough look later today.

> As for why "trusted keys" aren't used, they just aren't helpful here.  "Trusted
> keys" are based around a model where the kernel can request that keys be sealed
> and unsealed using a trust source, and the kernel gets access to the raw
> unsealed keys.  Hardware-wrapped inline encryption keys use a different model
> where the kernel never gets access to the raw keys.  They also have the concept
> of ephemeral wrapping which does not exist in "trusted keys".  And they need to
> be properly integrated with the inline encryption framework in the block layer.

Then what exactly does qcom_scm_derive_sw_secret() do? Does it rewrap
the key under some other key?
I had the feeling that there are two separate pieces of functionality
being stuffed into a single patchset and into a single solution.

First one is handling the keys. I keep on thinking that there should
be a separate software interface to unseal the key and rewrap it under
an ephemeral key. Some hardware might permit importing raw keys. Other
hardware might insist on generating the keys on-chip so that raw keys
can never be used. Anyway, the net result is the binary blob + cookie
for the ephemeral key.

Second part is the actual block interface. Gaurav wrote about
targeting fscrypt, but there should be no actual difference between
crypto targets. FDE or having a single partition encrypted should
probably work in the same way. Convert the key into blk_crypto_key
(including the cookie for the ephemeral key), program the key into the
slot, use the slot to en/decrypt hardware blocks.

My main point is that the decision on the key type should be coming
from the user. I can easily imagine a user, which wants to use
password / raw key for documents storage so that it is possible to
recover the data, hw-wrapped long-term key for app & data storage and
generated one-time random key for the swap, so that memory contents
can never be recovered after reboot / device capture.

Hello Eric

On 06/20/2024, 9:48 PM PDT, Eric Biggers wrote:
> On Thu, Jun 20, 2024 at 02:57:40PM +0300, Dmitry Baryshkov wrote:
> > > > >
> > > > > > Is it possible to use both kind of keys when working on standard
> mode?
> > > > > > If not, it should be the user who selects what type of keys to be
> used.
> > > > > > Enforcing this via DT is not a way to go.
> > > > > >
> > > > >
> > > > > Unfortunately, that support is not there yet. When you say user,
> > > > > do you mean to have it as a filesystem mount option?
> > > >
> > > > During cryptsetup time. When running e.g. cryptsetup I, as a user,
> > > > would like to be able to use either a hardware-wrapped key or a
> standard key.
> > > >
> > >
> > > What we are looking for with these patches is for per-file/folder
> encryption using fscrypt policies.
> > > Cryptsetup to my understanding supports only full-disk , and does
> > > not support FBE (File-Based)
> >
> > I must admit, I mostly used dm-crypt beforehand, so I had to look at
> > fscrypt now. Some of my previous comments might not be fully
> > applicable.
> >
> > > Hence the idea here is that we mount an unencrypted device (with the
> > > inlinecrypt option that indicates inline encryption is supported) And
> specify policies (links to keys) for different folders.
> > >
> > > > > The way the UFS/EMMC crypto layer is designed currently is that,
> > > > > this information is needed when the modules are loaded.
> > > > >
> > > > > https://lore.kernel.org/all/20231104211259.17448-2-ebiggers@kern
> > > > > el.org /#Z31drivers:ufs:core:ufshcd-crypto.c
> > > >
> > > > I see that the driver lists capabilities here. E.g. that it
> > > > supports HW-wrapped keys. But the line doesn't specify that standard
> keys are not supported.
> > > >
> > >
> > > Those are capabilities that are read from the storage controller.
> > > However, wrapped keys Are not a standard in the ICE JEDEC
> > > specification, and in most cases, is a value add coming from the SoC.
> > >
> > > QCOM SOC and firmware currently does not support both kinds of keys in
> the HWKM mode.
> > > That is something we are internally working on, but not available yet.
> >
> > I'd say this is a significant obstacle, at least from my point of
> > view. I understand that the default might be to use hw-wrapped keys,
> > but it should be possible for the user to select non-HW keys if the
> > ability to recover the data is considered to be important. Note, I'm
> > really pointing to the user here, not to the system integrator. So
> > using DT property or specifying kernel arguments to switch between
> > these modes is not really an option.
> >
> > But I'd really love to hear some feedback from linux-security and/or
> > linux-fscrypt here.
> >
> > In my humble opinion the user should be able to specify that the key
> > is wrapped using the hardware KMK. Then if the hardware has already
> > started using the other kind of keys, it should be able to respond
> > with -EINVAL / whatever else. Then the user can evict previously
> > programmed key and program a desired one.
> >
> > > > Also, I'd have expected that hw-wrapped keys are handled using
> > > > trusted keys mechanism (see security/keys/trusted-keys/). Could
> > > > you please point out why that's not the case?
> > > >
> > >
> > > I will evaluate this.
> > > But my initial response is that we currently cannot communicate to
> > > our TPM directly from HLOS, but goes through QTEE, and I don't think
> > > our qtee currently interfaces with the open source tee driver. The
> interface is through QCOM SCM driver.
> >
> > Note, this is just an API interface, see how it is implemented for the
> > CAAM hardware.
> >
> 
> The problem is that this patchset was sent out without the patches that add
> the block and filesystem-level framework for hardware-wrapped inline
> encryption keys, which it depends on.  So it's lacking context.  The proposed
> framework can be found at https://lore.kernel.org/linux-
> block/20231104211259.17448-1-ebiggers@kernel.org/T/#u
> 

I have only been adding the fscryp patch link as part of the cover letter - as a dependency.
https://lore.kernel.org/all/20240617005825.1443206-1-quic_gaurkash@quicinc.com/
If you would like me to include it in the patch series itself, I can do that as well.

> As for why "trusted keys" aren't used, they just aren't helpful here.  "Trusted
> keys" are based around a model where the kernel can request that keys be
> sealed and unsealed using a trust source, and the kernel gets access to the
> raw unsealed keys.  Hardware-wrapped inline encryption keys use a
> different model where the kernel never gets access to the raw keys.  They
> also have the concept of ephemeral wrapping which does not exist in
> "trusted keys".  And they need to be properly integrated with the inline
> encryption framework in the block layer.
> 
> - Eric

Regards,
Gaurav

Apologies, fixed incorrect email

> Hello Eric
> 
> On 06/20/2024, 9:48 PM PDT, Eric Biggers wrote:
> > On Thu, Jun 20, 2024 at 02:57:40PM +0300, Dmitry Baryshkov wrote:
> > > > > >
> > > > > > > Is it possible to use both kind of keys when working on
> > > > > > > standard
> > mode?
> > > > > > > If not, it should be the user who selects what type of keys
> > > > > > > to be
> > used.
> > > > > > > Enforcing this via DT is not a way to go.
> > > > > > >
> > > > > >
> > > > > > Unfortunately, that support is not there yet. When you say
> > > > > > user, do you mean to have it as a filesystem mount option?
> > > > >
> > > > > During cryptsetup time. When running e.g. cryptsetup I, as a
> > > > > user, would like to be able to use either a hardware-wrapped key
> > > > > or a
> > standard key.
> > > > >
> > > >
> > > > What we are looking for with these patches is for per-file/folder
> > encryption using fscrypt policies.
> > > > Cryptsetup to my understanding supports only full-disk , and does
> > > > not support FBE (File-Based)
> > >
> > > I must admit, I mostly used dm-crypt beforehand, so I had to look at
> > > fscrypt now. Some of my previous comments might not be fully
> > > applicable.
> > >
> > > > Hence the idea here is that we mount an unencrypted device (with
> > > > the inlinecrypt option that indicates inline encryption is
> > > > supported) And
> > specify policies (links to keys) for different folders.
> > > >
> > > > > > The way the UFS/EMMC crypto layer is designed currently is
> > > > > > that, this information is needed when the modules are loaded.
> > > > > >
> > > > > > https://lore.kernel.org/all/20231104211259.17448-2-ebiggers@ke
> > > > > > rn el.org /#Z31drivers:ufs:core:ufshcd-crypto.c
> > > > >
> > > > > I see that the driver lists capabilities here. E.g. that it
> > > > > supports HW-wrapped keys. But the line doesn't specify that
> > > > > standard
> > keys are not supported.
> > > > >
> > > >
> > > > Those are capabilities that are read from the storage controller.
> > > > However, wrapped keys Are not a standard in the ICE JEDEC
> > > > specification, and in most cases, is a value add coming from the SoC.
> > > >
> > > > QCOM SOC and firmware currently does not support both kinds of
> > > > keys in
> > the HWKM mode.
> > > > That is something we are internally working on, but not available yet.
> > >
> > > I'd say this is a significant obstacle, at least from my point of
> > > view. I understand that the default might be to use hw-wrapped keys,
> > > but it should be possible for the user to select non-HW keys if the
> > > ability to recover the data is considered to be important. Note, I'm
> > > really pointing to the user here, not to the system integrator. So
> > > using DT property or specifying kernel arguments to switch between
> > > these modes is not really an option.
> > >
> > > But I'd really love to hear some feedback from linux-security and/or
> > > linux-fscrypt here.
> > >
> > > In my humble opinion the user should be able to specify that the key
> > > is wrapped using the hardware KMK. Then if the hardware has already
> > > started using the other kind of keys, it should be able to respond
> > > with -EINVAL / whatever else. Then the user can evict previously
> > > programmed key and program a desired one.
> > >
> > > > > Also, I'd have expected that hw-wrapped keys are handled using
> > > > > trusted keys mechanism (see security/keys/trusted-keys/). Could
> > > > > you please point out why that's not the case?
> > > > >
> > > >
> > > > I will evaluate this.
> > > > But my initial response is that we currently cannot communicate to
> > > > our TPM directly from HLOS, but goes through QTEE, and I don't
> > > > think our qtee currently interfaces with the open source tee
> > > > driver. The
> > interface is through QCOM SCM driver.
> > >
> > > Note, this is just an API interface, see how it is implemented for
> > > the CAAM hardware.
> > >
> >
> > The problem is that this patchset was sent out without the patches
> > that add the block and filesystem-level framework for hardware-wrapped
> > inline encryption keys, which it depends on.  So it's lacking context.
> > The proposed framework can be found at https://lore.kernel.org/linux-
> > block/20231104211259.17448-1-ebiggers@kernel.org/T/#u
> >
> 
> I have only been adding the fscryp patch link as part of the cover letter - as a
> dependency.
> https://lore.kernel.org/all/20240617005825.1443206-1-
> quic_gaurkash@quicinc.com/
> If you would like me to include it in the patch series itself, I can do that as
> well.
> 
> > As for why "trusted keys" aren't used, they just aren't helpful here.
> > "Trusted keys" are based around a model where the kernel can request
> > that keys be sealed and unsealed using a trust source, and the kernel
> > gets access to the raw unsealed keys.  Hardware-wrapped inline
> > encryption keys use a different model where the kernel never gets
> > access to the raw keys.  They also have the concept of ephemeral
> > wrapping which does not exist in "trusted keys".  And they need to be
> > properly integrated with the inline encryption framework in the block layer.
> >
> > - Eric
> 
> Regards,
> Gaurav

On Fri, Jun 21, 2024 at 06:16:37PM +0300, Dmitry Baryshkov wrote:
> On Fri, 21 Jun 2024 at 07:47, Eric Biggers <ebiggers@kernel.org> wrote:
> >
> > On Thu, Jun 20, 2024 at 02:57:40PM +0300, Dmitry Baryshkov wrote:
> > > > > >
> > > > > > > Is it possible to use both kind of keys when working on standard mode?
> > > > > > > If not, it should be the user who selects what type of keys to be used.
> > > > > > > Enforcing this via DT is not a way to go.
> > > > > > >
> > > > > >
> > > > > > Unfortunately, that support is not there yet. When you say user, do
> > > > > > you mean to have it as a filesystem mount option?
> > > > >
> > > > > During cryptsetup time. When running e.g. cryptsetup I, as a user, would like
> > > > > to be able to use either a hardware-wrapped key or a standard key.
> > > > >
> > > >
> > > > What we are looking for with these patches is for per-file/folder encryption using fscrypt policies.
> > > > Cryptsetup to my understanding supports only full-disk , and does not support FBE (File-Based)
> > >
> > > I must admit, I mostly used dm-crypt beforehand, so I had to look at
> > > fscrypt now. Some of my previous comments might not be fully
> > > applicable.
> > >
> > > > Hence the idea here is that we mount an unencrypted device (with the inlinecrypt option that indicates inline encryption is supported)
> > > > And specify policies (links to keys) for different folders.
> > > >
> > > > > > The way the UFS/EMMC crypto layer is designed currently is that, this
> > > > > > information is needed when the modules are loaded.
> > > > > >
> > > > > > https://lore.kernel.org/all/20231104211259.17448-2-ebiggers@kernel.org
> > > > > > /#Z31drivers:ufs:core:ufshcd-crypto.c
> > > > >
> > > > > I see that the driver lists capabilities here. E.g. that it supports HW-wrapped
> > > > > keys. But the line doesn't specify that standard keys are not supported.
> > > > >
> > > >
> > > > Those are capabilities that are read from the storage controller. However, wrapped keys
> > > > Are not a standard in the ICE JEDEC specification, and in most cases, is a value add coming
> > > > from the SoC.
> > > >
> > > > QCOM SOC and firmware currently does not support both kinds of keys in the HWKM mode.
> > > > That is something we are internally working on, but not available yet.
> > >
> > > I'd say this is a significant obstacle, at least from my point of
> > > view. I understand that the default might be to use hw-wrapped keys,
> > > but it should be possible for the user to select non-HW keys if the
> > > ability to recover the data is considered to be important. Note, I'm
> > > really pointing to the user here, not to the system integrator. So
> > > using DT property or specifying kernel arguments to switch between
> > > these modes is not really an option.
> > >
> > > But I'd really love to hear some feedback from linux-security and/or
> > > linux-fscrypt here.
> > >
> > > In my humble opinion the user should be able to specify that the key
> > > is wrapped using the hardware KMK. Then if the hardware has already
> > > started using the other kind of keys, it should be able to respond
> > > with -EINVAL / whatever else. Then the user can evict previously
> > > programmed key and program a desired one.
> > >
> > > > > Also, I'd have expected that hw-wrapped keys are handled using trusted
> > > > > keys mechanism (see security/keys/trusted-keys/). Could you please point
> > > > > out why that's not the case?
> > > > >
> > > >
> > > > I will evaluate this.
> > > > But my initial response is that we currently cannot communicate to our TPM directly from HLOS, but
> > > > goes through QTEE, and I don't think our qtee currently interfaces with the open source tee
> > > > driver. The interface is through QCOM SCM driver.
> > >
> > > Note, this is just an API interface, see how it is implemented for the
> > > CAAM hardware.
> > >
> >
> > The problem is that this patchset was sent out without the patches that add the
> > block and filesystem-level framework for hardware-wrapped inline encryption
> > keys, which it depends on.  So it's lacking context.  The proposed framework can
> > be found at
> > https://lore.kernel.org/linux-block/20231104211259.17448-1-ebiggers@kernel.org/T/#u
> 
> Thank you. I have quickly skimmed through the patches, but I didn't
> review them thoroughly. Maybe the patchset already implements the
> interfaces that I'm thinking about. In such a case please excuse me. I
> will give it a more thorough look later today.
> 
> > As for why "trusted keys" aren't used, they just aren't helpful here.  "Trusted
> > keys" are based around a model where the kernel can request that keys be sealed
> > and unsealed using a trust source, and the kernel gets access to the raw
> > unsealed keys.  Hardware-wrapped inline encryption keys use a different model
> > where the kernel never gets access to the raw keys.  They also have the concept
> > of ephemeral wrapping which does not exist in "trusted keys".  And they need to
> > be properly integrated with the inline encryption framework in the block layer.
> 
> Then what exactly does qcom_scm_derive_sw_secret() do? Does it rewrap
> the key under some other key?

It derives a secret for functionality such as filenames encryption that can't
use inline encryption.

> I had the feeling that there are two separate pieces of functionality
> being stuffed into a single patchset and into a single solution.
> 
> First one is handling the keys. I keep on thinking that there should
> be a separate software interface to unseal the key and rewrap it under
> an ephemeral key.

There is.  That's what the BLKCRYPTOPREPAREKEY ioctl is for.

> Some hardware might permit importing raw keys.

That's what BLKCRYPTOIMPORTKEY is for.

> Other hardware might insist on generating the keys on-chip so that raw keys
> can never be used.

And that's what BLKCRYPTOGENERATEKEY is for.

> Second part is the actual block interface. Gaurav wrote about
> targeting fscrypt, but there should be no actual difference between
> crypto targets. FDE or having a single partition encrypted should
> probably work in the same way. Convert the key into blk_crypto_key
> (including the cookie for the ephemeral key), program the key into the
> slot, use the slot to en/decrypt hardware blocks.
> 
> My main point is that the decision on the key type should be coming
> from the user.

That's exactly how it works.  There is a block interface for specifying an
inline encryption key along with each bio.  The submitter of the bio can specify
either a standard key or a HW-wrapped key.

Again, take a look at the patchset
https://lore.kernel.org/linux-block/20231104211259.17448-1-ebiggers@kernel.org/T/#u.
That's where all this stuff is.

Thanks,

- Eric

On Fri, Jun 21, 2024 at 03:35:40PM +0000, Gaurav Kashyap wrote:
> Hello Eric
> 
> On 06/20/2024, 9:48 PM PDT, Eric Biggers wrote:
> > On Thu, Jun 20, 2024 at 02:57:40PM +0300, Dmitry Baryshkov wrote:
> > > > > >
> > > > > > > Is it possible to use both kind of keys when working on standard
> > mode?
> > > > > > > If not, it should be the user who selects what type of keys to be
> > used.
> > > > > > > Enforcing this via DT is not a way to go.
> > > > > > >
> > > > > >
> > > > > > Unfortunately, that support is not there yet. When you say user,
> > > > > > do you mean to have it as a filesystem mount option?
> > > > >
> > > > > During cryptsetup time. When running e.g. cryptsetup I, as a user,
> > > > > would like to be able to use either a hardware-wrapped key or a
> > standard key.
> > > > >
> > > >
> > > > What we are looking for with these patches is for per-file/folder
> > encryption using fscrypt policies.
> > > > Cryptsetup to my understanding supports only full-disk , and does
> > > > not support FBE (File-Based)
> > >
> > > I must admit, I mostly used dm-crypt beforehand, so I had to look at
> > > fscrypt now. Some of my previous comments might not be fully
> > > applicable.
> > >
> > > > Hence the idea here is that we mount an unencrypted device (with the
> > > > inlinecrypt option that indicates inline encryption is supported) And
> > specify policies (links to keys) for different folders.
> > > >
> > > > > > The way the UFS/EMMC crypto layer is designed currently is that,
> > > > > > this information is needed when the modules are loaded.
> > > > > >
> > > > > > https://lore.kernel.org/all/20231104211259.17448-2-ebiggers@kern
> > > > > > el.org /#Z31drivers:ufs:core:ufshcd-crypto.c
> > > > >
> > > > > I see that the driver lists capabilities here. E.g. that it
> > > > > supports HW-wrapped keys. But the line doesn't specify that standard
> > keys are not supported.
> > > > >
> > > >
> > > > Those are capabilities that are read from the storage controller.
> > > > However, wrapped keys Are not a standard in the ICE JEDEC
> > > > specification, and in most cases, is a value add coming from the SoC.
> > > >
> > > > QCOM SOC and firmware currently does not support both kinds of keys in
> > the HWKM mode.
> > > > That is something we are internally working on, but not available yet.
> > >
> > > I'd say this is a significant obstacle, at least from my point of
> > > view. I understand that the default might be to use hw-wrapped keys,
> > > but it should be possible for the user to select non-HW keys if the
> > > ability to recover the data is considered to be important. Note, I'm
> > > really pointing to the user here, not to the system integrator. So
> > > using DT property or specifying kernel arguments to switch between
> > > these modes is not really an option.
> > >
> > > But I'd really love to hear some feedback from linux-security and/or
> > > linux-fscrypt here.
> > >
> > > In my humble opinion the user should be able to specify that the key
> > > is wrapped using the hardware KMK. Then if the hardware has already
> > > started using the other kind of keys, it should be able to respond
> > > with -EINVAL / whatever else. Then the user can evict previously
> > > programmed key and program a desired one.
> > >
> > > > > Also, I'd have expected that hw-wrapped keys are handled using
> > > > > trusted keys mechanism (see security/keys/trusted-keys/). Could
> > > > > you please point out why that's not the case?
> > > > >
> > > >
> > > > I will evaluate this.
> > > > But my initial response is that we currently cannot communicate to
> > > > our TPM directly from HLOS, but goes through QTEE, and I don't think
> > > > our qtee currently interfaces with the open source tee driver. The
> > interface is through QCOM SCM driver.
> > >
> > > Note, this is just an API interface, see how it is implemented for the
> > > CAAM hardware.
> > >
> > 
> > The problem is that this patchset was sent out without the patches that add
> > the block and filesystem-level framework for hardware-wrapped inline
> > encryption keys, which it depends on.  So it's lacking context.  The proposed
> > framework can be found at https://lore.kernel.org/linux-
> > block/20231104211259.17448-1-ebiggers@kernel.org/T/#u
> > 
> 
> I have only been adding the fscryp patch link as part of the cover letter - as a dependency.
> https://lore.kernel.org/all/20240617005825.1443206-1-quic_gaurkash@quicinc.com/
> If you would like me to include it in the patch series itself, I can do that as well.
> 

I think including all prerequisite patches would be helpful for reviewers.

Thanks for continuing to work on this!

I still need to get ahold of a sm8650 based device and test this out.  Is the
SM8650 HDK the only option, or is there a sm8650 based phone with upstream
support yet?

- Eric

On Fri, 21 Jun 2024 at 18:39, Eric Biggers <ebiggers@kernel.org> wrote:
>
> On Fri, Jun 21, 2024 at 06:16:37PM +0300, Dmitry Baryshkov wrote:
> > On Fri, 21 Jun 2024 at 07:47, Eric Biggers <ebiggers@kernel.org> wrote:
> > >
> > > On Thu, Jun 20, 2024 at 02:57:40PM +0300, Dmitry Baryshkov wrote:
> > > > > > >
> > > > > > > > Is it possible to use both kind of keys when working on standard mode?
> > > > > > > > If not, it should be the user who selects what type of keys to be used.
> > > > > > > > Enforcing this via DT is not a way to go.
> > > > > > > >
> > > > > > >
> > > > > > > Unfortunately, that support is not there yet. When you say user, do
> > > > > > > you mean to have it as a filesystem mount option?
> > > > > >
> > > > > > During cryptsetup time. When running e.g. cryptsetup I, as a user, would like
> > > > > > to be able to use either a hardware-wrapped key or a standard key.
> > > > > >
> > > > >
> > > > > What we are looking for with these patches is for per-file/folder encryption using fscrypt policies.
> > > > > Cryptsetup to my understanding supports only full-disk , and does not support FBE (File-Based)
> > > >
> > > > I must admit, I mostly used dm-crypt beforehand, so I had to look at
> > > > fscrypt now. Some of my previous comments might not be fully
> > > > applicable.
> > > >
> > > > > Hence the idea here is that we mount an unencrypted device (with the inlinecrypt option that indicates inline encryption is supported)
> > > > > And specify policies (links to keys) for different folders.
> > > > >
> > > > > > > The way the UFS/EMMC crypto layer is designed currently is that, this
> > > > > > > information is needed when the modules are loaded.
> > > > > > >
> > > > > > > https://lore.kernel.org/all/20231104211259.17448-2-ebiggers@kernel.org
> > > > > > > /#Z31drivers:ufs:core:ufshcd-crypto.c
> > > > > >
> > > > > > I see that the driver lists capabilities here. E.g. that it supports HW-wrapped
> > > > > > keys. But the line doesn't specify that standard keys are not supported.
> > > > > >
> > > > >
> > > > > Those are capabilities that are read from the storage controller. However, wrapped keys
> > > > > Are not a standard in the ICE JEDEC specification, and in most cases, is a value add coming
> > > > > from the SoC.
> > > > >
> > > > > QCOM SOC and firmware currently does not support both kinds of keys in the HWKM mode.
> > > > > That is something we are internally working on, but not available yet.
> > > >
> > > > I'd say this is a significant obstacle, at least from my point of
> > > > view. I understand that the default might be to use hw-wrapped keys,
> > > > but it should be possible for the user to select non-HW keys if the
> > > > ability to recover the data is considered to be important. Note, I'm
> > > > really pointing to the user here, not to the system integrator. So
> > > > using DT property or specifying kernel arguments to switch between
> > > > these modes is not really an option.
> > > >
> > > > But I'd really love to hear some feedback from linux-security and/or
> > > > linux-fscrypt here.
> > > >
> > > > In my humble opinion the user should be able to specify that the key
> > > > is wrapped using the hardware KMK. Then if the hardware has already
> > > > started using the other kind of keys, it should be able to respond
> > > > with -EINVAL / whatever else. Then the user can evict previously
> > > > programmed key and program a desired one.
> > > >
> > > > > > Also, I'd have expected that hw-wrapped keys are handled using trusted
> > > > > > keys mechanism (see security/keys/trusted-keys/). Could you please point
> > > > > > out why that's not the case?
> > > > > >
> > > > >
> > > > > I will evaluate this.
> > > > > But my initial response is that we currently cannot communicate to our TPM directly from HLOS, but
> > > > > goes through QTEE, and I don't think our qtee currently interfaces with the open source tee
> > > > > driver. The interface is through QCOM SCM driver.
> > > >
> > > > Note, this is just an API interface, see how it is implemented for the
> > > > CAAM hardware.
> > > >
> > >
> > > The problem is that this patchset was sent out without the patches that add the
> > > block and filesystem-level framework for hardware-wrapped inline encryption
> > > keys, which it depends on.  So it's lacking context.  The proposed framework can
> > > be found at
> > > https://lore.kernel.org/linux-block/20231104211259.17448-1-ebiggers@kernel.org/T/#u
> >
> > Thank you. I have quickly skimmed through the patches, but I didn't
> > review them thoroughly. Maybe the patchset already implements the
> > interfaces that I'm thinking about. In such a case please excuse me. I
> > will give it a more thorough look later today.
> >
> > > As for why "trusted keys" aren't used, they just aren't helpful here.  "Trusted
> > > keys" are based around a model where the kernel can request that keys be sealed
> > > and unsealed using a trust source, and the kernel gets access to the raw
> > > unsealed keys.  Hardware-wrapped inline encryption keys use a different model
> > > where the kernel never gets access to the raw keys.  They also have the concept
> > > of ephemeral wrapping which does not exist in "trusted keys".  And they need to
> > > be properly integrated with the inline encryption framework in the block layer.
> >
> > Then what exactly does qcom_scm_derive_sw_secret() do? Does it rewrap
> > the key under some other key?
>
> It derives a secret for functionality such as filenames encryption that can't
> use inline encryption.
>
> > I had the feeling that there are two separate pieces of functionality
> > being stuffed into a single patchset and into a single solution.
> >
> > First one is handling the keys. I keep on thinking that there should
> > be a separate software interface to unseal the key and rewrap it under
> > an ephemeral key.
>
> There is.  That's what the BLKCRYPTOPREPAREKEY ioctl is for.
>
> > Some hardware might permit importing raw keys.
>
> That's what BLKCRYPTOIMPORTKEY is for.
>
> > Other hardware might insist on generating the keys on-chip so that raw keys
> > can never be used.
>
> And that's what BLKCRYPTOGENERATEKEY is for.

Again, this might be answered somewhere, but why can't we use keyctl
for handling the keys and then use a single IOCTL to point the block
device to the key in the keyring?

>
> > Second part is the actual block interface. Gaurav wrote about
> > targeting fscrypt, but there should be no actual difference between
> > crypto targets. FDE or having a single partition encrypted should
> > probably work in the same way. Convert the key into blk_crypto_key
> > (including the cookie for the ephemeral key), program the key into the
> > slot, use the slot to en/decrypt hardware blocks.
> >
> > My main point is that the decision on the key type should be coming
> > from the user.
>
> That's exactly how it works.  There is a block interface for specifying an
> inline encryption key along with each bio.  The submitter of the bio can specify
> either a standard key or a HW-wrapped key.

Not in this patchset. The ICE driver decides whether it can support
HW-wrapped keys or not and then fails to support other type of keys.

>
> Again, take a look at the patchset
> https://lore.kernel.org/linux-block/20231104211259.17448-1-ebiggers@kernel.org/T/#u.
> That's where all this stuff is.

I was mostly looking at the hardware-specific implementation.

On Fri, Jun 21, 2024 at 07:06:25PM +0300, Dmitry Baryshkov wrote:
> On Fri, 21 Jun 2024 at 18:39, Eric Biggers <ebiggers@kernel.org> wrote:
> >
> > On Fri, Jun 21, 2024 at 06:16:37PM +0300, Dmitry Baryshkov wrote:
> > > On Fri, 21 Jun 2024 at 07:47, Eric Biggers <ebiggers@kernel.org> wrote:
> > > >
> > > > On Thu, Jun 20, 2024 at 02:57:40PM +0300, Dmitry Baryshkov wrote:
> > > > > > > >
> > > > > > > > > Is it possible to use both kind of keys when working on standard mode?
> > > > > > > > > If not, it should be the user who selects what type of keys to be used.
> > > > > > > > > Enforcing this via DT is not a way to go.
> > > > > > > > >
> > > > > > > >
> > > > > > > > Unfortunately, that support is not there yet. When you say user, do
> > > > > > > > you mean to have it as a filesystem mount option?
> > > > > > >
> > > > > > > During cryptsetup time. When running e.g. cryptsetup I, as a user, would like
> > > > > > > to be able to use either a hardware-wrapped key or a standard key.
> > > > > > >
> > > > > >
> > > > > > What we are looking for with these patches is for per-file/folder encryption using fscrypt policies.
> > > > > > Cryptsetup to my understanding supports only full-disk , and does not support FBE (File-Based)
> > > > >
> > > > > I must admit, I mostly used dm-crypt beforehand, so I had to look at
> > > > > fscrypt now. Some of my previous comments might not be fully
> > > > > applicable.
> > > > >
> > > > > > Hence the idea here is that we mount an unencrypted device (with the inlinecrypt option that indicates inline encryption is supported)
> > > > > > And specify policies (links to keys) for different folders.
> > > > > >
> > > > > > > > The way the UFS/EMMC crypto layer is designed currently is that, this
> > > > > > > > information is needed when the modules are loaded.
> > > > > > > >
> > > > > > > > https://lore.kernel.org/all/20231104211259.17448-2-ebiggers@kernel.org
> > > > > > > > /#Z31drivers:ufs:core:ufshcd-crypto.c
> > > > > > >
> > > > > > > I see that the driver lists capabilities here. E.g. that it supports HW-wrapped
> > > > > > > keys. But the line doesn't specify that standard keys are not supported.
> > > > > > >
> > > > > >
> > > > > > Those are capabilities that are read from the storage controller. However, wrapped keys
> > > > > > Are not a standard in the ICE JEDEC specification, and in most cases, is a value add coming
> > > > > > from the SoC.
> > > > > >
> > > > > > QCOM SOC and firmware currently does not support both kinds of keys in the HWKM mode.
> > > > > > That is something we are internally working on, but not available yet.
> > > > >
> > > > > I'd say this is a significant obstacle, at least from my point of
> > > > > view. I understand that the default might be to use hw-wrapped keys,
> > > > > but it should be possible for the user to select non-HW keys if the
> > > > > ability to recover the data is considered to be important. Note, I'm
> > > > > really pointing to the user here, not to the system integrator. So
> > > > > using DT property or specifying kernel arguments to switch between
> > > > > these modes is not really an option.
> > > > >
> > > > > But I'd really love to hear some feedback from linux-security and/or
> > > > > linux-fscrypt here.
> > > > >
> > > > > In my humble opinion the user should be able to specify that the key
> > > > > is wrapped using the hardware KMK. Then if the hardware has already
> > > > > started using the other kind of keys, it should be able to respond
> > > > > with -EINVAL / whatever else. Then the user can evict previously
> > > > > programmed key and program a desired one.
> > > > >
> > > > > > > Also, I'd have expected that hw-wrapped keys are handled using trusted
> > > > > > > keys mechanism (see security/keys/trusted-keys/). Could you please point
> > > > > > > out why that's not the case?
> > > > > > >
> > > > > >
> > > > > > I will evaluate this.
> > > > > > But my initial response is that we currently cannot communicate to our TPM directly from HLOS, but
> > > > > > goes through QTEE, and I don't think our qtee currently interfaces with the open source tee
> > > > > > driver. The interface is through QCOM SCM driver.
> > > > >
> > > > > Note, this is just an API interface, see how it is implemented for the
> > > > > CAAM hardware.
> > > > >
> > > >
> > > > The problem is that this patchset was sent out without the patches that add the
> > > > block and filesystem-level framework for hardware-wrapped inline encryption
> > > > keys, which it depends on.  So it's lacking context.  The proposed framework can
> > > > be found at
> > > > https://lore.kernel.org/linux-block/20231104211259.17448-1-ebiggers@kernel.org/T/#u
> > >
> > > Thank you. I have quickly skimmed through the patches, but I didn't
> > > review them thoroughly. Maybe the patchset already implements the
> > > interfaces that I'm thinking about. In such a case please excuse me. I
> > > will give it a more thorough look later today.
> > >
> > > > As for why "trusted keys" aren't used, they just aren't helpful here.  "Trusted
> > > > keys" are based around a model where the kernel can request that keys be sealed
> > > > and unsealed using a trust source, and the kernel gets access to the raw
> > > > unsealed keys.  Hardware-wrapped inline encryption keys use a different model
> > > > where the kernel never gets access to the raw keys.  They also have the concept
> > > > of ephemeral wrapping which does not exist in "trusted keys".  And they need to
> > > > be properly integrated with the inline encryption framework in the block layer.
> > >
> > > Then what exactly does qcom_scm_derive_sw_secret() do? Does it rewrap
> > > the key under some other key?
> >
> > It derives a secret for functionality such as filenames encryption that can't
> > use inline encryption.
> >
> > > I had the feeling that there are two separate pieces of functionality
> > > being stuffed into a single patchset and into a single solution.
> > >
> > > First one is handling the keys. I keep on thinking that there should
> > > be a separate software interface to unseal the key and rewrap it under
> > > an ephemeral key.
> >
> > There is.  That's what the BLKCRYPTOPREPAREKEY ioctl is for.
> >
> > > Some hardware might permit importing raw keys.
> >
> > That's what BLKCRYPTOIMPORTKEY is for.
> >
> > > Other hardware might insist on generating the keys on-chip so that raw keys
> > > can never be used.
> >
> > And that's what BLKCRYPTOGENERATEKEY is for.
> 
> Again, this might be answered somewhere, but why can't we use keyctl
> for handling the keys and then use a single IOCTL to point the block
> device to the key in the keyring?

All the same functionality would need to be supported, and I think that
shoehorning it into the keyrings service instead of just adding new ioctls would
be more difficult.  The keyrings service was not designed for this use case.
We've already had a lot of problems trying to take advantage of the keyrings
service in fscrypt previously.  The keyrings service is something that sounds
useful but really isn't all that useful.

By "a single IOCTL to point the block device to the key in the keyring", you
seem to be referring to configuring full block device encryption with a single
key.  That's not something that's supported by the upstream kernel yet, and it's
not related to this patchset; currently only fscrypt supports inline encryption.
Support for it will be added at some point, which will likely indeed take the
form of an ioctl to set a key on a block device.  But that would be the case
even without HW-wrapped keys.  And *requiring* the key to be given in a keyring
(instead of just in a byte array passed to the ioctl) isn't very helpful, as it
just makes the API harder to use.  We've learned this from the fscrypt API
already where we actually had to move away from the keyrings service in order to
fix all the issues caused by it (see FS_IOC_ADD_ENCRYPTION_KEY).

> >
> > > Second part is the actual block interface. Gaurav wrote about
> > > targeting fscrypt, but there should be no actual difference between
> > > crypto targets. FDE or having a single partition encrypted should
> > > probably work in the same way. Convert the key into blk_crypto_key
> > > (including the cookie for the ephemeral key), program the key into the
> > > slot, use the slot to en/decrypt hardware blocks.
> > >
> > > My main point is that the decision on the key type should be coming
> > > from the user.
> >
> > That's exactly how it works.  There is a block interface for specifying an
> > inline encryption key along with each bio.  The submitter of the bio can specify
> > either a standard key or a HW-wrapped key.
> 
> Not in this patchset. The ICE driver decides whether it can support
> HW-wrapped keys or not and then fails to support other type of keys.
> 

Sure, that's just a matter of hardware capabilities though, right?  The block
layer provides a way for drivers to declare which inline encryption capabilities
they support.  They can declare they support standard keys, HW-wrapped keys,
both, or neither.  If Qualcomm SoCs can't support both types of keys at the same
time, that's unfortunate, but I'm not sure what your point is.  The user (e.g.
fscrypt) still has control over whether they use the functionality that the
hardware provides.

- Eric

On Fri, 21 Jun 2024 at 19:31, Eric Biggers <ebiggers@kernel.org> wrote:
>
> On Fri, Jun 21, 2024 at 07:06:25PM +0300, Dmitry Baryshkov wrote:
> > On Fri, 21 Jun 2024 at 18:39, Eric Biggers <ebiggers@kernel.org> wrote:
> > >
> > > On Fri, Jun 21, 2024 at 06:16:37PM +0300, Dmitry Baryshkov wrote:
> > > > On Fri, 21 Jun 2024 at 07:47, Eric Biggers <ebiggers@kernel.org> wrote:
> > > > >
> > > > > On Thu, Jun 20, 2024 at 02:57:40PM +0300, Dmitry Baryshkov wrote:
> > > > > > > > >
> > > > > > > > > > Is it possible to use both kind of keys when working on standard mode?
> > > > > > > > > > If not, it should be the user who selects what type of keys to be used.
> > > > > > > > > > Enforcing this via DT is not a way to go.
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > Unfortunately, that support is not there yet. When you say user, do
> > > > > > > > > you mean to have it as a filesystem mount option?
> > > > > > > >
> > > > > > > > During cryptsetup time. When running e.g. cryptsetup I, as a user, would like
> > > > > > > > to be able to use either a hardware-wrapped key or a standard key.
> > > > > > > >
> > > > > > >
> > > > > > > What we are looking for with these patches is for per-file/folder encryption using fscrypt policies.
> > > > > > > Cryptsetup to my understanding supports only full-disk , and does not support FBE (File-Based)
> > > > > >
> > > > > > I must admit, I mostly used dm-crypt beforehand, so I had to look at
> > > > > > fscrypt now. Some of my previous comments might not be fully
> > > > > > applicable.
> > > > > >
> > > > > > > Hence the idea here is that we mount an unencrypted device (with the inlinecrypt option that indicates inline encryption is supported)
> > > > > > > And specify policies (links to keys) for different folders.
> > > > > > >
> > > > > > > > > The way the UFS/EMMC crypto layer is designed currently is that, this
> > > > > > > > > information is needed when the modules are loaded.
> > > > > > > > >
> > > > > > > > > https://lore.kernel.org/all/20231104211259.17448-2-ebiggers@kernel.org
> > > > > > > > > /#Z31drivers:ufs:core:ufshcd-crypto.c
> > > > > > > >
> > > > > > > > I see that the driver lists capabilities here. E.g. that it supports HW-wrapped
> > > > > > > > keys. But the line doesn't specify that standard keys are not supported.
> > > > > > > >
> > > > > > >
> > > > > > > Those are capabilities that are read from the storage controller. However, wrapped keys
> > > > > > > Are not a standard in the ICE JEDEC specification, and in most cases, is a value add coming
> > > > > > > from the SoC.
> > > > > > >
> > > > > > > QCOM SOC and firmware currently does not support both kinds of keys in the HWKM mode.
> > > > > > > That is something we are internally working on, but not available yet.
> > > > > >
> > > > > > I'd say this is a significant obstacle, at least from my point of
> > > > > > view. I understand that the default might be to use hw-wrapped keys,
> > > > > > but it should be possible for the user to select non-HW keys if the
> > > > > > ability to recover the data is considered to be important. Note, I'm
> > > > > > really pointing to the user here, not to the system integrator. So
> > > > > > using DT property or specifying kernel arguments to switch between
> > > > > > these modes is not really an option.
> > > > > >
> > > > > > But I'd really love to hear some feedback from linux-security and/or
> > > > > > linux-fscrypt here.
> > > > > >
> > > > > > In my humble opinion the user should be able to specify that the key
> > > > > > is wrapped using the hardware KMK. Then if the hardware has already
> > > > > > started using the other kind of keys, it should be able to respond
> > > > > > with -EINVAL / whatever else. Then the user can evict previously
> > > > > > programmed key and program a desired one.
> > > > > >
> > > > > > > > Also, I'd have expected that hw-wrapped keys are handled using trusted
> > > > > > > > keys mechanism (see security/keys/trusted-keys/). Could you please point
> > > > > > > > out why that's not the case?
> > > > > > > >
> > > > > > >
> > > > > > > I will evaluate this.
> > > > > > > But my initial response is that we currently cannot communicate to our TPM directly from HLOS, but
> > > > > > > goes through QTEE, and I don't think our qtee currently interfaces with the open source tee
> > > > > > > driver. The interface is through QCOM SCM driver.
> > > > > >
> > > > > > Note, this is just an API interface, see how it is implemented for the
> > > > > > CAAM hardware.
> > > > > >
> > > > >
> > > > > The problem is that this patchset was sent out without the patches that add the
> > > > > block and filesystem-level framework for hardware-wrapped inline encryption
> > > > > keys, which it depends on.  So it's lacking context.  The proposed framework can
> > > > > be found at
> > > > > https://lore.kernel.org/linux-block/20231104211259.17448-1-ebiggers@kernel.org/T/#u
> > > >
> > > > Thank you. I have quickly skimmed through the patches, but I didn't
> > > > review them thoroughly. Maybe the patchset already implements the
> > > > interfaces that I'm thinking about. In such a case please excuse me. I
> > > > will give it a more thorough look later today.
> > > >
> > > > > As for why "trusted keys" aren't used, they just aren't helpful here.  "Trusted
> > > > > keys" are based around a model where the kernel can request that keys be sealed
> > > > > and unsealed using a trust source, and the kernel gets access to the raw
> > > > > unsealed keys.  Hardware-wrapped inline encryption keys use a different model
> > > > > where the kernel never gets access to the raw keys.  They also have the concept
> > > > > of ephemeral wrapping which does not exist in "trusted keys".  And they need to
> > > > > be properly integrated with the inline encryption framework in the block layer.
> > > >
> > > > Then what exactly does qcom_scm_derive_sw_secret() do? Does it rewrap
> > > > the key under some other key?
> > >
> > > It derives a secret for functionality such as filenames encryption that can't
> > > use inline encryption.
> > >
> > > > I had the feeling that there are two separate pieces of functionality
> > > > being stuffed into a single patchset and into a single solution.
> > > >
> > > > First one is handling the keys. I keep on thinking that there should
> > > > be a separate software interface to unseal the key and rewrap it under
> > > > an ephemeral key.
> > >
> > > There is.  That's what the BLKCRYPTOPREPAREKEY ioctl is for.
> > >
> > > > Some hardware might permit importing raw keys.
> > >
> > > That's what BLKCRYPTOIMPORTKEY is for.
> > >
> > > > Other hardware might insist on generating the keys on-chip so that raw keys
> > > > can never be used.
> > >
> > > And that's what BLKCRYPTOGENERATEKEY is for.
> >
> > Again, this might be answered somewhere, but why can't we use keyctl
> > for handling the keys and then use a single IOCTL to point the block
> > device to the key in the keyring?
>
> All the same functionality would need to be supported, and I think that
> shoehorning it into the keyrings service instead of just adding new ioctls would
> be more difficult.  The keyrings service was not designed for this use case.
> We've already had a lot of problems trying to take advantage of the keyrings
> service in fscrypt previously.  The keyrings service is something that sounds
> useful but really isn't all that useful.

I would be really interested in reading or listening to any kind of
summary or parts of the issues.
I'm slightly pushy towards keyctl / keyrings, because it already
provides support for different kinds of key wrapping and key
management. Encrypted keys, trusted keys - those are all kinds of key
management, which either will be missing or will have to be
reimplemented for block layers.

I know that keyrings are clumsy and not that logical, but then their
API needs to be improved. Just ignoring the existing mechanisms sounds
like a bad idea.

>
> By "a single IOCTL to point the block device to the key in the keyring", you
> seem to be referring to configuring full block device encryption with a single
> key.  That's not something that's supported by the upstream kernel yet, and it's
> not related to this patchset; currently only fscrypt supports inline encryption.

I see that dm has at least some provisioning and hooks for
CONFIG_BLK_INLINE_ENCRYPTION. Thus I thought that it's possible to use
inline encryption through DM.

> Support for it will be added at some point, which will likely indeed take the
> form of an ioctl to set a key on a block device.  But that would be the case
> even without HW-wrapped keys.  And *requiring* the key to be given in a keyring
> (instead of just in a byte array passed to the ioctl) isn't very helpful, as it
> just makes the API harder to use.  We've learned this from the fscrypt API
> already where we actually had to move away from the keyrings service in order to
> fix all the issues caused by it (see FS_IOC_ADD_ENCRYPTION_KEY).
>
> > >
> > > > Second part is the actual block interface. Gaurav wrote about
> > > > targeting fscrypt, but there should be no actual difference between
> > > > crypto targets. FDE or having a single partition encrypted should
> > > > probably work in the same way. Convert the key into blk_crypto_key
> > > > (including the cookie for the ephemeral key), program the key into the
> > > > slot, use the slot to en/decrypt hardware blocks.
> > > >
> > > > My main point is that the decision on the key type should be coming
> > > > from the user.
> > >
> > > That's exactly how it works.  There is a block interface for specifying an
> > > inline encryption key along with each bio.  The submitter of the bio can specify
> > > either a standard key or a HW-wrapped key.
> >
> > Not in this patchset. The ICE driver decides whether it can support
> > HW-wrapped keys or not and then fails to support other type of keys.
> >
>
> Sure, that's just a matter of hardware capabilities though, right?  The block
> layer provides a way for drivers to declare which inline encryption capabilities
> they support.  They can declare they support standard keys, HW-wrapped keys,
> both, or neither.  If Qualcomm SoCs can't support both types of keys at the same
> time, that's unfortunate, but I'm not sure what your poitnt is.  The user (e.g.
> fscrypt) still has control over whether they use the functionality that the
> hardware provides.

It's a matter of policy. Harware / firmware doesn't support using both
kinds of keys concurrently, if I understood Gaurav's explanations
correctly. But the user should be able to make a judgement and use
non-hw-wrapped keys if it fits their requirements. The driver should
not make this kind of judgement. Note, this is not an issue of your
original patchset, but it's a driver flaw in this patchset.

--
With best wishes
Dmitry

On Fri, Jun 21, 2024 at 08:49:56PM +0300, Dmitry Baryshkov wrote:
> On Fri, 21 Jun 2024 at 19:31, Eric Biggers <ebiggers@kernel.org> wrote:
> >
> > On Fri, Jun 21, 2024 at 07:06:25PM +0300, Dmitry Baryshkov wrote:
> > > On Fri, 21 Jun 2024 at 18:39, Eric Biggers <ebiggers@kernel.org> wrote:
> > > >
> > > > On Fri, Jun 21, 2024 at 06:16:37PM +0300, Dmitry Baryshkov wrote:
> > > > > On Fri, 21 Jun 2024 at 07:47, Eric Biggers <ebiggers@kernel.org> wrote:
> > > > > >
> > > > > > On Thu, Jun 20, 2024 at 02:57:40PM +0300, Dmitry Baryshkov wrote:
> > > > > > > > > >
> > > > > > > > > > > Is it possible to use both kind of keys when working on standard mode?
> > > > > > > > > > > If not, it should be the user who selects what type of keys to be used.
> > > > > > > > > > > Enforcing this via DT is not a way to go.
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Unfortunately, that support is not there yet. When you say user, do
> > > > > > > > > > you mean to have it as a filesystem mount option?
> > > > > > > > >
> > > > > > > > > During cryptsetup time. When running e.g. cryptsetup I, as a user, would like
> > > > > > > > > to be able to use either a hardware-wrapped key or a standard key.
> > > > > > > > >
> > > > > > > >
> > > > > > > > What we are looking for with these patches is for per-file/folder encryption using fscrypt policies.
> > > > > > > > Cryptsetup to my understanding supports only full-disk , and does not support FBE (File-Based)
> > > > > > >
> > > > > > > I must admit, I mostly used dm-crypt beforehand, so I had to look at
> > > > > > > fscrypt now. Some of my previous comments might not be fully
> > > > > > > applicable.
> > > > > > >
> > > > > > > > Hence the idea here is that we mount an unencrypted device (with the inlinecrypt option that indicates inline encryption is supported)
> > > > > > > > And specify policies (links to keys) for different folders.
> > > > > > > >
> > > > > > > > > > The way the UFS/EMMC crypto layer is designed currently is that, this
> > > > > > > > > > information is needed when the modules are loaded.
> > > > > > > > > >
> > > > > > > > > > https://lore.kernel.org/all/20231104211259.17448-2-ebiggers@kernel.org
> > > > > > > > > > /#Z31drivers:ufs:core:ufshcd-crypto.c
> > > > > > > > >
> > > > > > > > > I see that the driver lists capabilities here. E.g. that it supports HW-wrapped
> > > > > > > > > keys. But the line doesn't specify that standard keys are not supported.
> > > > > > > > >
> > > > > > > >
> > > > > > > > Those are capabilities that are read from the storage controller. However, wrapped keys
> > > > > > > > Are not a standard in the ICE JEDEC specification, and in most cases, is a value add coming
> > > > > > > > from the SoC.
> > > > > > > >
> > > > > > > > QCOM SOC and firmware currently does not support both kinds of keys in the HWKM mode.
> > > > > > > > That is something we are internally working on, but not available yet.
> > > > > > >
> > > > > > > I'd say this is a significant obstacle, at least from my point of
> > > > > > > view. I understand that the default might be to use hw-wrapped keys,
> > > > > > > but it should be possible for the user to select non-HW keys if the
> > > > > > > ability to recover the data is considered to be important. Note, I'm
> > > > > > > really pointing to the user here, not to the system integrator. So
> > > > > > > using DT property or specifying kernel arguments to switch between
> > > > > > > these modes is not really an option.
> > > > > > >
> > > > > > > But I'd really love to hear some feedback from linux-security and/or
> > > > > > > linux-fscrypt here.
> > > > > > >
> > > > > > > In my humble opinion the user should be able to specify that the key
> > > > > > > is wrapped using the hardware KMK. Then if the hardware has already
> > > > > > > started using the other kind of keys, it should be able to respond
> > > > > > > with -EINVAL / whatever else. Then the user can evict previously
> > > > > > > programmed key and program a desired one.
> > > > > > >
> > > > > > > > > Also, I'd have expected that hw-wrapped keys are handled using trusted
> > > > > > > > > keys mechanism (see security/keys/trusted-keys/). Could you please point
> > > > > > > > > out why that's not the case?
> > > > > > > > >
> > > > > > > >
> > > > > > > > I will evaluate this.
> > > > > > > > But my initial response is that we currently cannot communicate to our TPM directly from HLOS, but
> > > > > > > > goes through QTEE, and I don't think our qtee currently interfaces with the open source tee
> > > > > > > > driver. The interface is through QCOM SCM driver.
> > > > > > >
> > > > > > > Note, this is just an API interface, see how it is implemented for the
> > > > > > > CAAM hardware.
> > > > > > >
> > > > > >
> > > > > > The problem is that this patchset was sent out without the patches that add the
> > > > > > block and filesystem-level framework for hardware-wrapped inline encryption
> > > > > > keys, which it depends on.  So it's lacking context.  The proposed framework can
> > > > > > be found at
> > > > > > https://lore.kernel.org/linux-block/20231104211259.17448-1-ebiggers@kernel.org/T/#u
> > > > >
> > > > > Thank you. I have quickly skimmed through the patches, but I didn't
> > > > > review them thoroughly. Maybe the patchset already implements the
> > > > > interfaces that I'm thinking about. In such a case please excuse me. I
> > > > > will give it a more thorough look later today.
> > > > >
> > > > > > As for why "trusted keys" aren't used, they just aren't helpful here.  "Trusted
> > > > > > keys" are based around a model where the kernel can request that keys be sealed
> > > > > > and unsealed using a trust source, and the kernel gets access to the raw
> > > > > > unsealed keys.  Hardware-wrapped inline encryption keys use a different model
> > > > > > where the kernel never gets access to the raw keys.  They also have the concept
> > > > > > of ephemeral wrapping which does not exist in "trusted keys".  And they need to
> > > > > > be properly integrated with the inline encryption framework in the block layer.
> > > > >
> > > > > Then what exactly does qcom_scm_derive_sw_secret() do? Does it rewrap
> > > > > the key under some other key?
> > > >
> > > > It derives a secret for functionality such as filenames encryption that can't
> > > > use inline encryption.
> > > >
> > > > > I had the feeling that there are two separate pieces of functionality
> > > > > being stuffed into a single patchset and into a single solution.
> > > > >
> > > > > First one is handling the keys. I keep on thinking that there should
> > > > > be a separate software interface to unseal the key and rewrap it under
> > > > > an ephemeral key.
> > > >
> > > > There is.  That's what the BLKCRYPTOPREPAREKEY ioctl is for.
> > > >
> > > > > Some hardware might permit importing raw keys.
> > > >
> > > > That's what BLKCRYPTOIMPORTKEY is for.
> > > >
> > > > > Other hardware might insist on generating the keys on-chip so that raw keys
> > > > > can never be used.
> > > >
> > > > And that's what BLKCRYPTOGENERATEKEY is for.
> > >
> > > Again, this might be answered somewhere, but why can't we use keyctl
> > > for handling the keys and then use a single IOCTL to point the block
> > > device to the key in the keyring?
> >
> > All the same functionality would need to be supported, and I think that
> > shoehorning it into the keyrings service instead of just adding new ioctls would
> > be more difficult.  The keyrings service was not designed for this use case.
> > We've already had a lot of problems trying to take advantage of the keyrings
> > service in fscrypt previously.  The keyrings service is something that sounds
> > useful but really isn't all that useful.
> 
> I would be really interested in reading or listening to any kind of
> summary or parts of the issues.
> I'm slightly pushy towards keyctl / keyrings, because it already
> provides support for different kinds of key wrapping and key
> management. Encrypted keys, trusted keys - those are all kinds of key
> management, which either will be missing or will have to be
> reimplemented for block layers.
> 
> I know that keyrings are clumsy and not that logical, but then their
> API needs to be improved. Just ignoring the existing mechanisms sounds
> like a bad idea.

One thing to keep in mind is that keyring service keys can't be used directly
for storage encryption.  So if a component that manages storage encryption (such
as fscrypt or dm-crypt) is given a keyring service key, in practice it has to
extract the payload from the keyring service key, and not use the keyring
service key any further.  So the keyring service can, at most, serve as a way to
create and prepare the key, and after that it doesn't serve a purpose.

(fscrypt used to use the keyring service a bit more: it looked up a key whenever
a file was opened, and it supported evicting per-file keys by revoking the
corresponding keyring key.  But this turned out to be totally broken.  E.g., it
didn't provide the correct semantics for filesystem encryption where the key
should either be present or absent filesystem-wide.)

We do need the ability to create HW-wrapped keys in long-term wrapped form,
either via "generate" or "import", return those long-term wrapped keys to
userspace so that they can be stored on-disk, and convert them into
ephemerally-wrapped form so they can be used.  It probably would be possible to
support all of this through the keyrings service, but it would need a couple new
key types:

- One key type that can be instantiated with a raw key (or NULL to request
  generation of a key) and that automagically creates a long-term wrapped key
  and supports userspace reading it back.  This would be vaguely similar to
  "trusted", but without any support for using the key directly.

- One key type that can be instantiated using a long-term wrapped key which gets
  automagically converted to an ephemerally-wrapped key.  This would be what is
  passed to other kernel subsystems.  Functions specific to this key type would
  need to be provided for users to use.

I think it would be possible, but it feels like a bit of a shoehorned API.  The
ioctls are a more straightforward solution.

> >
> > By "a single IOCTL to point the block device to the key in the keyring", you
> > seem to be referring to configuring full block device encryption with a single
> > key.  That's not something that's supported by the upstream kernel yet, and it's
> > not related to this patchset; currently only fscrypt supports inline encryption.
> 
> I see that dm has at least some provisioning and hooks for
> CONFIG_BLK_INLINE_ENCRYPTION. Thus I thought that it's possible to use
> inline encryption through DM.

device-mapper supports passing through the inline encryption support of
underlying devices in certain cases, but it doesn't yet support using it itself.

> 
> > Support for it will be added at some point, which will likely indeed take the
> > form of an ioctl to set a key on a block device.  But that would be the case
> > even without HW-wrapped keys.  And *requiring* the key to be given in a keyring
> > (instead of just in a byte array passed to the ioctl) isn't very helpful, as it
> > just makes the API harder to use.  We've learned this from the fscrypt API
> > already where we actually had to move away from the keyrings service in order to
> > fix all the issues caused by it (see FS_IOC_ADD_ENCRYPTION_KEY).
> >
> > > >
> > > > > Second part is the actual block interface. Gaurav wrote about
> > > > > targeting fscrypt, but there should be no actual difference between
> > > > > crypto targets. FDE or having a single partition encrypted should
> > > > > probably work in the same way. Convert the key into blk_crypto_key
> > > > > (including the cookie for the ephemeral key), program the key into the
> > > > > slot, use the slot to en/decrypt hardware blocks.
> > > > >
> > > > > My main point is that the decision on the key type should be coming
> > > > > from the user.
> > > >
> > > > That's exactly how it works.  There is a block interface for specifying an
> > > > inline encryption key along with each bio.  The submitter of the bio can specify
> > > > either a standard key or a HW-wrapped key.
> > >
> > > Not in this patchset. The ICE driver decides whether it can support
> > > HW-wrapped keys or not and then fails to support other type of keys.
> > >
> >
> > Sure, that's just a matter of hardware capabilities though, right?  The block
> > layer provides a way for drivers to declare which inline encryption capabilities
> > they support.  They can declare they support standard keys, HW-wrapped keys,
> > both, or neither.  If Qualcomm SoCs can't support both types of keys at the same
> > time, that's unfortunate, but I'm not sure what your poitnt is.  The user (e.g.
> > fscrypt) still has control over whether they use the functionality that the
> > hardware provides.
> 
> It's a matter of policy. Harware / firmware doesn't support using both
> kinds of keys concurrently, if I understood Gaurav's explanations
> correctly. But the user should be able to make a judgement and use
> non-hw-wrapped keys if it fits their requirements. The driver should
> not make this kind of judgement. Note, this is not an issue of your
> original patchset, but it's a driver flaw in this patchset.

If the driver has to make a decision about which type of keys to support (due to
the hardware and firmware supporting both but not at the same time), I think
this will need to be done via a module parameter, e.g.
qcom_ice.hw_wrapped_keys=1 to support HW-wrapped keys instead of standard keys.

- Eric

On Fri, 21 Jun 2024 at 21:36, Eric Biggers <ebiggers@kernel.org> wrote:
>
> On Fri, Jun 21, 2024 at 08:49:56PM +0300, Dmitry Baryshkov wrote:
> > On Fri, 21 Jun 2024 at 19:31, Eric Biggers <ebiggers@kernel.org> wrote:
> > >
> > > On Fri, Jun 21, 2024 at 07:06:25PM +0300, Dmitry Baryshkov wrote:
> > > > On Fri, 21 Jun 2024 at 18:39, Eric Biggers <ebiggers@kernel.org> wrote:
> > > > >
> > > > > On Fri, Jun 21, 2024 at 06:16:37PM +0300, Dmitry Baryshkov wrote:
> > > > > > On Fri, 21 Jun 2024 at 07:47, Eric Biggers <ebiggers@kernel.org> wrote:
> > > > > > >
> > > > > > > On Thu, Jun 20, 2024 at 02:57:40PM +0300, Dmitry Baryshkov wrote:
> > > > > > > > > > >
> > > > > > > > > > > > Is it possible to use both kind of keys when working on standard mode?
> > > > > > > > > > > > If not, it should be the user who selects what type of keys to be used.
> > > > > > > > > > > > Enforcing this via DT is not a way to go.
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > Unfortunately, that support is not there yet. When you say user, do
> > > > > > > > > > > you mean to have it as a filesystem mount option?
> > > > > > > > > >
> > > > > > > > > > During cryptsetup time. When running e.g. cryptsetup I, as a user, would like
> > > > > > > > > > to be able to use either a hardware-wrapped key or a standard key.
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > What we are looking for with these patches is for per-file/folder encryption using fscrypt policies.
> > > > > > > > > Cryptsetup to my understanding supports only full-disk , and does not support FBE (File-Based)
> > > > > > > >
> > > > > > > > I must admit, I mostly used dm-crypt beforehand, so I had to look at
> > > > > > > > fscrypt now. Some of my previous comments might not be fully
> > > > > > > > applicable.
> > > > > > > >
> > > > > > > > > Hence the idea here is that we mount an unencrypted device (with the inlinecrypt option that indicates inline encryption is supported)
> > > > > > > > > And specify policies (links to keys) for different folders.
> > > > > > > > >
> > > > > > > > > > > The way the UFS/EMMC crypto layer is designed currently is that, this
> > > > > > > > > > > information is needed when the modules are loaded.
> > > > > > > > > > >
> > > > > > > > > > > https://lore.kernel.org/all/20231104211259.17448-2-ebiggers@kernel.org
> > > > > > > > > > > /#Z31drivers:ufs:core:ufshcd-crypto.c
> > > > > > > > > >
> > > > > > > > > > I see that the driver lists capabilities here. E.g. that it supports HW-wrapped
> > > > > > > > > > keys. But the line doesn't specify that standard keys are not supported.
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > Those are capabilities that are read from the storage controller. However, wrapped keys
> > > > > > > > > Are not a standard in the ICE JEDEC specification, and in most cases, is a value add coming
> > > > > > > > > from the SoC.
> > > > > > > > >
> > > > > > > > > QCOM SOC and firmware currently does not support both kinds of keys in the HWKM mode.
> > > > > > > > > That is something we are internally working on, but not available yet.
> > > > > > > >
> > > > > > > > I'd say this is a significant obstacle, at least from my point of
> > > > > > > > view. I understand that the default might be to use hw-wrapped keys,
> > > > > > > > but it should be possible for the user to select non-HW keys if the
> > > > > > > > ability to recover the data is considered to be important. Note, I'm
> > > > > > > > really pointing to the user here, not to the system integrator. So
> > > > > > > > using DT property or specifying kernel arguments to switch between
> > > > > > > > these modes is not really an option.
> > > > > > > >
> > > > > > > > But I'd really love to hear some feedback from linux-security and/or
> > > > > > > > linux-fscrypt here.
> > > > > > > >
> > > > > > > > In my humble opinion the user should be able to specify that the key
> > > > > > > > is wrapped using the hardware KMK. Then if the hardware has already
> > > > > > > > started using the other kind of keys, it should be able to respond
> > > > > > > > with -EINVAL / whatever else. Then the user can evict previously
> > > > > > > > programmed key and program a desired one.
> > > > > > > >
> > > > > > > > > > Also, I'd have expected that hw-wrapped keys are handled using trusted
> > > > > > > > > > keys mechanism (see security/keys/trusted-keys/). Could you please point
> > > > > > > > > > out why that's not the case?
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > I will evaluate this.
> > > > > > > > > But my initial response is that we currently cannot communicate to our TPM directly from HLOS, but
> > > > > > > > > goes through QTEE, and I don't think our qtee currently interfaces with the open source tee
> > > > > > > > > driver. The interface is through QCOM SCM driver.
> > > > > > > >
> > > > > > > > Note, this is just an API interface, see how it is implemented for the
> > > > > > > > CAAM hardware.
> > > > > > > >
> > > > > > >
> > > > > > > The problem is that this patchset was sent out without the patches that add the
> > > > > > > block and filesystem-level framework for hardware-wrapped inline encryption
> > > > > > > keys, which it depends on.  So it's lacking context.  The proposed framework can
> > > > > > > be found at
> > > > > > > https://lore.kernel.org/linux-block/20231104211259.17448-1-ebiggers@kernel.org/T/#u
> > > > > >
> > > > > > Thank you. I have quickly skimmed through the patches, but I didn't
> > > > > > review them thoroughly. Maybe the patchset already implements the
> > > > > > interfaces that I'm thinking about. In such a case please excuse me. I
> > > > > > will give it a more thorough look later today.
> > > > > >
> > > > > > > As for why "trusted keys" aren't used, they just aren't helpful here.  "Trusted
> > > > > > > keys" are based around a model where the kernel can request that keys be sealed
> > > > > > > and unsealed using a trust source, and the kernel gets access to the raw
> > > > > > > unsealed keys.  Hardware-wrapped inline encryption keys use a different model
> > > > > > > where the kernel never gets access to the raw keys.  They also have the concept
> > > > > > > of ephemeral wrapping which does not exist in "trusted keys".  And they need to
> > > > > > > be properly integrated with the inline encryption framework in the block layer.
> > > > > >
> > > > > > Then what exactly does qcom_scm_derive_sw_secret() do? Does it rewrap
> > > > > > the key under some other key?
> > > > >
> > > > > It derives a secret for functionality such as filenames encryption that can't
> > > > > use inline encryption.
> > > > >
> > > > > > I had the feeling that there are two separate pieces of functionality
> > > > > > being stuffed into a single patchset and into a single solution.
> > > > > >
> > > > > > First one is handling the keys. I keep on thinking that there should
> > > > > > be a separate software interface to unseal the key and rewrap it under
> > > > > > an ephemeral key.
> > > > >
> > > > > There is.  That's what the BLKCRYPTOPREPAREKEY ioctl is for.
> > > > >
> > > > > > Some hardware might permit importing raw keys.
> > > > >
> > > > > That's what BLKCRYPTOIMPORTKEY is for.
> > > > >
> > > > > > Other hardware might insist on generating the keys on-chip so that raw keys
> > > > > > can never be used.
> > > > >
> > > > > And that's what BLKCRYPTOGENERATEKEY is for.
> > > >
> > > > Again, this might be answered somewhere, but why can't we use keyctl
> > > > for handling the keys and then use a single IOCTL to point the block
> > > > device to the key in the keyring?
> > >
> > > All the same functionality would need to be supported, and I think that
> > > shoehorning it into the keyrings service instead of just adding new ioctls would
> > > be more difficult.  The keyrings service was not designed for this use case.
> > > We've already had a lot of problems trying to take advantage of the keyrings
> > > service in fscrypt previously.  The keyrings service is something that sounds
> > > useful but really isn't all that useful.
> >
> > I would be really interested in reading or listening to any kind of
> > summary or parts of the issues.
> > I'm slightly pushy towards keyctl / keyrings, because it already
> > provides support for different kinds of key wrapping and key
> > management. Encrypted keys, trusted keys - those are all kinds of key
> > management, which either will be missing or will have to be
> > reimplemented for block layers.
> >
> > I know that keyrings are clumsy and not that logical, but then their
> > API needs to be improved. Just ignoring the existing mechanisms sounds
> > like a bad idea.
>
> One thing to keep in mind is that keyring service keys can't be used directly
> for storage encryption.  So if a component that manages storage encryption (such
> as fscrypt or dm-crypt) is given a keyring service key, in practice it has to
> extract the payload from the keyring service key, and not use the keyring
> service key any further.  So the keyring service can, at most, serve as a way to
> create and prepare the key, and after that it doesn't serve a purpose.

Yes, this sounds good enough.

>
> (fscrypt used to use the keyring service a bit more: it looked up a key whenever
> a file was opened, and it supported evicting per-file keys by revoking the
> corresponding keyring key.  But this turned out to be totally broken.  E.g., it
> didn't provide the correct semantics for filesystem encryption where the key
> should either be present or absent filesystem-wide.)
>
> We do need the ability to create HW-wrapped keys in long-term wrapped form,
> either via "generate" or "import", return those long-term wrapped keys to
> userspace so that they can be stored on-disk, and convert them into
> ephemerally-wrapped form so they can be used.  It probably would be possible to
> support all of this through the keyrings service, but it would need a couple new
> key types:
>
> - One key type that can be instantiated with a raw key (or NULL to request
>   generation of a key) and that automagically creates a long-term wrapped key
>   and supports userspace reading it back.  This would be vaguely similar to
>   "trusted", but without any support for using the key directly.
>
> - One key type that can be instantiated using a long-term wrapped key which gets
>   automagically converted to an ephemerally-wrapped key.  This would be what is
>   passed to other kernel subsystems.  Functions specific to this key type would
>   need to be provided for users to use.

I think having one key type should be enough. The userspace loads /
generates&reads / wraps and reads back the 'exported' version wrapped
using the platform-specific key. In kernel the key is unsealed and
represented as binary key to be loaded to the hardware + a cookie for
the ephemeral key and device that have been used to wrap it. When
userspace asks the device to program the key, the cookie is verified
to match the device / ephemeral key and then the binary is programmed
to the hardware. Maybe it's enough to use the struct device as a
cookie.

> I think it would be possible, but it feels like a bit of a shoehorned API.  The
> ioctls are a more straightforward solution.

Are we going to have another set of IOCTLs for loading the encrypted
keys? keys sealed by TPM?

> > > By "a single IOCTL to point the block device to the key in the keyring", you
> > > seem to be referring to configuring full block device encryption with a single
> > > key.  That's not something that's supported by the upstream kernel yet, and it's
> > > not related to this patchset; currently only fscrypt supports inline encryption.
> >
> > I see that dm has at least some provisioning and hooks for
> > CONFIG_BLK_INLINE_ENCRYPTION. Thus I thought that it's possible to use
> > inline encryption through DM.
>
> device-mapper supports passing through the inline encryption support of
> underlying devices in certain cases, but it doesn't yet support using it itself.

I see. I was confused by the dm code then. Please excuse me.

>
> >
> > > Support for it will be added at some point, which will likely indeed take the
> > > form of an ioctl to set a key on a block device.  But that would be the case
> > > even without HW-wrapped keys.  And *requiring* the key to be given in a keyring
> > > (instead of just in a byte array passed to the ioctl) isn't very helpful, as it
> > > just makes the API harder to use.  We've learned this from the fscrypt API
> > > already where we actually had to move away from the keyrings service in order to
> > > fix all the issues caused by it (see FS_IOC_ADD_ENCRYPTION_KEY).
> > >
> > > > >
> > > > > > Second part is the actual block interface. Gaurav wrote about
> > > > > > targeting fscrypt, but there should be no actual difference between
> > > > > > crypto targets. FDE or having a single partition encrypted should
> > > > > > probably work in the same way. Convert the key into blk_crypto_key
> > > > > > (including the cookie for the ephemeral key), program the key into the
> > > > > > slot, use the slot to en/decrypt hardware blocks.
> > > > > >
> > > > > > My main point is that the decision on the key type should be coming
> > > > > > from the user.
> > > > >
> > > > > That's exactly how it works.  There is a block interface for specifying an
> > > > > inline encryption key along with each bio.  The submitter of the bio can specify
> > > > > either a standard key or a HW-wrapped key.
> > > >
> > > > Not in this patchset. The ICE driver decides whether it can support
> > > > HW-wrapped keys or not and then fails to support other type of keys.
> > > >
> > >
> > > Sure, that's just a matter of hardware capabilities though, right?  The block
> > > layer provides a way for drivers to declare which inline encryption capabilities
> > > they support.  They can declare they support standard keys, HW-wrapped keys,
> > > both, or neither.  If Qualcomm SoCs can't support both types of keys at the same
> > > time, that's unfortunate, but I'm not sure what your poitnt is.  The user (e.g.
> > > fscrypt) still has control over whether they use the functionality that the
> > > hardware provides.
> >
> > It's a matter of policy. Harware / firmware doesn't support using both
> > kinds of keys concurrently, if I understood Gaurav's explanations
> > correctly. But the user should be able to make a judgement and use
> > non-hw-wrapped keys if it fits their requirements. The driver should
> > not make this kind of judgement. Note, this is not an issue of your
> > original patchset, but it's a driver flaw in this patchset.
>
> If the driver has to make a decision about which type of keys to support (due to
> the hardware and firmware supporting both but not at the same time), I think
> this will need to be done via a module parameter, e.g.
> qcom_ice.hw_wrapped_keys=1 to support HW-wrapped keys instead of standard keys.

No, the user can not set modparams on  e.g. Android device. In my
opinion it should be first-come-first-serve. If the user wants
hw-wrapped keys (and the platform is fine with that), then further
attempts to use raw keys should fail. If the user loads a raw key,
further attempts to set hw-wrapped key should fail (maybe until the
last raw key has been evicted from the hw, if such thing is actually
supported).

On Fri, Jun 21, 2024 at 10:24:07PM +0300, Dmitry Baryshkov wrote:
> >
> > (fscrypt used to use the keyring service a bit more: it looked up a key whenever
> > a file was opened, and it supported evicting per-file keys by revoking the
> > corresponding keyring key.  But this turned out to be totally broken.  E.g., it
> > didn't provide the correct semantics for filesystem encryption where the key
> > should either be present or absent filesystem-wide.)
> >
> > We do need the ability to create HW-wrapped keys in long-term wrapped form,
> > either via "generate" or "import", return those long-term wrapped keys to
> > userspace so that they can be stored on-disk, and convert them into
> > ephemerally-wrapped form so they can be used.  It probably would be possible to
> > support all of this through the keyrings service, but it would need a couple new
> > key types:
> >
> > - One key type that can be instantiated with a raw key (or NULL to request
> >   generation of a key) and that automagically creates a long-term wrapped key
> >   and supports userspace reading it back.  This would be vaguely similar to
> >   "trusted", but without any support for using the key directly.
> >
> > - One key type that can be instantiated using a long-term wrapped key which gets
> >   automagically converted to an ephemerally-wrapped key.  This would be what is
> >   passed to other kernel subsystems.  Functions specific to this key type would
> >   need to be provided for users to use.
> 
> I think having one key type should be enough. The userspace loads /
> generates&reads / wraps and reads back the 'exported' version wrapped
> using the platform-specific key. In kernel the key is unsealed and
> represented as binary key to be loaded to the hardware + a cookie for
> the ephemeral key and device that have been used to wrap it. When
> userspace asks the device to program the key, the cookie is verified
> to match the device / ephemeral key and then the binary is programmed
> to the hardware. Maybe it's enough to use the struct device as a
> cookie.

The long-term wrapped key has to be wiped from memory as soon as it's no longer
needed.  So it's hard to see how overloading a key type in this way can work, as
the kernel can't know if userspace intends to read back the long-term wrapped
key or not.

> 
> > I think it would be possible, but it feels like a bit of a shoehorned API.  The
> > ioctls are a more straightforward solution.
> 
> Are we going to have another set of IOCTLs for loading the encrypted
> keys? keys sealed by TPM?

Those features aren't compatible with hardware-wrapped inline encryption keys,
so they're not really relevant here.  BLKCRYPTOIMPORTKEY could support importing
a keyring service key as an alternative to a raw key, of course.  But this would
just work similarly to fscrypt and dm-crypt where they just extract the payload,
and the keyring service key plays no further role.

> > > > Support for it will be added at some point, which will likely indeed take the
> > > > form of an ioctl to set a key on a block device.  But that would be the case
> > > > even without HW-wrapped keys.  And *requiring* the key to be given in a keyring
> > > > (instead of just in a byte array passed to the ioctl) isn't very helpful, as it
> > > > just makes the API harder to use.  We've learned this from the fscrypt API
> > > > already where we actually had to move away from the keyrings service in order to
> > > > fix all the issues caused by it (see FS_IOC_ADD_ENCRYPTION_KEY).
> > > >
> > > > > >
> > > > > > > Second part is the actual block interface. Gaurav wrote about
> > > > > > > targeting fscrypt, but there should be no actual difference between
> > > > > > > crypto targets. FDE or having a single partition encrypted should
> > > > > > > probably work in the same way. Convert the key into blk_crypto_key
> > > > > > > (including the cookie for the ephemeral key), program the key into the
> > > > > > > slot, use the slot to en/decrypt hardware blocks.
> > > > > > >
> > > > > > > My main point is that the decision on the key type should be coming
> > > > > > > from the user.
> > > > > >
> > > > > > That's exactly how it works.  There is a block interface for specifying an
> > > > > > inline encryption key along with each bio.  The submitter of the bio can specify
> > > > > > either a standard key or a HW-wrapped key.
> > > > >
> > > > > Not in this patchset. The ICE driver decides whether it can support
> > > > > HW-wrapped keys or not and then fails to support other type of keys.
> > > > >
> > > >
> > > > Sure, that's just a matter of hardware capabilities though, right?  The block
> > > > layer provides a way for drivers to declare which inline encryption capabilities
> > > > they support.  They can declare they support standard keys, HW-wrapped keys,
> > > > both, or neither.  If Qualcomm SoCs can't support both types of keys at the same
> > > > time, that's unfortunate, but I'm not sure what your poitnt is.  The user (e.g.
> > > > fscrypt) still has control over whether they use the functionality that the
> > > > hardware provides.
> > >
> > > It's a matter of policy. Harware / firmware doesn't support using both
> > > kinds of keys concurrently, if I understood Gaurav's explanations
> > > correctly. But the user should be able to make a judgement and use
> > > non-hw-wrapped keys if it fits their requirements. The driver should
> > > not make this kind of judgement. Note, this is not an issue of your
> > > original patchset, but it's a driver flaw in this patchset.
> >
> > If the driver has to make a decision about which type of keys to support (due to
> > the hardware and firmware supporting both but not at the same time), I think
> > this will need to be done via a module parameter, e.g.
> > qcom_ice.hw_wrapped_keys=1 to support HW-wrapped keys instead of standard keys.
> 
> No, the user can not set modparams on  e.g. Android device. In my
> opinion it should be first-come-first-serve. If the user wants
> hw-wrapped keys (and the platform is fine with that), then further
> attempts to use raw keys should fail. If the user loads a raw key,
> further attempts to set hw-wrapped key should fail (maybe until the
> last raw key has been evicted from the hw, if such thing is actually
> supported).

That's not going to work.  Upper layers need to know what the crypto
capabilities are before they decide to use them.  We can't randomly revoke
capabilities based on who happened to get there first, as a user might have
already checked the capabilities.  Yes, the module parameter is a litle
annoying, but it seems to be necessary here.  It is not a problem for Android
because the type of encryption an Android device uses is set by the build
anyway, which makes it no easier to change than module parameters.

- Eric

On Fri, Jun 21, 2024 at 08:14:41PM GMT, Eric Biggers wrote:
> On Fri, Jun 21, 2024 at 10:24:07PM +0300, Dmitry Baryshkov wrote:
> > >
> > > (fscrypt used to use the keyring service a bit more: it looked up a key whenever
> > > a file was opened, and it supported evicting per-file keys by revoking the
> > > corresponding keyring key.  But this turned out to be totally broken.  E.g., it
> > > didn't provide the correct semantics for filesystem encryption where the key
> > > should either be present or absent filesystem-wide.)
> > >
> > > We do need the ability to create HW-wrapped keys in long-term wrapped form,
> > > either via "generate" or "import", return those long-term wrapped keys to
> > > userspace so that they can be stored on-disk, and convert them into
> > > ephemerally-wrapped form so they can be used.  It probably would be possible to
> > > support all of this through the keyrings service, but it would need a couple new
> > > key types:
> > >
> > > - One key type that can be instantiated with a raw key (or NULL to request
> > >   generation of a key) and that automagically creates a long-term wrapped key
> > >   and supports userspace reading it back.  This would be vaguely similar to
> > >   "trusted", but without any support for using the key directly.
> > >
> > > - One key type that can be instantiated using a long-term wrapped key which gets
> > >   automagically converted to an ephemerally-wrapped key.  This would be what is
> > >   passed to other kernel subsystems.  Functions specific to this key type would
> > >   need to be provided for users to use.
> > 
> > I think having one key type should be enough. The userspace loads /
> > generates&reads / wraps and reads back the 'exported' version wrapped
> > using the platform-specific key. In kernel the key is unsealed and
> > represented as binary key to be loaded to the hardware + a cookie for
> > the ephemeral key and device that have been used to wrap it. When
> > userspace asks the device to program the key, the cookie is verified
> > to match the device / ephemeral key and then the binary is programmed
> > to the hardware. Maybe it's enough to use the struct device as a
> > cookie.
> 
> The long-term wrapped key has to be wiped from memory as soon as it's no longer
> needed.  So it's hard to see how overloading a key type in this way can work, as
> the kernel can't know if userspace intends to read back the long-term wrapped
> key or not.

Why? It should be user's decision. Pretty much in the same way as it's
done for all other keys.

> > > I think it would be possible, but it feels like a bit of a shoehorned API.  The
> > > ioctls are a more straightforward solution.
> > 
> > Are we going to have another set of IOCTLs for loading the encrypted
> > keys? keys sealed by TPM?
> 
> Those features aren't compatible with hardware-wrapped inline encryption keys,
> so they're not really relevant here.  BLKCRYPTOIMPORTKEY could support importing
> a keyring service key as an alternative to a raw key, of course.  But this would
> just work similarly to fscrypt and dm-crypt where they just extract the payload,
> and the keyring service key plays no further role.

Yes, extracting the payload is fine. As you wrote, dm-crypt and fscrypt
already do it in this way. But what I really don't like here is the idea
of having two different kinds of API having pretty close functionality.
In my opinion, all the keys should be handled via the existing keyrings
API and then imported via the BLKCRYPTOIMPORTKEY IOCTL. This way all
kinds of keys are handled in a similar way from user's point of view.

> > > > > Support for it will be added at some point, which will likely indeed take the
> > > > > form of an ioctl to set a key on a block device.  But that would be the case
> > > > > even without HW-wrapped keys.  And *requiring* the key to be given in a keyring
> > > > > (instead of just in a byte array passed to the ioctl) isn't very helpful, as it
> > > > > just makes the API harder to use.  We've learned this from the fscrypt API
> > > > > already where we actually had to move away from the keyrings service in order to
> > > > > fix all the issues caused by it (see FS_IOC_ADD_ENCRYPTION_KEY).
> > > > >
> > > > > > >
> > > > > > > > Second part is the actual block interface. Gaurav wrote about
> > > > > > > > targeting fscrypt, but there should be no actual difference between
> > > > > > > > crypto targets. FDE or having a single partition encrypted should
> > > > > > > > probably work in the same way. Convert the key into blk_crypto_key
> > > > > > > > (including the cookie for the ephemeral key), program the key into the
> > > > > > > > slot, use the slot to en/decrypt hardware blocks.
> > > > > > > >
> > > > > > > > My main point is that the decision on the key type should be coming
> > > > > > > > from the user.
> > > > > > >
> > > > > > > That's exactly how it works.  There is a block interface for specifying an
> > > > > > > inline encryption key along with each bio.  The submitter of the bio can specify
> > > > > > > either a standard key or a HW-wrapped key.
> > > > > >
> > > > > > Not in this patchset. The ICE driver decides whether it can support
> > > > > > HW-wrapped keys or not and then fails to support other type of keys.
> > > > > >
> > > > >
> > > > > Sure, that's just a matter of hardware capabilities though, right?  The block
> > > > > layer provides a way for drivers to declare which inline encryption capabilities
> > > > > they support.  They can declare they support standard keys, HW-wrapped keys,
> > > > > both, or neither.  If Qualcomm SoCs can't support both types of keys at the same
> > > > > time, that's unfortunate, but I'm not sure what your poitnt is.  The user (e.g.
> > > > > fscrypt) still has control over whether they use the functionality that the
> > > > > hardware provides.
> > > >
> > > > It's a matter of policy. Harware / firmware doesn't support using both
> > > > kinds of keys concurrently, if I understood Gaurav's explanations
> > > > correctly. But the user should be able to make a judgement and use
> > > > non-hw-wrapped keys if it fits their requirements. The driver should
> > > > not make this kind of judgement. Note, this is not an issue of your
> > > > original patchset, but it's a driver flaw in this patchset.
> > >
> > > If the driver has to make a decision about which type of keys to support (due to
> > > the hardware and firmware supporting both but not at the same time), I think
> > > this will need to be done via a module parameter, e.g.
> > > qcom_ice.hw_wrapped_keys=1 to support HW-wrapped keys instead of standard keys.
> > 
> > No, the user can not set modparams on  e.g. Android device. In my
> > opinion it should be first-come-first-serve. If the user wants
> > hw-wrapped keys (and the platform is fine with that), then further
> > attempts to use raw keys should fail. If the user loads a raw key,
> > further attempts to set hw-wrapped key should fail (maybe until the
> > last raw key has been evicted from the hw, if such thing is actually
> > supported).
> 
> That's not going to work.  Upper layers need to know what the crypto
> capabilities are before they decide to use them.  We can't randomly revoke
> capabilities based on who happened to get there first, as a user might have
> already checked the capabilities.  Yes, the module parameter is a litle
> annoying, but it seems to be necessary here.

Hmm. This is typical to have resource-limited capabilities. So yes, the
user checks the capabilities to identify whether the key type is
supported at all. But then _using_ the key might fail. For example
because all the hardware resources that are used by this key type are
already taken.

> It is not a problem for Android
> because the type of encryption an Android device uses is set by the build
> anyway, which makes it no easier to change than module parameters.

If AOSP misbehaves, it doesn't mean that we should follow the pattern.

On Fri, Jun 21, 2024 at 11:52:01PM +0300, Dmitry Baryshkov wrote:
> On Fri, Jun 21, 2024 at 08:14:41PM GMT, Eric Biggers wrote:
> > On Fri, Jun 21, 2024 at 10:24:07PM +0300, Dmitry Baryshkov wrote:
> > > >
> > > > (fscrypt used to use the keyring service a bit more: it looked up a key whenever
> > > > a file was opened, and it supported evicting per-file keys by revoking the
> > > > corresponding keyring key.  But this turned out to be totally broken.  E.g., it
> > > > didn't provide the correct semantics for filesystem encryption where the key
> > > > should either be present or absent filesystem-wide.)
> > > >
> > > > We do need the ability to create HW-wrapped keys in long-term wrapped form,
> > > > either via "generate" or "import", return those long-term wrapped keys to
> > > > userspace so that they can be stored on-disk, and convert them into
> > > > ephemerally-wrapped form so they can be used.  It probably would be possible to
> > > > support all of this through the keyrings service, but it would need a couple new
> > > > key types:
> > > >
> > > > - One key type that can be instantiated with a raw key (or NULL to request
> > > >   generation of a key) and that automagically creates a long-term wrapped key
> > > >   and supports userspace reading it back.  This would be vaguely similar to
> > > >   "trusted", but without any support for using the key directly.
> > > >
> > > > - One key type that can be instantiated using a long-term wrapped key which gets
> > > >   automagically converted to an ephemerally-wrapped key.  This would be what is
> > > >   passed to other kernel subsystems.  Functions specific to this key type would
> > > >   need to be provided for users to use.
> > > 
> > > I think having one key type should be enough. The userspace loads /
> > > generates&reads / wraps and reads back the 'exported' version wrapped
> > > using the platform-specific key. In kernel the key is unsealed and
> > > represented as binary key to be loaded to the hardware + a cookie for
> > > the ephemeral key and device that have been used to wrap it. When
> > > userspace asks the device to program the key, the cookie is verified
> > > to match the device / ephemeral key and then the binary is programmed
> > > to the hardware. Maybe it's enough to use the struct device as a
> > > cookie.
> > 
> > The long-term wrapped key has to be wiped from memory as soon as it's no longer
> > needed.  So it's hard to see how overloading a key type in this way can work, as
> > the kernel can't know if userspace intends to read back the long-term wrapped
> > key or not.
> 
> Why? It should be user's decision. Pretty much in the same way as it's
> done for all other keys.

Sorry, I don't understand what your point is supposed to be here.  It's
certainly not okay to leave the long-term wrapped key in memory, since that
destroys the security properties of hardware-wrapped keys.  So we need to
provide an API that makes it possible for the long-term wrapped key to be
zeroized.  The API you're proposing, as I understand it, wouldn't allow for that
because the long-term wrapped key would remain in memory as long as the keyring
service key exists, even when only the ephemerally-wrapped key is needed.

> 
> > > > I think it would be possible, but it feels like a bit of a shoehorned API.  The
> > > > ioctls are a more straightforward solution.
> > > 
> > > Are we going to have another set of IOCTLs for loading the encrypted
> > > keys? keys sealed by TPM?
> > 
> > Those features aren't compatible with hardware-wrapped inline encryption keys,
> > so they're not really relevant here.  BLKCRYPTOIMPORTKEY could support importing
> > a keyring service key as an alternative to a raw key, of course.  But this would
> > just work similarly to fscrypt and dm-crypt where they just extract the payload,
> > and the keyring service key plays no further role.
> 
> Yes, extracting the payload is fine. As you wrote, dm-crypt and fscrypt
> already do it in this way. But what I really don't like here is the idea
> of having two different kinds of API having pretty close functionality.
> In my opinion, all the keys should be handled via the existing keyrings
> API and then imported via the BLKCRYPTOIMPORTKEY IOCTL. This way all
> kinds of keys are handled in a similar way from user's point of view.

But in that case all the proposed new BLKCRYPTO* ioctls are still needed.  Your
suggestion would just make them harder to use by requiring users to copy their
key into a keyrings service key instead of just providing it directly in the
ioctl.

> 
> > > > > > Support for it will be added at some point, which will likely indeed take the
> > > > > > form of an ioctl to set a key on a block device.  But that would be the case
> > > > > > even without HW-wrapped keys.  And *requiring* the key to be given in a keyring
> > > > > > (instead of just in a byte array passed to the ioctl) isn't very helpful, as it
> > > > > > just makes the API harder to use.  We've learned this from the fscrypt API
> > > > > > already where we actually had to move away from the keyrings service in order to
> > > > > > fix all the issues caused by it (see FS_IOC_ADD_ENCRYPTION_KEY).
> > > > > >
> > > > > > > >
> > > > > > > > > Second part is the actual block interface. Gaurav wrote about
> > > > > > > > > targeting fscrypt, but there should be no actual difference between
> > > > > > > > > crypto targets. FDE or having a single partition encrypted should
> > > > > > > > > probably work in the same way. Convert the key into blk_crypto_key
> > > > > > > > > (including the cookie for the ephemeral key), program the key into the
> > > > > > > > > slot, use the slot to en/decrypt hardware blocks.
> > > > > > > > >
> > > > > > > > > My main point is that the decision on the key type should be coming
> > > > > > > > > from the user.
> > > > > > > >
> > > > > > > > That's exactly how it works.  There is a block interface for specifying an
> > > > > > > > inline encryption key along with each bio.  The submitter of the bio can specify
> > > > > > > > either a standard key or a HW-wrapped key.
> > > > > > >
> > > > > > > Not in this patchset. The ICE driver decides whether it can support
> > > > > > > HW-wrapped keys or not and then fails to support other type of keys.
> > > > > > >
> > > > > >
> > > > > > Sure, that's just a matter of hardware capabilities though, right?  The block
> > > > > > layer provides a way for drivers to declare which inline encryption capabilities
> > > > > > they support.  They can declare they support standard keys, HW-wrapped keys,
> > > > > > both, or neither.  If Qualcomm SoCs can't support both types of keys at the same
> > > > > > time, that's unfortunate, but I'm not sure what your poitnt is.  The user (e.g.
> > > > > > fscrypt) still has control over whether they use the functionality that the
> > > > > > hardware provides.
> > > > >
> > > > > It's a matter of policy. Harware / firmware doesn't support using both
> > > > > kinds of keys concurrently, if I understood Gaurav's explanations
> > > > > correctly. But the user should be able to make a judgement and use
> > > > > non-hw-wrapped keys if it fits their requirements. The driver should
> > > > > not make this kind of judgement. Note, this is not an issue of your
> > > > > original patchset, but it's a driver flaw in this patchset.
> > > >
> > > > If the driver has to make a decision about which type of keys to support (due to
> > > > the hardware and firmware supporting both but not at the same time), I think
> > > > this will need to be done via a module parameter, e.g.
> > > > qcom_ice.hw_wrapped_keys=1 to support HW-wrapped keys instead of standard keys.
> > > 
> > > No, the user can not set modparams on  e.g. Android device. In my
> > > opinion it should be first-come-first-serve. If the user wants
> > > hw-wrapped keys (and the platform is fine with that), then further
> > > attempts to use raw keys should fail. If the user loads a raw key,
> > > further attempts to set hw-wrapped key should fail (maybe until the
> > > last raw key has been evicted from the hw, if such thing is actually
> > > supported).
> > 
> > That's not going to work.  Upper layers need to know what the crypto
> > capabilities are before they decide to use them.  We can't randomly revoke
> > capabilities based on who happened to get there first, as a user might have
> > already checked the capabilities.  Yes, the module parameter is a litle
> > annoying, but it seems to be necessary here.
> 
> Hmm. This is typical to have resource-limited capabilities. So yes, the
> user checks the capabilities to identify whether the key type is
> supported at all. But then _using_ the key might fail. For example
> because all the hardware resources that are used by this key type are
> already taken.

That mustn't happen here, since finding out in the middle of an I/O request that
inline encryption isn't supported is too late.  That's what the crypto
capabilities in struct blk_crypto_profile are for -- to allow users to check
what is supported before trying to use it.

> 
> > It is not a problem for Android
> > because the type of encryption an Android device uses is set by the build
> > anyway, which makes it no easier to change than module parameters.
> 
> If AOSP misbehaves, it doesn't mean that we should follow the pattern.

It's not "misbehaving" -- it's just an example of a system that configures the
encryption centrally, which is common.  (And the reason I brought up that the
module parameter works for Android is because you claimed it wouldn't.)

Again, needing a module parameter is unfortunate but I don't see any realistic
way around it for these Qualcomm SoCs.

- Eric

Hey Eric

On 06/21/2024, 9:02 AM PDT, Eric Biggers wrote:
> On Fri, Jun 21, 2024 at 03:35:40PM +0000, Gaurav Kashyap wrote:
> > Hello Eric
> >
> > On 06/20/2024, 9:48 PM PDT, Eric Biggers wrote:
> > > On Thu, Jun 20, 2024 at 02:57:40PM +0300, Dmitry Baryshkov wrote:
> > > > > > >
> > > > > > > > Is it possible to use both kind of keys when working on
> > > > > > > > standard
> > > mode?
> > > > > > > > If not, it should be the user who selects what type of
> > > > > > > > keys to be
> > > used.
> > > > > > > > Enforcing this via DT is not a way to go.
> > > > > > > >
> > > > > > >
> > > > > > > Unfortunately, that support is not there yet. When you say
> > > > > > > user, do you mean to have it as a filesystem mount option?
> > > > > >
> > > > > > During cryptsetup time. When running e.g. cryptsetup I, as a
> > > > > > user, would like to be able to use either a hardware-wrapped
> > > > > > key or a
> > > standard key.
> > > > > >
> > > > >
> > > > > What we are looking for with these patches is for
> > > > > per-file/folder
> > > encryption using fscrypt policies.
> > > > > Cryptsetup to my understanding supports only full-disk , and
> > > > > does not support FBE (File-Based)
> > > >
> > > > I must admit, I mostly used dm-crypt beforehand, so I had to look
> > > > at fscrypt now. Some of my previous comments might not be fully
> > > > applicable.
> > > >
> > > > > Hence the idea here is that we mount an unencrypted device (with
> > > > > the inlinecrypt option that indicates inline encryption is
> > > > > supported) And
> > > specify policies (links to keys) for different folders.
> > > > >
> > > > > > > The way the UFS/EMMC crypto layer is designed currently is
> > > > > > > that, this information is needed when the modules are loaded.
> > > > > > >
> > > > > > > https://lore.kernel.org/all/20231104211259.17448-2-ebiggers@
> > > > > > > kern el.org /#Z31drivers:ufs:core:ufshcd-crypto.c
> > > > > >
> > > > > > I see that the driver lists capabilities here. E.g. that it
> > > > > > supports HW-wrapped keys. But the line doesn't specify that
> > > > > > standard
> > > keys are not supported.
> > > > > >
> > > > >
> > > > > Those are capabilities that are read from the storage controller.
> > > > > However, wrapped keys Are not a standard in the ICE JEDEC
> > > > > specification, and in most cases, is a value add coming from the SoC.
> > > > >
> > > > > QCOM SOC and firmware currently does not support both kinds of
> > > > > keys in
> > > the HWKM mode.
> > > > > That is something we are internally working on, but not available yet.
> > > >
> > > > I'd say this is a significant obstacle, at least from my point of
> > > > view. I understand that the default might be to use hw-wrapped
> > > > keys, but it should be possible for the user to select non-HW keys
> > > > if the ability to recover the data is considered to be important.
> > > > Note, I'm really pointing to the user here, not to the system
> > > > integrator. So using DT property or specifying kernel arguments to
> > > > switch between these modes is not really an option.
> > > >
> > > > But I'd really love to hear some feedback from linux-security
> > > > and/or linux-fscrypt here.
> > > >
> > > > In my humble opinion the user should be able to specify that the
> > > > key is wrapped using the hardware KMK. Then if the hardware has
> > > > already started using the other kind of keys, it should be able to
> > > > respond with -EINVAL / whatever else. Then the user can evict
> > > > previously programmed key and program a desired one.
> > > >
> > > > > > Also, I'd have expected that hw-wrapped keys are handled using
> > > > > > trusted keys mechanism (see security/keys/trusted-keys/).
> > > > > > Could you please point out why that's not the case?
> > > > > >
> > > > >
> > > > > I will evaluate this.
> > > > > But my initial response is that we currently cannot communicate
> > > > > to our TPM directly from HLOS, but goes through QTEE, and I
> > > > > don't think our qtee currently interfaces with the open source
> > > > > tee driver. The
> > > interface is through QCOM SCM driver.
> > > >
> > > > Note, this is just an API interface, see how it is implemented for
> > > > the CAAM hardware.
> > > >
> > >
> > > The problem is that this patchset was sent out without the patches
> > > that add the block and filesystem-level framework for
> > > hardware-wrapped inline encryption keys, which it depends on.  So
> > > it's lacking context.  The proposed framework can be found at
> > > https://lore.kernel.org/linux-
> > > block/20231104211259.17448-1-ebiggers@kernel.org/T/#u
> > >
> >
> > I have only been adding the fscryp patch link as part of the cover letter - as
> a dependency.
> > https://lore.kernel.org/all/20240617005825.1443206-1-quic_gaurkash@qui
> > cinc.com/ If you would like me to include it in the patch series
> > itself, I can do that as well.
> >
> 
> I think including all prerequisite patches would be helpful for reviewers.

Noted. I'll do that for the next patch.

> 
> Thanks for continuing to work on this!
> 
> I still need to get ahold of a sm8650 based device and test this out.  Is the
> SM8650 HDK the only option, or is there a sm8650 based phone with
> upstream support yet?

There are some devices released with SM8650 (Snapdragon 8 Gen 3). Sorry, I have
not kept track of which. I know the S24s were released with that. But there should be
more in the market.

> 
> - Eric

On 25/06/2024 06:58, Gaurav Kashyap (QUIC) wrote:
> 
> Hey Eric
> 
> On 06/21/2024, 9:02 AM PDT, Eric Biggers wrote:
>> On Fri, Jun 21, 2024 at 03:35:40PM +0000, Gaurav Kashyap wrote:
>>> Hello Eric
>>>
>>> On 06/20/2024, 9:48 PM PDT, Eric Biggers wrote:
>>>> On Thu, Jun 20, 2024 at 02:57:40PM +0300, Dmitry Baryshkov wrote:
>>>>>>>>
>>>>>>>>> Is it possible to use both kind of keys when working on
>>>>>>>>> standard
>>>> mode?
>>>>>>>>> If not, it should be the user who selects what type of
>>>>>>>>> keys to be
>>>> used.
>>>>>>>>> Enforcing this via DT is not a way to go.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Unfortunately, that support is not there yet. When you say
>>>>>>>> user, do you mean to have it as a filesystem mount option?
>>>>>>>
>>>>>>> During cryptsetup time. When running e.g. cryptsetup I, as a
>>>>>>> user, would like to be able to use either a hardware-wrapped
>>>>>>> key or a
>>>> standard key.
>>>>>>>
>>>>>>
>>>>>> What we are looking for with these patches is for
>>>>>> per-file/folder
>>>> encryption using fscrypt policies.
>>>>>> Cryptsetup to my understanding supports only full-disk , and
>>>>>> does not support FBE (File-Based)
>>>>>
>>>>> I must admit, I mostly used dm-crypt beforehand, so I had to look
>>>>> at fscrypt now. Some of my previous comments might not be fully
>>>>> applicable.
>>>>>
>>>>>> Hence the idea here is that we mount an unencrypted device (with
>>>>>> the inlinecrypt option that indicates inline encryption is
>>>>>> supported) And
>>>> specify policies (links to keys) for different folders.
>>>>>>
>>>>>>>> The way the UFS/EMMC crypto layer is designed currently is
>>>>>>>> that, this information is needed when the modules are loaded.
>>>>>>>>
>>>>>>>> https://lore.kernel.org/all/20231104211259.17448-2-ebiggers@
>>>>>>>> kern el.org /#Z31drivers:ufs:core:ufshcd-crypto.c
>>>>>>>
>>>>>>> I see that the driver lists capabilities here. E.g. that it
>>>>>>> supports HW-wrapped keys. But the line doesn't specify that
>>>>>>> standard
>>>> keys are not supported.
>>>>>>>
>>>>>>
>>>>>> Those are capabilities that are read from the storage controller.
>>>>>> However, wrapped keys Are not a standard in the ICE JEDEC
>>>>>> specification, and in most cases, is a value add coming from the SoC.
>>>>>>
>>>>>> QCOM SOC and firmware currently does not support both kinds of
>>>>>> keys in
>>>> the HWKM mode.
>>>>>> That is something we are internally working on, but not available yet.
>>>>>
>>>>> I'd say this is a significant obstacle, at least from my point of
>>>>> view. I understand that the default might be to use hw-wrapped
>>>>> keys, but it should be possible for the user to select non-HW keys
>>>>> if the ability to recover the data is considered to be important.
>>>>> Note, I'm really pointing to the user here, not to the system
>>>>> integrator. So using DT property or specifying kernel arguments to
>>>>> switch between these modes is not really an option.
>>>>>
>>>>> But I'd really love to hear some feedback from linux-security
>>>>> and/or linux-fscrypt here.
>>>>>
>>>>> In my humble opinion the user should be able to specify that the
>>>>> key is wrapped using the hardware KMK. Then if the hardware has
>>>>> already started using the other kind of keys, it should be able to
>>>>> respond with -EINVAL / whatever else. Then the user can evict
>>>>> previously programmed key and program a desired one.
>>>>>
>>>>>>> Also, I'd have expected that hw-wrapped keys are handled using
>>>>>>> trusted keys mechanism (see security/keys/trusted-keys/).
>>>>>>> Could you please point out why that's not the case?
>>>>>>>
>>>>>>
>>>>>> I will evaluate this.
>>>>>> But my initial response is that we currently cannot communicate
>>>>>> to our TPM directly from HLOS, but goes through QTEE, and I
>>>>>> don't think our qtee currently interfaces with the open source
>>>>>> tee driver. The
>>>> interface is through QCOM SCM driver.
>>>>>
>>>>> Note, this is just an API interface, see how it is implemented for
>>>>> the CAAM hardware.
>>>>>
>>>>
>>>> The problem is that this patchset was sent out without the patches
>>>> that add the block and filesystem-level framework for
>>>> hardware-wrapped inline encryption keys, which it depends on.  So
>>>> it's lacking context.  The proposed framework can be found at
>>>> https://lore.kernel.org/linux-
>>>> block/20231104211259.17448-1-ebiggers@kernel.org/T/#u
>>>>
>>>
>>> I have only been adding the fscryp patch link as part of the cover letter - as
>> a dependency.
>>> https://lore.kernel.org/all/20240617005825.1443206-1-quic_gaurkash@qui
>>> cinc.com/ If you would like me to include it in the patch series
>>> itself, I can do that as well.
>>>
>>
>> I think including all prerequisite patches would be helpful for reviewers.
> 
> Noted. I'll do that for the next patch.
> 
>>
>> Thanks for continuing to work on this!
>>
>> I still need to get ahold of a sm8650 based device and test this out.  Is the
>> SM8650 HDK the only option, or is there a sm8650 based phone with
>> upstream support yet?

Yes you should be able to buy the SM8650 HDK from lantronix:
https://www.lantronix.com/products/snapdragon-8-gen-3-mobile-hardware-development-kit/

It should be supported in v6.11

Neil

> 
> There are some devices released with SM8650 (Snapdragon 8 Gen 3). Sorry, I have
> not kept track of which. I know the S24s were released with that. But there should be
> more in the market.
> 
>>
>> - Eric