| Message ID | 20220915002235.v2.1.I7c0a79e9b3c52584f5b637fde5f1d6f807605806@changeid |
|---|---|
| State | New |
| Headers | show |
| Series | [v2,1/2] random: move add_hwgenerator_randomness()'s wait outside function | expand |
Hi Sven, On Thu, Sep 15, 2022 at 12:22:53AM +0000, Sven van Ashbrook wrote: > add_hwgenerator_randomness() currently blocks until more entropy > is needed. Move the blocking wait out of the function to the caller, > by letting the function return the number of jiffies needed to block. > > This is done to prepare the function's sole kernel caller from a > kthread to self-rearming delayed_work. Isn't Dominik working on the same thing, but slightly different? I recall he sent a patch recently, which looked pretty good, except it just needed to be split up. I'm waiting for his v2. Does this build on that? Jason
Am Thu, Sep 15, 2022 at 02:54:24PM -0400 schrieb Sven van Ashbrook: > Dominik, Jason, > > On Thu, Sep 15, 2022 at 11:44 AM Jason A. Donenfeld <Jason@zx2c4.com> wrote: > > > > Isn't Dominik working on the same thing, but slightly different? > > I don't believe Dominik is working on quite the same thing, but his > work will conflict with mine for sure: > https://lore.kernel.org/lkml/YxiXEJ6up6XEW8SM@zx2c4.com/T/ > > What are the odds that two people are making changes to the hwrng > kthread at the same time? > > I see two possible ways forward: > 1. Dominik rebases his patch against mine (iff mine finds favour). > This may simplify his patch > quite a bit, because the delayed_work abstraction tends to have fewer > footguns than > kthread. > > or > > 2. I rebase against Dominik's. > > Both are fine with me, just let me know what you think. Indeed, our patches address different issues. I'm fine with both approaches, i.e. my patches to be based on Sven's, or the other way round. Best, Dominik
Hi Dominik, On Fri, Sep 16, 2022 at 2:24 AM Dominik Brodowski <linux@dominikbrodowski.net> wrote: > > Indeed, our patches address different issues. I'm fine with both approaches, > i.e. my patches to be based on Sven's, or the other way round. Sounds good! May I suggest then that you try to rebase your patch on top of mine. With a bit of luck, it will become simpler, with fewer kthread related footguns. I am available to help review. If your patch becomes more complicated however, we'll work the other way around. How does that sound? Sven
On Fri, Sep 16, 2022 at 10:51 AM Jason A. Donenfeld <Jason@zx2c4.com> wrote: > > The other thing that occurred to me when reading this patch in context > of the other one is that this sleep you're removing here is not the > only sleep in the call chain. Each hwrng driver can also sleep, and > many do, sometimes for a long time, blocking until there's data > available, which might happen after minutes in some cases. So maybe > that's something to think about in context of this patchset -- that > just moving this to a delayed worker might not actually fix the issue > you're having with sleeps. > This is an excellent point. A look at tpm2_calc_ordinal_duration() reveals that tpm_transmit() may block for 300s at a time. So when we are using a WQ_FREEZABLE delayed_work, the PM may have to wait for up to 300s when draining the wq on suspend. That will introduce a lot of breakage in suspend/resume. Dominik: in light of this, please proceed with your patch, without rebasing it onto mine. + tpm maintainers Peter Huewe and Jarkko Sakkinen, a quick recap of the problem: - on ChromeOS we are seeing intermittent suspend/resume errors+warnings related to activity of the core's hwrng_fillfn. this kthread keeps runningduring suspend/resume. if this happens to kick off an bus (i2c) transaction while the bus driver is in suspend, this triggers a "Transfer while suspended" warning from the i2c core, followed by an error return: i2c_designware i2c_designware.1: Transfer while suspended tpm tpm0: i2c transfer failed (attempt 1/3): -108 [ snip 10s of transfer failed attempts] - in 2019, Stephen Boyd made an attempt at fixing this by making the hwrng_fillfn kthread freezable. But a freezable thread requires different API calls for scheduling, waiting, and timeout. This generated regressions, so the solution had to be reverted. https://patchwork.kernel.org/project/linux-crypto/patch/20190805233241.220521-1-swboyd@chromium.org/ - the current patch attempts to halt hwrng_fillfn during suspend by converting it to a self-rearming delayed_work. The PM drains all work before going into suspend. But, the potential minute-long blocking delays in tpm make this solution infeasible. Peter and Jarkko, can you think of a possible way forward to eliminate the warnings+errors? -Sven
On Mon, Sep 19, 2022 at 5:03 PM Sven van Ashbrook <svenva@chromium.org> wrote: > > On Fri, Sep 16, 2022 at 10:51 AM Jason A. Donenfeld <Jason@zx2c4.com> wrote: > > > > The other thing that occurred to me when reading this patch in context > > of the other one is that this sleep you're removing here is not the > > only sleep in the call chain. Each hwrng driver can also sleep, and > > many do, sometimes for a long time, blocking until there's data > > available, which might happen after minutes in some cases. So maybe > > that's something to think about in context of this patchset -- that > > just moving this to a delayed worker might not actually fix the issue > > you're having with sleeps. > > > > This is an excellent point. A look at tpm2_calc_ordinal_duration() > reveals that tpm_transmit() may block for 300s at a time. So when > we are using a WQ_FREEZABLE delayed_work, the PM may have to wait > for up to 300s when draining the wq on suspend. That will introduce > a lot of breakage in suspend/resume. > > Dominik: in light of this, please proceed with your patch, without > rebasing it onto mine. > > + tpm maintainers Peter Huewe and Jarkko Sakkinen, a quick recap of > the problem: > > - on ChromeOS we are seeing intermittent suspend/resume errors+warnings > related to activity of the core's hwrng_fillfn. this kthread keeps > runningduring suspend/resume. if this happens to kick off an bus (i2c) > transaction while the bus driver is in suspend, this triggers > a "Transfer while suspended" warning from the i2c core, followed by > an error return: > > i2c_designware i2c_designware.1: Transfer while suspended > tpm tpm0: i2c transfer failed (attempt 1/3): -108 > [ snip 10s of transfer failed attempts] > > - in 2019, Stephen Boyd made an attempt at fixing this by making the > hwrng_fillfn kthread freezable. But a freezable thread requires > different API calls for scheduling, waiting, and timeout. This > generated regressions, so the solution had to be reverted. > > https://patchwork.kernel.org/project/linux-crypto/patch/20190805233241.220521-1-swboyd@chromium.org/ > > - the current patch attempts to halt hwrng_fillfn during suspend by > converting it to a self-rearming delayed_work. The PM drains all > work before going into suspend. But, the potential minute-long > blocking delays in tpm make this solution infeasible. > > Peter and Jarkko, can you think of a possible way forward to eliminate > the warnings+errors? > > -Sven By the way, there was a recent ath9k patch that kind of went to a similar tune. The solution was to make ath9k's hwrng driver sleep using a hwrng-specific sleep function that allowed the core framework to cancel that sleep. Maybe that's a potential solution here, or something similar to it. Might be worth taking a look at that patch. I think it's in one of Herbert's trees. Jason
On Mon, Sep 19, 2022 at 11:06 AM Jason A. Donenfeld <Jason@zx2c4.com> wrote: > > By the way, there was a recent ath9k patch that kind of went to a > similar tune. [...] Maybe that's a potential solution here, or > something similar to it. Jason was kind enough to point me to the patch in question: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?id=36cb6494429bd64b27b7ff8b4af56f8e526da2b4 This patch limits the long sleep inside the fillfn kthread, by terminating the sleep on hwrng_unregister(). This doesn't appear like a viable approach for the suspend/resume issue? - there is a great multitude of tpm_msleep()/msleep() calls in the tpm's rng_get_data() path. They would all have to be made interruptible. - even if interrupted successfully, now the kthread must be blocked until after resume. If so, what is the point of using a non-freezable kthread.
diff --git a/drivers/char/hw_random/core.c b/drivers/char/hw_random/core.c index 16f227b995e8..3675122c6cce 100644 --- a/drivers/char/hw_random/core.c +++ b/drivers/char/hw_random/core.c @@ -491,6 +491,7 @@ static int __init register_miscdev(void) static int hwrng_fillfn(void *unused) { size_t entropy, entropy_credit = 0; /* in 1/1024 of a bit */ + unsigned long delay; long rc; while (!kthread_should_stop()) { @@ -526,8 +527,10 @@ static int hwrng_fillfn(void *unused) entropy_credit = entropy; /* Outside lock, sure, but y'know: randomness. */ - add_hwgenerator_randomness((void *)rng_fillbuf, rc, - entropy >> 10); + delay = add_hwgenerator_randomness((void *)rng_fillbuf, rc, + entropy >> 10); + if (delay > 0) + schedule_timeout_interruptible(delay); } hwrng_fill = NULL; return 0; diff --git a/drivers/char/random.c b/drivers/char/random.c index 79d7d4e4e582..5dc949298f92 100644 --- a/drivers/char/random.c +++ b/drivers/char/random.c @@ -686,7 +686,7 @@ static void __cold _credit_init_bits(size_t bits) * the above entropy accumulation routines: * * void add_device_randomness(const void *buf, size_t len); - * void add_hwgenerator_randomness(const void *buf, size_t len, size_t entropy); + * unsigned long add_hwgenerator_randomness(const void *buf, size_t len, size_t entropy); * void add_bootloader_randomness(const void *buf, size_t len); * void add_vmfork_randomness(const void *unique_vm_id, size_t len); * void add_interrupt_randomness(int irq); @@ -702,8 +702,8 @@ static void __cold _credit_init_bits(size_t bits) * available to them (particularly common in the embedded world). * * add_hwgenerator_randomness() is for true hardware RNGs, and will credit - * entropy as specified by the caller. If the entropy pool is full it will - * block until more entropy is needed. + * entropy as specified by the caller. Returns time delay in jiffies until + * more entropy is needed. * * add_bootloader_randomness() is called by bootloader drivers, such as EFI * and device tree, and credits its input depending on whether or not the @@ -857,10 +857,10 @@ EXPORT_SYMBOL(add_device_randomness); /* * Interface for in-kernel drivers of true hardware RNGs. - * Those devices may produce endless random bits and will be throttled + * Those devices may produce endless random bits and should be throttled * when our pool is full. */ -void add_hwgenerator_randomness(const void *buf, size_t len, size_t entropy) +unsigned long add_hwgenerator_randomness(const void *buf, size_t len, size_t entropy) { mix_pool_bytes(buf, len); credit_init_bits(entropy); @@ -869,8 +869,7 @@ void add_hwgenerator_randomness(const void *buf, size_t len, size_t entropy) * Throttle writing to once every CRNG_RESEED_INTERVAL, unless * we're not yet initialized. */ - if (!kthread_should_stop() && crng_ready()) - schedule_timeout_interruptible(CRNG_RESEED_INTERVAL); + return crng_ready() ? CRNG_RESEED_INTERVAL : 0; } EXPORT_SYMBOL_GPL(add_hwgenerator_randomness); diff --git a/include/linux/random.h b/include/linux/random.h index 3fec206487f6..6608b0fb4402 100644 --- a/include/linux/random.h +++ b/include/linux/random.h @@ -17,7 +17,7 @@ void __init add_bootloader_randomness(const void *buf, size_t len); void add_input_randomness(unsigned int type, unsigned int code, unsigned int value) __latent_entropy; void add_interrupt_randomness(int irq) __latent_entropy; -void add_hwgenerator_randomness(const void *buf, size_t len, size_t entropy); +unsigned long add_hwgenerator_randomness(const void *buf, size_t len, size_t entropy); #if defined(LATENT_ENTROPY_PLUGIN) && !defined(__CHECKER__) static inline void add_latent_entropy(void)
add_hwgenerator_randomness() currently blocks until more entropy is needed. Move the blocking wait out of the function to the caller, by letting the function return the number of jiffies needed to block. This is done to prepare the function's sole kernel caller from a kthread to self-rearming delayed_work. Signed-off-by: Sven van Ashbrook <svenva@chromium.org> --- Changes in v2: - justify patch as a preparation for next patch drivers/char/hw_random/core.c | 7 +++++-- drivers/char/random.c | 13 ++++++------- include/linux/random.h | 2 +- 3 files changed, 12 insertions(+), 10 deletions(-)