usb-storage: Add quirk to defeat Kindle's automatic unload

Message ID 20210317190654.GA497856@rowland.harvard.edu
State New
Headers show
Series
  • usb-storage: Add quirk to defeat Kindle's automatic unload
Related show

Commit Message

Alan Stern March 17, 2021, 7:06 p.m.
Matthias reports that the Amazon Kindle automatically removes its
emulated media if it doesn't receive another SCSI command within about
one second after a SYNCHRONIZE CACHE.  It does so even when the host
has sent a PREVENT MEDIUM REMOVAL command.  The reason for this
behavior isn't clear, although it's not hard to make some guesses.

At any rate, the results can be unexpected for anyone who tries to
access the Kindle in an unusual fashion, and in theory they can lead
to data loss (for example, if one file is closed and synchronized
while other files are still in the middle of being written).

To avoid such problems, this patch creates a new usb-storage quirks
flag telling the driver always to issue a REQUEST SENSE following a
SYNCHRONIZE CACHE command, and adds an unusual_devs entry for the
Kindle with the flag set.  This is sufficient to prevent the Kindle
from doing its automatic unload, without interfering with proper
operation.

Another possible way to deal with this would be to increase the
frequency of TEST UNIT READY polling that the kernel normally carries
out for removable-media storage devices.  However that would increase
the overall load on the system and it is not as reliable, because the
user can override the polling interval.  Changing the driver's
behavior is safer and has minimal overhead.

Reported-and-tested-by: Matthias Schwarzott <zzam@gentoo.org>
Signed-off-by: Alan Stern <stern@rowland.harvard.edu>
CC: <stable@vger.kernel.org>

---


[as1953]


 drivers/usb/storage/transport.c    |    7 +++++++
 drivers/usb/storage/unusual_devs.h |   12 ++++++++++++
 include/linux/usb_usual.h          |    2 ++
 3 files changed, 21 insertions(+)

Comments

Ulrich Windl March 18, 2021, 7:10 a.m. | #1
>>> Alan Stern <stern@rowland.harvard.edu> schrieb am 17.03.2021 um 20:06 in
Nachricht <20210317190654.GA497856@rowland.harvard.edu>:
> Matthias reports that the Amazon Kindle automatically removes its
> emulated media if it doesn't receive another SCSI command within about
> one second after a SYNCHRONIZE CACHE.  It does so even when the host
> has sent a PREVENT MEDIUM REMOVAL command.  The reason for this
> behavior isn't clear, although it's not hard to make some guesses.

Actually I think Amazon should fix the firmware.
It seems the main goal was to prevent the use of open software to manage the
content.

> 
> At any rate, the results can be unexpected for anyone who tries to
> access the Kindle in an unusual fashion, and in theory they can lead
> to data loss (for example, if one file is closed and synchronized
> while other files are still in the middle of being written).
> 
> To avoid such problems, this patch creates a new usb‑storage quirks
> flag telling the driver always to issue a REQUEST SENSE following a
> SYNCHRONIZE CACHE command, and adds an unusual_devs entry for the
> Kindle with the flag set.  This is sufficient to prevent the Kindle
> from doing its automatic unload, without interfering with proper
> operation.
> 
> Another possible way to deal with this would be to increase the
> frequency of TEST UNIT READY polling that the kernel normally carries
> out for removable‑media storage devices.  However that would increase
> the overall load on the system and it is not as reliable, because the
> user can override the polling interval.  Changing the driver's
> behavior is safer and has minimal overhead.
> 
> Reported‑and‑tested‑by: Matthias Schwarzott <zzam@gentoo.org>
> Signed‑off‑by: Alan Stern <stern@rowland.harvard.edu>
> CC: <stable@vger.kernel.org>
> 
> ‑‑‑
> 
> 
> [as1953]
> 
> 
>  drivers/usb/storage/transport.c    |    7 +++++++
>  drivers/usb/storage/unusual_devs.h |   12 ++++++++++++
>  include/linux/usb_usual.h          |    2 ++
>  3 files changed, 21 insertions(+)
> 
> Index: usb‑devel/drivers/usb/storage/transport.c
> ===================================================================
> ‑‑‑ usb‑devel.orig/drivers/usb/storage/transport.c
> +++ usb‑devel/drivers/usb/storage/transport.c
> @@ ‑656,6 +656,13 @@ void usb_stor_invoke_transport(struct sc
>  		need_auto_sense = 1;
>  	}
>  
> +	/* Some devices (Kindle) require another command after SYNC CACHE */
> +	if ((us‑>fflags & US_FL_SENSE_AFTER_SYNC) &&
> +			srb‑>cmnd[0] == SYNCHRONIZE_CACHE) {
> +		usb_stor_dbg(us, "‑‑ sense after SYNC CACHE\n");
> +		need_auto_sense = 1;
> +	}
> +
>  	/*
>  	 * If we have a failure, we're going to do a REQUEST_SENSE 
>  	 * automatically.  Note that we differentiate between a command
> Index: usb‑devel/drivers/usb/storage/unusual_devs.h
> ===================================================================
> ‑‑‑ usb‑devel.orig/drivers/usb/storage/unusual_devs.h
> +++ usb‑devel/drivers/usb/storage/unusual_devs.h
> @@ ‑2212,6 +2212,18 @@ UNUSUAL_DEV( 0x1908, 0x3335, 0x0200, 0x0
>  		US_FL_NO_READ_DISC_INFO ),
>  
>  /*
> + * Reported by Matthias Schwarzott <zzam@gentoo.org>
> + * The Amazon Kindle treats SYNCHRONIZE CACHE as an indication that
> + * the host may be finished with it, and automatically ejects its
> + * emulated media unless it receives another command within one second.
> + */
> +UNUSUAL_DEV( 0x1949, 0x0004, 0x0000, 0x9999,
> +		"Amazon",
> +		"Kindle",
> +		USB_SC_DEVICE, USB_PR_DEVICE, NULL,
> +		US_FL_SENSE_AFTER_SYNC ),
> +
> +/*
>   * Reported by Oliver Neukum <oneukum@suse.com>
>   * This device morphes spontaneously into another device if the access
>   * pattern of Windows isn't followed. Thus writable media would be dirty
> Index: usb‑devel/include/linux/usb_usual.h
> ===================================================================
> ‑‑‑ usb‑devel.orig/include/linux/usb_usual.h
> +++ usb‑devel/include/linux/usb_usual.h
> @@ ‑86,6 +86,8 @@
>  		/* lies about caching, so always sync */	\
>  	US_FLAG(NO_SAME, 0x40000000)				\
>  		/* Cannot handle WRITE_SAME */			\
> +	US_FLAG(SENSE_AFTER_SYNC, 0x80000000)			\
> +		/* Do REQUEST_SENSE after SYNCHRONIZE_CACHE */	\
>  
>  #define US_FLAG(name, value)	US_FL_##name = value ,
>  enum { US_DO_ALL_FLAGS };
> _______________________________________________
> systemd‑devel mailing list
> systemd‑devel@lists.freedesktop.org 
> https://lists.freedesktop.org/mailman/listinfo/systemd‑devel
Hans de Goede March 18, 2021, 11:39 a.m. | #2
Hi,

On 3/17/21 8:06 PM, Alan Stern wrote:
> Matthias reports that the Amazon Kindle automatically removes its

> emulated media if it doesn't receive another SCSI command within about

> one second after a SYNCHRONIZE CACHE.  It does so even when the host

> has sent a PREVENT MEDIUM REMOVAL command.  The reason for this

> behavior isn't clear, although it's not hard to make some guesses.

> 

> At any rate, the results can be unexpected for anyone who tries to

> access the Kindle in an unusual fashion, and in theory they can lead

> to data loss (for example, if one file is closed and synchronized

> while other files are still in the middle of being written).

> 

> To avoid such problems, this patch creates a new usb-storage quirks

> flag telling the driver always to issue a REQUEST SENSE following a

> SYNCHRONIZE CACHE command, and adds an unusual_devs entry for the

> Kindle with the flag set.  This is sufficient to prevent the Kindle

> from doing its automatic unload, without interfering with proper

> operation.

> 

> Another possible way to deal with this would be to increase the

> frequency of TEST UNIT READY polling that the kernel normally carries

> out for removable-media storage devices.  However that would increase

> the overall load on the system and it is not as reliable, because the

> user can override the polling interval.  Changing the driver's

> behavior is safer and has minimal overhead.

> 

> Reported-and-tested-by: Matthias Schwarzott <zzam@gentoo.org>

> Signed-off-by: Alan Stern <stern@rowland.harvard.edu>

> CC: <stable@vger.kernel.org>


Thanks, patch looks good to me:

Reviewed-by: Hans de Goede <hdegoede@redhat.com>


Regards,

Hans


> 

> ---

> 

> 

> [as1953]

> 

> 

>  drivers/usb/storage/transport.c    |    7 +++++++

>  drivers/usb/storage/unusual_devs.h |   12 ++++++++++++

>  include/linux/usb_usual.h          |    2 ++

>  3 files changed, 21 insertions(+)

> 

> Index: usb-devel/drivers/usb/storage/transport.c

> ===================================================================

> --- usb-devel.orig/drivers/usb/storage/transport.c

> +++ usb-devel/drivers/usb/storage/transport.c

> @@ -656,6 +656,13 @@ void usb_stor_invoke_transport(struct sc

>  		need_auto_sense = 1;

>  	}

>  

> +	/* Some devices (Kindle) require another command after SYNC CACHE */

> +	if ((us->fflags & US_FL_SENSE_AFTER_SYNC) &&

> +			srb->cmnd[0] == SYNCHRONIZE_CACHE) {

> +		usb_stor_dbg(us, "-- sense after SYNC CACHE\n");

> +		need_auto_sense = 1;

> +	}

> +

>  	/*

>  	 * If we have a failure, we're going to do a REQUEST_SENSE 

>  	 * automatically.  Note that we differentiate between a command

> Index: usb-devel/drivers/usb/storage/unusual_devs.h

> ===================================================================

> --- usb-devel.orig/drivers/usb/storage/unusual_devs.h

> +++ usb-devel/drivers/usb/storage/unusual_devs.h

> @@ -2212,6 +2212,18 @@ UNUSUAL_DEV( 0x1908, 0x3335, 0x0200, 0x0

>  		US_FL_NO_READ_DISC_INFO ),

>  

>  /*

> + * Reported by Matthias Schwarzott <zzam@gentoo.org>

> + * The Amazon Kindle treats SYNCHRONIZE CACHE as an indication that

> + * the host may be finished with it, and automatically ejects its

> + * emulated media unless it receives another command within one second.

> + */

> +UNUSUAL_DEV( 0x1949, 0x0004, 0x0000, 0x9999,

> +		"Amazon",

> +		"Kindle",

> +		USB_SC_DEVICE, USB_PR_DEVICE, NULL,

> +		US_FL_SENSE_AFTER_SYNC ),

> +

> +/*

>   * Reported by Oliver Neukum <oneukum@suse.com>

>   * This device morphes spontaneously into another device if the access

>   * pattern of Windows isn't followed. Thus writable media would be dirty

> Index: usb-devel/include/linux/usb_usual.h

> ===================================================================

> --- usb-devel.orig/include/linux/usb_usual.h

> +++ usb-devel/include/linux/usb_usual.h

> @@ -86,6 +86,8 @@

>  		/* lies about caching, so always sync */	\

>  	US_FLAG(NO_SAME, 0x40000000)				\

>  		/* Cannot handle WRITE_SAME */			\

> +	US_FLAG(SENSE_AFTER_SYNC, 0x80000000)			\

> +		/* Do REQUEST_SENSE after SYNCHRONIZE_CACHE */	\

>  

>  #define US_FLAG(name, value)	US_FL_##name = value ,

>  enum { US_DO_ALL_FLAGS };

>
Tomasz Torcz March 18, 2021, 1:50 p.m. | #3
Dnia Wed, Mar 17, 2021 at 03:06:54PM -0400, Alan Stern napisał(a):
> Matthias reports that the Amazon Kindle automatically removes its

> emulated media if it doesn't receive another SCSI command within about

> one second after a SYNCHRONIZE CACHE.  It does so even when the host

> has sent a PREVENT MEDIUM REMOVAL command.  The reason for this

> behavior isn't clear, although it's not hard to make some guesses.


  Could Kindle be fixed not to required such workaround? Is there a way
to open a bug with Amazon?

-- 
Tomasz Torcz                        Once you've read the dictionary,
@ttorcz:pipebreaker.pl              every other book is just a remix.
Alan Stern March 18, 2021, 3:03 p.m. | #4
On Thu, Mar 18, 2021 at 08:10:56AM +0100, Ulrich Windl wrote:
> >>> Alan Stern <stern@rowland.harvard.edu> schrieb am 17.03.2021 um 20:06 in

> Nachricht <20210317190654.GA497856@rowland.harvard.edu>:

> > Matthias reports that the Amazon Kindle automatically removes its

> > emulated media if it doesn't receive another SCSI command within about

> > one second after a SYNCHRONIZE CACHE.  It does so even when the host

> > has sent a PREVENT MEDIUM REMOVAL command.  The reason for this

> > behavior isn't clear, although it's not hard to make some guesses.

> 

> Actually I think Amazon should fix the firmware.


You are free to suggest to them that they change it.  Even if you do 
find the right people to ask about this, I'd be very surprised if they 
agreed to make the change.

> It seems the main goal was to prevent the use of open software to manage the

> content.


This is guesswork on your part, and I disagree.  I think the main goal 
was to improve the user experience by making the Kindle return to normal 
operating mode automatically when file transfers are finished, rather 
than requiring the user to do something extra.  But that's also just a 
guess.

Alan Stern
Alan Stern March 18, 2021, 3:07 p.m. | #5
On Thu, Mar 18, 2021 at 02:50:13PM +0100, Tomasz Torcz wrote:
> Dnia Wed, Mar 17, 2021 at 03:06:54PM -0400, Alan Stern napisał(a):

> > Matthias reports that the Amazon Kindle automatically removes its

> > emulated media if it doesn't receive another SCSI command within about

> > one second after a SYNCHRONIZE CACHE.  It does so even when the host

> > has sent a PREVENT MEDIUM REMOVAL command.  The reason for this

> > behavior isn't clear, although it's not hard to make some guesses.

> 

>   Could Kindle be fixed not to required such workaround? Is there a way

> to open a bug with Amazon?


You really should be asking people who work for Amazon.  I suspect that 
nobody who regularly reads this mailing list knows the answer.

(If you look through the MAINTAINERS file, you'll find there are a few 
kernel developers who do work for Amazon, or at least, have @amazon.com 
email addresses.  Try asking some of them.)

But even if the Kindle firmware gets changed (which I doubt will 
happen), we would still want to support all the old devices that aren't 
running the updated firmware.

Alan Stern

Patch

Index: usb-devel/drivers/usb/storage/transport.c
===================================================================
--- usb-devel.orig/drivers/usb/storage/transport.c
+++ usb-devel/drivers/usb/storage/transport.c
@@ -656,6 +656,13 @@  void usb_stor_invoke_transport(struct sc
 		need_auto_sense = 1;
 	}
 
+	/* Some devices (Kindle) require another command after SYNC CACHE */
+	if ((us->fflags & US_FL_SENSE_AFTER_SYNC) &&
+			srb->cmnd[0] == SYNCHRONIZE_CACHE) {
+		usb_stor_dbg(us, "-- sense after SYNC CACHE\n");
+		need_auto_sense = 1;
+	}
+
 	/*
 	 * If we have a failure, we're going to do a REQUEST_SENSE 
 	 * automatically.  Note that we differentiate between a command
Index: usb-devel/drivers/usb/storage/unusual_devs.h
===================================================================
--- usb-devel.orig/drivers/usb/storage/unusual_devs.h
+++ usb-devel/drivers/usb/storage/unusual_devs.h
@@ -2212,6 +2212,18 @@  UNUSUAL_DEV( 0x1908, 0x3335, 0x0200, 0x0
 		US_FL_NO_READ_DISC_INFO ),
 
 /*
+ * Reported by Matthias Schwarzott <zzam@gentoo.org>
+ * The Amazon Kindle treats SYNCHRONIZE CACHE as an indication that
+ * the host may be finished with it, and automatically ejects its
+ * emulated media unless it receives another command within one second.
+ */
+UNUSUAL_DEV( 0x1949, 0x0004, 0x0000, 0x9999,
+		"Amazon",
+		"Kindle",
+		USB_SC_DEVICE, USB_PR_DEVICE, NULL,
+		US_FL_SENSE_AFTER_SYNC ),
+
+/*
  * Reported by Oliver Neukum <oneukum@suse.com>
  * This device morphes spontaneously into another device if the access
  * pattern of Windows isn't followed. Thus writable media would be dirty
Index: usb-devel/include/linux/usb_usual.h
===================================================================
--- usb-devel.orig/include/linux/usb_usual.h
+++ usb-devel/include/linux/usb_usual.h
@@ -86,6 +86,8 @@ 
 		/* lies about caching, so always sync */	\
 	US_FLAG(NO_SAME, 0x40000000)				\
 		/* Cannot handle WRITE_SAME */			\
+	US_FLAG(SENSE_AFTER_SYNC, 0x80000000)			\
+		/* Do REQUEST_SENSE after SYNCHRONIZE_CACHE */	\
 
 #define US_FLAG(name, value)	US_FL_##name = value ,
 enum { US_DO_ALL_FLAGS };