| Message ID | 20201216024444epcms2p5e69281911dd675306c473df3d2cef8b2@epcms2p5 |
|---|---|
| Headers | show |
| Series | scsi: ufs: Add Host Performance Booster Support | expand |
On Wed, Dec 16, 2020 at 11:44:44AM +0900, Daejun Park wrote: > NAND flash memory-based storage devices use Flash Translation Layer (FTL) > to translate logical addresses of I/O requests to corresponding flash > memory addresses. Mobile storage devices typically have RAM with > constrained size, thus lack in memory to keep the whole mapping table. > Therefore, mapping tables are partially retrieved from NAND flash on > demand, causing random-read performance degradation. > > To improve random read performance, JESD220-3 (HPB v1.0) proposes HPB > (Host Performance Booster) which uses host system memory as a cache for the > FTL mapping table. By using HPB, FTL data can be read from host memory > faster than from NAND flash memory. > > The current version only supports the DCM (device control mode). > This patch consists of 3 parts to support HPB feature. > > 1) HPB probe and initialization process > 2) READ -> HPB READ using cached map information > 3) L2P (logical to physical) map management > > In the HPB probe and init process, the device information of the UFS is > queried. After checking supported features, the data structure for the HPB > is initialized according to the device information. > > A read I/O in the active sub-region where the map is cached is changed to > HPB READ by the HPB. > > The HPB manages the L2P map using information received from the > device. For active sub-region, the HPB caches through ufshpb_map > request. For the in-active region, the HPB discards the L2P map. > When a write I/O occurs in an active sub-region area, associated dirty > bitmap checked as dirty for preventing stale read. > > HPB is shown to have a performance improvement of 58 - 67% for random read > workload. [1] > > We measured the total start-up time of popular applications and observed > the difference by enabling the HPB. > Popular applications are 12 game apps and 24 non-game apps. Each target > applications were launched in order. The cycle consists of running 36 > applications in sequence. We repeated the cycle for observing performance > improvement by L2P mapping cache hit in HPB. > > The Following is experiment environment: > - kernel version: 4.4.0 > - UFS 2.1 (64GB) > > Result: > +-------+----------+----------+-------+ > | cycle | baseline | with HPB | diff | > +-------+----------+----------+-------+ > | 1 | 272.4 | 264.9 | -7.5 | > | 2 | 250.4 | 248.2 | -2.2 | > | 3 | 226.2 | 215.6 | -10.6 | > | 4 | 230.6 | 214.8 | -15.8 | > | 5 | 232.0 | 218.1 | -13.9 | > | 6 | 231.9 | 212.6 | -19.3 | > +-------+----------+----------+-------+ I feel this was burried in the 00 email, shouldn't it go into the 01 commit changelog so that you can see this? But why does the "cycle" matter here? Can you run a normal benchmark, like fio, on here so we can get some numbers we know how to compare to other systems with, and possible reproduct it ourselves? I'm sure fio will easily show random read performance increases, right? thanks, greg k-h
On Wed, Dec 16, 2020 at 11:45:32AM +0900, Daejun Park wrote: > > This is a patch for the HPB initialization and adds HPB function calls to > > UFS core driver. > > <snip> > > Your "subject" is odd, it has "Subject:" in it twice, did git > format-patch create that? > > thanks, > > greg k-h > Sorry, It is my mistake. Should I resend this patch with proper subject? Thansk, Daejun
On Thu, Dec 17, 2020 at 02:21:36PM +0900, Daejun Park wrote: > On Wed, Dec 16, 2020 at 11:45:32AM +0900, Daejun Park wrote: > > > This is a patch for the HPB initialization and adds HPB function calls to > > > UFS core driver. > > > > <snip> > > > > Your "subject" is odd, it has "Subject:" in it twice, did git > > format-patch create that? > > > > thanks, > > > > greg k-h > > > > Sorry, It is my mistake. > Should I resend this patch with proper subject? Eventually yes. Nothing anyone can do with this before 5.11-rc1 is out anyway, so you might want to wait. thanks, greg k-h
Hi, Greg > > NAND flash memory-based storage devices use Flash Translation Layer (FTL) > > to translate logical addresses of I/O requests to corresponding flash > > memory addresses. Mobile storage devices typically have RAM with > > constrained size, thus lack in memory to keep the whole mapping table. > > Therefore, mapping tables are partially retrieved from NAND flash on > > demand, causing random-read performance degradation. > > > > To improve random read performance, JESD220-3 (HPB v1.0) proposes HPB > > (Host Performance Booster) which uses host system memory as a cache for the > > FTL mapping table. By using HPB, FTL data can be read from host memory > > faster than from NAND flash memory. > > > > The current version only supports the DCM (device control mode). > > This patch consists of 3 parts to support HPB feature. > > > > 1) HPB probe and initialization process > > 2) READ -> HPB READ using cached map information > > 3) L2P (logical to physical) map management > > > > In the HPB probe and init process, the device information of the UFS is > > queried. After checking supported features, the data structure for the HPB > > is initialized according to the device information. > > > > A read I/O in the active sub-region where the map is cached is changed to > > HPB READ by the HPB. > > > > The HPB manages the L2P map using information received from the > > device. For active sub-region, the HPB caches through ufshpb_map > > request. For the in-active region, the HPB discards the L2P map. > > When a write I/O occurs in an active sub-region area, associated dirty > > bitmap checked as dirty for preventing stale read. > > > > HPB is shown to have a performance improvement of 58 - 67% for random read > > workload. [1] > > > > We measured the total start-up time of popular applications and observed > > the difference by enabling the HPB. > > Popular applications are 12 game apps and 24 non-game apps. Each target > > applications were launched in order. The cycle consists of running 36 > > applications in sequence. We repeated the cycle for observing performance > > improvement by L2P mapping cache hit in HPB. > > > > The Following is experiment environment: > > - kernel version: 4.4.0 > > - UFS 2.1 (64GB) > > > > Result: > > +-------+----------+----------+-------+ > > | cycle | baseline | with HPB | diff | > > +-------+----------+----------+-------+ > > | 1 | 272.4 | 264.9 | -7.5 | > > | 2 | 250.4 | 248.2 | -2.2 | > > | 3 | 226.2 | 215.6 | -10.6 | > > | 4 | 230.6 | 214.8 | -15.8 | > > | 5 | 232.0 | 218.1 | -13.9 | > > | 6 | 231.9 | 212.6 | -19.3 | > > +-------+----------+----------+-------+ > > I feel this was burried in the 00 email, shouldn't it go into the 01 > commit changelog so that you can see this? Sure, I will move this result to 01 commit log. > But why does the "cycle" matter here? I think iteration minimizes other factors that affect the start-up time of application. > Can you run a normal benchmark, like fio, on here so we can get some > numbers we know how to compare to other systems with, and possible > reproduct it ourselves? I'm sure fio will easily show random read > performance increases, right? Here is my iozone script: iozone -r 4k -+n -i2 -ecI -t 16 -l 16 -u 16 -s $IO_RANGE/16 -F mnt/tmp_1 mnt/tmp_2 mnt/tmp_3 mnt/tmp_4 mnt/tmp_5 mnt/tmp_6 mnt/tmp_7 mnt/tmp_8 mnt/tmp_9 mnt/tmp_10 mnt/tmp_11 mnt/tmp_12 mnt/tmp_13 mnt/tmp_14 mnt/tmp_15 mnt/tmp_16 Result: +----------+--------+---------+ | IO range | HPB on | HPB off | +----------+--------+---------+ | 1 GB | 294.8 | 300.87 | | 4 GB | 293.51 | 179.35 | | 8 GB | 294.85 | 162.52 | | 16 GB | 293.45 | 156.26 | | 32 GB | 277.4 | 153.25 | +----------+--------+---------+ Thanks, Daejun
On 12/17/20 5:05 PM, Daejun Park wrote: > Here is my iozone script: > iozone -r 4k -+n -i2 -ecI -t 16 -l 16 -u 16 > -s $IO_RANGE/16 -F mnt/tmp_1 mnt/tmp_2 mnt/tmp_3 mnt/tmp_4 > mnt/tmp_5 mnt/tmp_6 mnt/tmp_7 mnt/tmp_8 mnt/tmp_9 mnt/tmp_10 mnt/tmp_11 > mnt/tmp_12 mnt/tmp_13 mnt/tmp_14 mnt/tmp_15 mnt/tmp_16 > > Result: > +----------+--------+---------+ > | IO range | HPB on | HPB off | > +----------+--------+---------+ > | 1 GB | 294.8 | 300.87 | > | 4 GB | 293.51 | 179.35 | > | 8 GB | 294.85 | 162.52 | > | 16 GB | 293.45 | 156.26 | > | 32 GB | 277.4 | 153.25 | > +----------+--------+---------+ Hi Daejun, What are the units of the numbers in columns 2 and 3? Thanks, Bart.
Hi Daejun, On Wed, 2020-12-16 at 11:45 +0900, Daejun Park wrote: > This is a patch for the HPB initialization and adds HPB function calls to > UFS core driver. > > NAND flash-based storage devices, including UFS, have mechanisms to > translate logical addresses of IO requests to the corresponding physical > addresses of the flash storage. > In UFS, Logical-address-to-Physical-address (L2P) map data, which is > required to identify the physical address for the requested IOs, can only > be partially stored in SRAM from NAND flash. Due to this partial loading, > accessing the flash address area where the L2P information for that address > is not loaded in the SRAM can result in serious performance degradation. > > The basic concept of HPB is to cache L2P mapping entries in host system > memory so that both physical block address (PBA) and logical block address > (LBA) can be delivered in HPB read command. > The HPB READ command allows to read data faster than a read command in UFS > since it provides the physical address (HPB Entry) of the desired logical > block in addition to its logical address. The UFS device can access the > physical block in NAND directly without searching and uploading L2P mapping > table. This improves read performance because the NAND read operation for > uploading L2P mapping table is removed. > > In HPB initialization, the host checks if the UFS device supports HPB > feature and retrieves related device capabilities. Then, some HPB > parameters are configured in the device. > > Reviewed-by: Bart Van Assche <bvanassche@acm.org> > Reviewed-by: Can Guo <cang@codeaurora.org> > Acked-by: Avri Altman <Avri.Altman@wdc.com> > Tested-by: Bean Huo <beanhuo@micron.com> > Signed-off-by: Daejun Park <daejun7.park@samsung.com> > --- > Documentation/ABI/testing/sysfs-driver-ufs | 80 +++ > drivers/scsi/ufs/Kconfig | 9 + > drivers/scsi/ufs/Makefile | 1 + > drivers/scsi/ufs/ufs-sysfs.c | 18 + > drivers/scsi/ufs/ufs.h | 13 + > drivers/scsi/ufs/ufshcd.c | 48 ++ > drivers/scsi/ufs/ufshcd.h | 23 +- > drivers/scsi/ufs/ufshpb.c | 562 +++++++++++++++++++++ > drivers/scsi/ufs/ufshpb.h | 167 ++++++ > 9 files changed, 920 insertions(+), 1 deletion(-) > create mode 100644 drivers/scsi/ufs/ufshpb.c > create mode 100644 drivers/scsi/ufs/ufshpb.h > > diff --git a/Documentation/ABI/testing/sysfs-driver-ufs b/Documentation/ABI/testing/sysfs-driver-ufs > index d1a352194d2e..8b16a353392c 100644 > --- a/Documentation/ABI/testing/sysfs-driver-ufs > +++ b/Documentation/ABI/testing/sysfs-driver-ufs > @@ -1019,3 +1019,83 @@ Contact: Asutosh Das <asutoshd@codeaurora.org> > Description: This entry shows the configured size of WriteBooster buffer. > 0400h corresponds to 4GB. > The file is read only. > + > +What: /sys/bus/platform/drivers/ufshcd/*/device_descriptor/hpb_version > +Date: December 2020 > +Contact: Daejun Park <daejun7.park@samsung.com> > +Description: This entry shows the HPB specification version. > + The full information about the descriptor could be found at UFS > + HPB (Host Performance Booster) Extension specifications. > + Example: version 1.2.3 = 0123h > + The file is read only. > + > +What: /sys/bus/platform/drivers/ufshcd/*/device_descriptor/hpb_control > +Date: December 2020 > +Contact: Daejun Park <daejun7.park@samsung.com> > +Description: This entry shows an indication of the HPB control mode. > + 00h: Host control mode > + 01h: Device control mode > + The file is read only. > + > +What: /sys/bus/platform/drivers/ufshcd/*/geometry_descriptor/hpb_region_size > +Date: December 2020 > +Contact: Daejun Park <daejun7.park@samsung.com> > +Description: This entry shows the bHPBRegionSize which can be calculated > + as in the following (in bytes): > + HPB Region size = 512B * 2^bHPBRegionSize > + The file is read only. > + > +What: /sys/bus/platform/drivers/ufshcd/*/geometry_descriptor/hpb_number_lu > +Date: December 2020 > +Contact: Daejun Park <daejun7.park@samsung.com> > +Description: This entry shows the maximum number of HPB LU supported by > + the device. > + 00h: HPB is not supported by the device. > + 01h ~ 20h: Maximum number of HPB LU supported by the device > + The file is read only. > + > +What: /sys/bus/platform/drivers/ufshcd/*/geometry_descriptor/hpb_number_lu > +Date: December 2020 > +Contact: Daejun Park <daejun7.park@samsung.com> > +Description: This entry shows the maximum number of HPB LU supported by > + the device. > + 00h: HPB is not supported by the device. > + 01h ~ 20h: Maximum number of HPB LU supported by the device > + The file is read only. Please remove above duplicated item. Thanks, Stanley Chu
Hi Daejun, > > On Wed, 2020-12-16 at 11:45 +0900, Daejun Park wrote: > > This is a patch for the HPB initialization and adds HPB function calls to > > UFS core driver. > > > > NAND flash-based storage devices, including UFS, have mechanisms to > > translate logical addresses of IO requests to the corresponding physical > > addresses of the flash storage. > > In UFS, Logical-address-to-Physical-address (L2P) map data, which is > > required to identify the physical address for the requested IOs, can only > > be partially stored in SRAM from NAND flash. Due to this partial loading, > > accessing the flash address area where the L2P information for that address > > is not loaded in the SRAM can result in serious performance degradation. > > > > The basic concept of HPB is to cache L2P mapping entries in host system > > memory so that both physical block address (PBA) and logical block address > > (LBA) can be delivered in HPB read command. > > The HPB READ command allows to read data faster than a read command in UFS > > since it provides the physical address (HPB Entry) of the desired logical > > block in addition to its logical address. The UFS device can access the > > physical block in NAND directly without searching and uploading L2P mapping > > table. This improves read performance because the NAND read operation for > > uploading L2P mapping table is removed. > > > > In HPB initialization, the host checks if the UFS device supports HPB > > feature and retrieves related device capabilities. Then, some HPB > > parameters are configured in the device. > > > > Reviewed-by: Bart Van Assche <bvanassche@acm.org> > > Reviewed-by: Can Guo <cang@codeaurora.org> > > Acked-by: Avri Altman <Avri.Altman@wdc.com> > > Tested-by: Bean Huo <beanhuo@micron.com> > > Signed-off-by: Daejun Park <daejun7.park@samsung.com> > > --- > > Documentation/ABI/testing/sysfs-driver-ufs | 80 +++ > > drivers/scsi/ufs/Kconfig | 9 + > > drivers/scsi/ufs/Makefile | 1 + > > drivers/scsi/ufs/ufs-sysfs.c | 18 + > > drivers/scsi/ufs/ufs.h | 13 + > > drivers/scsi/ufs/ufshcd.c | 48 ++ > > drivers/scsi/ufs/ufshcd.h | 23 +- > > drivers/scsi/ufs/ufshpb.c | 562 +++++++++++++++++++++ > > drivers/scsi/ufs/ufshpb.h | 167 ++++++ > > 9 files changed, 920 insertions(+), 1 deletion(-) > > create mode 100644 drivers/scsi/ufs/ufshpb.c > > create mode 100644 drivers/scsi/ufs/ufshpb.h > > > > diff --git a/Documentation/ABI/testing/sysfs-driver-ufs b/Documentation/ABI/testing/sysfs-driver-ufs > > index d1a352194d2e..8b16a353392c 100644 > > --- a/Documentation/ABI/testing/sysfs-driver-ufs > > +++ b/Documentation/ABI/testing/sysfs-driver-ufs > > @@ -1019,3 +1019,83 @@ Contact: Asutosh Das <asutoshd@codeaurora.org> > > Description: This entry shows the configured size of WriteBooster buffer. > > 0400h corresponds to 4GB. > > The file is read only. > > + > > +What: /sys/bus/platform/drivers/ufshcd/*/device_descriptor/hpb_version > > +Date: December 2020 > > +Contact: Daejun Park <daejun7.park@samsung.com> > > +Description: This entry shows the HPB specification version. > > + The full information about the descriptor could be found at UFS > > + HPB (Host Performance Booster) Extension specifications. > > + Example: version 1.2.3 = 0123h > > + The file is read only. > > + > > +What: /sys/bus/platform/drivers/ufshcd/*/device_descriptor/hpb_control > > +Date: December 2020 > > +Contact: Daejun Park <daejun7.park@samsung.com> > > +Description: This entry shows an indication of the HPB control mode. > > + 00h: Host control mode > > + 01h: Device control mode > > + The file is read only. > > + > > +What: /sys/bus/platform/drivers/ufshcd/*/geometry_descriptor/hpb_region_size > > +Date: December 2020 > > +Contact: Daejun Park <daejun7.park@samsung.com> > > +Description: This entry shows the bHPBRegionSize which can be calculated > > + as in the following (in bytes): > > + HPB Region size = 512B * 2^bHPBRegionSize > > + The file is read only. > > + > > +What: /sys/bus/platform/drivers/ufshcd/*/geometry_descriptor/hpb_number_lu > > +Date: December 2020 > > +Contact: Daejun Park <daejun7.park@samsung.com> > > +Description: This entry shows the maximum number of HPB LU supported by > > + the device. > > + 00h: HPB is not supported by the device. > > + 01h ~ 20h: Maximum number of HPB LU supported by the device > > + The file is read only. > > + > > +What: /sys/bus/platform/drivers/ufshcd/*/geometry_descriptor/hpb_number_lu > > +Date: December 2020 > > +Contact: Daejun Park <daejun7.park@samsung.com> > > +Description: This entry shows the maximum number of HPB LU supported by > > + the device. > > + 00h: HPB is not supported by the device. > > + 01h ~ 20h: Maximum number of HPB LU supported by the device > > + The file is read only. > > Please remove above duplicated item. > > Thanks, > Stanley Chu OK, I will remove them. Thanks, Daejun
On 12/17/20 5:05 PM, Daejun Park wrote: > > Here is my iozone script: > > iozone -r 4k -+n -i2 -ecI -t 16 -l 16 -u 16 > > -s $IO_RANGE/16 -F mnt/tmp_1 mnt/tmp_2 mnt/tmp_3 mnt/tmp_4 > > mnt/tmp_5 mnt/tmp_6 mnt/tmp_7 mnt/tmp_8 mnt/tmp_9 mnt/tmp_10 mnt/tmp_11 > > mnt/tmp_12 mnt/tmp_13 mnt/tmp_14 mnt/tmp_15 mnt/tmp_16 > > > > Result: > > +----------+--------+---------+ > > | IO range | HPB on | HPB off | > > +----------+--------+---------+ > > | 1 GB | 294.8 | 300.87 | > > | 4 GB | 293.51 | 179.35 | > > | 8 GB | 294.85 | 162.52 | > > | 16 GB | 293.45 | 156.26 | > > | 32 GB | 277.4 | 153.25 | > > +----------+--------+---------+ > > Hi Daejun, > > What are the units of the numbers in columns 2 and 3? > > Thanks, > > Bart. > I forgot to add units, it is MB/s. Thanks Daejun