[0/5] lib/ring: templates to support custom element size

Message ID 20190828144614.25284-1-honnappa.nagarahalli@arm.com
Headers show
Series
  • lib/ring: templates to support custom element size
Related show

Message

Honnappa Nagarahalli Aug. 28, 2019, 2:46 p.m.
The current rte_ring hard-codes the type of the ring element to 'void *',
hence the size of the element is hard-coded to 32b/64b. Since the ring
element type is not an input to rte_ring APIs, it results in couple
of issues:

1) If an application requires to store an element which is not 64b, it
   needs to writes its own ring APIs similar to rte_event_ring APIs. This
   creates additional burden on the programmers, who simply end up making
   work-arounds and often waste memory.
2) If there are multiple libraries that store elements of the same
   type, currently they would have to write their own rte_ring APIs. This
   results in code duplication.

This patch consists of 4 parts:
1) New APIs to support configurable ring element size
   These will help reduce code duplication in the templates. I think these
   can be made internal (do not expose to DPDK applications, but expose to
   DPDK libraries), feedback needed.

2) rte_ring templates
   The templates provide an easy way to add new APIs for different ring
   element types/sizes which can be used by multiple libraries. These
   also allow for creating APIs to store elements of custom types
   (for ex: a structure)

   The template needs 4 parameters:
   a) RTE_RING_TMPLT_API_SUFFIX - This is used as a suffix to the
      rte_ring APIs.
      For ex: if RTE_RING_TMPLT_API_SUFFIX is '32b', the API name will be
      rte_ring_create_32b
   b) RTE_RING_TMPLT_ELEM_SIZE - Size of the ring element in bytes.
      For ex: sizeof(uint32_t)
   c) RTE_RING_TMPLT_ELEM_TYPE - Type of the ring element.
      For ex: uint32_t. If a common ring library does not use a standard
      data type, it should create its own type by defining a structure
      with standard data type. For ex: for an elment size of 96b, one
      could define a structure

      struct s_96b {
          uint32_t a[3];
      }
      The common library can use this structure to define
      RTE_RING_TMPLT_ELEM_TYPE.

      The application using this common ring library should define its
      element type as a union with the above structure.

      union app_element_type {
          struct s_96b v;
          struct app_element {
              uint16_t a;
              uint16_t b;
              uint32_t c;
              uint32_t d;
          }
      }
   d) RTE_RING_TMPLT_EXPERIMENTAL - Indicates if the new APIs being defined
      are experimental. Should be set to empty to remove the experimental
      tag.

   The ring library consists of some APIs that are defined as inline
   functions and some APIs that are non-inline functions. The non-inline
   functions are in rte_ring_template.c. However, this file needs to be
   included in other .c files. Any feedback on how to handle this is
   appreciated.

   Note that the templates help create the APIs that are dependent on the
   element size (for ex: rte_ring_create, enqueue/dequeue etc). Other APIs
   that do NOT depend on the element size do not need to be part of the
   template (for ex: rte_ring_dump, rte_ring_count, rte_ring_free_count
   etc).

3) APIs for 32b ring element size
   This uses the templates to create APIs to enqueue/dequeue elements of
   size 32b.

4) rte_hash libray is changed to use 32b ring APIs
   The 32b APIs are used in rte_hash library to store the free slot index
   and free bucket index.

This patch results in following checkpatch issue:
WARNING:UNSPECIFIED_INT: Prefer 'unsigned int' to bare use of 'unsigned'

The patch is following the rules in the existing code. Please let me know
if this needs to be fixed.

Honnappa Nagarahalli (5):
  lib/ring: apis to support configurable element size
  lib/ring: add template to support different element sizes
  tools/checkpatch: relax constraints on __rte_experimental
  lib/ring: add ring APIs to support 32b ring elements
  lib/hash: use ring with 32b element size to save memory

 devtools/checkpatches.sh             |  11 +-
 lib/librte_hash/rte_cuckoo_hash.c    |  55 ++---
 lib/librte_hash/rte_cuckoo_hash.h    |   2 +-
 lib/librte_ring/Makefile             |   9 +-
 lib/librte_ring/meson.build          |  11 +-
 lib/librte_ring/rte_ring.c           |  34 ++-
 lib/librte_ring/rte_ring.h           |  72 ++++++
 lib/librte_ring/rte_ring_32.c        |  19 ++
 lib/librte_ring/rte_ring_32.h        |  36 +++
 lib/librte_ring/rte_ring_template.c  |  46 ++++
 lib/librte_ring/rte_ring_template.h  | 330 +++++++++++++++++++++++++++
 lib/librte_ring/rte_ring_version.map |   4 +
 12 files changed, 582 insertions(+), 47 deletions(-)
 create mode 100644 lib/librte_ring/rte_ring_32.c
 create mode 100644 lib/librte_ring/rte_ring_32.h
 create mode 100644 lib/librte_ring/rte_ring_template.c
 create mode 100644 lib/librte_ring/rte_ring_template.h

-- 
2.17.1

Comments

Jerin Jacob Kollanukkaran Aug. 28, 2019, 3:12 p.m. | #1
> -----Original Message-----

> From: dev <dev-bounces@dpdk.org> On Behalf Of Honnappa Nagarahalli

> Sent: Wednesday, August 28, 2019 8:16 PM

> To: olivier.matz@6wind.com; yipeng1.wang@intel.com;

> sameh.gobriel@intel.com; bruce.richardson@intel.com;

> pablo.de.lara.guarch@intel.com; honnappa.nagarahalli@arm.com

> Cc: dev@dpdk.org; dharmik.thakkar@arm.com; gavin.hu@arm.com;

> ruifeng.wang@arm.com; nd@arm.com

> Subject: [dpdk-dev] [PATCH 0/5] lib/ring: templates to support custom element

> size

> 

> The current rte_ring hard-codes the type of the ring element to 'void *', hence

> the size of the element is hard-coded to 32b/64b. Since the ring element type is

> not an input to rte_ring APIs, it results in couple of issues:

> 

> 1) If an application requires to store an element which is not 64b, it

>    needs to writes its own ring APIs similar to rte_event_ring APIs. This

>    creates additional burden on the programmers, who simply end up making

>    work-arounds and often waste memory.


If we are taking this path, Could you change rte_event_ring implementation based
on new framework?



> 2) If there are multiple libraries that store elements of the same

>    type, currently they would have to write their own rte_ring APIs. This

>    results in code duplication.

> 

> This patch consists of 4 parts:

> 1) New APIs to support configurable ring element size

>    These will help reduce code duplication in the templates. I think these

>    can be made internal (do not expose to DPDK applications, but expose to

>    DPDK libraries), feedback needed.

> 

> 2) rte_ring templates

>    The templates provide an easy way to add new APIs for different ring

>    element types/sizes which can be used by multiple libraries. These

>    also allow for creating APIs to store elements of custom types

>    (for ex: a structure)

> 

>    The template needs 4 parameters:

>    a) RTE_RING_TMPLT_API_SUFFIX - This is used as a suffix to the

>       rte_ring APIs.

>       For ex: if RTE_RING_TMPLT_API_SUFFIX is '32b', the API name will be

>       rte_ring_create_32b

>    b) RTE_RING_TMPLT_ELEM_SIZE - Size of the ring element in bytes.

>       For ex: sizeof(uint32_t)

>    c) RTE_RING_TMPLT_ELEM_TYPE - Type of the ring element.

>       For ex: uint32_t. If a common ring library does not use a standard

>       data type, it should create its own type by defining a structure

>       with standard data type. For ex: for an elment size of 96b, one

>       could define a structure

> 

>       struct s_96b {

>           uint32_t a[3];

>       }

>       The common library can use this structure to define

>       RTE_RING_TMPLT_ELEM_TYPE.

> 

>       The application using this common ring library should define its

>       element type as a union with the above structure.

> 

>       union app_element_type {

>           struct s_96b v;

>           struct app_element {

>               uint16_t a;

>               uint16_t b;

>               uint32_t c;

>               uint32_t d;

>           }

>       }

>    d) RTE_RING_TMPLT_EXPERIMENTAL - Indicates if the new APIs being defined

>       are experimental. Should be set to empty to remove the experimental

>       tag.

> 

>    The ring library consists of some APIs that are defined as inline

>    functions and some APIs that are non-inline functions. The non-inline

>    functions are in rte_ring_template.c. However, this file needs to be

>    included in other .c files. Any feedback on how to handle this is

>    appreciated.

> 

>    Note that the templates help create the APIs that are dependent on the

>    element size (for ex: rte_ring_create, enqueue/dequeue etc). Other APIs

>    that do NOT depend on the element size do not need to be part of the

>    template (for ex: rte_ring_dump, rte_ring_count, rte_ring_free_count

>    etc).

> 

> 3) APIs for 32b ring element size

>    This uses the templates to create APIs to enqueue/dequeue elements of

>    size 32b.

> 

> 4) rte_hash libray is changed to use 32b ring APIs

>    The 32b APIs are used in rte_hash library to store the free slot index

>    and free bucket index.

> 

> This patch results in following checkpatch issue:

> WARNING:UNSPECIFIED_INT: Prefer 'unsigned int' to bare use of 'unsigned'

> 

> The patch is following the rules in the existing code. Please let me know if this

> needs to be fixed.

> 

> Honnappa Nagarahalli (5):

>   lib/ring: apis to support configurable element size

>   lib/ring: add template to support different element sizes

>   tools/checkpatch: relax constraints on __rte_experimental

>   lib/ring: add ring APIs to support 32b ring elements

>   lib/hash: use ring with 32b element size to save memory

> 

>  devtools/checkpatches.sh             |  11 +-

>  lib/librte_hash/rte_cuckoo_hash.c    |  55 ++---

>  lib/librte_hash/rte_cuckoo_hash.h    |   2 +-

>  lib/librte_ring/Makefile             |   9 +-

>  lib/librte_ring/meson.build          |  11 +-

>  lib/librte_ring/rte_ring.c           |  34 ++-

>  lib/librte_ring/rte_ring.h           |  72 ++++++

>  lib/librte_ring/rte_ring_32.c        |  19 ++

>  lib/librte_ring/rte_ring_32.h        |  36 +++

>  lib/librte_ring/rte_ring_template.c  |  46 ++++

> lib/librte_ring/rte_ring_template.h  | 330 +++++++++++++++++++++++++++

>  lib/librte_ring/rte_ring_version.map |   4 +

>  12 files changed, 582 insertions(+), 47 deletions(-)  create mode 100644

> lib/librte_ring/rte_ring_32.c  create mode 100644 lib/librte_ring/rte_ring_32.h

> create mode 100644 lib/librte_ring/rte_ring_template.c

>  create mode 100644 lib/librte_ring/rte_ring_template.h

> 

> --

> 2.17.1
Pavan Nikhilesh Bhagavatula Aug. 28, 2019, 3:16 p.m. | #2
Hi Honnappa, 

Great idea I think we can replace duplicated implementation lib/librte_eventdev/rte_event_ring.h which uses
element sizeof 16B.
 There are already a couple of SW eventdevice drivers using event_ring.

Pavan.

>-----Original Message-----

>From: dev <dev-bounces@dpdk.org> On Behalf Of Honnappa

>Nagarahalli

>Sent: Wednesday, August 28, 2019 8:16 PM

>To: olivier.matz@6wind.com; yipeng1.wang@intel.com;

>sameh.gobriel@intel.com; bruce.richardson@intel.com;

>pablo.de.lara.guarch@intel.com; honnappa.nagarahalli@arm.com

>Cc: dev@dpdk.org; dharmik.thakkar@arm.com; gavin.hu@arm.com;

>ruifeng.wang@arm.com; nd@arm.com

>Subject: [dpdk-dev] [PATCH 0/5] lib/ring: templates to support custom

>element size

>
Honnappa Nagarahalli Aug. 28, 2019, 10:59 p.m. | #3
<snip>

> Subject: RE: [dpdk-dev] [PATCH 0/5] lib/ring: templates to support custom

> element size

> 

> Hi Honnappa,

> 

> Great idea I think we can replace duplicated implementation

> lib/librte_eventdev/rte_event_ring.h which uses element sizeof 16B.

>  There are already a couple of SW eventdevice drivers using event_ring.

Thank you Pavan. I will take a look and get back.

> 

> Pavan.

> 

> >-----Original Message-----

> >From: dev <dev-bounces@dpdk.org> On Behalf Of Honnappa Nagarahalli

> >Sent: Wednesday, August 28, 2019 8:16 PM

> >To: olivier.matz@6wind.com; yipeng1.wang@intel.com;

> >sameh.gobriel@intel.com; bruce.richardson@intel.com;

> >pablo.de.lara.guarch@intel.com; honnappa.nagarahalli@arm.com

> >Cc: dev@dpdk.org; dharmik.thakkar@arm.com; gavin.hu@arm.com;

> >ruifeng.wang@arm.com; nd@arm.com

> >Subject: [dpdk-dev] [PATCH 0/5] lib/ring: templates to support custom

> >element size

> >