| Message ID | 1454035480-21776-2-git-send-email-bill.fischofer@linaro.org |
|---|---|
| State | Superseded |
| Headers | show |
On 29/01/16 02:44, Bill Fischofer wrote: > Add odp_cpu_socket_id() and odp_cpu_socket_id_cpu() routines > > Signed-off-by: Bill Fischofer <bill.fischofer@linaro.org> > --- > include/odp/api/cpu.h | 23 +++++++++++++++++++++++ > 1 file changed, 23 insertions(+) > > diff --git a/include/odp/api/cpu.h b/include/odp/api/cpu.h > index 4cbaf58..fe74825 100644 > --- a/include/odp/api/cpu.h > +++ b/include/odp/api/cpu.h > @@ -36,6 +36,29 @@ extern "C" { > int odp_cpu_id(void); > > /** > + * CPU socket id > + * > + * Returns the socket id associated with the calling CPU on NUMA systems. > + * Socket ID numbering is system specific. > + * > + * @return Socket ID of the calling CPU > + * @retval ODP_SOCKET_ID_ANY If the caller is not running on a NUMA system. > + */ > +uint32_t odp_cpu_socket_id(void); > + > +/** > + * CPU socket id of designated CPU > + * > + * Returns the socket id associated with a specified CPU on NUMA systems. > + * Socket ID numbering is system specific. > + * > + * @return Socket ID of the designated CPU > + * @retval ODP_SOCKET_ID_ANY If the specified CPU is unknown or caller is > + * not running on a NUMA system. Maybe worth to differentiate between the two scenario with different return value? To query for an invalid CPU sounds like a serious problem in the application while the other is part of normal operation. > + */ > +uint32_t odp_cpu_socket_id_cpu(int cpu_id); > + > +/** > * CPU count > * > * Report the number of CPU's available to this ODP program. >
We'll be discussing this RFC on Monday's ARCH call. I'm certainly open to adding this if it's deemed useful. I do note that DPDK defines SOCK_ID_ANY to be -1 so I'd like to minimize translation overhead for odp-dpdk in this area as well. On Fri, Jan 29, 2016 at 10:27 AM, Zoltan Kiss <zoltan.kiss@linaro.org> wrote: > > > On 29/01/16 02:44, Bill Fischofer wrote: > >> Add odp_cpu_socket_id() and odp_cpu_socket_id_cpu() routines >> >> Signed-off-by: Bill Fischofer <bill.fischofer@linaro.org> >> --- >> include/odp/api/cpu.h | 23 +++++++++++++++++++++++ >> 1 file changed, 23 insertions(+) >> >> diff --git a/include/odp/api/cpu.h b/include/odp/api/cpu.h >> index 4cbaf58..fe74825 100644 >> --- a/include/odp/api/cpu.h >> +++ b/include/odp/api/cpu.h >> @@ -36,6 +36,29 @@ extern "C" { >> int odp_cpu_id(void); >> >> /** >> + * CPU socket id >> + * >> + * Returns the socket id associated with the calling CPU on NUMA systems. >> + * Socket ID numbering is system specific. >> + * >> + * @return Socket ID of the calling CPU >> + * @retval ODP_SOCKET_ID_ANY If the caller is not running on a NUMA >> system. >> + */ >> +uint32_t odp_cpu_socket_id(void); >> + >> +/** >> + * CPU socket id of designated CPU >> + * >> + * Returns the socket id associated with a specified CPU on NUMA systems. >> + * Socket ID numbering is system specific. >> + * >> + * @return Socket ID of the designated CPU >> + * @retval ODP_SOCKET_ID_ANY If the specified CPU is unknown or caller is >> + * not running on a NUMA system. >> > > Maybe worth to differentiate between the two scenario with different > return value? To query for an invalid CPU sounds like a serious problem in > the application while the other is part of normal operation. > > > + */ >> +uint32_t odp_cpu_socket_id_cpu(int cpu_id); >> + >> +/** >> * CPU count >> * >> * Report the number of CPU's available to this ODP program. >> >>
The immediate need was voiced by the TRex folks, who are porting from DPDK, where socket_id is an explicit parameter for mempool creation. Since the ODP equivalent is odp_pool_create(), adding this to the odp_pool_param_t seems a natural "fit". I agree that in general applications should not worry about topology issues, however the question is how are these to be mapped into ODP APIs? An alternative approach we discussed last year was to introduce the concept of a pool group where a common group name/handle would be instantiated as multiple individual pools associated with each socket so that threads could use a common group and still get local memory, however this needs to be fleshed out. Were you thinking along those lines or did you have something else in mind? As I said, we should discuss this further in today's ARCH call and it's on the agenda. On Mon, Feb 1, 2016 at 4:04 AM, Savolainen, Petri (Nokia - FI/Espoo) < petri.savolainen@nokia.com> wrote: > How this ID would be used? Would it be added as a parameter to many of the > create calls (shm, pool, timer, crypto, …)? > > > > I’d like to avoid introduction of explicit “socket” IDs (if possible). > Application should not care too much on physical format/layout of the > circuit boards or SoCs. In general, the problem is about how **user** > could control clustering and get information on cost of crossing various > cluster borders. Do we need API calls and IDs, or just implementation > defined named resources that user can pass down (e.g. on command line). > > > > For example, an application could run in SMP setup on a blade that have 6 > SoCs connected like this (with 2-3 coherent IO links toward each neighbor). > Each SoC would have clustering inside: e.g. ring based interconnect, 8x8 > CPUs in clusters, 3-4 cache levels, DDR memory controllers and HW > accelerators spread around the interconnect. Also virtualization/VMs could > hide some of these details (which virtual CPU is close to which virtual > interface, etc). > > > > > > DDR DDR > > | | > > SoC1 SoC2 SoC3 > > + ------ + ------ + > > | | | > > + ------ + ------ + > > SoC4 SoC5 SoC6 > > | | | > > eth0 eth1 DDR > > > > > > Memory/HW accelerator access latency would vary a lot depending on the > location of the source CPU and target device (memory). E.g. latency from > CPU:X/SoC1 to DDR:Y on SoC2, SoC3 and SoC6 would be quite different. Also > it could be hard to tell without testing, if optimal placement for a HW > buffer pool used for packet input from eth0 and output to eth1 would be > SoC4, 5 or 6 (if in addition application SW runs on SoC6). > > > > -Petri > > > > > > > > *From:* lng-odp [mailto:lng-odp-bounces@lists.linaro.org] *On Behalf Of *EXT > Bill Fischofer > *Sent:* Saturday, January 30, 2016 12:40 AM > *To:* Zoltan Kiss > *Cc:* LNG ODP Mailman List > *Subject:* Re: [lng-odp] [RFC API-NEXT PATCH 2/2] api: cpu: add routines > for obtaining socket ids > > > > We'll be discussing this RFC on Monday's ARCH call. I'm certainly open to > adding this if it's deemed useful. I do note that DPDK defines SOCK_ID_ANY > to be -1 so I'd like to minimize translation overhead for odp-dpdk in this > area as well. > > > > On Fri, Jan 29, 2016 at 10:27 AM, Zoltan Kiss <zoltan.kiss@linaro.org> > wrote: > > > > On 29/01/16 02:44, Bill Fischofer wrote: > > Add odp_cpu_socket_id() and odp_cpu_socket_id_cpu() routines > > Signed-off-by: Bill Fischofer <bill.fischofer@linaro.org> > --- > include/odp/api/cpu.h | 23 +++++++++++++++++++++++ > 1 file changed, 23 insertions(+) > > diff --git a/include/odp/api/cpu.h b/include/odp/api/cpu.h > index 4cbaf58..fe74825 100644 > --- a/include/odp/api/cpu.h > +++ b/include/odp/api/cpu.h > @@ -36,6 +36,29 @@ extern "C" { > int odp_cpu_id(void); > > /** > + * CPU socket id > + * > + * Returns the socket id associated with the calling CPU on NUMA systems. > + * Socket ID numbering is system specific. > + * > + * @return Socket ID of the calling CPU > + * @retval ODP_SOCKET_ID_ANY If the caller is not running on a NUMA > system. > + */ > +uint32_t odp_cpu_socket_id(void); > + > +/** > + * CPU socket id of designated CPU > + * > + * Returns the socket id associated with a specified CPU on NUMA systems. > + * Socket ID numbering is system specific. > + * > + * @return Socket ID of the designated CPU > + * @retval ODP_SOCKET_ID_ANY If the specified CPU is unknown or caller is > + * not running on a NUMA system. > > > > Maybe worth to differentiate between the two scenario with different > return value? To query for an invalid CPU sounds like a serious problem in > the application while the other is part of normal operation. > > > > + */ > +uint32_t odp_cpu_socket_id_cpu(int cpu_id); > + > +/** > * CPU count > * > * Report the number of CPU's available to this ODP program. > > >
diff --git a/include/odp/api/cpu.h b/include/odp/api/cpu.h index 4cbaf58..fe74825 100644 --- a/include/odp/api/cpu.h +++ b/include/odp/api/cpu.h @@ -36,6 +36,29 @@ extern "C" { int odp_cpu_id(void); /** + * CPU socket id + * + * Returns the socket id associated with the calling CPU on NUMA systems. + * Socket ID numbering is system specific. + * + * @return Socket ID of the calling CPU + * @retval ODP_SOCKET_ID_ANY If the caller is not running on a NUMA system. + */ +uint32_t odp_cpu_socket_id(void); + +/** + * CPU socket id of designated CPU + * + * Returns the socket id associated with a specified CPU on NUMA systems. + * Socket ID numbering is system specific. + * + * @return Socket ID of the designated CPU + * @retval ODP_SOCKET_ID_ANY If the specified CPU is unknown or caller is + * not running on a NUMA system. + */ +uint32_t odp_cpu_socket_id_cpu(int cpu_id); + +/** * CPU count * * Report the number of CPU's available to this ODP program.
Add odp_cpu_socket_id() and odp_cpu_socket_id_cpu() routines Signed-off-by: Bill Fischofer <bill.fischofer@linaro.org> --- include/odp/api/cpu.h | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+)