| Message ID | 1469494176-5436-1-git-send-email-bill.fischofer@linaro.org |
|---|---|
| State | New |
| Headers | show |
On 07/25 19:49:36, Bill Fischofer wrote: > Add ODP_THREAD_SERVICE as a third type of ODP thread along with related > cpumask APIs that allow cores to be designated for running service threads. > Service threads are intended to be used internally by ODP implementations > or library functions such as OpenFastPath rather than ODP applications. > > Signed-off-by: Bill Fischofer <bill.fischofer@linaro.org> > --- > include/odp/api/spec/cpumask.h | 13 +++++++++++++ > include/odp/api/spec/init.h | 17 +++++++++++++++++ > include/odp/api/spec/thread.h | 15 ++++++++++++++- > 3 files changed, 44 insertions(+), 1 deletion(-) > > diff --git a/include/odp/api/spec/cpumask.h b/include/odp/api/spec/cpumask.h > index 22d8e8f..dca0255 100644 > --- a/include/odp/api/spec/cpumask.h > +++ b/include/odp/api/spec/cpumask.h > @@ -233,6 +233,19 @@ int odp_cpumask_default_worker(odp_cpumask_t *mask, int num); > int odp_cpumask_default_control(odp_cpumask_t *mask, int num); > > /** > + * Default cpumask for service threads > + * > + * Initializes cpumask with CPUs available for ODP service threads. Sets up > + * to 'num' CPUs and returns the count actually set. Use zero for all available > + * CPUs. > + * > + * @param[out] mask CPU mask to initialize > + * @param num Number of service threads, zero for all available CPUs > + * @return Actual number of CPUs used to create the mask > + */ > +int odp_cpumask_default_service(odp_cpumask_t *mask, int num); Here, the application is telling the implementation (up to) how many cores it may use to run its background threads on. And, in return, the implementation will provide a core mask back to the application saying that it will take over those cores to run its background threads. Is this understanding close? > +/** > * Report all the available CPUs > * > * All the available CPUs include both worker CPUs and control CPUs > diff --git a/include/odp/api/spec/init.h b/include/odp/api/spec/init.h > index 154cdf8..a39bff2 100644 > --- a/include/odp/api/spec/init.h > +++ b/include/odp/api/spec/init.h > @@ -123,6 +123,12 @@ typedef struct odp_init_t { > Valid range is from 0 to platform specific maximum. Set both > num_worker and num_control to zero for default number of threads. */ > int num_control; > + /** Maximum number of service threads available to the ODP > + implementation. Valid range is from 0 to platform specific > + maximum. Set to 0 (recommended) to accept default number of > + threads. ODP init may fail or certain functions may not be > + available if this number is set too low. */ > + int num_service; > /** Pointer to bit mask mapping CPUs available to this ODP instance > for running worker threads. > Initialize to a NULL pointer to use default CPU mapping. > @@ -149,6 +155,17 @@ typedef struct odp_init_t { > worker and control masks do not overlap. > */ > const odp_cpumask_t *control_cpus; > + /** Pointer to bit mask mapping CPUs available to this ODP instance > + for running ODP service threads. > + Initialize to a NULL pointer to use default CPU mapping. > + When the mask is defined, odp_cpumask_default_service() > + uses it instead of returning a default mask. > + Applications code should not access this cpumask directly. > + Valid range of CPUs and optimal CPU selection > + are platform specific, but generally it is recommended that > + service masks do not overlap either worker or control masks. > + */ > + const odp_cpumask_t *service_cpus; Is this the mask that was obtained from the implementation via odp_cpumask_default_service(...)? If so, does it need to be passed back to the implementation? > /** Replacement for the default log fn */ > odp_log_func_t log_fn; > /** Replacement for the default abort fn */ > diff --git a/include/odp/api/spec/thread.h b/include/odp/api/spec/thread.h > index 689ba59..0f2963f 100644 > --- a/include/odp/api/spec/thread.h > +++ b/include/odp/api/spec/thread.h > @@ -54,7 +54,20 @@ typedef enum odp_thread_type_e { > * use system calls, share the CPU with other threads and be interrupt > * driven. > */ > - ODP_THREAD_CONTROL > + ODP_THREAD_CONTROL, > + > + /** > + * Service thread > + * > + * Service threads are used internally by ODP implementations and/or > + * libraries. They may process packets or perform other housekeeping > + * functions. These threads may prefer to be bound to dedicated cores > + * but may also share cores depending on resource availabilty. ODP > + * applications should not use service threads but should be aware > + * of their existence since they may subtract from the total number > + * of cores available to the ODP application. The service thread solution is focused on reserving a dedicated group of cores for the implementation to run its background threads on. Is a third thread type required to negotiate a core mask during init time? > + */ > + ODP_THREAD_SERVICE, > } odp_thread_type_t; > > > -- > 2.7.4 >
Regards, Bala On 26 July 2016 at 09:18, Brian Brooks <brian.brooks@linaro.org> wrote: > On 07/25 19:49:36, Bill Fischofer wrote: >> Add ODP_THREAD_SERVICE as a third type of ODP thread along with related >> cpumask APIs that allow cores to be designated for running service threads. >> Service threads are intended to be used internally by ODP implementations >> or library functions such as OpenFastPath rather than ODP applications. >> >> Signed-off-by: Bill Fischofer <bill.fischofer@linaro.org> >> --- >> include/odp/api/spec/cpumask.h | 13 +++++++++++++ >> include/odp/api/spec/init.h | 17 +++++++++++++++++ >> include/odp/api/spec/thread.h | 15 ++++++++++++++- >> 3 files changed, 44 insertions(+), 1 deletion(-) >> >> diff --git a/include/odp/api/spec/cpumask.h b/include/odp/api/spec/cpumask.h >> index 22d8e8f..dca0255 100644 >> --- a/include/odp/api/spec/cpumask.h >> +++ b/include/odp/api/spec/cpumask.h >> @@ -233,6 +233,19 @@ int odp_cpumask_default_worker(odp_cpumask_t *mask, int num); >> int odp_cpumask_default_control(odp_cpumask_t *mask, int num); >> >> /** >> + * Default cpumask for service threads >> + * >> + * Initializes cpumask with CPUs available for ODP service threads. Sets up >> + * to 'num' CPUs and returns the count actually set. Use zero for all available >> + * CPUs. >> + * >> + * @param[out] mask CPU mask to initialize >> + * @param num Number of service threads, zero for all available CPUs >> + * @return Actual number of CPUs used to create the mask >> + */ >> +int odp_cpumask_default_service(odp_cpumask_t *mask, int num); > > Here, the application is telling the implementation (up to) how many cores it > may use to run its background threads on. And, in return, the implementation > will provide a core mask back to the application saying that it will take over > those cores to run its background threads. Is this understanding close? My opinion is that in case of service core the ODP implementation will just inform the application as to which are the cores which have been reserved by this implementation to run its service threads. The application need not send any feedback back to the implementation with respect to service cores. Regards, Bala > >> +/** >> * Report all the available CPUs >> * >> * All the available CPUs include both worker CPUs and control CPUs >> diff --git a/include/odp/api/spec/init.h b/include/odp/api/spec/init.h >> index 154cdf8..a39bff2 100644 >> --- a/include/odp/api/spec/init.h >> +++ b/include/odp/api/spec/init.h >> @@ -123,6 +123,12 @@ typedef struct odp_init_t { >> Valid range is from 0 to platform specific maximum. Set both >> num_worker and num_control to zero for default number of threads. */ >> int num_control; >> + /** Maximum number of service threads available to the ODP >> + implementation. Valid range is from 0 to platform specific >> + maximum. Set to 0 (recommended) to accept default number of >> + threads. ODP init may fail or certain functions may not be >> + available if this number is set too low. */ >> + int num_service; >> /** Pointer to bit mask mapping CPUs available to this ODP instance >> for running worker threads. >> Initialize to a NULL pointer to use default CPU mapping. >> @@ -149,6 +155,17 @@ typedef struct odp_init_t { >> worker and control masks do not overlap. >> */ >> const odp_cpumask_t *control_cpus; >> + /** Pointer to bit mask mapping CPUs available to this ODP instance >> + for running ODP service threads. >> + Initialize to a NULL pointer to use default CPU mapping. >> + When the mask is defined, odp_cpumask_default_service() >> + uses it instead of returning a default mask. >> + Applications code should not access this cpumask directly. >> + Valid range of CPUs and optimal CPU selection >> + are platform specific, but generally it is recommended that >> + service masks do not overlap either worker or control masks. >> + */ >> + const odp_cpumask_t *service_cpus; > > Is this the mask that was obtained from the implementation via > odp_cpumask_default_service(...)? If so, does it need to be passed back > to the implementation? > >> /** Replacement for the default log fn */ >> odp_log_func_t log_fn; >> /** Replacement for the default abort fn */ >> diff --git a/include/odp/api/spec/thread.h b/include/odp/api/spec/thread.h >> index 689ba59..0f2963f 100644 >> --- a/include/odp/api/spec/thread.h >> +++ b/include/odp/api/spec/thread.h >> @@ -54,7 +54,20 @@ typedef enum odp_thread_type_e { >> * use system calls, share the CPU with other threads and be interrupt >> * driven. >> */ >> - ODP_THREAD_CONTROL >> + ODP_THREAD_CONTROL, >> + >> + /** >> + * Service thread >> + * >> + * Service threads are used internally by ODP implementations and/or >> + * libraries. They may process packets or perform other housekeeping >> + * functions. These threads may prefer to be bound to dedicated cores >> + * but may also share cores depending on resource availabilty. ODP >> + * applications should not use service threads but should be aware >> + * of their existence since they may subtract from the total number >> + * of cores available to the ODP application. > > The service thread solution is focused on reserving a dedicated group of > cores for the implementation to run its background threads on. Is a third > thread type required to negotiate a core mask during init time? > >> + */ >> + ODP_THREAD_SERVICE, >> } odp_thread_type_t; >> >> >> -- >> 2.7.4 >>
control thread mask and worker thread mask should give a hint to implementation which cores might be used. How to spread load inside that 2 masks it's the deal of implementation. I think it's a little bit overload to create one more mask for service threads.... Maxim. On 07/26/16 07:50, Bala Manoharan wrote: > Regards, > Bala > > > On 26 July 2016 at 09:18, Brian Brooks <brian.brooks@linaro.org> wrote: >> On 07/25 19:49:36, Bill Fischofer wrote: >>> Add ODP_THREAD_SERVICE as a third type of ODP thread along with related >>> cpumask APIs that allow cores to be designated for running service threads. >>> Service threads are intended to be used internally by ODP implementations >>> or library functions such as OpenFastPath rather than ODP applications. >>> >>> Signed-off-by: Bill Fischofer <bill.fischofer@linaro.org> >>> --- >>> include/odp/api/spec/cpumask.h | 13 +++++++++++++ >>> include/odp/api/spec/init.h | 17 +++++++++++++++++ >>> include/odp/api/spec/thread.h | 15 ++++++++++++++- >>> 3 files changed, 44 insertions(+), 1 deletion(-) >>> >>> diff --git a/include/odp/api/spec/cpumask.h b/include/odp/api/spec/cpumask.h >>> index 22d8e8f..dca0255 100644 >>> --- a/include/odp/api/spec/cpumask.h >>> +++ b/include/odp/api/spec/cpumask.h >>> @@ -233,6 +233,19 @@ int odp_cpumask_default_worker(odp_cpumask_t *mask, int num); >>> int odp_cpumask_default_control(odp_cpumask_t *mask, int num); >>> >>> /** >>> + * Default cpumask for service threads >>> + * >>> + * Initializes cpumask with CPUs available for ODP service threads. Sets up >>> + * to 'num' CPUs and returns the count actually set. Use zero for all available >>> + * CPUs. >>> + * >>> + * @param[out] mask CPU mask to initialize >>> + * @param num Number of service threads, zero for all available CPUs >>> + * @return Actual number of CPUs used to create the mask >>> + */ >>> +int odp_cpumask_default_service(odp_cpumask_t *mask, int num); >> Here, the application is telling the implementation (up to) how many cores it >> may use to run its background threads on. And, in return, the implementation >> will provide a core mask back to the application saying that it will take over >> those cores to run its background threads. Is this understanding close? > My opinion is that in case of service core the ODP implementation will > just inform the application as to which are the cores which have been > reserved by this implementation to run its service threads. The > application need not send any feedback back to the implementation with > respect to service cores. > > Regards, > Bala >>> +/** >>> * Report all the available CPUs >>> * >>> * All the available CPUs include both worker CPUs and control CPUs >>> diff --git a/include/odp/api/spec/init.h b/include/odp/api/spec/init.h >>> index 154cdf8..a39bff2 100644 >>> --- a/include/odp/api/spec/init.h >>> +++ b/include/odp/api/spec/init.h >>> @@ -123,6 +123,12 @@ typedef struct odp_init_t { >>> Valid range is from 0 to platform specific maximum. Set both >>> num_worker and num_control to zero for default number of threads. */ >>> int num_control; >>> + /** Maximum number of service threads available to the ODP >>> + implementation. Valid range is from 0 to platform specific >>> + maximum. Set to 0 (recommended) to accept default number of >>> + threads. ODP init may fail or certain functions may not be >>> + available if this number is set too low. */ >>> + int num_service; >>> /** Pointer to bit mask mapping CPUs available to this ODP instance >>> for running worker threads. >>> Initialize to a NULL pointer to use default CPU mapping. >>> @@ -149,6 +155,17 @@ typedef struct odp_init_t { >>> worker and control masks do not overlap. >>> */ >>> const odp_cpumask_t *control_cpus; >>> + /** Pointer to bit mask mapping CPUs available to this ODP instance >>> + for running ODP service threads. >>> + Initialize to a NULL pointer to use default CPU mapping. >>> + When the mask is defined, odp_cpumask_default_service() >>> + uses it instead of returning a default mask. >>> + Applications code should not access this cpumask directly. >>> + Valid range of CPUs and optimal CPU selection >>> + are platform specific, but generally it is recommended that >>> + service masks do not overlap either worker or control masks. >>> + */ >>> + const odp_cpumask_t *service_cpus; >> Is this the mask that was obtained from the implementation via >> odp_cpumask_default_service(...)? If so, does it need to be passed back >> to the implementation? >> >>> /** Replacement for the default log fn */ >>> odp_log_func_t log_fn; >>> /** Replacement for the default abort fn */ >>> diff --git a/include/odp/api/spec/thread.h b/include/odp/api/spec/thread.h >>> index 689ba59..0f2963f 100644 >>> --- a/include/odp/api/spec/thread.h >>> +++ b/include/odp/api/spec/thread.h >>> @@ -54,7 +54,20 @@ typedef enum odp_thread_type_e { >>> * use system calls, share the CPU with other threads and be interrupt >>> * driven. >>> */ >>> - ODP_THREAD_CONTROL >>> + ODP_THREAD_CONTROL, >>> + >>> + /** >>> + * Service thread >>> + * >>> + * Service threads are used internally by ODP implementations and/or >>> + * libraries. They may process packets or perform other housekeeping >>> + * functions. These threads may prefer to be bound to dedicated cores >>> + * but may also share cores depending on resource availabilty. ODP >>> + * applications should not use service threads but should be aware >>> + * of their existence since they may subtract from the total number >>> + * of cores available to the ODP application. >> The service thread solution is focused on reserving a dedicated group of >> cores for the implementation to run its background threads on. Is a third >> thread type required to negotiate a core mask during init time? >> >>> + */ >>> + ODP_THREAD_SERVICE, >>> } odp_thread_type_t; >>> >>> >>> -- >>> 2.7.4 >>>
diff --git a/include/odp/api/spec/cpumask.h b/include/odp/api/spec/cpumask.h index 22d8e8f..dca0255 100644 --- a/include/odp/api/spec/cpumask.h +++ b/include/odp/api/spec/cpumask.h @@ -233,6 +233,19 @@ int odp_cpumask_default_worker(odp_cpumask_t *mask, int num); int odp_cpumask_default_control(odp_cpumask_t *mask, int num); /** + * Default cpumask for service threads + * + * Initializes cpumask with CPUs available for ODP service threads. Sets up + * to 'num' CPUs and returns the count actually set. Use zero for all available + * CPUs. + * + * @param[out] mask CPU mask to initialize + * @param num Number of service threads, zero for all available CPUs + * @return Actual number of CPUs used to create the mask + */ +int odp_cpumask_default_service(odp_cpumask_t *mask, int num); + +/** * Report all the available CPUs * * All the available CPUs include both worker CPUs and control CPUs diff --git a/include/odp/api/spec/init.h b/include/odp/api/spec/init.h index 154cdf8..a39bff2 100644 --- a/include/odp/api/spec/init.h +++ b/include/odp/api/spec/init.h @@ -123,6 +123,12 @@ typedef struct odp_init_t { Valid range is from 0 to platform specific maximum. Set both num_worker and num_control to zero for default number of threads. */ int num_control; + /** Maximum number of service threads available to the ODP + implementation. Valid range is from 0 to platform specific + maximum. Set to 0 (recommended) to accept default number of + threads. ODP init may fail or certain functions may not be + available if this number is set too low. */ + int num_service; /** Pointer to bit mask mapping CPUs available to this ODP instance for running worker threads. Initialize to a NULL pointer to use default CPU mapping. @@ -149,6 +155,17 @@ typedef struct odp_init_t { worker and control masks do not overlap. */ const odp_cpumask_t *control_cpus; + /** Pointer to bit mask mapping CPUs available to this ODP instance + for running ODP service threads. + Initialize to a NULL pointer to use default CPU mapping. + When the mask is defined, odp_cpumask_default_service() + uses it instead of returning a default mask. + Applications code should not access this cpumask directly. + Valid range of CPUs and optimal CPU selection + are platform specific, but generally it is recommended that + service masks do not overlap either worker or control masks. + */ + const odp_cpumask_t *service_cpus; /** Replacement for the default log fn */ odp_log_func_t log_fn; /** Replacement for the default abort fn */ diff --git a/include/odp/api/spec/thread.h b/include/odp/api/spec/thread.h index 689ba59..0f2963f 100644 --- a/include/odp/api/spec/thread.h +++ b/include/odp/api/spec/thread.h @@ -54,7 +54,20 @@ typedef enum odp_thread_type_e { * use system calls, share the CPU with other threads and be interrupt * driven. */ - ODP_THREAD_CONTROL + ODP_THREAD_CONTROL, + + /** + * Service thread + * + * Service threads are used internally by ODP implementations and/or + * libraries. They may process packets or perform other housekeeping + * functions. These threads may prefer to be bound to dedicated cores + * but may also share cores depending on resource availabilty. ODP + * applications should not use service threads but should be aware + * of their existence since they may subtract from the total number + * of cores available to the ODP application. + */ + ODP_THREAD_SERVICE, } odp_thread_type_t;
Add ODP_THREAD_SERVICE as a third type of ODP thread along with related cpumask APIs that allow cores to be designated for running service threads. Service threads are intended to be used internally by ODP implementations or library functions such as OpenFastPath rather than ODP applications. Signed-off-by: Bill Fischofer <bill.fischofer@linaro.org> --- include/odp/api/spec/cpumask.h | 13 +++++++++++++ include/odp/api/spec/init.h | 17 +++++++++++++++++ include/odp/api/spec/thread.h | 15 ++++++++++++++- 3 files changed, 44 insertions(+), 1 deletion(-) -- 2.7.4