doc: userguide: add portability and usage info for odp time apis

Message ID 20170213224753.15361-1-bill.fischofer@linaro.org
State New
Headers show

Commit Message

Bill Fischofer Feb. 13, 2017, 10:47 p.m.
Clarify and expand on portability and performance considerations
regarding the use of the ODP time APIs in fulfillment of JIRA
issue https://projects.linaro.org/browse/ODP-575

Signed-off-by: Bill Fischofer <bill.fischofer@linaro.org>

---
 doc/users-guide/users-guide.adoc | 32 +++++++++++++++++++++-----------
 1 file changed, 21 insertions(+), 11 deletions(-)

-- 
2.11.0.295.gd7dffce

Comments

Brian Brooks Feb. 16, 2017, 9:20 p.m. | #1
On Mon, Feb 13, 2017 at 4:47 PM, Bill Fischofer
<bill.fischofer@linaro.org> wrote:
> Clarify and expand on portability and performance considerations

> regarding the use of the ODP time APIs in fulfillment of JIRA

> issue https://projects.linaro.org/browse/ODP-575

>

> Signed-off-by: Bill Fischofer <bill.fischofer@linaro.org>

> ---

>  doc/users-guide/users-guide.adoc | 32 +++++++++++++++++++++-----------

>  1 file changed, 21 insertions(+), 11 deletions(-)

>

> diff --git a/doc/users-guide/users-guide.adoc b/doc/users-guide/users-guide.adoc

> index 41c57d1c..05bade8c 100755

> --- a/doc/users-guide/users-guide.adoc

> +++ b/doc/users-guide/users-guide.adoc

> @@ -362,31 +362,41 @@ PktIOs are represented by handles of abstract type `odp_pktio_t`.

>

>  === Time

>  The time API is used to measure time intervals and track time flow of an

> -application and presents a convenient way to get access to a time source.

> -The time API consists of two main parts: local time API and global time API.

> +application and presents a convenient way to get access to an

> +implementation-defined time source. The time API consists of two main parts:

> +local time API and global time API.

>

>  ==== Local time

> -The local time API is designed to be used within one thread and can be faster

> -than the global time API. The local time API cannot be used between threads as

> -time consistency is not guaranteed, and in some cases that's enough.

> -So, local time stamps are local to the calling thread and must not be shared

> -with other threads. Current local time can be read with `odp_time_local()`.

> +The local time API is designed to be used within one thread


How do I convert a local timestamp into a global one?

What happens if the thread migrates to a different core? Thinking cloud...

What happens if a timestamp taken on one thread is used by another thread?
Thinking multithreaded applications that use shared memory synchronization...

> and obtaining

> +local time may be more efficient in some implementations than global

> +time.


To elicit a response... I claim this is *not* true on ARMv8 and modern
x86 chips.  So, no need for global vs. local notion of time.

> Local time stamps are local to the calling thread and should not be

> +shared with other threads, as local time is not guaranteed to be consistent

> +between threads. Current local time can be read with `odp_time_local()`.

>

>  ==== Global time

>  The global time API is designed to be used for tracking time between threads.

> -So, global time stamps can be shared between threads. Current global time can

> -be read with `odp_time_global()`.

> +So, global time stamps may safely be shared between threads. Current global

> +time can be read with `odp_time_global()`.

>

> -Both, local and global time is not wrapped during the application life cycle.

> +Both local and global time is not wrapped during the application life cycle.

>  The time API includes functions to operate with time, such as `odp_time_diff()`,

>  `odp_time_sum()`, `odp_time_cmp()`, conversion functions like

>  `odp_time_to_ns()`, `odp_time_local_from_ns()`, `odp_time_global_from_ns()`.

>  To get rate of time source `odp_time_local_res()`, `odp_time_global_res()`

>  are used. To wait, `odp_time_wait_ns()` and `odp_time_wait_until()` are used,

> -during witch a thread potentially busy loop the entire wait time.

> +during which a thread potentially busy loops the entire wait time.

>

>  The `odp_time_t` opaque type represents local or global timestamps.

>

> +==== Portability Considerations

> +The ODP Time APIs are designed to permit high-precision relative time

> +measurement within an ODP application. No attempt is made to correlate an

> +`odp_time_t` object with "wall time" or any other external time reference.


I think Time should represent "wall time".

> +As defined by the ODP specification, `odp_time_t` values are required to

> +be unique over a span of at least 10 years.


A 32 bit 1 GHz counter wraps in 4.29s.
A 32 bit 12.5 Hz counter wraps in 10 yrs. This is 80ms granularity. :(

A 64 bit 1 GHz counter wraps in 584 yrs.
A 64 bit 3 GHz counter wraps in 194 yrs.

ARMv8 Generic Timer specifies a min width of 56 bits but also min roll-over
of 40 yrs.  If hardware counter is 56 bits, the counter frequency can be no
faster than ~54 MHz (19ns granularity) to achieve >10 yr wrap.  A wider
counter allows for faster frequency, but 19ns granularity is quite usable.

> Most implementations will choose

> +to implement time values using 64-bit values,


I think odp_time_t needs to be larger than 32 bits; no other way to maintain
~10yr wrap at sensible granularity.  That means a 64 bit in C API.  It can be
defined at the universal API layer to be a uint64_t for all.  This type does
not need to be defined differently for each OS or ISA or SoC combo. That is
the point of an API and abstraction layer.

> whose wrap times exceed 500

> +years, making wrapping concerns not relevant to ODP applications.

> +

>  === Timer

>  Timers are how ODP applications measure and respond to the passage of time.

>  Timers are drawn from specialized pools called timer pools that have their

> --

> 2.11.0.295.gd7dffce

>
Bill Fischofer Feb. 16, 2017, 9:56 p.m. | #2
On Thu, Feb 16, 2017 at 3:20 PM, Brian Brooks <brian.brooks@linaro.org> wrote:
> On Mon, Feb 13, 2017 at 4:47 PM, Bill Fischofer

> <bill.fischofer@linaro.org> wrote:

>> Clarify and expand on portability and performance considerations

>> regarding the use of the ODP time APIs in fulfillment of JIRA

>> issue https://projects.linaro.org/browse/ODP-575

>>

>> Signed-off-by: Bill Fischofer <bill.fischofer@linaro.org>

>> ---

>>  doc/users-guide/users-guide.adoc | 32 +++++++++++++++++++++-----------

>>  1 file changed, 21 insertions(+), 11 deletions(-)

>>

>> diff --git a/doc/users-guide/users-guide.adoc b/doc/users-guide/users-guide.adoc

>> index 41c57d1c..05bade8c 100755

>> --- a/doc/users-guide/users-guide.adoc

>> +++ b/doc/users-guide/users-guide.adoc

>> @@ -362,31 +362,41 @@ PktIOs are represented by handles of abstract type `odp_pktio_t`.

>>

>>  === Time

>>  The time API is used to measure time intervals and track time flow of an

>> -application and presents a convenient way to get access to a time source.

>> -The time API consists of two main parts: local time API and global time API.

>> +application and presents a convenient way to get access to an

>> +implementation-defined time source. The time API consists of two main parts:

>> +local time API and global time API.

>>

>>  ==== Local time

>> -The local time API is designed to be used within one thread and can be faster

>> -than the global time API. The local time API cannot be used between threads as

>> -time consistency is not guaranteed, and in some cases that's enough.

>> -So, local time stamps are local to the calling thread and must not be shared

>> -with other threads. Current local time can be read with `odp_time_local()`.

>> +The local time API is designed to be used within one thread

>

> How do I convert a local timestamp into a global one?


You'd have to go through the ns conversion routines:

local->global == odp_time_global_from_ns(odp_time_to_ns(local_time));

global->local == odp_time_local_from_ns(odp_time_to_ns(global_time));

Though I'm not sure what the use case would be for such conversions.

>

> What happens if the thread migrates to a different core? Thinking cloud...

>

> What happens if a timestamp taken on one thread is used by another thread?

> Thinking multithreaded applications that use shared memory synchronization...


Presumably odp_time_local() should only be used on cores that are
isolated to a specific CPU. "Floating" threads should use
odp_time_global(). However, if you're being preempted and timesliced
onto another core, it's not clear that odp_time_diff() is doing to be
meaningful in such cases no matter what time you're using.

>

>> and obtaining

>> +local time may be more efficient in some implementations than global

>> +time.

>

> To elicit a response... I claim this is *not* true on ARMv8 and modern

> x86 chips.  So, no need for global vs. local notion of time.


That's why this is a "may". I agree that the use case for the concept
of local time is weak, but presumably this is a bit of "NUMA
insurance" in the API?

>

>> Local time stamps are local to the calling thread and should not be

>> +shared with other threads, as local time is not guaranteed to be consistent

>> +between threads. Current local time can be read with `odp_time_local()`.

>>

>>  ==== Global time

>>  The global time API is designed to be used for tracking time between threads.

>> -So, global time stamps can be shared between threads. Current global time can

>> -be read with `odp_time_global()`.

>> +So, global time stamps may safely be shared between threads. Current global

>> +time can be read with `odp_time_global()`.

>>

>> -Both, local and global time is not wrapped during the application life cycle.

>> +Both local and global time is not wrapped during the application life cycle.

>>  The time API includes functions to operate with time, such as `odp_time_diff()`,

>>  `odp_time_sum()`, `odp_time_cmp()`, conversion functions like

>>  `odp_time_to_ns()`, `odp_time_local_from_ns()`, `odp_time_global_from_ns()`.

>>  To get rate of time source `odp_time_local_res()`, `odp_time_global_res()`

>>  are used. To wait, `odp_time_wait_ns()` and `odp_time_wait_until()` are used,

>> -during witch a thread potentially busy loop the entire wait time.

>> +during which a thread potentially busy loops the entire wait time.

>>

>>  The `odp_time_t` opaque type represents local or global timestamps.

>>

>> +==== Portability Considerations

>> +The ODP Time APIs are designed to permit high-precision relative time

>> +measurement within an ODP application. No attempt is made to correlate an

>> +`odp_time_t` object with "wall time" or any other external time reference.

>

> I think Time should represent "wall time".


Wall time refers to a specific epoch as well as flow rate. These
values cannot be converted into a calendar date/time, hence this
disclaimer.

>

>> +As defined by the ODP specification, `odp_time_t` values are required to

>> +be unique over a span of at least 10 years.

>

> A 32 bit 1 GHz counter wraps in 4.29s.

> A 32 bit 12.5 Hz counter wraps in 10 yrs. This is 80ms granularity. :(

>

> A 64 bit 1 GHz counter wraps in 584 yrs.

> A 64 bit 3 GHz counter wraps in 194 yrs.

>

> ARMv8 Generic Timer specifies a min width of 56 bits but also min roll-over

> of 40 yrs.  If hardware counter is 56 bits, the counter frequency can be no

> faster than ~54 MHz (19ns granularity) to achieve >10 yr wrap.  A wider

> counter allows for faster frequency, but 19ns granularity is quite usable.


These are all implementation choices. ODP does not specify this, only
the minimum wrap time to provide assurance that applications need not
concern themselves with time wraps.

>

>> Most implementations will choose

>> +to implement time values using 64-bit values,

>

> I think odp_time_t needs to be larger than 32 bits; no other way to maintain

> ~10yr wrap at sensible granularity.  That means a 64 bit in C API.  It can be

> defined at the universal API layer to be a uint64_t for all.  This type does

> not need to be defined differently for each OS or ISA or SoC combo. That is

> the point of an API and abstraction layer.


Most probably true, but again the ODP API does not specify this. A
defined ODP ABI may well do exactly this, however.

>

>> whose wrap times exceed 500

>> +years, making wrapping concerns not relevant to ODP applications.

>> +

>>  === Timer

>>  Timers are how ODP applications measure and respond to the passage of time.

>>  Timers are drawn from specialized pools called timer pools that have their

>> --

>> 2.11.0.295.gd7dffce

>>
Maxim Uvarov Feb. 17, 2017, 10 a.m. | #3
On 17 February 2017 at 00:56, Bill Fischofer <bill.fischofer@linaro.org>
wrote:

> On Thu, Feb 16, 2017 at 3:20 PM, Brian Brooks <brian.brooks@linaro.org>

> wrote:

> > On Mon, Feb 13, 2017 at 4:47 PM, Bill Fischofer

> > <bill.fischofer@linaro.org> wrote:

> >> Clarify and expand on portability and performance considerations

> >> regarding the use of the ODP time APIs in fulfillment of JIRA

> >> issue https://projects.linaro.org/browse/ODP-575

> >>

> >> Signed-off-by: Bill Fischofer <bill.fischofer@linaro.org>

> >> ---

> >>  doc/users-guide/users-guide.adoc | 32 +++++++++++++++++++++-----------

> >>  1 file changed, 21 insertions(+), 11 deletions(-)

> >>

> >> diff --git a/doc/users-guide/users-guide.adoc

> b/doc/users-guide/users-guide.adoc

> >> index 41c57d1c..05bade8c 100755

> >> --- a/doc/users-guide/users-guide.adoc

> >> +++ b/doc/users-guide/users-guide.adoc

> >> @@ -362,31 +362,41 @@ PktIOs are represented by handles of abstract

> type `odp_pktio_t`.

> >>

> >>  === Time

> >>  The time API is used to measure time intervals and track time flow of

> an

> >> -application and presents a convenient way to get access to a time

> source.

> >> -The time API consists of two main parts: local time API and global

> time API.

> >> +application and presents a convenient way to get access to an

> >> +implementation-defined time source. The time API consists of two main

> parts:

> >> +local time API and global time API.

> >>

> >>  ==== Local time

> >> -The local time API is designed to be used within one thread and can be

> faster

> >> -than the global time API. The local time API cannot be used between

> threads as

> >> -time consistency is not guaranteed, and in some cases that's enough.

> >> -So, local time stamps are local to the calling thread and must not be

> shared

> >> -with other threads. Current local time can be read with

> `odp_time_local()`.

> >> +The local time API is designed to be used within one thread

> >

> > How do I convert a local timestamp into a global one?

>

> You'd have to go through the ns conversion routines:

>

> local->global == odp_time_global_from_ns(odp_time_to_ns(local_time));

>

> global->local == odp_time_local_from_ns(odp_time_to_ns(global_time));

>

> Though I'm not sure what the use case would be for such conversions.

>

> >

> > What happens if the thread migrates to a different core? Thinking

> cloud...

> >

> > What happens if a timestamp taken on one thread is used by another

> thread?

> > Thinking multithreaded applications that use shared memory

> synchronization...

>

> Presumably odp_time_local() should only be used on cores that are

> isolated to a specific CPU. "Floating" threads should use

> odp_time_global(). However, if you're being preempted and timesliced

> onto another core, it's not clear that odp_time_diff() is doing to be

> meaningful in such cases no matter what time you're using.

>

> >

> >> and obtaining

> >> +local time may be more efficient in some implementations than global

> >> +time.

> >

> > To elicit a response... I claim this is *not* true on ARMv8 and modern

> > x86 chips.  So, no need for global vs. local notion of time.

>

> That's why this is a "may". I agree that the use case for the concept

> of local time is weak, but presumably this is a bit of "NUMA

> insurance" in the API?

>

> >

> >> Local time stamps are local to the calling thread and should not be

> >> +shared with other threads, as local time is not guaranteed to be

> consistent

> >> +between threads. Current local time can be read with

> `odp_time_local()`.

> >>

> >>  ==== Global time

> >>  The global time API is designed to be used for tracking time between

> threads.

> >> -So, global time stamps can be shared between threads. Current global

> time can

> >> -be read with `odp_time_global()`.

> >> +So, global time stamps may safely be shared between threads. Current

> global

> >> +time can be read with `odp_time_global()`.

> >>

> >> -Both, local and global time is not wrapped during the application life

> cycle.

> >> +Both local and global time is not wrapped during the application life

> cycle.

> >>  The time API includes functions to operate with time, such as

> `odp_time_diff()`,

> >>  `odp_time_sum()`, `odp_time_cmp()`, conversion functions like

> >>  `odp_time_to_ns()`, `odp_time_local_from_ns()`,

> `odp_time_global_from_ns()`.

> >>  To get rate of time source `odp_time_local_res()`,

> `odp_time_global_res()`

> >>  are used. To wait, `odp_time_wait_ns()` and `odp_time_wait_until()`

> are used,

> >> -during witch a thread potentially busy loop the entire wait time.

> >> +during which a thread potentially busy loops the entire wait time.

> >>

> >>  The `odp_time_t` opaque type represents local or global timestamps.

> >>

> >> +==== Portability Considerations

> >> +The ODP Time APIs are designed to permit high-precision relative time

> >> +measurement within an ODP application. No attempt is made to correlate

> an

> >> +`odp_time_t` object with "wall time" or any other external time

> reference.

> >

> > I think Time should represent "wall time".

>

> Wall time refers to a specific epoch as well as flow rate. These

> values cannot be converted into a calendar date/time, hence this

> disclaimer.

>

> >

> >> +As defined by the ODP specification, `odp_time_t` values are required

> to

> >> +be unique over a span of at least 10 years.

> >

> > A 32 bit 1 GHz counter wraps in 4.29s.

> > A 32 bit 12.5 Hz counter wraps in 10 yrs. This is 80ms granularity. :(

> >

> > A 64 bit 1 GHz counter wraps in 584 yrs.

> > A 64 bit 3 GHz counter wraps in 194 yrs.

> >

> > ARMv8 Generic Timer specifies a min width of 56 bits but also min

> roll-over

> > of 40 yrs.  If hardware counter is 56 bits, the counter frequency can be

> no

> > faster than ~54 MHz (19ns granularity) to achieve >10 yr wrap.  A wider

> > counter allows for faster frequency, but 19ns granularity is quite

> usable.

>

> These are all implementation choices. ODP does not specify this, only

> the minimum wrap time to provide assurance that applications need not

> concern themselves with time wraps.

>

>


global time can be slow. I think ODP should not implement raw access to hw
timers for that.
At least for general implementation. Time subsystem is very complex in
linux kernel, synchronization
along cores and updating time at right events. I think we need to get all
this work done by kernel and
do not touch it. Also please do not forget that global time can be adjusted
by ntp or ptp. And that adjustments
can be done periodically  due to unstable clock sources, temperature of
pulse generator and etc. I will
be really complex task to move all that logic to odp (even if for beginning
it looks simple). And benefit
of that move is very smooth...

For local time is more simple due to it's access to some per cpu time
counter. It never adjusted. And
you can use it only inside binded threads.

cpu_cycles()   which perf use also per cpu and needed to take some
execution metric. Can be used also only
for binded threads.

Maxim.



> >

> >> Most implementations will choose

> >> +to implement time values using 64-bit values,

> >

> > I think odp_time_t needs to be larger than 32 bits; no other way to

> maintain

> > ~10yr wrap at sensible granularity.  That means a 64 bit in C API.  It

> can be

> > defined at the universal API layer to be a uint64_t for all.  This type

> does

> > not need to be defined differently for each OS or ISA or SoC combo. That

> is

> > the point of an API and abstraction layer.

>

> Most probably true, but again the ODP API does not specify this. A

> defined ODP ABI may well do exactly this, however.

>

> >

> >> whose wrap times exceed 500

> >> +years, making wrapping concerns not relevant to ODP applications.

> >> +

> >>  === Timer

> >>  Timers are how ODP applications measure and respond to the passage of

> time.

> >>  Timers are drawn from specialized pools called timer pools that have

> their

> >> --

> >> 2.11.0.295.gd7dffce

> >>

>

Patch hide | download patch | download mbox

diff --git a/doc/users-guide/users-guide.adoc b/doc/users-guide/users-guide.adoc
index 41c57d1c..05bade8c 100755
--- a/doc/users-guide/users-guide.adoc
+++ b/doc/users-guide/users-guide.adoc
@@ -362,31 +362,41 @@  PktIOs are represented by handles of abstract type `odp_pktio_t`.
 
 === Time
 The time API is used to measure time intervals and track time flow of an
-application and presents a convenient way to get access to a time source.
-The time API consists of two main parts: local time API and global time API.
+application and presents a convenient way to get access to an
+implementation-defined time source. The time API consists of two main parts:
+local time API and global time API.
 
 ==== Local time
-The local time API is designed to be used within one thread and can be faster
-than the global time API. The local time API cannot be used between threads as
-time consistency is not guaranteed, and in some cases that's enough.
-So, local time stamps are local to the calling thread and must not be shared
-with other threads. Current local time can be read with `odp_time_local()`.
+The local time API is designed to be used within one thread and obtaining
+local time may be more efficient in some implementations than global
+time. Local time stamps are local to the calling thread and should not be
+shared with other threads, as local time is not guaranteed to be consistent
+between threads. Current local time can be read with `odp_time_local()`.
 
 ==== Global time
 The global time API is designed to be used for tracking time between threads.
-So, global time stamps can be shared between threads. Current global time can
-be read with `odp_time_global()`.
+So, global time stamps may safely be shared between threads. Current global
+time can be read with `odp_time_global()`.
 
-Both, local and global time is not wrapped during the application life cycle.
+Both local and global time is not wrapped during the application life cycle.
 The time API includes functions to operate with time, such as `odp_time_diff()`,
 `odp_time_sum()`, `odp_time_cmp()`, conversion functions like
 `odp_time_to_ns()`, `odp_time_local_from_ns()`, `odp_time_global_from_ns()`.
 To get rate of time source `odp_time_local_res()`, `odp_time_global_res()`
 are used. To wait, `odp_time_wait_ns()` and `odp_time_wait_until()` are used,
-during witch a thread potentially busy loop the entire wait time.
+during which a thread potentially busy loops the entire wait time.
 
 The `odp_time_t` opaque type represents local or global timestamps.
 
+==== Portability Considerations
+The ODP Time APIs are designed to permit high-precision relative time
+measurement within an ODP application. No attempt is made to correlate an
+`odp_time_t` object with "wall time" or any other external time reference.
+As defined by the ODP specification, `odp_time_t` values are required to
+be unique over a span of at least 10 years. Most implementations will choose
+to implement time values using 64-bit values, whose wrap times exceed 500
+years, making wrapping concerns not relevant to ODP applications.
+
 === Timer
 Timers are how ODP applications measure and respond to the passage of time.
 Timers are drawn from specialized pools called timer pools that have their