diff mbox

doc: userguide: add portability and usage info for odp time apis

Message ID 20170213224753.15361-1-bill.fischofer@linaro.org
State Accepted
Commit f4f7679d16917d9a7c1b2220e351fd27733ee96b
Headers show

Commit Message

Bill Fischofer Feb. 13, 2017, 10:47 p.m. UTC
Clarify and expand on portability and performance considerations
regarding the use of the ODP time APIs in fulfillment of JIRA
issue https://projects.linaro.org/browse/ODP-575

Signed-off-by: Bill Fischofer <bill.fischofer@linaro.org>

---
 doc/users-guide/users-guide.adoc | 32 +++++++++++++++++++++-----------
 1 file changed, 21 insertions(+), 11 deletions(-)

-- 
2.11.0.295.gd7dffce

Comments

Brian Brooks Feb. 16, 2017, 9:20 p.m. UTC | #1
On Mon, Feb 13, 2017 at 4:47 PM, Bill Fischofer
<bill.fischofer@linaro.org> wrote:
> Clarify and expand on portability and performance considerations

> regarding the use of the ODP time APIs in fulfillment of JIRA

> issue https://projects.linaro.org/browse/ODP-575

>

> Signed-off-by: Bill Fischofer <bill.fischofer@linaro.org>

> ---

>  doc/users-guide/users-guide.adoc | 32 +++++++++++++++++++++-----------

>  1 file changed, 21 insertions(+), 11 deletions(-)

>

> diff --git a/doc/users-guide/users-guide.adoc b/doc/users-guide/users-guide.adoc

> index 41c57d1c..05bade8c 100755

> --- a/doc/users-guide/users-guide.adoc

> +++ b/doc/users-guide/users-guide.adoc

> @@ -362,31 +362,41 @@ PktIOs are represented by handles of abstract type `odp_pktio_t`.

>

>  === Time

>  The time API is used to measure time intervals and track time flow of an

> -application and presents a convenient way to get access to a time source.

> -The time API consists of two main parts: local time API and global time API.

> +application and presents a convenient way to get access to an

> +implementation-defined time source. The time API consists of two main parts:

> +local time API and global time API.

>

>  ==== Local time

> -The local time API is designed to be used within one thread and can be faster

> -than the global time API. The local time API cannot be used between threads as

> -time consistency is not guaranteed, and in some cases that's enough.

> -So, local time stamps are local to the calling thread and must not be shared

> -with other threads. Current local time can be read with `odp_time_local()`.

> +The local time API is designed to be used within one thread


How do I convert a local timestamp into a global one?

What happens if the thread migrates to a different core? Thinking cloud...

What happens if a timestamp taken on one thread is used by another thread?
Thinking multithreaded applications that use shared memory synchronization...

> and obtaining

> +local time may be more efficient in some implementations than global

> +time.


To elicit a response... I claim this is *not* true on ARMv8 and modern
x86 chips.  So, no need for global vs. local notion of time.

> Local time stamps are local to the calling thread and should not be

> +shared with other threads, as local time is not guaranteed to be consistent

> +between threads. Current local time can be read with `odp_time_local()`.

>

>  ==== Global time

>  The global time API is designed to be used for tracking time between threads.

> -So, global time stamps can be shared between threads. Current global time can

> -be read with `odp_time_global()`.

> +So, global time stamps may safely be shared between threads. Current global

> +time can be read with `odp_time_global()`.

>

> -Both, local and global time is not wrapped during the application life cycle.

> +Both local and global time is not wrapped during the application life cycle.

>  The time API includes functions to operate with time, such as `odp_time_diff()`,

>  `odp_time_sum()`, `odp_time_cmp()`, conversion functions like

>  `odp_time_to_ns()`, `odp_time_local_from_ns()`, `odp_time_global_from_ns()`.

>  To get rate of time source `odp_time_local_res()`, `odp_time_global_res()`

>  are used. To wait, `odp_time_wait_ns()` and `odp_time_wait_until()` are used,

> -during witch a thread potentially busy loop the entire wait time.

> +during which a thread potentially busy loops the entire wait time.

>

>  The `odp_time_t` opaque type represents local or global timestamps.

>

> +==== Portability Considerations

> +The ODP Time APIs are designed to permit high-precision relative time

> +measurement within an ODP application. No attempt is made to correlate an

> +`odp_time_t` object with "wall time" or any other external time reference.


I think Time should represent "wall time".

> +As defined by the ODP specification, `odp_time_t` values are required to

> +be unique over a span of at least 10 years.


A 32 bit 1 GHz counter wraps in 4.29s.
A 32 bit 12.5 Hz counter wraps in 10 yrs. This is 80ms granularity. :(

A 64 bit 1 GHz counter wraps in 584 yrs.
A 64 bit 3 GHz counter wraps in 194 yrs.

ARMv8 Generic Timer specifies a min width of 56 bits but also min roll-over
of 40 yrs.  If hardware counter is 56 bits, the counter frequency can be no
faster than ~54 MHz (19ns granularity) to achieve >10 yr wrap.  A wider
counter allows for faster frequency, but 19ns granularity is quite usable.

> Most implementations will choose

> +to implement time values using 64-bit values,


I think odp_time_t needs to be larger than 32 bits; no other way to maintain
~10yr wrap at sensible granularity.  That means a 64 bit in C API.  It can be
defined at the universal API layer to be a uint64_t for all.  This type does
not need to be defined differently for each OS or ISA or SoC combo. That is
the point of an API and abstraction layer.

> whose wrap times exceed 500

> +years, making wrapping concerns not relevant to ODP applications.

> +

>  === Timer

>  Timers are how ODP applications measure and respond to the passage of time.

>  Timers are drawn from specialized pools called timer pools that have their

> --

> 2.11.0.295.gd7dffce

>
Bill Fischofer Feb. 16, 2017, 9:56 p.m. UTC | #2
On Thu, Feb 16, 2017 at 3:20 PM, Brian Brooks <brian.brooks@linaro.org> wrote:
> On Mon, Feb 13, 2017 at 4:47 PM, Bill Fischofer

> <bill.fischofer@linaro.org> wrote:

>> Clarify and expand on portability and performance considerations

>> regarding the use of the ODP time APIs in fulfillment of JIRA

>> issue https://projects.linaro.org/browse/ODP-575

>>

>> Signed-off-by: Bill Fischofer <bill.fischofer@linaro.org>

>> ---

>>  doc/users-guide/users-guide.adoc | 32 +++++++++++++++++++++-----------

>>  1 file changed, 21 insertions(+), 11 deletions(-)

>>

>> diff --git a/doc/users-guide/users-guide.adoc b/doc/users-guide/users-guide.adoc

>> index 41c57d1c..05bade8c 100755

>> --- a/doc/users-guide/users-guide.adoc

>> +++ b/doc/users-guide/users-guide.adoc

>> @@ -362,31 +362,41 @@ PktIOs are represented by handles of abstract type `odp_pktio_t`.

>>

>>  === Time

>>  The time API is used to measure time intervals and track time flow of an

>> -application and presents a convenient way to get access to a time source.

>> -The time API consists of two main parts: local time API and global time API.

>> +application and presents a convenient way to get access to an

>> +implementation-defined time source. The time API consists of two main parts:

>> +local time API and global time API.

>>

>>  ==== Local time

>> -The local time API is designed to be used within one thread and can be faster

>> -than the global time API. The local time API cannot be used between threads as

>> -time consistency is not guaranteed, and in some cases that's enough.

>> -So, local time stamps are local to the calling thread and must not be shared

>> -with other threads. Current local time can be read with `odp_time_local()`.

>> +The local time API is designed to be used within one thread

>

> How do I convert a local timestamp into a global one?


You'd have to go through the ns conversion routines:

local->global == odp_time_global_from_ns(odp_time_to_ns(local_time));

global->local == odp_time_local_from_ns(odp_time_to_ns(global_time));

Though I'm not sure what the use case would be for such conversions.

>

> What happens if the thread migrates to a different core? Thinking cloud...

>

> What happens if a timestamp taken on one thread is used by another thread?

> Thinking multithreaded applications that use shared memory synchronization...


Presumably odp_time_local() should only be used on cores that are
isolated to a specific CPU. "Floating" threads should use
odp_time_global(). However, if you're being preempted and timesliced
onto another core, it's not clear that odp_time_diff() is doing to be
meaningful in such cases no matter what time you're using.

>

>> and obtaining

>> +local time may be more efficient in some implementations than global

>> +time.

>

> To elicit a response... I claim this is *not* true on ARMv8 and modern

> x86 chips.  So, no need for global vs. local notion of time.


That's why this is a "may". I agree that the use case for the concept
of local time is weak, but presumably this is a bit of "NUMA
insurance" in the API?

>

>> Local time stamps are local to the calling thread and should not be

>> +shared with other threads, as local time is not guaranteed to be consistent

>> +between threads. Current local time can be read with `odp_time_local()`.

>>

>>  ==== Global time

>>  The global time API is designed to be used for tracking time between threads.

>> -So, global time stamps can be shared between threads. Current global time can

>> -be read with `odp_time_global()`.

>> +So, global time stamps may safely be shared between threads. Current global

>> +time can be read with `odp_time_global()`.

>>

>> -Both, local and global time is not wrapped during the application life cycle.

>> +Both local and global time is not wrapped during the application life cycle.

>>  The time API includes functions to operate with time, such as `odp_time_diff()`,

>>  `odp_time_sum()`, `odp_time_cmp()`, conversion functions like

>>  `odp_time_to_ns()`, `odp_time_local_from_ns()`, `odp_time_global_from_ns()`.

>>  To get rate of time source `odp_time_local_res()`, `odp_time_global_res()`

>>  are used. To wait, `odp_time_wait_ns()` and `odp_time_wait_until()` are used,

>> -during witch a thread potentially busy loop the entire wait time.

>> +during which a thread potentially busy loops the entire wait time.

>>

>>  The `odp_time_t` opaque type represents local or global timestamps.

>>

>> +==== Portability Considerations

>> +The ODP Time APIs are designed to permit high-precision relative time

>> +measurement within an ODP application. No attempt is made to correlate an

>> +`odp_time_t` object with "wall time" or any other external time reference.

>

> I think Time should represent "wall time".


Wall time refers to a specific epoch as well as flow rate. These
values cannot be converted into a calendar date/time, hence this
disclaimer.

>

>> +As defined by the ODP specification, `odp_time_t` values are required to

>> +be unique over a span of at least 10 years.

>

> A 32 bit 1 GHz counter wraps in 4.29s.

> A 32 bit 12.5 Hz counter wraps in 10 yrs. This is 80ms granularity. :(

>

> A 64 bit 1 GHz counter wraps in 584 yrs.

> A 64 bit 3 GHz counter wraps in 194 yrs.

>

> ARMv8 Generic Timer specifies a min width of 56 bits but also min roll-over

> of 40 yrs.  If hardware counter is 56 bits, the counter frequency can be no

> faster than ~54 MHz (19ns granularity) to achieve >10 yr wrap.  A wider

> counter allows for faster frequency, but 19ns granularity is quite usable.


These are all implementation choices. ODP does not specify this, only
the minimum wrap time to provide assurance that applications need not
concern themselves with time wraps.

>

>> Most implementations will choose

>> +to implement time values using 64-bit values,

>

> I think odp_time_t needs to be larger than 32 bits; no other way to maintain

> ~10yr wrap at sensible granularity.  That means a 64 bit in C API.  It can be

> defined at the universal API layer to be a uint64_t for all.  This type does

> not need to be defined differently for each OS or ISA or SoC combo. That is

> the point of an API and abstraction layer.


Most probably true, but again the ODP API does not specify this. A
defined ODP ABI may well do exactly this, however.

>

>> whose wrap times exceed 500

>> +years, making wrapping concerns not relevant to ODP applications.

>> +

>>  === Timer

>>  Timers are how ODP applications measure and respond to the passage of time.

>>  Timers are drawn from specialized pools called timer pools that have their

>> --

>> 2.11.0.295.gd7dffce

>>
Maxim Uvarov Feb. 17, 2017, 10 a.m. UTC | #3
On 17 February 2017 at 00:56, Bill Fischofer <bill.fischofer@linaro.org>
wrote:

> On Thu, Feb 16, 2017 at 3:20 PM, Brian Brooks <brian.brooks@linaro.org>

> wrote:

> > On Mon, Feb 13, 2017 at 4:47 PM, Bill Fischofer

> > <bill.fischofer@linaro.org> wrote:

> >> Clarify and expand on portability and performance considerations

> >> regarding the use of the ODP time APIs in fulfillment of JIRA

> >> issue https://projects.linaro.org/browse/ODP-575

> >>

> >> Signed-off-by: Bill Fischofer <bill.fischofer@linaro.org>

> >> ---

> >>  doc/users-guide/users-guide.adoc | 32 +++++++++++++++++++++-----------

> >>  1 file changed, 21 insertions(+), 11 deletions(-)

> >>

> >> diff --git a/doc/users-guide/users-guide.adoc

> b/doc/users-guide/users-guide.adoc

> >> index 41c57d1c..05bade8c 100755

> >> --- a/doc/users-guide/users-guide.adoc

> >> +++ b/doc/users-guide/users-guide.adoc

> >> @@ -362,31 +362,41 @@ PktIOs are represented by handles of abstract

> type `odp_pktio_t`.

> >>

> >>  === Time

> >>  The time API is used to measure time intervals and track time flow of

> an

> >> -application and presents a convenient way to get access to a time

> source.

> >> -The time API consists of two main parts: local time API and global

> time API.

> >> +application and presents a convenient way to get access to an

> >> +implementation-defined time source. The time API consists of two main

> parts:

> >> +local time API and global time API.

> >>

> >>  ==== Local time

> >> -The local time API is designed to be used within one thread and can be

> faster

> >> -than the global time API. The local time API cannot be used between

> threads as

> >> -time consistency is not guaranteed, and in some cases that's enough.

> >> -So, local time stamps are local to the calling thread and must not be

> shared

> >> -with other threads. Current local time can be read with

> `odp_time_local()`.

> >> +The local time API is designed to be used within one thread

> >

> > How do I convert a local timestamp into a global one?

>

> You'd have to go through the ns conversion routines:

>

> local->global == odp_time_global_from_ns(odp_time_to_ns(local_time));

>

> global->local == odp_time_local_from_ns(odp_time_to_ns(global_time));

>

> Though I'm not sure what the use case would be for such conversions.

>

> >

> > What happens if the thread migrates to a different core? Thinking

> cloud...

> >

> > What happens if a timestamp taken on one thread is used by another

> thread?

> > Thinking multithreaded applications that use shared memory

> synchronization...

>

> Presumably odp_time_local() should only be used on cores that are

> isolated to a specific CPU. "Floating" threads should use

> odp_time_global(). However, if you're being preempted and timesliced

> onto another core, it's not clear that odp_time_diff() is doing to be

> meaningful in such cases no matter what time you're using.

>

> >

> >> and obtaining

> >> +local time may be more efficient in some implementations than global

> >> +time.

> >

> > To elicit a response... I claim this is *not* true on ARMv8 and modern

> > x86 chips.  So, no need for global vs. local notion of time.

>

> That's why this is a "may". I agree that the use case for the concept

> of local time is weak, but presumably this is a bit of "NUMA

> insurance" in the API?

>

> >

> >> Local time stamps are local to the calling thread and should not be

> >> +shared with other threads, as local time is not guaranteed to be

> consistent

> >> +between threads. Current local time can be read with

> `odp_time_local()`.

> >>

> >>  ==== Global time

> >>  The global time API is designed to be used for tracking time between

> threads.

> >> -So, global time stamps can be shared between threads. Current global

> time can

> >> -be read with `odp_time_global()`.

> >> +So, global time stamps may safely be shared between threads. Current

> global

> >> +time can be read with `odp_time_global()`.

> >>

> >> -Both, local and global time is not wrapped during the application life

> cycle.

> >> +Both local and global time is not wrapped during the application life

> cycle.

> >>  The time API includes functions to operate with time, such as

> `odp_time_diff()`,

> >>  `odp_time_sum()`, `odp_time_cmp()`, conversion functions like

> >>  `odp_time_to_ns()`, `odp_time_local_from_ns()`,

> `odp_time_global_from_ns()`.

> >>  To get rate of time source `odp_time_local_res()`,

> `odp_time_global_res()`

> >>  are used. To wait, `odp_time_wait_ns()` and `odp_time_wait_until()`

> are used,

> >> -during witch a thread potentially busy loop the entire wait time.

> >> +during which a thread potentially busy loops the entire wait time.

> >>

> >>  The `odp_time_t` opaque type represents local or global timestamps.

> >>

> >> +==== Portability Considerations

> >> +The ODP Time APIs are designed to permit high-precision relative time

> >> +measurement within an ODP application. No attempt is made to correlate

> an

> >> +`odp_time_t` object with "wall time" or any other external time

> reference.

> >

> > I think Time should represent "wall time".

>

> Wall time refers to a specific epoch as well as flow rate. These

> values cannot be converted into a calendar date/time, hence this

> disclaimer.

>

> >

> >> +As defined by the ODP specification, `odp_time_t` values are required

> to

> >> +be unique over a span of at least 10 years.

> >

> > A 32 bit 1 GHz counter wraps in 4.29s.

> > A 32 bit 12.5 Hz counter wraps in 10 yrs. This is 80ms granularity. :(

> >

> > A 64 bit 1 GHz counter wraps in 584 yrs.

> > A 64 bit 3 GHz counter wraps in 194 yrs.

> >

> > ARMv8 Generic Timer specifies a min width of 56 bits but also min

> roll-over

> > of 40 yrs.  If hardware counter is 56 bits, the counter frequency can be

> no

> > faster than ~54 MHz (19ns granularity) to achieve >10 yr wrap.  A wider

> > counter allows for faster frequency, but 19ns granularity is quite

> usable.

>

> These are all implementation choices. ODP does not specify this, only

> the minimum wrap time to provide assurance that applications need not

> concern themselves with time wraps.

>

>


global time can be slow. I think ODP should not implement raw access to hw
timers for that.
At least for general implementation. Time subsystem is very complex in
linux kernel, synchronization
along cores and updating time at right events. I think we need to get all
this work done by kernel and
do not touch it. Also please do not forget that global time can be adjusted
by ntp or ptp. And that adjustments
can be done periodically  due to unstable clock sources, temperature of
pulse generator and etc. I will
be really complex task to move all that logic to odp (even if for beginning
it looks simple). And benefit
of that move is very smooth...

For local time is more simple due to it's access to some per cpu time
counter. It never adjusted. And
you can use it only inside binded threads.

cpu_cycles()   which perf use also per cpu and needed to take some
execution metric. Can be used also only
for binded threads.

Maxim.



> >

> >> Most implementations will choose

> >> +to implement time values using 64-bit values,

> >

> > I think odp_time_t needs to be larger than 32 bits; no other way to

> maintain

> > ~10yr wrap at sensible granularity.  That means a 64 bit in C API.  It

> can be

> > defined at the universal API layer to be a uint64_t for all.  This type

> does

> > not need to be defined differently for each OS or ISA or SoC combo. That

> is

> > the point of an API and abstraction layer.

>

> Most probably true, but again the ODP API does not specify this. A

> defined ODP ABI may well do exactly this, however.

>

> >

> >> whose wrap times exceed 500

> >> +years, making wrapping concerns not relevant to ODP applications.

> >> +

> >>  === Timer

> >>  Timers are how ODP applications measure and respond to the passage of

> time.

> >>  Timers are drawn from specialized pools called timer pools that have

> their

> >> --

> >> 2.11.0.295.gd7dffce

> >>

>
Bill Fischofer April 13, 2017, 1:58 p.m. UTC | #4
Ping. This patch seems to have stalled. Does it need revision or just a review?

On Fri, Feb 17, 2017 at 4:00 AM, Maxim Uvarov <maxim.uvarov@linaro.org> wrote:
>

>

> On 17 February 2017 at 00:56, Bill Fischofer <bill.fischofer@linaro.org>

> wrote:

>>

>> On Thu, Feb 16, 2017 at 3:20 PM, Brian Brooks <brian.brooks@linaro.org>

>> wrote:

>> > On Mon, Feb 13, 2017 at 4:47 PM, Bill Fischofer

>> > <bill.fischofer@linaro.org> wrote:

>> >> Clarify and expand on portability and performance considerations

>> >> regarding the use of the ODP time APIs in fulfillment of JIRA

>> >> issue https://projects.linaro.org/browse/ODP-575

>> >>

>> >> Signed-off-by: Bill Fischofer <bill.fischofer@linaro.org>

>> >> ---

>> >>  doc/users-guide/users-guide.adoc | 32 +++++++++++++++++++++-----------

>> >>  1 file changed, 21 insertions(+), 11 deletions(-)

>> >>

>> >> diff --git a/doc/users-guide/users-guide.adoc

>> >> b/doc/users-guide/users-guide.adoc

>> >> index 41c57d1c..05bade8c 100755

>> >> --- a/doc/users-guide/users-guide.adoc

>> >> +++ b/doc/users-guide/users-guide.adoc

>> >> @@ -362,31 +362,41 @@ PktIOs are represented by handles of abstract

>> >> type `odp_pktio_t`.

>> >>

>> >>  === Time

>> >>  The time API is used to measure time intervals and track time flow of

>> >> an

>> >> -application and presents a convenient way to get access to a time

>> >> source.

>> >> -The time API consists of two main parts: local time API and global

>> >> time API.

>> >> +application and presents a convenient way to get access to an

>> >> +implementation-defined time source. The time API consists of two main

>> >> parts:

>> >> +local time API and global time API.

>> >>

>> >>  ==== Local time

>> >> -The local time API is designed to be used within one thread and can be

>> >> faster

>> >> -than the global time API. The local time API cannot be used between

>> >> threads as

>> >> -time consistency is not guaranteed, and in some cases that's enough.

>> >> -So, local time stamps are local to the calling thread and must not be

>> >> shared

>> >> -with other threads. Current local time can be read with

>> >> `odp_time_local()`.

>> >> +The local time API is designed to be used within one thread

>> >

>> > How do I convert a local timestamp into a global one?

>>

>> You'd have to go through the ns conversion routines:

>>

>> local->global == odp_time_global_from_ns(odp_time_to_ns(local_time));

>>

>> global->local == odp_time_local_from_ns(odp_time_to_ns(global_time));

>>

>> Though I'm not sure what the use case would be for such conversions.

>>

>> >

>> > What happens if the thread migrates to a different core? Thinking

>> > cloud...

>> >

>> > What happens if a timestamp taken on one thread is used by another

>> > thread?

>> > Thinking multithreaded applications that use shared memory

>> > synchronization...

>>

>> Presumably odp_time_local() should only be used on cores that are

>> isolated to a specific CPU. "Floating" threads should use

>> odp_time_global(). However, if you're being preempted and timesliced

>> onto another core, it's not clear that odp_time_diff() is doing to be

>> meaningful in such cases no matter what time you're using.

>>

>> >

>> >> and obtaining

>> >> +local time may be more efficient in some implementations than global

>> >> +time.

>> >

>> > To elicit a response... I claim this is *not* true on ARMv8 and modern

>> > x86 chips.  So, no need for global vs. local notion of time.

>>

>> That's why this is a "may". I agree that the use case for the concept

>> of local time is weak, but presumably this is a bit of "NUMA

>> insurance" in the API?

>>

>> >

>> >> Local time stamps are local to the calling thread and should not be

>> >> +shared with other threads, as local time is not guaranteed to be

>> >> consistent

>> >> +between threads. Current local time can be read with

>> >> `odp_time_local()`.

>> >>

>> >>  ==== Global time

>> >>  The global time API is designed to be used for tracking time between

>> >> threads.

>> >> -So, global time stamps can be shared between threads. Current global

>> >> time can

>> >> -be read with `odp_time_global()`.

>> >> +So, global time stamps may safely be shared between threads. Current

>> >> global

>> >> +time can be read with `odp_time_global()`.

>> >>

>> >> -Both, local and global time is not wrapped during the application life

>> >> cycle.

>> >> +Both local and global time is not wrapped during the application life

>> >> cycle.

>> >>  The time API includes functions to operate with time, such as

>> >> `odp_time_diff()`,

>> >>  `odp_time_sum()`, `odp_time_cmp()`, conversion functions like

>> >>  `odp_time_to_ns()`, `odp_time_local_from_ns()`,

>> >> `odp_time_global_from_ns()`.

>> >>  To get rate of time source `odp_time_local_res()`,

>> >> `odp_time_global_res()`

>> >>  are used. To wait, `odp_time_wait_ns()` and `odp_time_wait_until()`

>> >> are used,

>> >> -during witch a thread potentially busy loop the entire wait time.

>> >> +during which a thread potentially busy loops the entire wait time.

>> >>

>> >>  The `odp_time_t` opaque type represents local or global timestamps.

>> >>

>> >> +==== Portability Considerations

>> >> +The ODP Time APIs are designed to permit high-precision relative time

>> >> +measurement within an ODP application. No attempt is made to correlate

>> >> an

>> >> +`odp_time_t` object with "wall time" or any other external time

>> >> reference.

>> >

>> > I think Time should represent "wall time".

>>

>> Wall time refers to a specific epoch as well as flow rate. These

>> values cannot be converted into a calendar date/time, hence this

>> disclaimer.

>>

>> >

>> >> +As defined by the ODP specification, `odp_time_t` values are required

>> >> to

>> >> +be unique over a span of at least 10 years.

>> >

>> > A 32 bit 1 GHz counter wraps in 4.29s.

>> > A 32 bit 12.5 Hz counter wraps in 10 yrs. This is 80ms granularity. :(

>> >

>> > A 64 bit 1 GHz counter wraps in 584 yrs.

>> > A 64 bit 3 GHz counter wraps in 194 yrs.

>> >

>> > ARMv8 Generic Timer specifies a min width of 56 bits but also min

>> > roll-over

>> > of 40 yrs.  If hardware counter is 56 bits, the counter frequency can be

>> > no

>> > faster than ~54 MHz (19ns granularity) to achieve >10 yr wrap.  A wider

>> > counter allows for faster frequency, but 19ns granularity is quite

>> > usable.

>>

>> These are all implementation choices. ODP does not specify this, only

>> the minimum wrap time to provide assurance that applications need not

>> concern themselves with time wraps.

>>

>

>

> global time can be slow. I think ODP should not implement raw access to hw

> timers for that.

> At least for general implementation. Time subsystem is very complex in linux

> kernel, synchronization

> along cores and updating time at right events. I think we need to get all

> this work done by kernel and

> do not touch it. Also please do not forget that global time can be adjusted

> by ntp or ptp. And that adjustments

> can be done periodically  due to unstable clock sources, temperature of

> pulse generator and etc. I will

> be really complex task to move all that logic to odp (even if for beginning

> it looks simple). And benefit

> of that move is very smooth...

>

> For local time is more simple due to it's access to some per cpu time

> counter. It never adjusted. And

> you can use it only inside binded threads.

>

> cpu_cycles()   which perf use also per cpu and needed to take some execution

> metric. Can be used also only

> for binded threads.

>

> Maxim.

>

>

>>

>> >

>> >> Most implementations will choose

>> >> +to implement time values using 64-bit values,

>> >

>> > I think odp_time_t needs to be larger than 32 bits; no other way to

>> > maintain

>> > ~10yr wrap at sensible granularity.  That means a 64 bit in C API.  It

>> > can be

>> > defined at the universal API layer to be a uint64_t for all.  This type

>> > does

>> > not need to be defined differently for each OS or ISA or SoC combo. That

>> > is

>> > the point of an API and abstraction layer.

>>

>> Most probably true, but again the ODP API does not specify this. A

>> defined ODP ABI may well do exactly this, however.

>>

>> >

>> >> whose wrap times exceed 500

>> >> +years, making wrapping concerns not relevant to ODP applications.

>> >> +

>> >>  === Timer

>> >>  Timers are how ODP applications measure and respond to the passage of

>> >> time.

>> >>  Timers are drawn from specialized pools called timer pools that have

>> >> their

>> >> --

>> >> 2.11.0.295.gd7dffce

>> >>

>

>
Brian Brooks June 22, 2017, 9:50 p.m. UTC | #5
On 06/29 16:21:47, Maxim Uvarov wrote:
> Hello Bill,

> 

> patch is good. Please see my notes bellow which I think reasonable.

> 

> 

> On 02/14/17 01:47, Bill Fischofer wrote:

> > Clarify and expand on portability and performance considerations

> > regarding the use of the ODP time APIs in fulfillment of JIRA

> 

> fulfilment

> 

> > issue https://projects.linaro.org/browse/ODP-575

> > 

> > Signed-off-by: Bill Fischofer <bill.fischofer@linaro.org>

> > ---

> >  doc/users-guide/users-guide.adoc | 32 +++++++++++++++++++++-----------

> >  1 file changed, 21 insertions(+), 11 deletions(-)

> > 

> > diff --git a/doc/users-guide/users-guide.adoc b/doc/users-guide/users-guide.adoc

> > index 41c57d1c..05bade8c 100755

> > --- a/doc/users-guide/users-guide.adoc

> > +++ b/doc/users-guide/users-guide.adoc

> > @@ -362,31 +362,41 @@ PktIOs are represented by handles of abstract type `odp_pktio_t`.

> >  

> >  === Time

> >  The time API is used to measure time intervals and track time flow of an

> > -application and presents a convenient way to get access to a time source.

> > -The time API consists of two main parts: local time API and global time API.

> > +application and presents a convenient way to get access to an

> > +implementation-defined time source. The time API consists of two main parts:

> > +local time API and global time API.

> >  

> >  ==== Local time

> > -The local time API is designed to be used within one thread and can be faster

> > -than the global time API. The local time API cannot be used between threads as

> > -time consistency is not guaranteed, and in some cases that's enough.

> > -So, local time stamps are local to the calling thread and must not be shared

> > -with other threads. Current local time can be read with `odp_time_local()`.

> > +The local time API is designed to be used within one thread and obtaining

> > +local time may be more efficient in some implementations than global

> > +time.

> 

> 

> The local time API is designed to be used within one odp worker which is

> bind to specific cpu core.

> 

> 

> > Local time stamps are local to the calling thread and should not be

> > +shared with other threads, as local time is not guaranteed to be consistent

> > +between threads. Current local time can be read with `odp_time_local()`.

> >  

> >  ==== Global time

> >  The global time API is designed to be used for tracking time between threads.

> > -So, global time stamps can be shared between threads. Current global time can

> > -be read with `odp_time_global()`.

> > +So, global time stamps may safely be shared between threads. Current global

> > +time can be read with `odp_time_global()`.

> 

> also odp workers and control threads.

> 

> >  

> > -Both, local and global time is not wrapped during the application life cycle.

> > +Both local and global time is not wrapped during the application life cycle.

> >  The time API includes functions to operate with time, such as `odp_time_diff()`,

> >  `odp_time_sum()`, `odp_time_cmp()`, conversion functions like

> >  `odp_time_to_ns()`, `odp_time_local_from_ns()`, `odp_time_global_from_ns()`.

> >  To get rate of time source `odp_time_local_res()`, `odp_time_global_res()`

> >  are used. To wait, `odp_time_wait_ns()` and `odp_time_wait_until()` are used,

> > -during witch a thread potentially busy loop the entire wait time.

> > +during which a thread potentially busy loops the entire wait time.

> >  

> >  The `odp_time_t` opaque type represents local or global timestamps.

> >  

> > +==== Portability Considerations

> > +The ODP Time APIs are designed to permit high-precision relative time

> > +measurement within an ODP application. No attempt is made to correlate an

> > +`odp_time_t` object with "wall time" or any other external time reference.

> > +As defined by the ODP specification, `odp_time_t` values are required to

> > +be unique over a span of at least 10 years. Most implementations will choose

> > +to implement time values using 64-bit values, whose wrap times exceed 500

> > +years, making wrapping concerns not relevant to ODP applications.

> > +

> 

> Yes that is good addition. That means that odp time is not adjusted (ptp

> or ntp) and can not go backwards. I.e. each next call you will get

> updated incremented value. No time adjustments.


I was confused by the bit about no correlation with wall time. What are the
Time APIs good for then? ;) If it is more accurate to say something about
clock drift, synchronization, thread vs. cpu time then that would be much better!

> Best regards,

> Maxim.

> 

> >  === Timer

> >  Timers are how ODP applications measure and respond to the passage of time.

> >  Timers are drawn from specialized pools called timer pools that have their

> > 

>
Maxim Uvarov June 29, 2017, 1:21 p.m. UTC | #6
Hello Bill,

patch is good. Please see my notes bellow which I think reasonable.


On 02/14/17 01:47, Bill Fischofer wrote:
> Clarify and expand on portability and performance considerations

> regarding the use of the ODP time APIs in fulfillment of JIRA


fulfilment

> issue https://projects.linaro.org/browse/ODP-575

> 

> Signed-off-by: Bill Fischofer <bill.fischofer@linaro.org>

> ---

>  doc/users-guide/users-guide.adoc | 32 +++++++++++++++++++++-----------

>  1 file changed, 21 insertions(+), 11 deletions(-)

> 

> diff --git a/doc/users-guide/users-guide.adoc b/doc/users-guide/users-guide.adoc

> index 41c57d1c..05bade8c 100755

> --- a/doc/users-guide/users-guide.adoc

> +++ b/doc/users-guide/users-guide.adoc

> @@ -362,31 +362,41 @@ PktIOs are represented by handles of abstract type `odp_pktio_t`.

>  

>  === Time

>  The time API is used to measure time intervals and track time flow of an

> -application and presents a convenient way to get access to a time source.

> -The time API consists of two main parts: local time API and global time API.

> +application and presents a convenient way to get access to an

> +implementation-defined time source. The time API consists of two main parts:

> +local time API and global time API.

>  

>  ==== Local time

> -The local time API is designed to be used within one thread and can be faster

> -than the global time API. The local time API cannot be used between threads as

> -time consistency is not guaranteed, and in some cases that's enough.

> -So, local time stamps are local to the calling thread and must not be shared

> -with other threads. Current local time can be read with `odp_time_local()`.

> +The local time API is designed to be used within one thread and obtaining

> +local time may be more efficient in some implementations than global

> +time.



The local time API is designed to be used within one odp worker which is
bind to specific cpu core.


> Local time stamps are local to the calling thread and should not be

> +shared with other threads, as local time is not guaranteed to be consistent

> +between threads. Current local time can be read with `odp_time_local()`.

>  

>  ==== Global time

>  The global time API is designed to be used for tracking time between threads.

> -So, global time stamps can be shared between threads. Current global time can

> -be read with `odp_time_global()`.

> +So, global time stamps may safely be shared between threads. Current global

> +time can be read with `odp_time_global()`.


also odp workers and control threads.

>  

> -Both, local and global time is not wrapped during the application life cycle.

> +Both local and global time is not wrapped during the application life cycle.

>  The time API includes functions to operate with time, such as `odp_time_diff()`,

>  `odp_time_sum()`, `odp_time_cmp()`, conversion functions like

>  `odp_time_to_ns()`, `odp_time_local_from_ns()`, `odp_time_global_from_ns()`.

>  To get rate of time source `odp_time_local_res()`, `odp_time_global_res()`

>  are used. To wait, `odp_time_wait_ns()` and `odp_time_wait_until()` are used,

> -during witch a thread potentially busy loop the entire wait time.

> +during which a thread potentially busy loops the entire wait time.

>  

>  The `odp_time_t` opaque type represents local or global timestamps.

>  

> +==== Portability Considerations

> +The ODP Time APIs are designed to permit high-precision relative time

> +measurement within an ODP application. No attempt is made to correlate an

> +`odp_time_t` object with "wall time" or any other external time reference.

> +As defined by the ODP specification, `odp_time_t` values are required to

> +be unique over a span of at least 10 years. Most implementations will choose

> +to implement time values using 64-bit values, whose wrap times exceed 500

> +years, making wrapping concerns not relevant to ODP applications.

> +


Yes that is good addition. That means that odp time is not adjusted (ptp
or ntp) and can not go backwards. I.e. each next call you will get
updated incremented value. No time adjustments.


Best regards,
Maxim.

>  === Timer

>  Timers are how ODP applications measure and respond to the passage of time.

>  Timers are drawn from specialized pools called timer pools that have their

>
Bill Fischofer June 29, 2017, 9:07 p.m. UTC | #7
On Thu, Jun 22, 2017 at 4:50 PM, Brian Brooks <brian.brooks@arm.com> wrote:
> On 06/29 16:21:47, Maxim Uvarov wrote:

>> Hello Bill,

>>

>> patch is good. Please see my notes bellow which I think reasonable.

>>

>>

>> On 02/14/17 01:47, Bill Fischofer wrote:

>> > Clarify and expand on portability and performance considerations

>> > regarding the use of the ODP time APIs in fulfillment of JIRA

>>

>> fulfilment


fulfillment is the correct (American) english spelling. Fulfilment is
considered an acceptable alternate spelling, though Google flags it.

>>

>> > issue https://projects.linaro.org/browse/ODP-575

>> >

>> > Signed-off-by: Bill Fischofer <bill.fischofer@linaro.org>

>> > ---

>> >  doc/users-guide/users-guide.adoc | 32 +++++++++++++++++++++-----------

>> >  1 file changed, 21 insertions(+), 11 deletions(-)

>> >

>> > diff --git a/doc/users-guide/users-guide.adoc b/doc/users-guide/users-guide.adoc

>> > index 41c57d1c..05bade8c 100755

>> > --- a/doc/users-guide/users-guide.adoc

>> > +++ b/doc/users-guide/users-guide.adoc

>> > @@ -362,31 +362,41 @@ PktIOs are represented by handles of abstract type `odp_pktio_t`.

>> >

>> >  === Time

>> >  The time API is used to measure time intervals and track time flow of an

>> > -application and presents a convenient way to get access to a time source.

>> > -The time API consists of two main parts: local time API and global time API.

>> > +application and presents a convenient way to get access to an

>> > +implementation-defined time source. The time API consists of two main parts:

>> > +local time API and global time API.

>> >

>> >  ==== Local time

>> > -The local time API is designed to be used within one thread and can be faster

>> > -than the global time API. The local time API cannot be used between threads as

>> > -time consistency is not guaranteed, and in some cases that's enough.

>> > -So, local time stamps are local to the calling thread and must not be shared

>> > -with other threads. Current local time can be read with `odp_time_local()`.

>> > +The local time API is designed to be used within one thread and obtaining

>> > +local time may be more efficient in some implementations than global

>> > +time.

>>

>>

>> The local time API is designed to be used within one odp worker which is

>> bind to specific cpu core.


These APIs are not restricted to use by worker threads nor are they
restricted to pinned threads. Local time is simply time designed to be
used within a single thread.

>>

>>

>> > Local time stamps are local to the calling thread and should not be

>> > +shared with other threads, as local time is not guaranteed to be consistent

>> > +between threads. Current local time can be read with `odp_time_local()`.

>> >

>> >  ==== Global time

>> >  The global time API is designed to be used for tracking time between threads.

>> > -So, global time stamps can be shared between threads. Current global time can

>> > -be read with `odp_time_global()`.

>> > +So, global time stamps may safely be shared between threads. Current global

>> > +time can be read with `odp_time_global()`.

>>

>> also odp workers and control threads.


The API doesn't care what type of thread calls it.

>>

>> >

>> > -Both, local and global time is not wrapped during the application life cycle.

>> > +Both local and global time is not wrapped during the application life cycle.

>> >  The time API includes functions to operate with time, such as `odp_time_diff()`,

>> >  `odp_time_sum()`, `odp_time_cmp()`, conversion functions like

>> >  `odp_time_to_ns()`, `odp_time_local_from_ns()`, `odp_time_global_from_ns()`.

>> >  To get rate of time source `odp_time_local_res()`, `odp_time_global_res()`

>> >  are used. To wait, `odp_time_wait_ns()` and `odp_time_wait_until()` are used,

>> > -during witch a thread potentially busy loop the entire wait time.

>> > +during which a thread potentially busy loops the entire wait time.

>> >

>> >  The `odp_time_t` opaque type represents local or global timestamps.

>> >

>> > +==== Portability Considerations

>> > +The ODP Time APIs are designed to permit high-precision relative time

>> > +measurement within an ODP application. No attempt is made to correlate an

>> > +`odp_time_t` object with "wall time" or any other external time reference.

>> > +As defined by the ODP specification, `odp_time_t` values are required to

>> > +be unique over a span of at least 10 years. Most implementations will choose

>> > +to implement time values using 64-bit values, whose wrap times exceed 500

>> > +years, making wrapping concerns not relevant to ODP applications.

>> > +

>>

>> Yes that is good addition. That means that odp time is not adjusted (ptp

>> or ntp) and can not go backwards. I.e. each next call you will get

>> updated incremented value. No time adjustments.

>

> I was confused by the bit about no correlation with wall time. What are the

> Time APIs good for then? ;) If it is more accurate to say something about

> clock drift, synchronization, thread vs. cpu time then that would be much better!


They are not correlated with wall time in that you cannot take the
output from these calls and convert them to a calendar date / time in
any portable manner. They simply record the monotonic passage of
elapsed time and can be used to measure time differences between point
A and point B as taking so many nanoseconds.

>

>> Best regards,

>> Maxim.

>>

>> >  === Timer

>> >  Timers are how ODP applications measure and respond to the passage of time.

>> >  Timers are drawn from specialized pools called timer pools that have their

>> >

>>
Maxim Uvarov July 24, 2017, 4:38 p.m. UTC | #8
Merged as discussed today.

Maxim.

On 06/30/17 00:07, Bill Fischofer wrote:
> On Thu, Jun 22, 2017 at 4:50 PM, Brian Brooks <brian.brooks@arm.com> wrote:

>> On 06/29 16:21:47, Maxim Uvarov wrote:

>>> Hello Bill,

>>>

>>> patch is good. Please see my notes bellow which I think reasonable.

>>>

>>>

>>> On 02/14/17 01:47, Bill Fischofer wrote:

>>>> Clarify and expand on portability and performance considerations

>>>> regarding the use of the ODP time APIs in fulfillment of JIRA

>>>

>>> fulfilment

> 

> fulfillment is the correct (American) english spelling. Fulfilment is

> considered an acceptable alternate spelling, though Google flags it.

> 

>>>

>>>> issue https://projects.linaro.org/browse/ODP-575

>>>>

>>>> Signed-off-by: Bill Fischofer <bill.fischofer@linaro.org>

>>>> ---

>>>>  doc/users-guide/users-guide.adoc | 32 +++++++++++++++++++++-----------

>>>>  1 file changed, 21 insertions(+), 11 deletions(-)

>>>>

>>>> diff --git a/doc/users-guide/users-guide.adoc b/doc/users-guide/users-guide.adoc

>>>> index 41c57d1c..05bade8c 100755

>>>> --- a/doc/users-guide/users-guide.adoc

>>>> +++ b/doc/users-guide/users-guide.adoc

>>>> @@ -362,31 +362,41 @@ PktIOs are represented by handles of abstract type `odp_pktio_t`.

>>>>

>>>>  === Time

>>>>  The time API is used to measure time intervals and track time flow of an

>>>> -application and presents a convenient way to get access to a time source.

>>>> -The time API consists of two main parts: local time API and global time API.

>>>> +application and presents a convenient way to get access to an

>>>> +implementation-defined time source. The time API consists of two main parts:

>>>> +local time API and global time API.

>>>>

>>>>  ==== Local time

>>>> -The local time API is designed to be used within one thread and can be faster

>>>> -than the global time API. The local time API cannot be used between threads as

>>>> -time consistency is not guaranteed, and in some cases that's enough.

>>>> -So, local time stamps are local to the calling thread and must not be shared

>>>> -with other threads. Current local time can be read with `odp_time_local()`.

>>>> +The local time API is designed to be used within one thread and obtaining

>>>> +local time may be more efficient in some implementations than global

>>>> +time.

>>>

>>>

>>> The local time API is designed to be used within one odp worker which is

>>> bind to specific cpu core.

> 

> These APIs are not restricted to use by worker threads nor are they

> restricted to pinned threads. Local time is simply time designed to be

> used within a single thread.

> 

>>>

>>>

>>>> Local time stamps are local to the calling thread and should not be

>>>> +shared with other threads, as local time is not guaranteed to be consistent

>>>> +between threads. Current local time can be read with `odp_time_local()`.

>>>>

>>>>  ==== Global time

>>>>  The global time API is designed to be used for tracking time between threads.

>>>> -So, global time stamps can be shared between threads. Current global time can

>>>> -be read with `odp_time_global()`.

>>>> +So, global time stamps may safely be shared between threads. Current global

>>>> +time can be read with `odp_time_global()`.

>>>

>>> also odp workers and control threads.

> 

> The API doesn't care what type of thread calls it.

> 

>>>

>>>>

>>>> -Both, local and global time is not wrapped during the application life cycle.

>>>> +Both local and global time is not wrapped during the application life cycle.

>>>>  The time API includes functions to operate with time, such as `odp_time_diff()`,

>>>>  `odp_time_sum()`, `odp_time_cmp()`, conversion functions like

>>>>  `odp_time_to_ns()`, `odp_time_local_from_ns()`, `odp_time_global_from_ns()`.

>>>>  To get rate of time source `odp_time_local_res()`, `odp_time_global_res()`

>>>>  are used. To wait, `odp_time_wait_ns()` and `odp_time_wait_until()` are used,

>>>> -during witch a thread potentially busy loop the entire wait time.

>>>> +during which a thread potentially busy loops the entire wait time.

>>>>

>>>>  The `odp_time_t` opaque type represents local or global timestamps.

>>>>

>>>> +==== Portability Considerations

>>>> +The ODP Time APIs are designed to permit high-precision relative time

>>>> +measurement within an ODP application. No attempt is made to correlate an

>>>> +`odp_time_t` object with "wall time" or any other external time reference.

>>>> +As defined by the ODP specification, `odp_time_t` values are required to

>>>> +be unique over a span of at least 10 years. Most implementations will choose

>>>> +to implement time values using 64-bit values, whose wrap times exceed 500

>>>> +years, making wrapping concerns not relevant to ODP applications.

>>>> +

>>>

>>> Yes that is good addition. That means that odp time is not adjusted (ptp

>>> or ntp) and can not go backwards. I.e. each next call you will get

>>> updated incremented value. No time adjustments.

>>

>> I was confused by the bit about no correlation with wall time. What are the

>> Time APIs good for then? ;) If it is more accurate to say something about

>> clock drift, synchronization, thread vs. cpu time then that would be much better!

> 

> They are not correlated with wall time in that you cannot take the

> output from these calls and convert them to a calendar date / time in

> any portable manner. They simply record the monotonic passage of

> elapsed time and can be used to measure time differences between point

> A and point B as taking so many nanoseconds.

> 

>>

>>> Best regards,

>>> Maxim.

>>>

>>>>  === Timer

>>>>  Timers are how ODP applications measure and respond to the passage of time.

>>>>  Timers are drawn from specialized pools called timer pools that have their

>>>>

>>>
diff mbox

Patch

diff --git a/doc/users-guide/users-guide.adoc b/doc/users-guide/users-guide.adoc
index 41c57d1c..05bade8c 100755
--- a/doc/users-guide/users-guide.adoc
+++ b/doc/users-guide/users-guide.adoc
@@ -362,31 +362,41 @@  PktIOs are represented by handles of abstract type `odp_pktio_t`.
 
 === Time
 The time API is used to measure time intervals and track time flow of an
-application and presents a convenient way to get access to a time source.
-The time API consists of two main parts: local time API and global time API.
+application and presents a convenient way to get access to an
+implementation-defined time source. The time API consists of two main parts:
+local time API and global time API.
 
 ==== Local time
-The local time API is designed to be used within one thread and can be faster
-than the global time API. The local time API cannot be used between threads as
-time consistency is not guaranteed, and in some cases that's enough.
-So, local time stamps are local to the calling thread and must not be shared
-with other threads. Current local time can be read with `odp_time_local()`.
+The local time API is designed to be used within one thread and obtaining
+local time may be more efficient in some implementations than global
+time. Local time stamps are local to the calling thread and should not be
+shared with other threads, as local time is not guaranteed to be consistent
+between threads. Current local time can be read with `odp_time_local()`.
 
 ==== Global time
 The global time API is designed to be used for tracking time between threads.
-So, global time stamps can be shared between threads. Current global time can
-be read with `odp_time_global()`.
+So, global time stamps may safely be shared between threads. Current global
+time can be read with `odp_time_global()`.
 
-Both, local and global time is not wrapped during the application life cycle.
+Both local and global time is not wrapped during the application life cycle.
 The time API includes functions to operate with time, such as `odp_time_diff()`,
 `odp_time_sum()`, `odp_time_cmp()`, conversion functions like
 `odp_time_to_ns()`, `odp_time_local_from_ns()`, `odp_time_global_from_ns()`.
 To get rate of time source `odp_time_local_res()`, `odp_time_global_res()`
 are used. To wait, `odp_time_wait_ns()` and `odp_time_wait_until()` are used,
-during witch a thread potentially busy loop the entire wait time.
+during which a thread potentially busy loops the entire wait time.
 
 The `odp_time_t` opaque type represents local or global timestamps.
 
+==== Portability Considerations
+The ODP Time APIs are designed to permit high-precision relative time
+measurement within an ODP application. No attempt is made to correlate an
+`odp_time_t` object with "wall time" or any other external time reference.
+As defined by the ODP specification, `odp_time_t` values are required to
+be unique over a span of at least 10 years. Most implementations will choose
+to implement time values using 64-bit values, whose wrap times exceed 500
+years, making wrapping concerns not relevant to ODP applications.
+
 === Timer
 Timers are how ODP applications measure and respond to the passage of time.
 Timers are drawn from specialized pools called timer pools that have their