There is a cruel standards author out there somewhere who said that SNTP could be used to achieve micro-second level synchronization. This has led far too many people into mistaking it for an easy task.
First, consider the timing issue of transmitting the SNTP query/response over ethernet. The message size is about 100 bytes, including prefix and trailer sync signals. So, on a 100 Mbit/s link this is 8 microseconds. But, if it's a switch there is another prefix related delay. And, if there is another message in transit at that moment the switch will buffer the SNTP message until that other message finishes. A maximum length message (1500 bytes including prefix and trailer) will take 120 microseconds. Since the switch will start the next message as soon as possible, this introduces a variation of 0-120 microseconds. SNTP does not include the statistical estimation information needed to remove these variations.
To potentially reach microsecond level synchronization with query/response you must do the following:
- Have a direct connection between client and source with no other systems or any other use of that link. It can only be a one to one connection. This eliminates the variability from other traffic. (NTP has statistical estimator logic, so the variability can be removed.)
- OR, have a single purpose network with only source broadcast and all client query eliminated. This also eliminates that potential variability. This allows one source to send time to other clients. The clients must dedicate a network device to the exclusive use of the time system.
But, that is just eliminating the network variability. What about system clock variability? How long does it take for the OS to process the incoming network message and adjust its internal time? When an application issues a time query to the OS, how long is the delay between request and response? how accurate is the internal time computation?
The OS network delays can be measured. You just need a high precision interval timer, logic analyzer, and strict configuration control over the ethernet switch. Then you measure these delays for your system. The biggest difficulty is within the OS. Real time OS usually have stable interrupt response characteristics, so once you measure it, you can then use that as a correction factor. But, general purpose OS, such as Windows, do not have a predictable interrupt response latency at the microsecond level. If the OS gets really busy on high priority internal tasks it can take hundreds of microseconds to respond. The network is usually treated as a low priority interrupt by general purpose OS.
Again, NTP uses statistical estimators to reduce this variability.
The next issue is, how accurate is the internal time? The SNTP broadcasts can't be too frequent. If you are constantly pinging the systems with high priority SNTP traffic, there won't be any time for the primary function. The practical limit is usually 100-1000 pings per second. So for at least a few milliseconds the local system clock needs to provide that microsecond level time measurement. How will it do it?
The very old clock methods were simple but inaccurate. The clock would just be incremented when the clock tick arrived. If the SNTP broadcast is 100/s, then you increment the clock by 10ms each time. This works fine if all you care about is 1s accuracy and don't mind that everything that happens during that 10ms interval is assigned to the exact same time. For lots of administrative tasks this is a good solution. It's cheap and easy.
For the microsecond level timing none of the purely tick related approaches will work. There is no OS that can both get useful work done and process a million timing ticks per second. Even the network cannot manage that. You need a gigabit network before it can carry that many tick messages. Instead, you need a local free running clock that provides a short term timer for periods of about a second.
Most simple CPUs and systems have such a timer. The original PC had a separate interval timer chip that had three clocks. One was programmed to provide an interrupting clock tick at a slow interval of 10-100 ticks per second. This was used to maintain the long duration system clock. Another could be programmed to increment an internal timer much more rapidly. A typical increment rate might be every 10 microseconds. This rate was usually limited by the available hardware crystal timing source. The same capabilities are present in modern CPUs, usually with a faster hardware crystal and often integrated with other components onto a single chip.
In a system like that, when the application needs a time marker, it issues an OS request. The OS uses the long duration system timer to get a base time and then corrects this by reading the high speed timer value and computing the proper offset. If the OS has a proper real time priority structure this all happens with a fairly stable speed. You can measure it with proper interval timer, logic analyzer, and similar equipment. You get microsecond level accuracy. Another excellent and easy to use high speed timer is the CPU cycle counter available on most CPUs. But, multiprocessor CPUs and power saving through clock speed changes make this impractical on most new CPUs. This is one reason that measurement equipment tends to avoid the very high performance CPUs.
The NTP systems will use the same approaches to obtain accuracy. They use statistical estimation theory to eliminate some of the variability, so they can be used on a wider variety of operating systems and often without the use of logic analyzers to measure internal computer state. (With new integrated processors it is hard to get the access needed to use a logic analyzer for these purposes.) But we're dealing with the claim of microsecond level accuracy from SNTP.
The NTP systems also enable the use of atomic clock controlled pulse generators. These can be attached to internal interval timers or they can be used as simple edge pulse timers. SNTP lacks the logic to manage combining of clocks into a single more accurate time estimate. The NTP use of atomic clocks as internal interval timers or as edge timers is based on using the atomic clock to measure the drift rate of the other clocks. It is normal for the internal timer clocks of computers to be inaccurate by up to 1 ppm. This can usually be ignored for administrative purposes. One ppm is a drift of 1 second every two weeks. It's stable enough to be accurate through a very long network dropout. When using NTP and an atomic clock the drift rate can be measured, and it can be removed every second. (Most simple atomic clocks are set at 1 pulse per second). With a measured drift rate there is a residual drift due to temperature and atmospheric pressure changes to the crystal. These are usually under 0.01 ppm. So you've met the sub microsecond requirement. SNTP does not include the information needed to do this.
So we have a cruel standards writer. Yes, you can make microsecond level accuracy with SNTP. Of course you also need specialized network design, logic analyzers, a real time OS, precision interval timers, and a lot of custom product engineering.
First, consider the timing issue of transmitting the SNTP query/response over ethernet. The message size is about 100 bytes, including prefix and trailer sync signals. So, on a 100 Mbit/s link this is 8 microseconds. But, if it's a switch there is another prefix related delay. And, if there is another message in transit at that moment the switch will buffer the SNTP message until that other message finishes. A maximum length message (1500 bytes including prefix and trailer) will take 120 microseconds. Since the switch will start the next message as soon as possible, this introduces a variation of 0-120 microseconds. SNTP does not include the statistical estimation information needed to remove these variations.
To potentially reach microsecond level synchronization with query/response you must do the following:
- Have a direct connection between client and source with no other systems or any other use of that link. It can only be a one to one connection. This eliminates the variability from other traffic. (NTP has statistical estimator logic, so the variability can be removed.)
- OR, have a single purpose network with only source broadcast and all client query eliminated. This also eliminates that potential variability. This allows one source to send time to other clients. The clients must dedicate a network device to the exclusive use of the time system.
But, that is just eliminating the network variability. What about system clock variability? How long does it take for the OS to process the incoming network message and adjust its internal time? When an application issues a time query to the OS, how long is the delay between request and response? how accurate is the internal time computation?
The OS network delays can be measured. You just need a high precision interval timer, logic analyzer, and strict configuration control over the ethernet switch. Then you measure these delays for your system. The biggest difficulty is within the OS. Real time OS usually have stable interrupt response characteristics, so once you measure it, you can then use that as a correction factor. But, general purpose OS, such as Windows, do not have a predictable interrupt response latency at the microsecond level. If the OS gets really busy on high priority internal tasks it can take hundreds of microseconds to respond. The network is usually treated as a low priority interrupt by general purpose OS.
Again, NTP uses statistical estimators to reduce this variability.
The next issue is, how accurate is the internal time? The SNTP broadcasts can't be too frequent. If you are constantly pinging the systems with high priority SNTP traffic, there won't be any time for the primary function. The practical limit is usually 100-1000 pings per second. So for at least a few milliseconds the local system clock needs to provide that microsecond level time measurement. How will it do it?
The very old clock methods were simple but inaccurate. The clock would just be incremented when the clock tick arrived. If the SNTP broadcast is 100/s, then you increment the clock by 10ms each time. This works fine if all you care about is 1s accuracy and don't mind that everything that happens during that 10ms interval is assigned to the exact same time. For lots of administrative tasks this is a good solution. It's cheap and easy.
For the microsecond level timing none of the purely tick related approaches will work. There is no OS that can both get useful work done and process a million timing ticks per second. Even the network cannot manage that. You need a gigabit network before it can carry that many tick messages. Instead, you need a local free running clock that provides a short term timer for periods of about a second.
Most simple CPUs and systems have such a timer. The original PC had a separate interval timer chip that had three clocks. One was programmed to provide an interrupting clock tick at a slow interval of 10-100 ticks per second. This was used to maintain the long duration system clock. Another could be programmed to increment an internal timer much more rapidly. A typical increment rate might be every 10 microseconds. This rate was usually limited by the available hardware crystal timing source. The same capabilities are present in modern CPUs, usually with a faster hardware crystal and often integrated with other components onto a single chip.
In a system like that, when the application needs a time marker, it issues an OS request. The OS uses the long duration system timer to get a base time and then corrects this by reading the high speed timer value and computing the proper offset. If the OS has a proper real time priority structure this all happens with a fairly stable speed. You can measure it with proper interval timer, logic analyzer, and similar equipment. You get microsecond level accuracy. Another excellent and easy to use high speed timer is the CPU cycle counter available on most CPUs. But, multiprocessor CPUs and power saving through clock speed changes make this impractical on most new CPUs. This is one reason that measurement equipment tends to avoid the very high performance CPUs.
The NTP systems will use the same approaches to obtain accuracy. They use statistical estimation theory to eliminate some of the variability, so they can be used on a wider variety of operating systems and often without the use of logic analyzers to measure internal computer state. (With new integrated processors it is hard to get the access needed to use a logic analyzer for these purposes.) But we're dealing with the claim of microsecond level accuracy from SNTP.
The NTP systems also enable the use of atomic clock controlled pulse generators. These can be attached to internal interval timers or they can be used as simple edge pulse timers. SNTP lacks the logic to manage combining of clocks into a single more accurate time estimate. The NTP use of atomic clocks as internal interval timers or as edge timers is based on using the atomic clock to measure the drift rate of the other clocks. It is normal for the internal timer clocks of computers to be inaccurate by up to 1 ppm. This can usually be ignored for administrative purposes. One ppm is a drift of 1 second every two weeks. It's stable enough to be accurate through a very long network dropout. When using NTP and an atomic clock the drift rate can be measured, and it can be removed every second. (Most simple atomic clocks are set at 1 pulse per second). With a measured drift rate there is a residual drift due to temperature and atmospheric pressure changes to the crystal. These are usually under 0.01 ppm. So you've met the sub microsecond requirement. SNTP does not include the information needed to do this.
So we have a cruel standards writer. Yes, you can make microsecond level accuracy with SNTP. Of course you also need specialized network design, logic analyzers, a real time OS, precision interval timers, and a lot of custom product engineering.
Comments