Hardware clock (PHC) and phc2sys#

Why#

Even with a well-configured real-time kernel, a timestamp taken in software (when the kernel processes the packet) accumulates 10 to 100 µs of non-deterministic latency between the signal’s physical arrival on the network cable and the moment the timestamp is actually recorded: PHY reception, DMA to RAM, interrupt handling, network stack traversal. To serve clients with better than ten-microsecond precision, this latency needs to be taken out of the equation, by having the network card itself timestamp the packet, before any software processing.

How#

The PHC (PTP Hardware Clock)#

The Raspberry Pi 5’s network controller exposes a dedicated hardware clock accessible via /dev/ptp0, the PHC, standardized by IEEE 1588. Chrony can use it directly to timestamp incoming and outgoing NTP packets at the silicon level, independent of kernel latency.

# In chrony.conf
hwtimestamp end0 rxfilter all txcomp 0 rxcomp 0 minpoll 0

hwtimestamp end0 alone is incomplete: without these explicit options, chrony has to guess the best settings, which doesn’t always work reliably. rxfilter all requests hardware timestamping for every received packet, not just those the controller’s filter recognizes as PTP (otherwise ordinary NTP traffic wouldn’t get the hardware timestamp at all). txcomp 0 enables hardware timestamping at transmit completion, with no additional software compensation (0). rxcomp 0 does the same on reception: no correction is applied afterward, the raw hardware timestamp is taken as-is. minpoll 0 imposes no artificial floor on measurement frequency beyond the standard 2⁰ = 1 second granularity.

phc2sys: keeping the PHC aligned#

The PHC needs to be kept synchronized against the system clock (itself disciplined by chrony from the PPS). That’s phc2sys’s job, from the linuxptp package:

cat > /etc/systemd/system/phc2sys-chrony.service <<'EOF'
[Unit]
Description=Synchronize PHC to system clock
After=chrony.service
Wants=chrony.service

[Service]
Type=simple
ExecStart=/usr/sbin/phc2sys -s CLOCK_REALTIME -c /dev/ptp0 -O 0 -R 1 -q
Restart=always
RestartSec=5

CPUSchedulingPolicy=fifo
CPUSchedulingPriority=90
CPUAffinity=1

[Install]
WantedBy=multi-user.target
EOF

systemctl enable --now phc2sys-chrony.service

-s CLOCK_REALTIME names the system clock (disciplined by chrony from the PPS) as the reference source for the PHC, -c /dev/ptp0 the target to align. -O 0 sets the TAI-UTC offset to zero: that’s not an oversight of leap seconds, chrony (via leap-seconds.list, dedicated chapter) already handles that aspect, phc2sys doesn’t need to duplicate it. -q (quiet) suppresses the default logging, otherwise the /var/log tmpfs (Filesystem chapter) would quickly fill up with one line per measurement.

Pitfalls to avoid#

Requires=chrony.service silently breaks phc2sys in production. A systemd Requires= dependency automatically stops the dependent service on every state transition of its dependency, including a plain restart or a temporary chrony failure. A real incident on this project: phc2sys would silently stop on every chrony cycle, leaving the PHC to drift at 94 µs per second with no visible alert. The fix: Wants=chrony.service (a soft dependency that doesn’t cascade a stop), combined with Restart=always for resilience against real crashes.

phc2sys’s measurement rate -R should be reduced, not maximized. The PHC is a single hardware counter, read and written concurrently by two processes: phc2sys, which writes it to keep it aligned, and chrony, which reads it to timestamp incoming packets. The network driver doesn’t explicitly serialize these accesses. Real-world measurements: at -R 16 (linuxptp’s default), only 63% of packets get a hardware timestamp (the rest silently fall back to a less precise software timestamp); at -R 1, that rate climbs to 86%. Reducing the write frequency reduces contention, at the cost of a marginally higher PHC offset, but one that stays well under a microsecond.

• Expecting near-100% hardware TX timestamping. On the Pi 5, the network driver only sets the hardware flag required for TX timestamping on packets recognized as PTP by the hardware filter, “normal” NTP packets don’t get it. This is a documented driver bug (raspberrypi/linux, issue #5904), not a configuration mistake: the hardware TX timestamping rate stays near 0% regardless of configuration. The real client-side impact is modest (tens of microseconds at worst, largely filtered out by the client’s own PLL).

Verify#

# The PHC should be detected on the interface
ethtool -T end0
# Should show "PTP Hardware Clock: 0" (or another number, not "none")

# Instantaneous offset between the PHC and the system clock
phc_ctl /dev/ptp0 cmp
# e.g.: "offset from CLOCK_REALTIME is 14ns"

A single phc_ctl cmp reading is an unfiltered snapshot, so it’s occasionally subject to a one-off scheduling spike (a few hundred ns to a few µs). For a reliable diagnostic, repeat the reading several times at real-time priority (chrt -f 99) and take the median, the exact same statistical filtering logic chrony uses for the PPS (Chrony chapter).

# Effective hardware timestamping rate, from chrony's side
chronyc -h /run/chrony/chronyd.sock serverstats | grep -i hardware

Current limitation: the PPS is not hardware-timestamped#

This Pi 5’s PHC timestamps network packets, but it cannot timestamp the PPS: the onboard network controller (Cadence GEM via the RP1 southbridge) exposes no timing pins, and the Pi 5 provides no “PPS IN” input routed to the network card. There is therefore no hardware entry point to inject the GNSS PPS into the PHC. Direct check:

cat /sys/class/ptp/ptp0/n_external_timestamps   # 0: no EXTTS input
cat /sys/class/ptp/ptp0/n_periodic_outputs      # 0: no PEROUT output
cat /sys/class/ptp/ptp0/n_programmable_pins     # 0: no SDP pin

The pps_available = 1 in the same directory only denotes an internal PPS emitted by the PHC to the kernel API, not an input for an external signal. As a result, the GNSS PPS can only be captured through the GPIO, via pps-gpio and a kernel interrupt, whose latency (~5 to 30 µs on PREEMPT_RT, Real-time kernel chapter) sets the system’s noise floor. That is the fundamental reason performance sits “around the microsecond” and not at the nanosecond level.

[ALTERNATIVE] Hardware-timestamp the PPS with a timing-pin NIC. A network card exposing its SDP pins would lift this limitation. The TimeHATv6 PCIe HAT (Intel i226 NIC) exposes them: an SMA input (SDP1) timestamps the PPS edge in silicon (EXTTS), an SMA output (SDP0) generates a PPS (PEROUT), and an M.2 slot houses a GNSS module whose 1PPS is routed directly to the NIC; a built-in TCXO provides holdover during a network outage. The PHC would then become a true disciplining reference, potentially at the tens-of-nanoseconds level, rather than a mere packet timestamper. A promising evolution path.

The PPS sawtooth error#

Even with the PPS hardware-timestamped, one finer error source would remain, specific to the receiver itself: the sawtooth error.

The receiver can only place the PPS edge on a tick of its internal clock, a quantized grid. The exact UTC second almost always falls between two ticks; the receiver keeps the nearest one, and a small offset remains on every pulse. Since the internal frequency is not an exact integer multiple of one hertz, this offset drifts slowly from second to second, reaches half a clock period, then flips to the other side and starts over, hence the sawtooth pattern:

 err
+T/2 ┤   /|   /|   /|
   0 ┤  / |  / |  / |
     │ /  | /  | /  |
-T/2 ┤/   |/   |/   |
     └────┴────┴────┴──►  t

Its amplitude is bounded by half the period of the timepulse’s reference clock: a few nanoseconds to a few tens of nanoseconds on this receiver (the M9N’s “timepulse ~30 ns RMS” spec already includes it).

This sawtooth is inherent to the PPS generation, not its capture: hardware timestamping (the HAT above) removes the interrupt jitter, but not the sawtooth, already baked into the edge. The two corrections are therefore complementary.

Compensating it. The receiver measures how far it missed each second and publishes that value, the quantization error. On u-blox, it is carried by the binary UBX-TIM-TP message (qErr field, in picoseconds, signed, one value per pulse): just read qErr, associate it with the right pulse, and subtract it from the edge’s timestamp to bring the sawtooth down to the sub-nanosecond level. ntpsec includes this correction for u-blox; chrony does not do it natively, it would need an intermediary reading UBX-TIM-TP.

On this server, the chain only uses NMEA (GxRMC/GxGGA, NMEA sentences chapter), which does not carry qErr: compensating the sawtooth would require enabling the binary UBX protocol, thus stepping outside the project’s “NMEA only, reproducible everywhere” choice. The gain (~ns) would anyway stay hidden today under the roughly microsecond noise floor of the PPS/GPIO chain. The logical order toward the nanosecond is therefore: hardware timestamping first, and only then the sawtooth correction.

📚 Going further

Hardware timestamping is standardized by IEEE 1588 (Precision Time Protocol, latest revision 2019). Linux exposes it via the SO_TIMESTAMPING socket option (since kernel 2.6.30): when chrony enables hwtimestamp end0, it asks the kernel, via setsockopt(), for receive and transmit timestamps to be taken directly in the network card’s silicon rather than in the software stack, typically achieving sub-100ns precision once PHC/chrony contention is under control (measurements published by M. Lichvar, chrony’s maintainer, at Linux.conf.au 2020).