(Emacs: -*- indented-text -*-)


			  Implementation of
	Nanosecond Time and a PPS API for the Linux 2.4 Kernel

	      Copyright (c) 1996 - 2003 by Ulrich Windl
		 <Ulrich.Windl@rz.uni-regensburg.de>
			   29th August 2003

		 This file describes ``PPSkit 2.1'',
	      a small collection of files to support the
	      ``Kernel Model for Precision Timekeeping''
		      as described in RFC-1589,
		later updates known as ``nanokernel'',
		  combined with an implementation of
	     the ``PPS API v1'' as described in RFC-2783.

		 Mainly these algorithms are used by
	  NTPv3 (Network Time Protocol, RFC1305) and NTPv4.

Overview:
--------

This collection contains:

     0) A new kernel that keeps time in nanoseconds (instead of
        microseconds).  These changes come along with several other
        improvements and cleanups.

     1) Extensions and bugfixes for adjtimex() to adjust the value of
        `tickadj' (defaults to 500/HZ). This can be useful when the
        adjtime() doesn't work at all or is too slow (1ms on Alpha
        architecture, 0.5ms per second by default on i386
        architecture).

     2) New code to support PPS (pulse-per-second) clock
        synchronization in the kernel (also known as ``nanokernel #4'').

     3) Example implementation for the serial driver (implements
        detection of pulse on DCD pin).  The implementation uses the
        new PPS API for portability and nanosecond accuracy.

     4) Some utilities and documentation files (like this).


Performance data:
----------------

On my Pentium-100 with an Intel Triton II/VX chipset I have measured
some (rather short-term) performance data (using my Meinberg GPS 167
as PPS source).  The user-visible time resolution is about 3µs.

---------------------------------------------
Raw offset correction |	Jitter    (all in ns)
samples	mean	sigma |	samples	mean	sigma
---------------------------------------------
 2713	19.08	82.12	2713	1462	1792
 1743	 2.872	18.36	1743	 947.3	1047
24717  -26.85  142.82  24717    2561    3418
 6651   -9.29   53.36   6651    2438    2744
[missing]
 5510  -74.95  102.04   5510    2777    4136 [*See note*]
(when ignoring about the first 500 samples we get)
 2213	 7.604	 9.74	2213	1364	1460
 1367	 0.600	 5.93	1367	 944.1	1056
24217  -27.16   77.97  24217    2566    3386
 6651   -9.29   53.36   6651    2438    2744 [*See note*]
 5234  -62.84   90.18   5234    2757    4174
(this is PPSkit-0.9.3 with Linux-2.2.16 and GPS without SA after warmup)
 8174  -33.63   55.06   8174    2780    1359

[*Note*]: All lines before this one probably have a wrongly computed sigma!


A short History of Changes and Plans:
------------------------------------

* In Linux 2.0.30 there was a module-hook for a function ``hardpps''
  to handle the timekeeping stuff.  As the code for timekeeping does
  not depend on the way the signal is fed into the hardware, but only
  on how the signal is detected, it was decided to add the PPS
  timekeeping code to kernel/time.c and to remove that hook.

* The new ``hardpps'' routine expects the time of the pulse to be
  passed as parameters.

* The interrupt routine of the serial driver collects precision
  timestamps that can be retrieved via the PPS API.  They can also be
  automatically passed to a ``hardpps()'' kernel routine to discipline
  the clock.  Care has been taken that the serial driver is still
  usable without PPS applications.

* The xntp package contains code to support a CIOGETEV ioctl that
  reads precision time-stamps of external events on the carrier detect
  line of a serial port.  CIOGETEV has been implemented as a new line
  discipline that fills the structure `ppsclockev'.  That structure
  contains an event count and a ``struct timeval'' of the last event.
  The code has been adapted from a more complete solution made by
  Harald Koenig.  [Removed in 2.0]

* A former implementation of the FreeBSD-like ``TIOCDCDTIMESTAMP'' has
  been removed again, because it required a change to the
  serial_struct, thus causing incompatibilities for existing binaries.
  Also, the CIOGETEV gets the same data.  H. Peter Anvin had
  implemented that function around Linux 2.1.40.

* The adjtimex() system call has been extended to allow reading and
  modifying of ``tickadj''.  This is not strictly required for
  accurate clock operation, but is rather helpful if the default slew
  rate of 0.5ms per second is too slow for a larger correction made by
  ``adjtime()''.

* The PPSkit (0.4) has been merged into Linux-2.2 shortly before it
  came out.  Thus 2.2.0 has some essential time fixes that older
  versions did not have (these fixes entered 2.0 when 2.1 already
  existed).

* In preparation for exchanging the major part of the NTP code, the
  code has been restructured for PPSkit-0.5.  The next generation will
  have 64bit quantities measuring nanoseconds.  Currently noone in the
  kernel provides nanoseconds, but we'll be prepared.

* In PPSkit-0.6 the kernel was converted to nanoseconds by brute
  force.  Along with the new 64bit time variables came the new kernel
  clock model known as the ``nanokernel'' (because it features
  nanoseconds and ``STA_NANO'').  It is planned to bring some of the
  related fixes back to the stable standard kernel.

* PPSkit-0.7 finally brought the PPS API ("03" dated 1999-02-11) as
  well as an important fix for the jitter in the time stamps (Many
  thanks to Reg Clemens).  The reliability of the ``hardpps()''
  routine has been improved.

* PPSkit-0.8 needed a new but incompatible implementation of the PPS
  API ("05" dated 1999-08-17).

* PPSkit-0.9 featured an updated ``hardpps()'' routine from
  ``nanokernel #3'', dated ``1999-08-29''.  It is possible to tune the
  kernel time zone and periodic updates of the RTC chip (which are
  done whenever the system time changes).

* PPSkit-1.0 featured a revised clock model (``nanokernel #4'') and
  revised conversion routines for the i386 architecture.

* PPSkit-2.0, targeted for Linux-2.4.0, drops ``CIOGETEV'' that is
  obsoleted by ``PPS API''.

* PPSkit-2.1 introduced a new utility, PPSspy.pl, to watch the PPS-related
  kernel variables.

Notes on Usage:
--------------

* The old, now obsolete, code to activate PPS event processing has
  been replaced with the PPS API.  The old code was like the
  following:

	struct serial_struct ss;
	ioctl(fd, TIOCGSERIAL, &ss);	/* enable pulse detection on DCD */
	ss.flags |= ASYNC_PPS_CD_POS;	/* or ``ASYNC_PPS_CD_NEG'' */
	ioctl(fd, TIOCSSERIAL, &ss);

  The new code uses the functions ``time_pps_create'' and
  ``time_pps_set_param'' to activate capturing of events.  The
  following code is from ``draft-mogul-pps-api-05.txt'' (comments and
  corrections added):

      #include <sys/timepps.h>

      int fd;
      pps_handle_t handle;	/* handle for PPS sources */
      pps_params_t params;	/* selection of events, options */
      pps_info_t infobuf;	/* status of events */
      struct timespec timeout;	/* waiting time */
      int mode;			/* supported mode bits */

      /* Open a file descriptor and enable PPS on rising edges */
      fd = open(PPSfilename, O_RDWR, 0);
      time_pps_create(fd, &handle);
      time_pps_getcap(handle, &mode);
      if ((mode & PPS_CAPTUREASSERT) == 0) {
          fprintf(stderr, "%s cannot currently CAPTUREASSERT; mode=%#04x\n",
                PPSfilename, mode);
          exit(1);
      }
      time_pps_getparams(handle, &params);
      params.mode |= PPS_CAPTURE_ASSERT;	/* capture ASSERT events */
      params.mode |= PPS_TSFMT_TSPEC;	/* use ``struct timespec'' format */
      time_pps_setparams(handle, &params);

      /* create a zero-valued timeout */
      timeout.tv_sec = 0;
      timeout.tv_nsec = 0;

      /* loop, printing the most recent timestamp every second or so */
      while (1) {
          sleep(1);
          time_pps_fetch(handle, PPS_TSFMT_TSPEC, &infobuf, &timeout);
          printf("Assert timestamp: %d.%09d, sequence: %ld\n",
                 infobuf.assert_timestamp.tv_sec,
                 infobuf.assert_timestamp.tv_nsec,
                 infobuf.assert_sequence);
      }

  Implementation note: The ASSERT event has been defined in terms of
  the ``DCD'' modem status register bit of the UART, i.e. `assert''
  means the bit has been set (the opposite applies to ``clear''
  events).  The ECHO feature has been implemented using the ``RTS''
  bit in the modem status register of the UART.  An active event will
  be signalled by clearing the bit, i.e. for an echoed ``assert''
  event the bit will be cleared and for an echoed ``clear'' event the
  bit also will be cleared. The bit will be set if all echoed event
  become inactive, i.e. if echoing both events, the bit will stay
  cleared after the first event..

  With ntp-4.0.99f the PPS event processing code is activated
  automatically with a correct configuration.  This is at least true
  for the PARSE based drivers.  Here is an example what ntptime may
  report (on a Celeron-500):

ntp_gettime() returns code 0 (OK)
  time bdf4ec7c.1c2afdd0  Wed, Dec 27 2000 23:32:28.110, (.110031991),
  maximum error 1985 us, estimated error 0 us.
ntp_adjtime() returns code 0 (OK)
  modes 0x0 (),
  offset 0.108 us, frequency 115.251 ppm, interval 128 s,
  maximum error 1985 us, estimated error 0 us,
  status 0x2107 (PLL,PPSFREQ,PPSTIME,PPSSIGNAL,NANO),
  time constant 6, precision 2.878 us, tolerance 496 ppm,
  pps frequency 115.251 ppm, stability 0.017 ppm, jitter 2.605 us,
  intervals 88, jitter exceeded 69, stability exceeded 4, errors 0.

  ``ntpq -p'' says something like:

     remote           refid      st t when poll reach   delay   offset  jitter
==============================================================================
+GENERIC(1)      .GPS.            0 l   51   64  377    0.000    0.003   0.000
oPPS(1)          .PPS.            0 l   60   64  377    0.000    0.004   0.000

* The current implementation still leaves some room for improvement, while
  already working fine at several places.  As the additional code is
  usually inactive, only a few additional CPU cycles are needed.  Still,
  nanoseconds don't come for nothing.  Measurements on my Pentium 100MHz
  have shown that the `hardware_pps' routine takes between 250 and 12000
  CPU cycles per call (with debugging messages enabled, of course). For my
  Celeron-500 the minimum is about 230 cycles, while the maximum is at
  about 50000 cycles.

* I once wrote the test program `gen_pps.c' that uses the RTC driver
  (see /proc/rtc) to create a PPS pulse on the UART's RTS pin (pin 4
  on "COM2" (25 pins)).  The program needs the device name for the
  port to use (e.g. /dev/cua0). I made a very simple cable between the
  DB9 and DB25 connector:

  (input)	(output)
  DB9:		DB25:
  ---		----
  1 -----------	4	(CD, RTS)
  5 -----------	7	(GND, GND)

  Unfortunately the RTC driver seems to have some problems (last seen
  in 2.0.36): From time to time the kernel says: ``rtc: lost some
  interrupts at 2Hz.'' (This is because updating the RTC will reset
  the divider for the periodic interrupt)
  [You should get a real PPS source, not this cheap trick]

* Yet another program (enable_pps.c) can be used to enable detection
  of the PPS signal on the CD pin and exercise the kernel clock
  machinery.  That program expects standard input to be redirected
  from the desired port, event processing will be disabled unless
  specific options are specified.

  I activate PPS processing using the command (ASSERT event, hardpps)
  ``enable_pps -tah -j -k </dev/refclock-1'' (You can use this command
  to gather raw statistics about your pulses).

* The programs are just my test programs to validate the code; don't
  expect them to synchronize your time!  Even worse, they may
  de-adjust your kernel clock badly.  Use ntp-4.0.99 (or later) for
  nice and stable NTP support.

* Usually you need some level converter to connect the TTL level
  output of a clock to the CD input of the serial port.  A friend of
  mine and I have developed a simple converter that is powered from
  the serial port's status lines.  There's also some information about
  a sample device (`gadget box') on the NTP home page
  http://www.ntp.org/.

* To get a quick start with ntp configuration and PPS read the
  documentation (that comes with ntp) on ``PPS'' (pps.html) and
  ``enable pps'' (prefer.html) and ``refclock ATOM'' (driver22.html).
  You are kindly advised to read ``debug.html'' before reporting
  problems to the NTP developers or to the newsgroup
  comp.protocols.time.ntp.  There's also an older FAQ, and a newer
  one, and a very new one...  (see http://www.ntp.org/ntpfaq/NTP-a-faq.htm)

* Updating the RTC: The RTC is updated automatically if either the
  system time is set, or if ``STA_UNSYNC'' in ``time_status'' is
  clear.  You can disable the feature if you set ``rtc_update'' to
  zero.  If ``rtc_runs_localtime'' is non-zero, ``sys_tz'' is used to
  compute the local time.  By default, RTC is set to UTC.

  NOTE 1: In my SuSE Linux 6.3 system the ``tz_minuteswest'' were -120
          instead of -60 for MET.  I had to correct that value via
          ``echo -60 0 >/proc/sys/kernel/time/timezone''.

  WARNING: The sysctl interface does only limited error checking, so be
           very careful when writing variables!  I warned you, so
           don't complain.

* I use the following lines to set up the system during boot, assuming
  the correct timezone has been set before:

timezone=$(date +%z | sed -e 's/\([0-9][0-9]\)\([0-9][0-9]\)/(60*10#\1+10#\2)/')
TIMESYSCTL=/proc/sys/kernel/time
[ -w $TIMESYSCTL/timezone ] && echo $((-$timezone)) 0 >$TIMESYSCTL/timezone
[ -w $TIMESYSCTL/rtc_runs_localtime ] && echo 1 >$TIMESYSCTL/rtc_runs_localtime
