[chrony-dev] [PATCH v2] sys_posix: support SCHED_FIFO and mlockall on more OSs

["Stefan R. Filipek" <srfilipek@xxxxxxxxx>]

Real-time scheduling and memory locking is available on posix compliant
OSs. This patch centralizes this functionality and brings support to
FreeBSD, NetBSD, and Solaris.
---
 configure            |  27 ++++++++---
 doc/chrony.conf.adoc |  46 +++++++++---------
 sys.c                |  13 +++--
 sys_linux.c          |  67 --------------------------
 sys_linux.h          |   4 --
 sys_posix.c          | 111 +++++++++++++++++++++++++++++++++++++++++++
 sys_posix.h          |  36 ++++++++++++++
 7 files changed, 198 insertions(+), 106 deletions(-)
 create mode 100644 sys_posix.c
 create mode 100644 sys_posix.h

diff --git a/configure b/configure
index c434127..9e21a8b 100755
--- a/configure
+++ b/configure
@@ -396,7 +396,7 @@ SYSTEM=${OPERATINGSYSTEM}-${MACHINE}
 
 case $OPERATINGSYSTEM in
     Linux)
-        EXTRA_OBJECTS="sys_generic.o sys_linux.o sys_timex.o"
+        EXTRA_OBJECTS="sys_generic.o sys_linux.o sys_timex.o sys_posix.o"
         [ $try_libcap != "0" ] && try_libcap=1
         try_rtc=1
         [ $try_seccomp != "0" ] && try_seccomp=1
@@ -411,7 +411,9 @@ case $OPERATINGSYSTEM in
         # recvmmsg() seems to be broken on FreeBSD 11.0 and it's just
         # a wrapper around recvmsg()
         try_recvmmsg=0
-        EXTRA_OBJECTS="sys_generic.o sys_netbsd.o sys_timex.o"
+        EXTRA_OBJECTS="sys_generic.o sys_netbsd.o sys_timex.o sys_posix.o"
+        try_setsched=1
+        try_lockmem=1
         add_def FREEBSD
         if [ $feat_droproot = "1" ]; then
           add_def FEAT_PRIVDROP
@@ -420,8 +422,10 @@ case $OPERATINGSYSTEM in
         echo "Configuring for $SYSTEM"
     ;;
     NetBSD)
-        EXTRA_OBJECTS="sys_generic.o sys_netbsd.o sys_timex.o"
+        EXTRA_OBJECTS="sys_generic.o sys_netbsd.o sys_timex.o sys_posix.o"
         try_clockctl=1
+        try_setsched=1
+        try_lockmem=1
         add_def NETBSD
         echo "Configuring for $SYSTEM"
     ;;
@@ -446,9 +450,11 @@ case $OPERATINGSYSTEM in
         echo "Configuring for macOS (" $SYSTEM "macOS version" $VERSION ")"
     ;;
     SunOS)
-        EXTRA_OBJECTS="sys_generic.o sys_solaris.o sys_timex.o"
+        EXTRA_OBJECTS="sys_generic.o sys_solaris.o sys_timex.o sys_posix.o"
         EXTRA_LIBS="-lsocket -lnsl -lresolv"
         EXTRA_CLI_LIBS="-lsocket -lnsl -lresolv"
+        try_setsched=1
+        try_lockmem=1
         add_def SOLARIS
         # These are needed to have msg_control in struct msghdr
         add_def __EXTENSIONS__
@@ -800,13 +806,20 @@ fi
 if [ $try_lockmem = "1" ] && \
   test_code \
     'mlockall()' \
-    'sys/mman.h sys/resource.h' '' '' '
-     struct rlimit rlim;
-     setrlimit(RLIMIT_MEMLOCK, &rlim);
+    'sys/mman.h' '' '' '
      mlockall(MCL_CURRENT|MCL_FUTURE);'
 then
   add_def HAVE_MLOCKALL
 fi
+if [ $try_lockmem = "1" ] && \
+  test_code \
+    'setrlimit(RLIMIT_MEMLOCK, ...)' \
+    'sys/resource.h' '' '' '
+     struct rlimit rlim;
+     setrlimit(RLIMIT_MEMLOCK, &rlim);'
+then
+  add_def HAVE_SETRLIMIT_MEMLOCK
+fi
 
 if [ $feat_forcednsretry = "1" ]
 then
diff --git a/doc/chrony.conf.adoc b/doc/chrony.conf.adoc
index 7aa2116..aa5cafe 100644
--- a/doc/chrony.conf.adoc
+++ b/doc/chrony.conf.adoc
@@ -2063,11 +2063,11 @@ file when the <<chronyc.adoc#rekey,*rekey*>> command is issued by *chronyc*).
 
 [[lock_all]]*lock_all*::
 The *lock_all* directive will lock chronyd into RAM so that it will never be
-paged out. This mode is only supported on Linux. This directive uses the Linux
-*mlockall()* system call to prevent *chronyd* from ever being swapped out. This
-should result in lower and more consistent latency. It should not have
-significant impact on performance as *chronyd's* memory usage is modest. The
-*mlockall(2)* man page has more details.
+paged out. This mode is supported on Linux, FreeBSD, NetBSD, and Solaris.  This
+directive uses the POSIX *mlockall()* system call to prevent *chronyd* from
+ever being swapped out. This should result in lower and more consistent
+latency. It should not have significant impact on performance as *chronyd's*
+memory usage is modest. The *mlockall(2)* man page has more details.
 
 [[pidfile]]*pidfile* _file_::
 Unless *chronyd* is started with the *-Q* option, it writes its process ID
@@ -2081,26 +2081,26 @@ pidfile /run/chronyd.pid
 ----
 
 [[sched_priority]]*sched_priority* _priority_::
-On Linux, the *sched_priority* directive will select the SCHED_FIFO real-time
-scheduler at the specified priority (which must be between 0 and 100). On
-macOS, this option must have either a value of 0 (the default) to disable the
-thread time constraint policy or 1 for the policy to be enabled. Other systems
-do not support this option.
-+
-On Linux, this directive uses the *sched_setscheduler()* system call to
-instruct the kernel to use the SCHED_FIFO first-in, first-out real-time
-scheduling policy for *chronyd* with the specified priority. This means that
-whenever *chronyd* is ready to run it will run, interrupting whatever else is
-running unless it is a higher priority real-time process. This should not
-impact performance as *chronyd* resource requirements are modest, but it should
-result in lower and more consistent latency since *chronyd* will not need to
-wait for the scheduler to get around to running it. You should not use this
-unless you really need it. The *sched_setscheduler(2)* man page has more
-details.
+On Linux, FreeBSD, NetBSD, and Solaris, the *sched_priority* directive will
+select the SCHED_FIFO real-time scheduler at the specified priority (which must
+be between 0 and 100). On macOS, this option must have either a value of 0 (the
+default) to disable the thread time constraint policy or 1 for the policy to be
+enabled.
++
+On systems other than macOS, this directive uses the *pthread_setschedparam()*
+system call to instruct the kernel to use the SCHED_FIFO first-in, first-out
+real-time scheduling policy for *chronyd* with the specified priority. This
+means that whenever *chronyd* is ready to run it will run, interrupting
+whatever else is running unless it is a higher priority real-time process. This
+should not impact performance as *chronyd* resource requirements are modest,
+but it should result in lower and more consistent latency since *chronyd* will
+not need to wait for the scheduler to get around to running it. You should not
+use this unless you really need it. The *pthread_setschedparam(3)* man page has
+more details.
 +
 On macOS, this directive uses the *thread_policy_set()* kernel call to
-specify real-time scheduling. As noted for Linux, you should not use this
-directive unless you really need it.
+specify real-time scheduling. As noted above, you should not use this directive
+unless you really need it.
 
 [[user]]*user* _user_::
 The *user* directive sets the name of the system user to which *chronyd* will
diff --git a/sys.c b/sys.c
index 2c42db1..a49ea5f 100644
--- a/sys.c
+++ b/sys.c
@@ -35,10 +35,13 @@
 
 #if defined(LINUX)
 #include "sys_linux.h"
+#include "sys_posix.h"
 #elif defined(SOLARIS)
 #include "sys_solaris.h"
+#include "sys_posix.h"
 #elif defined(NETBSD) || defined(FREEBSD)
 #include "sys_netbsd.h"
+#include "sys_posix.h"
 #elif defined(MACOSX)
 #include "sys_macosx.h"
 #endif
@@ -124,10 +127,10 @@ void SYS_EnableSystemCallFilter(int level)
 
 void SYS_SetScheduler(int SchedPriority)
 {
-#if defined(LINUX) && defined(HAVE_PTHREAD_SETSCHEDPARAM)
-  SYS_Linux_SetScheduler(SchedPriority);
-#elif defined(MACOSX)
+#if defined(MACOSX)
   SYS_MacOSX_SetScheduler(SchedPriority);
+#elif defined(HAVE_PTHREAD_SETSCHEDPARAM)
+  SYS_Posix_SetScheduler(SchedPriority);
 #else
   LOG_FATAL("scheduler priority setting not supported");
 #endif
@@ -137,8 +140,8 @@ void SYS_SetScheduler(int SchedPriority)
 
 void SYS_LockMemory(void)
 {
-#if defined(LINUX) && defined(HAVE_MLOCKALL)
-  SYS_Linux_MemLockAll(1);
+#if defined(HAVE_MLOCKALL)
+  SYS_Posix_MemLockAll(1);
 #else
   LOG_FATAL("memory locking not supported");
 #endif
diff --git a/sys_linux.c b/sys_linux.c
index 9e4ab3f..898dc7a 100644
--- a/sys_linux.c
+++ b/sys_linux.c
@@ -33,16 +33,6 @@
 
 #include <sys/utsname.h>
 
-#if defined(HAVE_PTHREAD_SETSCHEDPARAM)
-#  include <pthread.h>
-#  include <sched.h>
-#endif
-
-#if defined(HAVE_MLOCKALL)
-#  include <sys/mman.h>
-#include <sys/resource.h>
-#endif
-
 #if defined(FEAT_PHC) || defined(HAVE_LINUX_TIMESTAMPING)
 #include <linux/ptp_clock.h>
 #endif
@@ -633,63 +623,6 @@ add_failed:
 
 /* ================================================== */
 
-#if defined(HAVE_PTHREAD_SETSCHEDPARAM)
-  /* Install SCHED_FIFO real-time scheduler with specified priority */
-void SYS_Linux_SetScheduler(int SchedPriority)
-{
-  int pmax, pmin;
-  struct sched_param sched;
-
-  if (SchedPriority < 1 || SchedPriority > 99) {
-    LOG_FATAL("Bad scheduler priority: %d", SchedPriority);
-  } else {
-    sched.sched_priority = SchedPriority;
-    pmax = sched_get_priority_max(SCHED_FIFO);
-    pmin = sched_get_priority_min(SCHED_FIFO);
-    if ( SchedPriority > pmax ) {
-      sched.sched_priority = pmax;
-    }
-    else if ( SchedPriority < pmin ) {
-      sched.sched_priority = pmin;
-    }
-    if ( pthread_setschedparam(pthread_self(), SCHED_FIFO, &sched) == -1 ) {
-      LOG(LOGS_ERR, "pthread_setschedparam() failed");
-    }
-    else {
-      DEBUG_LOG("Enabled SCHED_FIFO with priority %d",
-          sched.sched_priority);
-    }
-  }
-}
-#endif /* HAVE_PTHREAD_SETSCHEDPARAM  */
-
-#if defined(HAVE_MLOCKALL)
-/* Lock the process into RAM so that it will never be swapped out */ 
-void SYS_Linux_MemLockAll(int LockAll)
-{
-  struct rlimit rlim;
-  if (LockAll == 1 ) {
-    /* Make sure that we will be able to lock all the memory we need */
-    /* even after dropping privileges.  This does not actually reaerve any memory */
-    rlim.rlim_max = RLIM_INFINITY;
-    rlim.rlim_cur = RLIM_INFINITY;
-    if (setrlimit(RLIMIT_MEMLOCK, &rlim) < 0) {
-      LOG(LOGS_ERR, "setrlimit() failed: not locking into RAM");
-    }
-    else {
-      if (mlockall(MCL_CURRENT|MCL_FUTURE) < 0) {
-	LOG(LOGS_ERR, "mlockall() failed");
-      }
-      else {
-	DEBUG_LOG("Successfully locked into RAM");
-      }
-    }
-  }
-}
-#endif /* HAVE_MLOCKALL */
-
-/* ================================================== */
-
 int
 SYS_Linux_CheckKernelVersion(int req_major, int req_minor)
 {
diff --git a/sys_linux.h b/sys_linux.h
index 799ae9a..551a186 100644
--- a/sys_linux.h
+++ b/sys_linux.h
@@ -35,10 +35,6 @@ extern void SYS_Linux_DropRoot(uid_t uid, gid_t gid, int clock_control);
 
 extern void SYS_Linux_EnableSystemCallFilter(int level);
 
-extern void SYS_Linux_MemLockAll(int LockAll);
-
-extern void SYS_Linux_SetScheduler(int SchedPriority);
-
 extern int SYS_Linux_CheckKernelVersion(int req_major, int req_minor);
 
 extern int SYS_Linux_OpenPHC(const char *path, int phc_index);
diff --git a/sys_posix.c b/sys_posix.c
new file mode 100644
index 0000000..c40dfbf
--- /dev/null
+++ b/sys_posix.c
@@ -0,0 +1,111 @@
+/*
+  chronyd/chronyc - Programs for keeping computer clocks accurate.
+
+ **********************************************************************
+ * Copyright (C) Richard P. Curnow  1997-2003
+ * Copyright (C) John G. Hasler  2009
+ * Copyright (C) Miroslav Lichvar  2009-2012, 2014-2018
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, write to the Free Software Foundation, Inc.,
+ * 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
+ *
+ **********************************************************************
+
+  =======================================================================
+
+  This module is for POSIX compliant operating systems.
+
+  */
+
+#include "config.h"
+
+#include "sysincl.h"
+
+#include <sys/utsname.h>
+
+#if defined(HAVE_PTHREAD_SETSCHEDPARAM)
+#include <pthread.h>
+#include <sched.h>
+#endif
+
+#if defined(HAVE_MLOCKALL)
+#include <sys/mman.h>
+#endif
+#if defined(HAVE_SETRLIMIT_MEMLOCK)
+#include <sys/resource.h>
+#endif
+
+#include "sys_posix.h"
+#include "conf.h"
+#include "local.h"
+#include "logging.h"
+#include "util.h"
+
+/* ================================================== */
+
+#if defined(HAVE_PTHREAD_SETSCHEDPARAM)
+/* Install SCHED_FIFO real-time scheduler with specified priority */
+void SYS_Posix_SetScheduler(int SchedPriority)
+{
+  int pmax, pmin;
+  struct sched_param sched;
+
+  if (SchedPriority < 1 || SchedPriority > 99) {
+    LOG_FATAL("Bad scheduler priority: %d", SchedPriority);
+  } else {
+    sched.sched_priority = SchedPriority;
+    pmax = sched_get_priority_max(SCHED_FIFO);
+    pmin = sched_get_priority_min(SCHED_FIFO);
+    if (SchedPriority > pmax) {
+      sched.sched_priority = pmax;
+    } else if (SchedPriority < pmin) {
+      sched.sched_priority = pmin;
+    }
+
+    if (pthread_setschedparam(pthread_self(), SCHED_FIFO, &sched) == -1) {
+      LOG(LOGS_ERR, "pthread_setschedparam() failed");
+    } else {
+      DEBUG_LOG("Enabled SCHED_FIFO with priority %d", sched.sched_priority);
+    }
+  }
+}
+#endif /* HAVE_PTHREAD_SETSCHEDPARAM  */
+
+/* ================================================== */
+
+#if defined(HAVE_MLOCKALL)
+/* Lock the process into RAM so that it will never be swapped out */
+void SYS_Posix_MemLockAll(int LockAll)
+{
+  if (LockAll == 1 ) {
+#if defined(HAVE_SETRLIMIT_MEMLOCK)
+    /* Ensure we can reserve as much as we need */
+    struct rlimit rlim;
+    rlim.rlim_max = RLIM_INFINITY;
+    rlim.rlim_cur = RLIM_INFINITY;
+    if (setrlimit(RLIMIT_MEMLOCK, &rlim) < 0) {
+      LOG(LOGS_ERR, "setrlimit() failed");
+      return;
+    }
+#endif
+
+    if (mlockall(MCL_CURRENT|MCL_FUTURE) < 0) {
+      LOG(LOGS_ERR, "mlockall() failed");
+    } else {
+      DEBUG_LOG("Successfully locked into RAM");
+    }
+
+  }
+}
+#endif /* HAVE_MLOCKALL */
+
diff --git a/sys_posix.h b/sys_posix.h
new file mode 100644
index 0000000..2138c16
--- /dev/null
+++ b/sys_posix.h
@@ -0,0 +1,36 @@
+/*
+  chronyd/chronyc - Programs for keeping computer clocks accurate.
+
+ **********************************************************************
+ * Copyright (C) Richard P. Curnow  1997-2003
+ * Copyright (C) John G. Hasler  2009
+ * Copyright (C) Miroslav Lichvar  2009-2012, 2014-2018
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License along
+ * with this program; if not, write to the Free Software Foundation, Inc.,
+ * 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
+ *
+ **********************************************************************
+
+  =======================================================================
+
+  The header file for shared Posix functionality
+  */
+
+#ifndef GOT_SYS_POSIX_H
+#define GOT_SYS_POSIX_H
+
+extern void SYS_Posix_MemLockAll(int LockAll);
+
+extern void SYS_Posix_SetScheduler(int SchedPriority);
+
+#endif  /* GOT_SYS_POSIX_H */
-- 
2.21.0

Hi,

Updated patch based on feedback. The documentation changes are actually
quite minor and can be best viewed locally with git diff --word-diff.

I re-tested on the aforementioned systems.

Let me know if you see room for improvement.

Regards,
Stefan


-- 
To unsubscribe email chrony-dev-request@xxxxxxxxxxxxxxxxxxxx with "unsubscribe" in the subject.
For help email chrony-dev-request@xxxxxxxxxxxxxxxxxxxx with "help" in the subject.
Trouble?  Email listmaster@xxxxxxxxxxxxxxxxxxxx.