samba-dc: Add sysvolsync script

Samba AD DC does not implement [DFS-R for replication of the SYSVOL][0]
contents.  This does not make much of a difference to me, since
the SYSVOL is really only used for Group Policy.  Windows machines may
log an error if they cannot access the (basically empty) GPO files, but
that's pretty much the only effect if the SYSVOL is in sync between
domain controllers.

Unfortunately, there is one side-effect of the missing DFS-R
functionality that does matter.  On domain controllers, all user,
computer, and group accounts need to have Unix UID/GID numbers mapped.
This is different than regular member machines, which only need UID/GID
numbers for users that will/are allowed to log into them.  LDAP entries
only have ID numbers mapped for the latter class of users, which does
not include machine accounts.  As a result, Samba falls back to
generating local ID numbers for the rest of the accounts.  Those ID
numbers are stored in a local database file,
`/var/lib/samba/private/idmap.ldb`.  It would seem that it wouldn't
actually matter if accounts have different ID numbers on different
domain controllers, but there are evidently [situations][1] where DCs
refuse to allocate ID numbers at all, which can cause authentication to
fail.  As such, the `idmap.ldb` file needs to be kept in sync.

If we're going to go through the effort of synchronizing `idmap.ldb`, we
might as well keep the SYSVOL in sync as well.  To that end, I've
written a script to synchronize both the SYSVOL contents and the
`idmap.ldb` file.  It performs a simple one-way synchronization using
`rsync` from the DC with the PDC emulator role, as discovered using DNS
SRV records.  To ensure the `idmap.ldb` file is in a consistent state,
it only copies the most recent backup file.  If the copied file differs
from the local one, the script stops Samba and restores the local
database from the backup.  It then flushes Samba's caches and restarts
the service.  Finally, it fixes the NT ACLs on the contents of the
SYSVOL.

Since the contents of the SYSVOL are owned by root, naturally the
synchronization process has to run as root as well.  To attempt to limit
the scope of control this would give the process, we use as much of the
systemd sandbox capabilities as possible.  Further, the SSH key pairs
the DCs use to authenticate to one another are restricted to only
running rsync.  As such, the `sysvolsync` script itself cannot run
`tdbbackup` to back up `idmap.ldb`.  To handle that, I've created a
systemd service and corresponding timer unit to run `tdbbackup`
periodically.

I considered for a long time how to best implement this process, and
although I chose this naïve implementation, I am not exactly happy with
it.  Since I do not fully understand *why* keeping
the `idmap.ldb` file in sync is necessary, there are undoubtedly cases
where blindly copying it from the PDC emulator is not correct.  There
are definitely cases where the contents of the SYSVOL can be updated on
a DC besides the PDC emulator, but again, we should not run into them
because we don't really use the SYSVOL at all.  In the end, I think this
solution is good enough for our needs, without being so complicated

[0]: https://wiki.samba.org/index.php?title=SysVol_replication_(DFS-R)&oldid=18120
[1]: https://lists.samba.org/archive/samba/2021-November/238370.html

roles/samba-dc/files/idmap-backup.service

@@ -0,0 +1,38 @@
+[Unit]
+Description=Back up Samba idmap database
+
+[Service]
+Type=oneshot
+ExecStart=/usr/bin/tdbbackup -s .bak /var/lib/samba/private/idmap.ldb
+ReadWritePaths=/var/lib/samba/private
+InaccessiblePaths=/etc
+
+CapabilityBoundingSet=
+DeviceAllow=
+DevicePolicy=closed
+IPAddressAllow=
+IPAddressDeny=any
+LockPersonality=yes
+MemoryDenyWriteExecute=yes
+NoNewPrivileges=yes
+PrivateDevices=yes
+PrivateUsers=yes
+PrivateTmp=yes
+ProcSubset=pid
+ProtectClock=yes
+ProtectControlGroups=yes
+ProtectHome=yes
+ProtectHostname=yes
+ProtectKernelLogs=yes
+ProtectKernelModules=yes
+ProtectKernelTunables=yes
+ProtectProc=invisible
+ProtectSystem=strict
+RestrictAddressFamilies=
+RestrictNamespaces=yes
+RestrictRealtime=yes
+RestrictSUIDSGID=yes
+SystemCallArchitectures=native
+SystemCallFilter=@system-service
+SystemCallFilter=~@privileged @resources
+UMask=0077

roles/samba-dc/files/idmap-backup.timer

@@ -0,0 +1,10 @@
+[Unit]
+Description=Periodically back up Samba idmap database
+
+[Timer]
+OnActiveSec=1
+OnCalendar=hourly
+RandomizedDelaySec=10min
+
+[Install]
+WantedBy=timers.target

roles/samba-dc/files/sysvolsync-server.sh

@@ -0,0 +1,13 @@
+#!/bin/sh
+
+set -- ${SSH_ORIGINAL_COMMAND}
+if [ "$1" != rsync ] && [ "$1" != /usr/bin/rsync ]; then
+    echo 'Only rsync is allowed' >&2
+    exit 2
+fi
+if [ "$2" != --server ]; then
+    echo 'rsync must be run in server mode' >&2
+    exit 2
+fi
+
+eval exec "${SSH_ORIGINAL_COMMAND}"

roles/samba-dc/files/sysvolsync.service

@@ -0,0 +1,44 @@
+[Unit]
+Description=Sync Samba AD sysvol
+Wants=network-online.target
+After=network-online.target
+
+[Service]
+Type=oneshot
+ExecStart=/usr/local/sbin/sysvolsync
+
+CacheDirectory=%N
+RuntimeDirectory=%N
+ReadWritePaths=%t/%N %C/%N /var/lib/samba
+TemporaryFileSystem=/etc/ssh
+BindReadOnlyPaths=/etc/ssh/ssh_config /etc/ssh/ssh_config.d
+# Doesn't work: SELinux AVC denial when starting unit
+#InaccessiblePaths=/etc/shadow
+
+CapabilityBoundingSet=CAP_CHOWN CAP_DAC_OVERRIDE CAP_SYS_ADMIN CAP_FOWNER
+DeviceAllow=
+DevicePolicy=closed
+LockPersonality=yes
+MemoryDenyWriteExecute=yes
+NoNewPrivileges=yes
+PrivateDevices=yes
+PrivateTmp=yes
+ProcSubset=pid
+ProtectClock=yes
+ProtectControlGroups=yes
+ProtectHome=yes
+ProtectHostname=yes
+ProtectKernelLogs=yes
+ProtectKernelModules=yes
+ProtectKernelTunables=yes
+ProtectProc=invisible
+ProtectSystem=strict
+RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX
+RestrictNamespaces=yes
+RestrictRealtime=yes
+RestrictSUIDSGID=yes
+SystemCallArchitectures=native
+SystemCallFilter=@system-service
+SystemCallFilter=~@privileged @resources
+SystemCallFilter=@chown
+UMask=0077

roles/samba-dc/files/sysvolsync.sh

@@ -0,0 +1,81 @@
+#!/bin/sh
+# vim: set sw=4 ts=4 sts=4 et :
+
+SYSVOL=/var/lib/samba/sysvol
+IDMAP_LDB=/var/lib/samba/private/idmap.ldb
+
+case "${DEBUG}" in
+yes|YES|y|Y|on|ON|1|true|TRUE|t|T)
+    unset DEBUG
+    DEBUG=1
+    ;;
+*)
+    unset DEBUG
+    ;;
+esac
+
+debug() {
+    [ -z "${DEBUG}" ] || echo "$*" >&2
+}
+
+info() {
+    echo "$*" >&2
+}
+
+get_pdc() {
+    dig @localhost +short -t srv _ldap._tcp.pdc._msdcs.$(dnsdomainname) \
+        | sort -n \
+        | awk '{print $4;exit}'
+}
+
+fqdn=$(hostname -f)
+pdc=$(get_pdc)
+pdc="${pdc%.}"
+if [ "${pdc}" = "${fqdn}" ]; then
+    debug 'Skipping SYSVOL sync on PDC emulator'
+    exit 0
+fi
+if [ -z "${pdc}" ]; then
+    echo 'Could not identify PDC emulator' >&2
+    exit 1
+fi
+debug "Found PDC emulator: ${pdc}"
+
+ssh_config=/var/cache/sysvolsync/ssh_config
+debug "Generating configuration file: ${ssh_config}"
+cat > "${ssh_config}" <<EOF
+User=root
+BatchMode=yes
+IdentityFile=/var/lib/samba/private/sysvolsync.key
+UserKnownHostsFile=/var/cache/sysvolsync/ssh_known_hosts
+ControlMaster=auto
+ControlPersist=yes
+ControlPath=/run/sysvolsync/sshcp
+EOF
+
+debug "Opening SSH connection to ${pdc}"
+ssh -F "${ssh_config}" -fN "${pdc}" || exit
+trap 'ssh -F "${ssh_config}" -q -O exit "${pdc}"' INT TERM QUIT EXIT
+
+export RSYNC_RSH="ssh -F ${ssh_config}"
+debug "Synchronizing SYSVOL from ${pdc}"
+rsync -a${DEBUG+i}HAXS --delete "${pdc}:${SYSVOL}/" "${SYSVOL}"
+debug "Copying idmap.ldb from ${pdc}"
+rsync -a${DEBUG+i} --delete "${pdc}:${IDMAP_LDB}.bak" "${IDMAP_LDB}.new"
+st_new=$(stat -c %Y "${IDMAP_LDB}.new")
+st_cur=$(stat -c %Y "${IDMAP_LDB}")
+if [ "${st_new}" -ne "${st_cur}" ]; then
+    info "Got updated idmap.ldb from ${pdc}"
+    info 'Stopping Samba service'
+    systemctl stop samba || exit
+    mv "${IDMAP_LDB}.new" "${IDMAP_LDB}"
+    info 'Flushing idmap cache'
+    net cache flush
+    info 'Restarting Samba service'
+    systemctl start samba || exit
+    info 'Resetting SYSVOL ACLs'
+    samba-tool ntacl sysvolreset
+else
+    debug 'Local idmap.ldb is up-to-date'
+    exit 0
+fi

roles/samba-dc/files/sysvolsync.sshd.conf

@@ -0,0 +1,2 @@
+Match User root
+    AuthorizedKeysFile .ssh/authorized_keys .ssh/authorized_keys_sysvolsync

roles/samba-dc/files/sysvolsync.timer

@@ -0,0 +1,9 @@
+[Unit]
+Description=Periodically sync Samba AD sysvol
+
+[Timer]
+OnActiveSec=1
+OnCalendar=hourly
+
+[Install]
+WantedBy=timers.target

roles/samba-dc/handlers/main.yml

@@ -7,7 +7,28 @@
     var=samba_dc_provision.admin_password
 - name: save firewalld configuration
   command: firewall-cmd --runtime-to-permanent
+
+- name: sync sysvol
+  systemd:
+    name: sysvolsync
+    state: started
+
 - name: restart samba
   service:
     name=samba
     state=restarted
+
+- name: restart sysvolsync.timer
+  systemd:
+    name: sysvolsync.timer
+    state: restarted
+
+- name: restart idmap-backup.timer
+  systemd:
+    name: idmap-backup.timer
+    state: restarted
+
+- name: reload sshd
+  service:
+    name: sshd
+    state: reloaded

roles/samba-dc/tasks/main.yml

@@ -35,6 +35,9 @@
   notify:
   - restore samba file contexts
   - display generated admin password
+  - sync sysvol
+  tags:
+  - provision-domain
 
 - name: ensure samba certificate is installed
   copy:
@@ -79,6 +82,136 @@
   service:
     name=samba
     enabled=yes
+
+- name: ensure sysvolsync ssh key exists
+  openssh_keypair:
+    type: ed25519
+    comment: '{{ inventory_hostname }}'
+    path: /var/lib/samba/private/sysvolsync.key
+    owner: root
+    group: root
+    mode: u=r,go=
+    state: present
+  register: sysvolsync_key
+  tags:
+  - sysvolsync
+- name: ensure sysvolsync key is trusted on other domain controllers
+  delegate_to: '{{ item }}'
+  authorized_key:
+    user: root
+    path: /root/.ssh/authorized_keys_sysvolsync
+    key: >-
+      {{ sysvolsync_key.public_key }}
+    key_options: command="/usr/local/libexec/sysvolsync-server"
+    state: present
+  # openssh_keypair module doesn't return public_key in check mode
+  when: not ansible_check_mode
+  loop: '{{ groups["samba-dc"] }}'
+  tags:
+  - sysvolsync
+- name: ensure sysvolsync cache directory exists
+  file:
+    path: /var/cache/sysvolsync
+    mode: u=rwx,go=rx
+    state: directory
+  tags:
+  - sysvolsync
+- name: ensure sysvolsync ssh host key database is populated
+  template:
+    src: sysvolsync.ssh_known_hosts.j2
+    dest: /var/cache/sysvolsync/ssh_known_hosts
+    mode: u=rw,go=r
+  tags:
+  - sysvolsync
+  - ssh_known_hosts
+
+- name: ensure sysvolsync script is installed
+  copy:
+    src: sysvolsync.sh
+    dest: /usr/local/sbin/sysvolsync
+    mode: u=rwx,go=rx
+  notify:
+  - restart sysvolsync.timer
+  tags:
+  - sysvolsync
+- name: ensure sysvolsync systemd units are installed
+  copy:
+    src: '{{ item }}'
+    dest: /etc/systemd/system
+    mode: u=rw,go=r
+  loop:
+  - sysvolsync.service
+  - sysvolsync.timer
+  notify:
+  - reload systemd
+  - restart sysvolsync.timer
+  tags:
+  - sysvolsync
+  - systemd
+- name: ensure sysvolsync timer unit is enabled
+  systemd:
+    name: sysvolsync.timer
+    enabled: true
+  tags:
+  - sysvolsync
+  - service
+- name: ensure sysvolsync timer unit is running
+  systemd:
+    name: sysvolsync.timer
+    state: started
+  tags:
+  - sysvolsync
+  - service
+
+- name: ensure sysvolsync-server script is installed
+  copy:
+    src: sysvolsync-server.sh
+    dest: /usr/local/libexec/sysvolsync-server
+    mode: u=rwx,go=rx
+  tags:
+  - sysvolsync
+- name: ensure sshd is configured for sysvolsync
+  copy:
+    src: sysvolsync.sshd.conf
+    dest: /etc/ssh/sshd_config.d/80-sysvolsync.conf
+    mode: u=rw,go=r
+  notify:
+  - reload sshd
+  tags:
+  - sysvolsync
+
+- name: ensure idmap-backup systemd units are installed
+  copy:
+    src: '{{ item }}'
+    dest: /etc/systemd/system/
+    mode: u=rw,go=r
+  loop:
+  - idmap-backup.service
+  - idmap-backup.timer
+  notify:
+  - reload systemd
+  - restart idmap-backup.timer
+  tags:
+  - idmap-backup
+  - systemd
+- name: ensure idmap-backup timer unit is enabled
+  systemd:
+    name: idmap-backup.timer
+    enabled: true
+  tags:
+  - idmap-backup
+  - service
+- name: ensure idmap-backup timer unit is running
+  systemd:
+    name: idmap-backup.timer
+    state: started
+  tags:
+  - idmap-backup
+  - service
+
+- name: flush_handlers
+  meta: flush_handlers
+
 - name: ensure samba is running
   service:
     name=samba
@@ -117,3 +250,4 @@
     mode: u=rw,go=r
   tags:
   - logrotate
+

roles/samba-dc/templates/sysvolsync.ssh_known_hosts.j2

@@ -0,0 +1,11 @@
+{% for host in groups['samba-dc'] %}
+{% if hostvars[host].ansible_ssh_host_key_ecdsa_public|d %}
+{{ host }} ecdsa-sha2-nistp256 {{ hostvars[host].ansible_ssh_host_key_ecdsa_public }}
+{% endif %}
+{% if hostvars[host].ansible_ssh_host_key_rsa_public|d %}
+{{ host }} ssh-rsa {{ hostvars[host].ansible_ssh_host_key_rsa_public }}
+{% endif %}
+{% if hostvars[host].ansible_ssh_host_key_ed25519_public|d %}
+{{ host }} ssh-ed25519 {{ hostvars[host].ansible_ssh_host_key_ed25519_public }}
+{% endif %}
+{% endfor %}

roles/samba-dc/vars/defaults.yml

@@ -4,7 +4,9 @@ samba_dc_packages:
 - krb5-workstation
 - ldb-tools
 - openldap-clients
+- openssh-clients
 - samba-dc
 - samba-winbind-clients
 - tdb-tools
+- rsync
 samba_bind_dlz_pkg: samba-dc-bind-dlz