Restructure role, add Systemd timer option. By @conloos (#112)

* add full path

* Update Readme.me: reorder optional Arguments, update cron -> systemd timer

* remove ssh_key_file; change cron to timer

* Removed cronie from package installation because systemd timer is used

* docker.sh - Stops all or selected containers to save the persistent data intact. The containers are started in reverse order

* Created arguments_specs.yml

* Role restructured:
  - if needed creation of a service user incl. creation of the ssh-key,
  - add the ssh key to authorized_keys,
  - auto init of the repos,
  - creation and start of systemd timer and services and
  - installation of the Docker helperscript.

* restructure role add import logic

* cleanup: user backup_user

* - "borg_source_directories" is not longer a required Argument
- add "borg_keys_directory" to load key from Service user during starting borgmatic by sudo

* Add borgmatic_initialization_repo (bool) as option to disable init of repo

* cleanup

* fix ansible-lint errors and warnings

* fix letter turner

* add option: borgmatic_timer

* add:
  - borgmatic_timer_systemd: true
readd:
  - borgmatic_cron_name: "borgmatic"

* - renamed borgmatic_cron_name to borgmatic_timer_cron_name to be more convergent.
- Change recommendations implemented by m3nu so that creation of a timer (systemd or cron) is optional and can be selected via borgmatic_timer.

* Add description to borgmatic_timer_cron_name and borgmatic_timer

* Add variable borg_cron_package to install the cron-packages in case of using timer: cron

* reworked timer install logic

* reworked timer install logic

* Add comments for running backup with service account

* add new parameters for tests

* Switch created to perform the backup as root or service account. If a service account is to be used, it will be created.

* Refactored: Check for ssh-key if not present, genereate them.

* Refactored

* Refactored

* renamed tasks/03_configure.yml to tasks/04_create_links_to_borg_and_borgmatic.yml

* Refactored

* Refactored

* add example for service account

* Update Python version for testing

* No auto init

* Add description to install_backup

* Add description to install_backup

* set coverage back to: m3nu.ansible_role_borgbackup

* The initialization of the repository must be activated and does not take place automatically.

* The initialization of the repository must be activated and does not take place automatically.

* Removed install_backup as var (bool) to prevent that this role run

* Rename backup_ssh_command to borg_ssh_command, tis was a double definition

* Renamed backup_repository to borg_repository and add better explanations

* remove copy ssh-keys and cert parts

* Add comments to borg_ssh_key_file and borg_ssh_key_type

* Set allways the borg_ssh_key_file and borg_ssh_command to load the right ssh-key. Add borg_ssh_key_type to select the key type by user

* Add borg_ssh_key_type

* renamed id_rsa to backup

* generate ssh-keys (backup and backup.pub) and add better explanation

* Print out key if borgmatic_initialization_repo is false

* Remove 'su - {{ borgbackup_user }} -c' to execute the borgmatic by the right user

* Add Check frequency, therefore, we no longer need to distinguish between normal and large repos

* Add link to Article

* renamed backup_ssh_command and backup_ssh_key_file to borg_ssh_command and borg_ssh_key_file

* Removed: borgmatic_initialization_repo

* Removed: borgmatic_initialization_repo

* Removed: borgmatic_initialization_repo

* revert changes

* Add Full Automation

* polishing

* rename backup.timer and bakup.service to borgmatic.timer and borgmatic.service

* remove debug

* Try to find services in ansible_facts

* Forgot to install Cron

* change borg_ssh_key_type to ed25519

* remove conditional checks

* - add hint to using a service user
- renamed: borg_ssh_key_file to borg_ssh_key_file_path
- removed advanced example

* add borg_ssh_key_name, renamed borg_ssh_key_file to borg_ssh_key_file_path

* removed static pointing to ~/.ssh/backup SSH private key

* Add README-Advanced-Examples.md for storing more examples

* Fix test idempotence

* Dont symlink when using distro packages

* Remove old test targets, consistent wording, remove tag

* Remove helper scripts, fix absolute path

* Fix cron job, add assert to prevent duplicate timers

* nit-pick

* Create bin links as root, no borg_ssh_command by default.

* Add breaking changes note to README

---------

Co-authored-by: Manu <manu@snapdragon.cc>

tasks/01_install.yml

@@ -0,0 +1,29 @@
+---
+- name: Install borgbackup
+  block:
+    - name: Ensure legacy hooks aren't used
+      ansible.builtin.assert:
+        that:
+          - borgmatic_failure_command is undefined
+          - borgmatic_before_backup_command is undefined
+          - borgmatic_after_backup_command is undefined
+        msg: Please use the new borgmatic_hooks variable instead of individual before/after/failure hooks.
+
+    - name: Include OS-specific variables
+      include_vars: "{{ item }}"
+      with_first_found:
+        - "{{ ansible_distribution }}-{{ ansible_distribution_major_version }}.yml"
+        - "{{ ansible_os_family }}-{{ ansible_distribution_major_version }}.yml"
+        - "{{ ansible_distribution }}.yml"
+        - "{{ ansible_os_family }}.yml"
+        - "{{ ansible_lsb.id }}.yml"
+
+    - name: Install general dependencies (openssh)
+      ansible.builtin.package:
+        name: "{{ borg_dep_packages }}"
+        state: present
+
+    - name: Install Borg and Borgmatic
+      ansible.builtin.include_tasks:
+        file: noauto_install_{{ borg_install_method }}.yml
+...

tasks/02_user_management.yml

@@ -0,0 +1,25 @@
+---
+# So in different positions in that role we need the user home
+# Since we cannot be sure that this FSH is compatible we will determine it.
+- name: Get home dir
+  when:
+    - borg_user == "root"
+  block:
+    - name: Get home if borg_user == "root"
+      ansible.builtin.user:
+        name: "{{ borg_user }}"
+        state: present
+      register: user_info
+      changed_when: false
+      check_mode: true  # Important, otherwise user will be created
+
+    - name: Save the user_info, we need them for the home_dir
+      ansible.builtin.set_fact:
+        backup_user_info: "{{ user_info }}"
+
+- name: Create user if borg_user != "root"
+  when:
+    - borg_user != "root"
+  ansible.builtin.include_tasks:
+    file: noauto_create_backup_user_and_group.yml
+...

tasks/03_create_key.yml

@@ -0,0 +1,28 @@
+---
+- name: Create SSH key (if neeeded) for {{ borg_user }}
+  block:
+    - name: Ensure directory exist
+      ansible.builtin.file:
+        path: "{{ backup_user_info.home }}/.ssh/"
+        state: directory
+        mode: "0700"
+        owner: "{{ borg_user }}"
+        group: "{{ borg_group }}"
+
+    - name: Generate an OpenSSH keypair
+      community.crypto.openssh_keypair:
+        path: "{{ borg_ssh_key_file_path }}"
+        mode: "0600"
+        type: "{{ borg_ssh_key_type }}"
+        owner: "{{ borg_user }}"
+        group: "{{ borg_group }}"
+
+    - name: Read SSH key
+      ansible.builtin.slurp:
+        src: "{{ borg_ssh_key_file_path }}.pub"
+      register: backup_local_ssh_key
+
+    - name: Print key
+      ansible.builtin.debug:
+        msg: "The generated key is: {{ backup_local_ssh_key['content'] | b64decode }}"
+...

tasks/05_configure.yml

@@ -0,0 +1,19 @@
+---
+- name: Add Borgmatic config file
+  block:
+    - name: Ensure /etc/borgmatic exists
+      ansible.builtin.file:
+        path: /etc/borgmatic
+        state: directory
+        mode: "0700"
+        owner: "{{ borg_user }}"
+        group: "{{ borg_group }}"
+
+    - name: Add Borgmatic configuration
+      ansible.builtin.template:
+        src: config.yaml.j2
+        dest: "/etc/borgmatic/{{ borgmatic_config_name }}"
+        mode: "0600"
+        owner: "{{ borg_user }}"
+        group: "{{ borg_group }}"
+...

tasks/07_install_timer.yml

@@ -0,0 +1,8 @@
+---
+- name: Install timer to run Borgmatic
+  when:
+    - borgmatic_timer is defined and borgmatic_timer | length > 0
+  block:
+    - name: Start timer install script
+      ansible.builtin.include_tasks:
+        file: noauto_create_timer_{{ borgmatic_timer }}.yml

tasks/install_package.yml

@@ -1,18 +0,0 @@
----
-- name: Check if EPEL repo is enabled, if installation from distro is requested
-  when: ansible_os_family == 'RedHat'
-  block:
-  - name: Get list of installed packages
-    ansible.builtin.package_facts:
-      manager: auto
-  - name: Ensure EPEL is enabled
-    ansible.builtin.assert:
-      that:
-        - "'epel-release' in ansible_facts.packages"
-      fail_msg: Need EPEL repo to install via distro package.
-
-- name: Install borgmatic and borg via distribution package manager
-  package:
-    name: "{{ item }}"
-    state: present
-  loop: "{{ borg_distro_packages }}"

tasks/install_pip.yml

@@ -1,39 +0,0 @@
----
-- name: Install build dependencies
-  package:
-    name: "{{ borg_pip_packages }}"
-    state: present
-
-- name: Create virtualenv for borg  # noqa package-latest
-  pip:
-    name:
-    - pip
-    - setuptools
-    state: latest
-    virtualenv: /opt/borgmatic
-    virtualenv_command: "{{ python_bin }} -m venv"
-
-- name: Install dependent Python Packages
-  pip:
-    name: "{{ borg_dependent_python_packages }}"
-    virtualenv: /opt/borgmatic
-  when: borg_dependent_python_packages is defined
-
-- name: Install main Python Packages
-  pip:
-    name: "{{ item.name }}"
-    version: "{{ item.version | default(omit, true) }}"
-    virtualenv: /opt/borgmatic
-  when: borg_python_packages is defined
-  loop: "{{ borg_python_packages }}"
-
-- name: Create borgmatic command in /usr/local/bin
-  copy:
-    content: |
-      #!/bin/bash
-      . /opt/borgmatic/bin/activate
-      borgmatic "$@"
-    dest: /usr/local/bin/borgmatic
-    owner: root
-    group: root
-    mode: "0755"

tasks/main.yml

@@ -1,96 +1,7 @@
 ---
-- name: Ensure legacy hooks aren't used
-  assert:
-    that:
-      - borgmatic_failure_command is undefined
-      - borgmatic_before_backup_command is undefined
-      - borgmatic_after_backup_command is undefined
-    msg: Please use the new borgmatic_hooks variable instead of individual before/after/failure hooks.
-
-- name: Include OS-specific variables
-  include_vars: "{{ item }}"
-  with_first_found:
-    - "{{ ansible_distribution }}-{{ ansible_distribution_major_version }}.yml"
-    - "{{ ansible_os_family }}-{{ ansible_distribution_major_version }}.yml"
-    - "{{ ansible_distribution }}.yml"
-    - "{{ ansible_os_family }}.yml"
-    - "{{ ansible_lsb.id }}.yml"
-
-- name: Install general dependencies (cron and openssh)
-  package:
-    name: "{{ borg_dep_packages }}"
-    state: present
-
-- name: Install Borg and Borgmatic
-  ansible.builtin.include_tasks:
-    file: install_{{ borg_install_method }}.yml
-
-- name: Ensure root has SSH key.
-  user:
-    name: "root"
-    generate_ssh_key: yes
-    ssh_key_file: "{{ ssh_key_file }}"
-    ssh_key_type: ed25519
-  register: root_user
-
-- name: Print key created for root user (use for remote repo)
-  debug:
-    var: root_user['ssh_public_key']
-
-- name: Ensure /etc/borgmatic exists
-  file:
-    path: /etc/borgmatic
-    state: directory
-    mode: 0700
-    owner: root
-
-- name: Add Borgmatic configuration
-  template:
-    src: config.yaml.j2
-    dest: "/etc/borgmatic/{{ borgmatic_config_name }}"
-    mode: 0600
-
-- name: Add cron-job for borgmatic (large repo, create and check separate)
-  block:
-    - name: Add cron job for regular create and prune
-      cron:
-        name: "{{ borgmatic_cron_name }}"
-        hour: "{{ borgmatic_cron_hour }}"
-        minute: "{{ borgmatic_cron_minute }}"
-        user: "root"
-        cron_file: "{{ borgmatic_cron_name }}"
-        job: "borgmatic -c /etc/borgmatic/{{ borgmatic_config_name }} -C -p"
-    - name: Add cron job for infrequent checks
-      cron:
-        name: "{{ borgmatic_cron_name }}-check"
-        day: "{{ borgmatic_cron_checks_day }}"
-        hour: "{{ borgmatic_cron_checks_hour }}"
-        minute: "{{ borgmatic_cron_checks_minute }}"
-        user: "root"
-        cron_file: "{{ borgmatic_cron_name }}"
-        job: "borgmatic -c /etc/borgmatic/{{ borgmatic_config_name }} -k"
-  when: borgmatic_large_repo
-
-- name: Add cron-job for borgmatic (normal-sized repo)
-  block:
-    - name: Add cron job for create, check and prune
-      cron:
-        name: "{{ borgmatic_cron_name }}"
-        hour: "{{ borgmatic_cron_hour }}"
-        minute: "{{ borgmatic_cron_minute }}"
-        user: "root"
-        cron_file: "{{ borgmatic_cron_name }}"
-        job: "borgmatic -c /etc/borgmatic/{{ borgmatic_config_name }}"
-    - name: Ensure separate check cron job is absent
-      cron:
-        name: "{{ borgmatic_cron_name }}-check"
-        state: absent
-  when: not borgmatic_large_repo
-
-- name: Set PATH for borgmatic cron job.
-  cron:
-    user: "root"
-    cron_file: "{{ borgmatic_cron_name }}"
-    name: PATH
-    env: yes
-    value: /usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
+- name: Add and run all plays
+  include_tasks: "{{ bak_element }}"
+  with_items: "{{ lookup('ansible.builtin.fileglob', '*.yml').split(',') | reject('search', 'main.yml') | reject('search', 'noauto_*') | sort }}"
+  loop_control:
+    loop_var: bak_element
+...

tasks/noauto_create_backup_user_and_group.yml

@@ -0,0 +1,34 @@
+---
+- name: Setup backup environment
+  when:
+    - backup_create_local_user is not defined or backup_create_local_user
+    - borg_user != "root"
+  block:
+    - name: Add local backup group
+      ansible.builtin.group:
+        name: "{{ borg_group }}"
+        state: present
+
+    - name: Add local backup user
+      ansible.builtin.user:
+        name: "{{ borg_user }}"
+        shell: "/bin/bash"
+        groups: "{{ borg_group }}"
+        comment: "Backup User Account"
+        append: true
+      register: user_info
+
+    - name: Save the user_info, we need them for the home_dir
+      ansible.builtin.set_fact:
+        backup_user_info: "{{ user_info }}"
+
+    - name: Add sudo users
+      community.general.sudoers:
+        name: "backup"
+        state: present
+        user: "{{ borg_user }}"
+        nopassword: true
+        commands:
+          - "/opt/borgmatic/bin/borg"
+          - "/usr/local/bin/borgmatic -c /etc/borgmatic/{{ borgmatic_config_name }}"
+...

tasks/noauto_create_timer_cron.yml

@@ -0,0 +1,31 @@
+---
+- name: Ensure Cron is installed
+  ansible.builtin.package:
+    name: "{{ borg_cron_package }}"
+    state: present
+
+- name: Add Cron job for borgmatic
+  block:
+    - name: Add single Cron job for borgmatic
+      cron:
+        name: "{{ borgmatic_timer_cron_name }}"
+        hour: "{{ borgmatic_timer_hour }}"
+        minute: "{{ borgmatic_timer_minute }}"
+        user: "{{ borg_user }}"
+        cron_file: "{{ borgmatic_timer_cron_name }}"
+        job: "borgmatic -c /etc/borgmatic/{{ borgmatic_config_name }}"
+
+    - name: Ensure separate check cron job is absent
+      cron:
+        name: "{{ borgmatic_timer_cron_name }}-check"
+        cron_file: "{{ borgmatic_timer_cron_name }}"
+        state: absent
+
+- name: Set PATH for borgmatic cron job.
+  cron:
+    user: "{{ borg_user }}"
+    cron_file: "{{ borgmatic_timer_cron_name }}"
+    name: PATH
+    env: yes
+    value: /usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
+...

tasks/noauto_create_timer_systemd.yml

@@ -0,0 +1,63 @@
+---
+- name: Register existence of Borgmatic cron file.
+  cron:
+    name: "{{ borgmatic_timer_cron_name }}"
+    cron_file: "{{ borgmatic_timer_cron_name }}"
+    state: absent
+  check_mode: true
+  register: cron_file_exists
+
+- name: Ensure no Borgmatic Cron file exists.
+  ansible.builtin.assert:
+    that:
+      - not cron_file_exists.changed
+    fail_msg: Found an existing Borgmatic Cron job. Please remove before using Systemd timer.
+
+- name: Create borgbackup timer
+  block:
+    - name: Copy systemd files
+      ansible.builtin.template:
+        src: "{{ item.src }}"
+        dest: "{{ item.dest }}"
+        owner: root
+        group: root
+        backup: true
+        mode: "{{ item.mode }}"
+      with_items:
+        - { src: "borgmatic.timer.j2", dest: "/usr/lib/systemd/system/borgmatic.timer", mode: "0644" }
+        - { src: "borgmatic.service.j2", dest: "/usr/lib/systemd/system/borgmatic.service", mode: "0644" }
+
+    - name: Populate service facts
+      ansible.builtin.service_facts:
+
+      # If the role is running and the repo is not yet initialized, an error will occur.
+      # Therefore the service is stopped by default and must be started manually.
+    - name: Stop fresh installed borgmatic.timer and borgmatic.service
+      when: "'borgmatic.service' not in ansible_facts.services"
+      block:
+        - name: Set borgmatic services to stopped - newly installed
+          ansible.builtin.systemd:
+            name: "{{ item }}"
+            state: stopped
+            enabled: false
+            masked: false
+            daemon_reload: true
+          when: item in ansible_facts.services
+          with_items:
+            - borgmatic.service
+
+        # bug: Need own section without masked else the timer are skipped
+        - name: Set borgmatic timers to stopped - newly installed
+          ansible.builtin.systemd:
+            name: "{{ item }}"
+            state: stopped
+            enabled: false
+            daemon_reload: true
+          with_items:
+            - "borgmatic.timer"
+
+        - name: Show hints
+          when: "'backup_init_repo' not in ansible_run_tags"
+          ansible.builtin.debug:
+            msg: "Attention: Since the repo was not initialized automatically, the systemd service (borgmatic.service) and the timer (borgmatic.timer) are not activated."
+...

tasks/noauto_install_package.yml

@@ -0,0 +1,21 @@
+---
+- name: Install borgbackup by distro
+  block:
+    - name: Check if EPEL repo is enabled, if installation from distro is requested
+      when: ansible_os_family == 'RedHat'
+      block:
+      - name: Get list of installed packages
+        ansible.builtin.package_facts:
+          manager: auto
+      - name: Ensure EPEL is enabled
+        ansible.builtin.assert:
+          that:
+            - "'epel-release' in ansible_facts.packages"
+          fail_msg: Need EPEL repo to install via distro package.
+
+    - name: Install borgmatic and borg via distribution package manager
+      ansible.builtin.package:
+        name: "{{ item }}"
+        state: present
+      loop: "{{ borg_distro_packages }}"
+...

tasks/noauto_install_pip.yml

@@ -0,0 +1,51 @@
+---
+- name: Install Borg and Borgmatic via pip
+  block:
+    - name: Install build dependencies
+      ansible.builtin.package:
+        name: "{{ borg_pip_packages }}"
+        state: present
+
+    - name: Create virtualenv for borg  # noqa package-latest
+      ansible.builtin.pip:
+        name:
+          - pip
+          - setuptools
+        state: latest
+        virtualenv: "{{ borg_venv_path }}"
+        virtualenv_command: "{{ python_bin }} -m venv"
+
+    - name: Install dependent Python packages
+      ansible.builtin.pip:
+        name: "{{ borg_dependent_python_packages }}"
+        virtualenv: "{{ borg_venv_path }}"
+      when: borg_dependent_python_packages is defined
+
+    - name: Install main Python packages
+      ansible.builtin.pip:
+        name: "{{ item.name }}"
+        version: "{{ item.version | default(omit, true) }}"
+        virtualenv: "{{ borg_venv_path }}"
+      when: borg_python_packages is defined
+      loop: "{{ borg_python_packages }}"
+
+- name: Create links to Borgmatic and Borg binaries
+  block:
+    - name: Create borgmatic command in /usr/local/bin
+      ansible.builtin.copy:
+        content: |
+          #!/bin/bash
+          . "{{ borg_venv_path }}"/bin/activate
+          borgmatic "$@"
+        dest: /usr/local/bin/borgmatic
+        mode: "0755"
+
+    - name: Create borg command in /usr/local/bin
+      ansible.builtin.copy:
+        content: |
+          #!/bin/bash
+          . "{{ borg_venv_path }}"/bin/activate
+          borg "$@"
+        dest: /usr/local/bin/borg
+        mode: "0755"
+...