Merge branch 'slurm_upgrade' into 'master'

Slurm upgrade and new rocky sql server

See merge request !595

roles/mysql/README.md

+# MYSQL FOR SLURM
+
+The Mysql for slurm is problematic, as configuring the database is no longer simple.
+Ansible commands such as mysql_user do not work in a brand new system, as it appears
+mysql/maria db must run in a root shell and Ansible breaks at this point.
+
+The preferred approach.
+
+1) Run this role. It may/may not break when trying to modify the database as the root password is not set.
+2) Login to sql machine. sudo as root
+3) Run **/usr/bin/mysql_secure_installation**  Set the root password to what is found in our Ansible scripts, i.e.
+```
+    ansible-vault edit vars/passwords.yml 
+    #Key to change is "sqlrootPasswd"
+```
+4). The datbase by default is in */var/lib* which is on a small Unix disk partition. You need to change it.
+    Several options exist.
+
+1. Set up a soft pointer, i.e.  **/var/lib/mysql -> /vdb/mysql** OR
+1. Modify **/etc/my.cnf.d/mariadb-server.cnf** and point to a new location.
+e.g.
+```
+[mysqld]
+datadir=/mnt/vdb1/var_lib_mysql
+socket=/mnt/vdb1/var_lib_mysql/mysql.sock
+log-error=/var/log/mariadb/mariadb.log
+pid-file=/run/mariadb/mariadb.pid
+```
+
+Note. The client uses the default socket */var/lib/mysql/mysql.sock* so you need to modify **/etc/my.cnf.d/mysql-clients.cnf** and add the same socket.
+To avoid errors like:
+```
+ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)
+```
+You must go:
+```
+[mysql]
+socket=/mnt/vdb1/var_lib_mysql/mysql.sock
+```
+
+5) The procures will be something like this:
+
+1. **systemctl stop mariadb**
+1. **mkdir /mnt/vdb1/var_lib_mysql**
+1. **chown mysql:mysql /mnt/vdb1/var_lib_mysql**
+1. **cp -rp /var/lib/mysql/\* /mnt/vdb1/var_lib_mysql**
+1. **systemctl start mariadb**
+1. Use **systemctl status mariadb** and view **/var/log/mariadb/mariadb.log** to trace errors
+
+6) None of the above is in the ansible scripts. The *mysql* role is tagged as **never** due to these complications
+7) The input parameters to this role have been modified, so users can hardcode the SLURM controllers directly.
+
+Sample usage (for a mysql server)
+```
+- role: mysql
+      vars:
+         mysql_type: "mysql_server"
+         mysql_root_password: "{{ sqlrootPasswd }}"
+         mysql_user_name: "slurmdb"
+         mysql_user_db_name: "slurm_acct_db"
+         current_slurm_mgmt_nodes_fqdn: " {{ current_slurm_mgmt_nodes_fqdn }} "
+         current_slurm_mgmt_nodes_short: " {{ current_slurm_mgmt_nodes_short }} "
+         mysql_user_password: "{{ slurmdb_passwd }}"
+  tags: [ database, mysql, never ]
+```
+
+where
+
+```
+- set_fact:
+         current_slurm_mgmt_nodes_fqdn:
+                  - 'm3-mgmt3.massive.org.au'
+                  - 'm3-mgmt4.massive.org.au'
+         current_slurm_mgmt_nodes_short:
+                  - 'm3-mgmt3'
+                  - 'm3-mgmt4'
+```
+

roles/mysql/tasks/Rocky_9_mysql_server.yml

+---
+- name: CHECKING VARS inside role
+  debug:
+        msg: "[DEBUG]  current_slurm_mgmt_nodes_fqdn is {{ current_slurm_mgmt_nodes_fqdn }}  and current_slurm_mgmt_nodes_short is {{ current_slurm_mgmt_nodes_short }}"
+ 
+- name: Installing MySQL RedHat
+  yum: name={{ item }}
+  with_items: "{{ server_packages }}"
+  become: true
+  when: ansible_os_family == "Rocky"
+
+- name: make sure mysql conf directory exists
+  file: dest=/etc/mysql/conf.d state=directory
+  become: true
+  register: mysqldb_confdir_create
+
+- name: "Starting MySQL"
+  service: name={{ sqlServiceName }} state=started enabled=true
+  become: true
+
+- name: Check that the slurm_acct_db_directory exists
+  stat:
+    path: /var/lib/mysql/slurm_acct_db/   #defined in /vars/filesystems.yaml
+  register: slurm_acct_db_directory_result
+
+# this will only work if a completely fresh db gets installed because it gets shipped with a blank root pw
+# thsi command has to be done manually after mysql installed as it is impossible due to issues with mariadb only resetting from a root shell
+# See README.md for this role
+#- name: "update mysql root password {{ mysql_root_password }} for all root accounts"
+  #mysql_user: name=root host=localhost password={{ mysql_root_password }} login_user=root login_password={{ mysql_root_password }}
+  #become: true
+  #become_user: root
+  #tags: [never]
+  #when: not slurm_acct_db_directory_result.stat.exists and mysqldb_confdir_create.changed
+  #when: not slurm_acct_db_directory_result.stat.exists 
+
+- name: "Adding user database {{ mysql_user_db_name }} "
+  mysql_db: name={{ mysql_user_db_name }} state=present login_user=root login_password={{ mysql_root_password }}
+  become: true
+  become_user: root
+
+
+- name: "Giving priviliges to user {{ mysql_user_name }}"
+  mysql_user: name={{ mysql_user_name }} host={{ mysql_user_host }} password={{ mysql_user_password }} login_user=root login_password={{ mysql_root_password }} priv={{ mysql_user_db_name }}.*:ALL,GRANT state=present
+  when: mysql_user_host is defined
+
+- name: "Giving priviliges to user for nodes identifed by hostname ( short)"
+  mysql_user: name={{ mysql_user_name }} host={{ item }} password={{ mysql_user_password }} login_user=root login_password={{ mysql_root_password }} priv={{ mysql_user_db_name }}.*:ALL,GRANT state=present
+  with_items: "{{ current_slurm_mgmt_nodes_short }}"
+  when: current_slurm_mgmt_nodes_short is defined  
+
+- name: "[Example only] Giving priviliges to user with hardcoded hostname"
+  mysql_user: name={{ mysql_user_name }} host='HARDCODE YOUR HOSTNAME' password={{ mysql_user_password }} login_user=root login_password={{ mysql_root_password }} priv={{ mysql_user_db_name }}.*:ALL,GRANT state=present
+  tags: [never]
+
+- name: "Giving priviliges to user for nodes identifed by hostname ( fqdn)"
+  mysql_user: name={{ mysql_user_name }} host={{ item  }} password={{ mysql_user_password }} login_user=root login_password={{ mysql_root_password }} priv={{ mysql_user_db_name }}.*:ALL,GRANT state=present
+  with_items: "{{ current_slurm_mgmt_nodes_fqdn }}"
+  when: current_slurm_mgmt_nodes_fqdn is defined
+
+
+#COMMENT. I think this is redundant but left in as it does no harm
+- name: "Giving priviliges to user with item.domain"
+  mysql_user: 
+    name: "{{ mysql_user_name }}"
+    host: "{{ item }}.{{ domain }}" 
+    password: "{{ mysql_user_password }}"
+    login_user: root 
+    login_password: "{{ mysql_root_password }}"
+    priv: "{{ mysql_user_db_name }}.*:ALL,GRANT" 
+    state: present 
+  with_items: "{{ current_slurm_mgmt_nodes_short }}"
+
+- debug:
+    msg: "{{ item }}"
+  with_items: "{{ current_slurm_mgmt_nodes_fqdn }}"
+  tags: [never]

roles/mysql/vars/Rocky_9.yml

+server_packages:
+  - python3
+  - python3-devel
+  - python3-PyMySQL
+  - mariadb-server
+
+client_packages:
+  - mariadb
+  - python3
+  - python3-PyMySQL
+
+sqlServiceName: mariadb