-
Notifications
You must be signed in to change notification settings - Fork 3
Updating
Rocks updates should be built into "rolls", simplifying rollbacks, and ensuring cluster nodes are configured consistently. For updates, there are two main groups, point release upgrades, and their associated updates.
A point release upgrade, will upgrade the CentOS distribution from 7.4 to 7.6. In the case of this example, it upgrades to 7.6.1810.
rocks create mirror http://centos.s.uw.edu/centos/7.6.1810/os/x86_64/Packages/ rollname=CentOS_7_6_1810
rocks add roll CentOS_7_6_1810-7.0-0.x86_64.disk1.iso
rocks enable roll CentOS_7_6_1810
cd /export/rocks/install
rocks create distro
A point release update, includes all updates up to the current date for a specific point release, such as 7.6.1810. To reduce confusion, the roll name should include both the point release, and date it was made. For example, the update roll generated on 2019-04-03 for 7.6.1810 would be named CentOS_7_6_1810_Update_2019_04_03.
rocks create mirror http://centos.s.uw.edu/centos/7.6.1810/updates/x86_64/Packages/ rollname=CentOS_7_6_1810_Update_2019_04_03
rocks add roll CentOS_7_6_1810_Update_2019_04_03-7.0-0.x86_64.disk1.iso
rocks enable roll CentOS_7_6_1810_Update_2019_04_03
cd /export/rocks/install
rocks create distro
Updating the head node is simply a matter of running yum, and rebooting.
yum clean all
yum check-update
yum update
reboot
A compute node can be updated using the same method as the head node. However, the preferred way is to simply redeploy or reinstall the node. Nodes can be reinstalled remotely, and the process is completely autonomous. Below is an example of reinstalling the node compute-0-0:
rocks set host boot compute-0-0 action=install
ssh compute-0-0 "shutdown -r now"
Once the node has finished reinstalling, you'll need to update the SSH host key cache:
rocks sync config
During an update, you may find that the sysctl.conf on the head node is reset to default. As a result, when the node is rebooted IP forwarding will be disabled, causing the compute nodes to no longer have internet access. This can be fixed by appending the following to /etc/sysctl.conf:
net.ipv4.ip_forward = 1
After updating the head node, hostbased authentication stopped working. The problem was traced to ssh-keysign no longer being setuid.
chmod u+s /usr/libexec/openssh/ssh-keysign