Move swapfile to data volume (/var/lib/scylla)

[syuu1228]

Related with #491, we want to reduce the image size, but if we do that we will not have enough disk space for swapfile.
To fix the problem, we need to move swapfile to the data volume (/var/lib/scylla) to avoid running out disk space of rootfs.

see: #491 (comment)

[avikivity]

It's not a good idea, wasting fast+expensive NVMe space.

Maybe we can add an EBS volume to the AMI.

[avikivity]

I asked bard, it said this:

To configure an AMI to create an EBS volume when launched as an instance, you can use block device mappings. Block device mappings specify how block devices are mapped to an instance when it launches. You can use block device mappings to attach EBS volumes to an instance at launch, and you can also use them to configure the instance's root volume.

To attach an EBS volume to an instance at launch, you need to create a block device mapping that specifies the following information:

• The device name of the EBS volume. This is the name that the instance will use to refer to the volume.
• The ID of the EBS volume. You can find the ID of an EBS volume in the Amazon EC2 console or using the AWS CLI.
• The size of the EBS volume. The size of the volume must be greater than zero.
• The volume type. The volume type determines the performance of the volume. There are three types of EBS volumes: standard, provisioned IOPS, and cold.
• Whether the volume should be deleted when the instance is terminated. If you set this parameter to true, the volume will be deleted when the instance is terminated. If you set this parameter to false, the volume will be preserved.

To configure the instance's root volume, you need to create a block device mapping that specifies the following information:

• The device name of the root volume. This is the name that the instance will use to refer to the root volume.
• The ID of the snapshot that contains the root volume. You can find the ID of a snapshot in the Amazon EC2 console or using the AWS CLI.
• The size of the root volume. The size of the root volume must be greater than zero.
• The volume type. The volume type determines the performance of the volume. There are three types of EBS volumes: standard, provisioned IOPS, and cold.

Once you have created the block device mappings for the EBS volumes that you want to attach to the instance, you can create the AMI. To create an AMI from an instance, you can use the following methods:

• Use the Amazon EC2 console.
• Use the AWS CLI.
• Use the EC2 API.

Once the AMI has been created, you can launch it as an instance. When the instance launches, the EBS volumes that you specified in the block device mappings will be attached to the instance.

[fruch]

It's not a good idea, wasting fast+expensive NVMe space.

Maybe we can add an EBS volume to the AMI.

bard is nice

but, the issue is that now you need to teach whom is using the image, that he need todo something else
to get swap as we recommend

the whole reason it was done in build time, was to save time and complexity for cloud deployment
the price for it was bigger AMIs.

we can't take a step back, without syncing it with whom is depending on it.

[avikivity]

It's not a good idea, wasting fast+expensive NVMe space.
Maybe we can add an EBS volume to the AMI.

bard is nice

but, the issue is that now you need to teach whom is using the image, that he need todo something else to get swap as we recommend

The idea is to configure the AMI to do it, if it's possible.

the whole reason it was done in build time, was to save time and complexity for cloud deployment the price for it was bigger AMIs.

we can't take a step back, without syncing it with whom is depending on it.

[avikivity]

Bard:

To create an AMI that creates a new EBS volume and attaches to it, you can use a combination of user data and cloud-init to automate the process. Here's a general outline of the steps involved:

1. Create a script to create the EBS volume and attach it to the instance: Write a script that uses the AWS CLI or boto3 library to create a new EBS volume of the desired size and attach it to the instance. This script should be placed in the user data section of the instance configuration.

2. Create an AMI from the instance: Once the script has successfully created and attached the EBS volume, create an AMI from the instance. This AMI will capture the state of the instance, including the attached EBS volume.

3. Launch new instances from the AMI: When you launch new instances from the AMI, the EBS volume will be automatically attached to each instance. This ensures that each instance has the same storage capacity and configuration.

Here's an example of a user data script that uses the AWS CLI to create a new EBS volume of 100GB and attach it to the instance:

#!/bin/bash

# Set the AWS region
aws_region=us-east-1

# Create a new EBS volume of 100GB
aws ec2 create-volume --volume-type gp2 --size 100 --region $aws_region > volume.json

# Extract the volume ID from the JSON output
volume_id=$(cat volume.json | jq -r '.VolumeId')

# Attach the EBS volume to the instance
aws ec2 attach-volume --instance-id $(curl -s http://169.254.169.254/latest/meta-data/instance-id) --volume-id $volume_id --region $aws_region

# Format and mount the EBS volume
mkfs.ext4 /dev/xvdf
mount /dev/xvdf /mnt/ebs

To create an AMI from the instance, you can use the AWS CLI or the Amazon EC2 console. Once the AMI is created, you can launch new instances from it and the EBS volume will be automatically attached to each instance.

[avikivity]

I think bard is crap

[avikivity]

but, we can certainly create a swap volume in cloud-init on the first boot.

[fruch]

but, we can certainly create a swap volume in cloud-init on the first boot.

1. would slow a bit that first boot
2. users (i.e. scylla-cloud) should be aware and set the size of the root disk big enough (i.e. 30Gb is they need it as before)

I wonder how long until we roll over, and move it back to build time to speed things up :)

[avikivity]

Why does the root disk need to change?

[fruch]

Why does the root disk need to change?

Where is the swap now in our images ?

[syuu1228]

We can add additional EBS volume with following change on packer configuration file:

diff --git a/packer/scylla.json b/packer/scylla.json
index f06cc7c..012e4b1 100644
--- a/packer/scylla.json
+++ b/packer/scylla.json
@@ -36,6 +36,12 @@
         {
           "device_name": "/dev/sdi",
           "virtual_name": "ephemeral7"
+        },
+        {
+          "device_name": "/dev/sdj",
+          "volume_type": "gp3",
+          "volume_size": 20,
+          "delete_on_termination": "true"
         }
       ],
       "ami_name": "{{user `image_name`| clean_resource_name}}",

With the configuration file, machine-image will have additional disk by default.
However, I haven't find a way to do same on GCE.
Maybe user have to manually attach additional disk.

[fruch]

We can add additional EBS volume with following change on packer configuration file:

diff --git a/packer/scylla.json b/packer/scylla.json
index f06cc7c..012e4b1 100644
--- a/packer/scylla.json
+++ b/packer/scylla.json
@@ -36,6 +36,12 @@
         {
           "device_name": "/dev/sdi",
           "virtual_name": "ephemeral7"
+        },
+        {
+          "device_name": "/dev/sdj",
+          "volume_type": "gp3",
+          "volume_size": 20,
+          "delete_on_termination": "true"
         }
       ],
       "ami_name": "{{user `image_name`| clean_resource_name}}",

With the configuration file, machine-image will have additional disk by default.
However, I haven't find a way to do same on GCE.
Maybe user have to manually attach additional disk.

How is it different from sizing the root disk ? both would be saved and we go back to the sizing, but you'll have two snapshots ? or I'm missing something

[yaronkaikov]

I don't understand why we should calculate the swap based on any disk size. Based on 069318b , there is a simple calculation to the relevant size of the swap, the max value should be 16Gb, how long does it take to create it during first boot as @avikivity suggested?

[syuu1228]

Closing, since using the data volume for swap seems not good idea, as @avikivity commented on the thread.