Creating Custom FreeBSD AMIs for Jenkins - Part 2

    FreeBSD Jenkins ec2

After creating a brand new AMI from the latest and greatest FreeBSD, we would be remiss if we did not properly test that it boots and can do a basic build.

This article is continuation of Creating Custom FreeBSD AMIs for Jenkins - Part 1.

This job will be broken down into the following sections:

  1. Copy the env.txt file to populate AMI ID with
  2. Inject the environment variables from the env.txt file
  3. Test the image to make sure it contains the packages needed and the compiler works
  4. Promote the AMI ID to the main AMI used for all the other jobs
  5. Clean up the old AMIs that are not needed anymore

Create a new Freestyle Project job in Jenkins and name it something like new-base-AMI-test.

Copy the env.txt file

Under the Build section of the job select Add build step and choose Copy artifacts from another project.

Set Project name to:

new-base-AMI-build/ARCH=${ARCH},BUILD_NODE=aws,REL=freebsdmain

Set Which build to:

Latest successful build

Set Artifacts to copy to:

env.txt

Inject the environment variables from env.txt

Under the Build section of the job select Add build step and choose Inject environment variables.

Set the Properties File Path to match the last line above env.txt and leave the Properties Content section blank.

This will be used to populate the AMI ID in a future step.

Test the image

Under the Build section of the job select Add build step and choose Execute shell.

For some basic testing, check to make sure the packages necessary are installed and the complier can compile a basic “Hello World” program:

uname -a

pkg info

PKGS="bsdec2-image-upload git jq nginx openjdk12 poudriere-devel rsync"
for p in ${PKGS}; do
	pkg info ${p}
done

echo '#include <stdio.h>' >> test.c
echo 'int main() {' >> test.c
echo '	printf("Hello World!\n");' >> test.c
echo '	return 0;' >> test.c
echo '}' >> test.c
cc -o test test.c
./test

Promote the AMI ID to be used by the other jobs

Under the Build section select Add build step and choose Execute system Groovy script.

This will modify the test AMI configured in the cloud section of Jenkins so make sure the names line up:

import jenkins.model.*;
import hudson.model.*
import hudson.AbortException
import hudson.plugins.ec2.*;

def config = new HashMap()
config.putAll(binding.variables)
def logger = config['out']

def envvars = new HashMap()
envvars.putAll(build.getEnvironment(listener))
def newami = envvars['NEWAMI']
def arch = envvars['ARCH']


Jenkins.instance.clouds.each {
  println('cloud: ' + it.displayName)
  if (it.displayName == 'engineering-aws') {
    it.getTemplates().each {
      if (it.description == 'FreeBSD-main-' + arch) {
        println('description: ' + it.description)
        def oldami = it.getAmi()
        if (oldami == newami) {
          println("Current AMI: " + oldami + "; new AMI: " + newami)
          throw new AbortException("AMIs are the same")
        }
        else {
          println("Current AMI id: " + oldami)
          it.setAmi(newami)
          println("New AMI: " + it.getAmi())
        }
      }
    }
  }
}

Jenkins.instance.save()

Clean up old AMIs

Since AWS charges for every little bit and to keep a clean list of images remove any old AMIs that are not needed anymore.

Under the Build section choose Add build step and select the type to be Execute shell.

export AWS_DEFAULT_REGION=us-east-2
export AWS_DEFAULT_OUTPUT=json
env
if [ "${ARCH}" = "amd64" ]; then
	T=x86_64
else
	T=arm64
fi

# deregister old AMIs
for ami in $(aws ec2 describe-images --owners self --filters Name=architecture,Values=${T} | jq '.Images[].ImageId' | sed -e 's/"//g'); do
	[ "${ami}" = "${NEWAMI}" ] && continue
	# Check name matches
	NAME=$(aws ec2 describe-images --image-ids ${ami} | jq '.Images[].Name' | sed -e 's/"//g')
	#Strip down "FreeBSD main-aarch64-24" to "main-aarch64"
	NAME=${NAME##FreeBSD }
	NAME=${NAME%-*}
	[ "${NAME}" != "main-${ARCH}" ] && continue
	SNAP=$(aws ec2 describe-images --image-ids ${ami} | jq '.Images[].BlockDeviceMappings[] | select(.DeviceName == "/dev/sda1") | .Ebs.SnapshotId' | sed -e 's/"//g')
	echo "Removing AMI: ${ami}; Name: ${NAME}; Snap: ${SNAP}"
	aws ec2 deregister-image --image-id ${ami}
	# delete its snap
	sleep 1
	aws ec2 delete-snapshot --snapshot-id ${SNAP}
done

Configure the build job to trigger this job

In the new-base-AMI-build job add a build trigger to trigger the new-base-AMI-test job whenever the build is successful.

brd