sycured: consultant's blog
Jenkins worker (agent) in FreeBSD jail

Jenkins worker (agent) in FreeBSD jail

sycured's photo
sycured
·

3 min read

Jails are Linux containers in FreeBSD's world. It's time to build it.

Why running it in jail?

Jails are like Linux containers so we can create specific jail like this schema:

  • jw-rust: build rust projects
  • jw-python: build python projects
  • jw-latex: build latex documents

The jail is like your docker base image, you install your requirements and you build.

We can snapshot a jail so post creation, you can create a snapshot and periodically, you restore this snapshot so you get back to a clean jail environment.

Host configuration for jails

I'll use Bastille to manage my jails.

pkg install -y bastille

My host is running FreeBSD 13.1-RELEASE so I'll use the same version for my jails, so firstly, I need to have it available:

bastille bootstrap 13.1-RELEASE

Now, it's time to create the network interface bastille0 with the NAT

sysrc cloned_interfaces="lo1"
sysrc ifconfig_lo1_name="bastille0"
sysrc pf_enable="YES"
sysrc pf_rules="/etc/pf.conf"
sysrc bastille_enable="YES"

If you already have an /etc/pf.conf file, you need to take the part needed for jails and nothing more This configuration is only a minimalist one.

# external interface.
ext_if="vtnet0"

table <jails> persist

set skip on lo
scrub in on $ext_if all fragment reassemble

nat on $ext_if from <jails> to any -> ($ext_if:0)
rdr-anchor "rdr/*"

pass out quick modulate state
antispoof for $ext_if inet
antispoof for $ext_if inet6

It's time to restart pf and start the new interface:

service pf restart
service netif cloneup

Add swapfile (optional)

Swap isn't bad because it can avoid having the OOM-killer kill the last compilation step for example.

Let's create a 8GB…

dd if=/dev/zero of=/usr/swap.bin bs=1M count=8192
chmod 0600 /usr/swap.bin
echo "md99 none swap sw,file=/usr/swap.bin,late 0 0" >> /etc/fstab
swapon -aL

I put the option late because the swapfile is a lower priority than other entries in fstab at boot time.

If the option "late" is specified, the file system will be automatically mounted at a stage of system startup after remote mount points are mounted.

Jenkins worker (agent)

Let's create the jail:

  • name: jw-rust
  • release used: 13.1-RELEASE
  • ip address: 192.168.0.2
bastille create jw-rust 13.1-RELEASE 192.168.0.2

Open a shell

bastille cmd jenkins-worker sh

Now, it's time to finalize the setup

# Switch to latest (optional)
sed -i '' 's/quarterly/latest/' /etc/pkg/FreeBSD.conf

# Install Java needed for the agent
pkg install -y openjdk19

# Install your stack
pkg install -y rust

#create jenkins user
pw user add -n jenkins -d /home/jenkins -m -s /bin/sh

# get agent.jar from the server
fetch http://MY_JENKINS_INTERNAL_NAME:8080/jnlpJars/agent.jar
mv agent.jar /usr/local/

# create the service file
cat > /etc/rc.d/jenkins_agent << EOF
#!/bin/sh
#
# PROVIDE: jenkins_agent
# REQUIRE: LOGIN
# KEYWORD: shutdown
#
# Note:
# Set "jenkins_agent_enable=yes" in either /etc/rc.conf, /etc/rc.conf.local to make
# this script actually do something.
#

. /etc/rc.subr

name="jenkins_agent"
rcvar="${name}_enable"

start_cmd="${name}_start"
stop_cmd="${name}_stop"

load_rc_config $name

jenkins_agent_start() {
        if [ ! -f /var/run/jenkins_agent.pid ]
        then
                echo -n "Starting jenkins agent"
                su jenkins -c "cd /usr/local && /usr/local/openjdk19/bin/java -jar agent.jar -jnlpUrl http://JENKINS_INTERNAL_NAME:8080/manage/computer/MY_WORKER_NAME/jenkins-agent.jnlp -secret SECRET_FROM_JENKINS -workDir '/home/jenkins' &"
                ps aux | grep jenkins | grep "\-jar agent" | cut -d " " -f2 > /var/run/jenkins_agent.pid
                echo "."
        else
                echo "jenkins agent is already running!"
        fi
}

jenkins_agent_stop() {
        if [ ! -f /var/run/jenkins_agent.pid ]
        then
                echo "jenkins agent is not running"
        else
                echo -n "Stopping jenkins agent"
                kill `cat /var/run/jenkins_agent.pid`
                rm /var/run/jenkins_agent.pid
                echo "."
        fi
}

run_rc_command "$1"
EOF

chmod +x /etc/rc.d/jenkins_agent

# set the service to start on boot
sysrc jenkins_agent_enable=yes

# start it now
service jenkins_agent start

Conclusion

We don't need Docker on FreeBSD and it's easy to know why with this setup. BastilleBSD can create jails using templates so the entire setup done manually can be fully automated which replaces Docker.

Happy building on FreeBSD ;)

JenkinsFreeBSD