Goal is to distribute a Java application as a Debian package and allow users to upgrade the package to newer version from within the application itself. Application is running as a systemd service, meaning itself and processes which it spawns are isolated in a control groups "container".
This article does not describe a real world implementation. Instead, it can be used as a template for someone trying to achieve similar results.
Prerequisites
- Debian package installs a Java application to a target system which is the executed as a systemd service.
- Java application runs with privileges of an "ordinary" user.
- Package will be distributed via APT and is upgraded by
apt upgrade
command. - The application will be distributed as a single package and runs as a single service. So, "manager" packages or services are not allowed.
- Build will be executed on Ubuntu 18.04 with
build-essential
,debhelper
,devscripts
andopenjdk-11-jdk
packages installed. - Target system for the application is Ubuntu 18.04.
Implementation
Part of this article is the Debian Package Demo project which can be used for testing of following techniques.
Application and service
In the demo project all Java application's sources and configurations are located in src/
directory under the project's base directory. In this demo scenario the Java application is represented by a simple DebianPackageDemo.java
which invokes upgrade mechanism every 30 seconds.
Next to the application file lies systemd service configuration. There is not too much into it except for, the application will be installed in the /usr/lib/debian-package-demo/
directory and this directory will also be application's working directory and home directory of debian-package-demo
user.
systemd configuration will be later installed to the /lib/systemd/system/
directory.
Extract from the systemd debian-package-demo.service
configuration.
...
[Service]
User=debian-package-demo
# Application is installed in the following directory.
WorkingDirectory=/usr/lib/debian-package-demo/
ExecStart=/usr/bin/java DebianPackageDemo
# Successful exit code of JVM is not 0, but 143
SuccessExitStatus=143
...
Debian package
To create a Debian package, the project must contain debian/
directory with apropriate files in it.
Because debuild
, tool for building Debian packages from source files, generates its output to a parent directory of a directory in which debian/
is located, in the demo application there is a package/debian/
structure so all files are generated into project's base dir instead of directory one level above it. This also means that all Debian package commands should be executed in package/
directory.
-
control
file contains basic information about the package (name, description) as well as list of dependencies needed to build the package and to install it on a target machine.In case of this demo, only debhelper and JDK are needed for building the application and JRE for running it. Following example shows bare minimum of source package control fields needed for creating a valid control file.
Source: debian-package-demo Maintainer: Some Name <some.name@email.com> Build-Depends: debhelper (>= 9), openjdk-11-jdk (>= 11) Standards-Version: 3.9.8 Package: debian-package-demo Architecture: any Depends: openjdk-11-jre (>= 11) Description: Debian Package Demo application. More description of Debian Package Demo application.
-
changelog
file contains information about versions and releases of the application. This file has defined structure and even though it is possible to modify it manually it is highly encouraged to usedch
command for its manipulation.By navigating to
package/
directory, you can use following commands for releasing new version.# Add new record to changelog. -U use standard version number, ie: 1.0.1, -i insert new version. dch -Ui "New version comment" # Mark latest version as released. Use empty string "" to NOT add new comment. dhc -r ""
-
rules
is a makefile used for application building. Those who are not familiar with makefiles, should take in consider that makefile is similar to shell scripts, but there are some differences described in GNU make documentation. For example only tabs and no spaces are allowed in front of commands, variables can be defined in a particular pars of the script, and they are referred to by using normal brackets$(VARIABLE)
, etc.In the demo application the makefile
build
target is used for compiling the application andinstall
target for copying it's files to a directory from which it will be picked up bydebuild
and packed into a generated Debian package.The usual way to create
rules
file is not to write it from scratch, but to usedh
command sequencer for every target and then use target hooks for targets you want to override. See the "any" target pattern%:
.#!/usr/bin/make -f SOURCE = $(CURDIR)/../src TARGET = $(CURDIR)/debian/debian-package-demo %: dh $@ override_dh_auto_build: javac $(SOURCE)/DebianPackageDemo.java override_dh_auto_install: # Created directory structure represents directories where files will be installed on a target machine. mkdir -p $(TARGET)/usr/lib/debian-package-demo mkdir -p $(TARGET)/lib/systemd/system mkdir -p $(TARGET)/etc/sudoers.d/ cp $(SOURCE)/DebianPackageDemo.class $(TARGET)/usr/lib/debian-package-demo cp $(SOURCE)/debian-package-demo.service $(TARGET)/lib/systemd/system # Reason for following files will be explained later. cp $(SOURCE)/debian-package-demo-upgrade.sh $(TARGET)/usr/lib/debian-package-demo cp $(SOURCE)/debian-package-demo-sudoers $(TARGET)/etc/sudoers.d
-
copyright
file contains licence. -
compat
(compatibility) file contains a version ofdebhelper
needed to make the package.
Invoking the upgrade from within the application
Upgrade process can be described by following steps:
- The Java application creates
apt upgrade
process. Theapt upgrade
command is wrapped in thedebian-package-demo-upgrade.sh
shell script. - At the start of upgrade, APT will stop the application itself. This is handled by
preinst
script. - Then APT will copy files from deb package to their target locations.
- APT will start service in
postinst
script.
First of all, because APT has to be called with elevated permissions, to be able to install a package, sudoers file has to be created. Due to this file, debian-package-demo user running the application will be able to execute apt upgrade
command without password. If better security is required the sudo
command can read passwords from standard input if -S switch is provided.
# Upgrade the application without password.
debian-package-demo ALL=NOPASSWD: /usr/bin/apt -y upgrade debian-package-demo
Next, the Java application cannot simply create child process which will stop it. Even if created process is detached. Because systemd service is running in a container, every child process created in that container will be destroyed as soon as the service itself is stopped.
To bypass this behaviour, one-off user task is scheduled by systemd-run
from the Java application with activation delay 1 second. This task will be later picked up by systemd and executed in a new, separated container.
In the Java application there is code which invokes upgrade:
String scheduleUpgradeCommand = "systemd-run --user --on-active=1 /usr/lib/debian-package-demo/debian-package-demo-upgrade.sh";
ProcessBuilder processBuilder = new ProcessBuilder("/bin/sh", "-c", scheduleUpgradeCommand);
// Environment variable XDG_RUNTIME_DIR has to be set for
// systemd-run to run correctly. Otherwise "Failed to create
// bus connection" error will occur.
processBuilder.environment().putIfAbsent("XDG_RUNTIME_DIR", "/run/user/" + new UnixSystem().getUid());
// Redirect process I/O to the application's I/O so it's output can be
// observed by journalctl.
processBuilder.inheritIO();
processBuilder.start();
Shell script debian-package-demo-upgrade.sh
contains simple command:
#!/bin/sh -e
sudo apt -y upgrade debian-package-demo
Finally, the Debian package should contain preinst
, prerm
and postinst
scripts which will stop and start systemd service during installation and eventually set-up the environment for the application.
Post installation script postinst
creates user for the application and enables systemd session lingering for the user.
Scheduled upgrade task is created as a systemd user timer which needs systemd user session to be present. The session is usually created during user's login and destroyed on its logout. In case of demo application debian-package-demo user will never login, so the session has to be created on system startup. This can be achieved by so called session lingering which will create the session on system boot.
#!/bin/sh -e
userName=debian-package-demo
serviceName=debian-package-demo
if ! id ${userName} > /dev/null 2>&1; then
adduser --quiet --system ${userName} --home /usr/lib/${serviceName}
# Because the Java application will schedule one-off systemd user
# services systemd session has to exist for user even if the user
# is not logged-in. To ensure this session lingering will be enabled
# for user.
loginctl enable-linger ${userName}
fi
systemctl daemon-reload
if ! systemctl is-active --quiet ${serviceName}; then
echo "Starting ${serviceName} service..."
systemctl enable ${serviceName}
systemctl start ${serviceName}
fi
Scripts preinst
and prerm
are similar to postinst
.
Build and distribution of the package
To build and distribute first version of the package and then to release a new version and upgrade it we need to:
- Build Debian package by calling
debuild
. - Move generated files to repository directory and generate repository index by
dpkg-scanpackages
. - Add the repository to target systems APT sources list.
- Install the package and start the application / service.
- Release a new version.
- Let the application to upgrade itself.
The demo application has following directory structure.
. <-- Debian Package Demo project's base dir
├── package
│ └── debian
│ ├── changelog
│ ├── compat
│ ├── control
│ ├── copyright
│ ├── postinst
│ ├── preinst
│ ├── prerm
│ └── rules
├── repository
└── src
├── DebianPackageDemo.java
├── debian-package-demo.service
├── debian-package-demo-sudoers
└── debian-package-demo-upgrade.sh
In package/
directory call the debuild -us -uc
command which will compile Java application and build Debian package files into its parent directory. Switches -us -uc will cause unsigned package to be built which is acceptable for testing purposes, but it is highly recommended to sign packages in production environment.
Next step is to move generated debian-packages-demo*
files to repository/
directory and then to generate repository index. This is done by the dpkg-scanpackages -m . /dev/null > Packages
command.
Let's assume the application will be installed by APT on a machine where it is build. Then the repository directory can be added to APT sources list as a simple file repository.
# Because packages are not signed, the repository has to be marked as trusted.
deb [trusted=yes] file:///debian-packages-demo-base-dir/repository ./
Now the demo project can be installed.
apt update
apt install debian-package-demo
To release a new version of the application version of the Debian package has to be increased, new package has to be build and moved to repository.
- Use the
dch
command to increase version as described before. - Build the package by
debuild
and move generated files to therepository/
directory. - Regenerate repository index by
dpkg-scanpackages
. - Call
apt update
to update sources list and wait for the application pick up latest version by invoking theapt upgrade
command.
Troubleshooting
-
If your "package experiments" led you to the point where a package cannot be removed or reinstalled because APT complains about "something is horribly broken", then the package can be "forcefully" removed from the system by directly calling of the underlying dpkg package manager. Note: APT is "just" a wrapper which downloads relevant packages from a repository to a target computer. Then it calls dpkg which installs, upgrades or removes it.
dpkg --force-remove-reinstreq -r debian-package-demo
-
Standard I/O from the processes created by Java application are redirected to the Java application's I/O itself. Then, the output can be observed in the systemd log of the application. If the application is not able to "upgrade itself" it is a good idea to look into it.
journalctl -u debian-package-demo
-
If the application reports a relatively common error "Failed to create bus connection: No such file or directory" make sure lingered systemd session for debian-package-demo user exists. In this case the
/run/user/$UUID
directory must exist, where UUID is a number ID of the user. Also, an environment variable$XDG_RUNTIME_DIR
must point to that directory, and the variable must be available for the environment executingsystemd-run
.