Situation
Suppose you are running Ubuntu LTS or Debian (Old)Stable, but you want the newer version of a package that's available in a newer distribution release (e.g. Ubuntu non-LTS or Debian testing). Then this how-to is for you.
This type of backport will be a no-change-of-sources rebuild. While it's not always trivial, it turns out to be working quite well usually.
First of all, you might want to check if the package is already backported for you in Debian/Ubuntu. Debian offers quite some backports already. For example in stretch-backports, at the timme of writing there are more than 1400 packages available.
Requirements
- Launchpad account.
- An SSH key.
- A GPG key. Not covered in this post.
Example in this blog with CCID
For this article I'll be using the CCID library, a piece of software for using smartcard readers. At the time of writing, Debian testing has version 1.4.27 and Ubuntu Xenial (16.04) has version 1.4.22 and there's no newer package available in xenial-backports.
At the time of writing Ubuntu Zesty (17.04) is the next non-LTS stable distribution release. This distribution offers version 1.4.26. That's nice, but we want the newer version for improved hardware support in our stable software stack rather than upgrading to a newer Ubuntu version just for this. That's what we're going to do here.
In this full example I'll use Ubuntu Xenial as local system to download/build the backport and upload the sources to Launchpad to get them built for many Ubuntu distributions.
Install Dependencies
In order to verify the downloaded sources, I'll advice to download the GPG keys of the Debian maintainers from a trusted source.
Ubuntu has packaged them nicely for you in the debian-keyring package.
Apart from that we need devscripts for the general package tooling.
$ sudo apt-get install debian-keyring devscripts
You don't need build dependencies here if you want to have Launchpad build your package in the cloud.
In case you want to build it on your local machine, you can install it with sudo apt-get build-dep <package> if present already in your current distribution.
Please note that if new requirements are required for new releases these are not included yet since your APT installation doesn't know that.
Configure devscripts
Configure the keyring to use the installed Debian developers keyring.
For example, in ~/.devscripts:
DSCVERIFY_KEYRINGS="/etc/apt/trusted.gpg:/usr/share/keyrings/debian-maintainers.gpg:~/.gnupg/pubring.gpg"
The syntax is like PATH, colon-separated.
In the above example it's a 'sane' list of keyrings on your system that you probably want to check against.
Set your username/email, e.g.
$ export DEBEMAIL="gertvdijk@gmail.com"
$ export DEBFULLNAME="Gert van Dijk"
Or, as a permanent solution to the above, put them in your environment variables and get a new login shell.
$ echo 'export DEBEMAIL="gertvdijk@gmail.com"' >> ~./profile
$ echo 'export DEBFULLNAME="Gert van Dijk"' >> ~./profile
Create the PPA on Launchpad
If you don't have a Launchpad account yet, create one.
Configure an SSH key for authentication on your profile page. If you don't have an SSH key yet, see this other blog post of me. Create an RSA key for Launchpad, because it does not support ed25519 keys at the time of writing (error: "Invalid key"). So, for this purpose, we'll be creating an extra RSA key just for Launchpad. E.g.:
$ ssh-keygen -o -a 100 -t rsa -b 2048 -f ~/.ssh/id_rsa_launchpad
Use the the public key for your Launchpad profile:
$ cat ~/.ssh/id_rsa_launchpad.pub
Add the output of the command above in the box for "Public key line".
Also upload your GPG key to your Launchpad profile in the "OpenPGP keys" section. If you don't have one yet, I can recommend the article by Alex Cabal.
After configuring your SSH and GPG keys, you should see a button "Create a new PPA" on your profile in the "Personal package archives" section. Follow those steps online and keep the URL in mind for the next step.
Configure dput
From the Launchpad PPA Upload page, configure your ~/.dput.cf.
E.g. for my ccid-backports PPA I chose SFTP (using SSH):
[ccid-backports]
fqdn = ppa.launchpad.net
incoming = ~gertvdijk/ubuntu/ccid-backports/
method = sftp
login = gertvdijk
ssh_config_options = IdentityFile ~/.ssh/id_rsa_launchpad
Get sources
Find the link to the .dsc file on the Debian packages website for libccid in the testing distribution. On the right hand side you shoul see that link.
And use that URL for dget, e.g.:
$ dget http://http.debian.net/debian/pool/main/c/ccid/ccid_1.4.27-1.dsc
The above should now "just work" without errors/warnings in the output, because the sources are signed by a Debian developer whose key is present in the debian-keyring package.
Sources should be available unpacked now. Change the working directory to there:
$ cd ccid-1.4.27
Edit the changelog file
Set the version in the Debian changelog with the appropriate target distribution and a locally suffixed version string. In this case using Xenial:
$ dch --local ppa~xenial --distribution xenial "No-change rebuild from current Debian testing version for Ubuntu xenial."
The above is setting all required information on the command line.
Alternatively, go full interactive and edit the file using dch -i.
This opens a text editor in a way similar to what you might be used to when editing the sudoers file with visudo.
I'm not using the --rebuild option, since I'm not able to control the suffix label (buildX where X is a counter).
Instead, I want to be explicit about this being a version in a PPA.
I believe this could help users identifying possible issues later on.
Build the source package
Meet debuild.
It's the universal way to build a package from its sources, provided it's 'packaged' by your distribution.
As we're going to build the binary package in the Launchpad cloud, we will build a signed source package and have Launchpad build the binaries.
Without the original sources, only diff-upload:
$ debuild -S
This approach is very efficient, but will only be accepted after upload if Launchpad can access the original upstream tarball of the sources itself. This is the case usually, but in some cases for packages not synced into Ubuntu yet or for unreleased versions you can run:
$ debuild -S -sa
The above command will ask for your GPG keyring to be unlocked for signing the changes file.
Output will be in the parent directory:
$ ls -al ../*.changes
-rw-r--r-- 1 gert gert 1933 Aug 7 17:37 ../ccid_1.4.27-1ppa~xenial1_source.changes
Upload the sources to Launchpad
$ dput <ppa-name> <changes-file>
E.g.:
$ dput ccid-backports ../ccid_1.4.27-1ppa~xenial1_source.changes
This should upload the packages. Reporting of the acceptance will be communicated over email. Keep an eye on your email inbox.
In the report by email you'll get either:
Accepted:
OK: ccid_1.4.27.orig.tar.bz2
OK: ccid_1.4.27-1ppa˜xenial1.debian.tar.xz
OK: ccid_1.4.27-1ppa˜xenial1.dsc
-> Component: main Section: libs
[...]
or:
Rejected:
Unable to find ccid_1.4.27.orig.tar.gz in upload or distribution.
Files specified in DSC are broken or missing, skipping package unpack verification.
[...]
For the latter case, rebuild the package with the full sources included (-sa) in the step above and increase the version in the Changelog because Launchpad will complain the version has already been uploaded before:
$ dput ccid-backports ../ccid_1.4.27-1ppa~xenial1_source.changes
Package has already been uploaded to ccid-backports on ppa.launchpad.net
Nothing more to do for ccid_1.4.27-1ppa~xenial1_source.changes
Go to the Launchpad PPA page to see the build progress.
You're done now, hopefully, if the package compiles!
I've published the libccid packages from this example in my PPA.
How users can use your PPA
Follow the instructions on the PPA page.
Usually, using the add-apt-repository tool approach is the easiest.
E.g.:
$ sudo add-apt-repository ppa:gertvdijk/ccid-backports
$ sudo apt-get update
Security updates and PPAs
In case Ubuntu publishes a security update, it's usually done without upgrading the upstream version of the package. However, by installing your PPA, the newer version available in your PPA will get precendence over the one available in the security repository as the version compare wins for the newer major version.
This can be tweaked by APT 'pinning', but the real solution would be to stay up-to-date and publish new releases regularly for your users.
Build for more distributions
In case you want to build for more Ubuntu distributions, first remove the newly created changelog entry in debian/changelog.
After that, repeat the process of the changelog entry above.
Then rebuild the source package and upload the same way.
Launchpad simply builds the package for the target distribution specified in the Changelog.
Build/Install the binary locally
Before uploading and publishing the package you may want to test it locally. Install the build dependencies and run this:
$ sudo apt-get build-dep libccid
$ debuild -uc -us -b
The -uc/-us and -b options are for "unsigned changes/sources file" and "binary only" respectively.
It will create a binary package in the parent directory.
Install with dpkg -i:
$ sudo dpkg -i ../libccid_1.4.27-1ppa~xenial1_amd64.deb
$ sudo apt-get install -f # in case of dependency errors to be resolved
Other useful tools
apt-cache policy
$ apt-cache policy <package>
Shows the options the APT package management sees and what version is chosen for installation.
apt-get source
$ apt-get source <package>
Downloads the sources of a given binary package in the current working directory.
This assumes the presence of deb-src lines in your /etc/apt/sources.list.
Docker
I find Docker very powerful to experiment with packages in other distribution releases. Very quickly I can check the presence of packages, have a peek at the sources and try to build them.
It's too broad to go into detail in this post, though.
Another example: OpenVPN
OpenVPN 2.4.0 appeared to be buggy for me and I noticed 2.4.3 available in stretch-backports worked for me instead. Building it for Ubuntu Xenial failed for me at first. Fortunately, it turns out that some build dependencies adjustments were all that was required to get it built.
--- upstream/openvpn-2.4.3/debian/control 2017-06-30 15:39:56.000000000 +0200
+++ my/openvpn-2.4.3/debian/control 2017-08-07 23:06:35.852621110 +0200
@@ -4,7 +4,7 @@
Maintainer: Bernhard Schmidt <berni@debian.org>
Uploaders: Jörg Frings-Fürst <debian@jff-webhosting.net>
Build-Depends:
- debhelper (>= 10),
+ debhelper (>= 9),
dh-systemd (>= 1.5),
dpkg-dev (>= 1.16.1),
iproute2 [linux-any],
@@ -12,7 +12,7 @@
liblzo2-dev,
libpam0g-dev,
libpkcs11-helper1-dev,
- libssl1.0-dev,
+ libssl-dev,
libsystemd-dev [linux-any],
net-tools [!linux-any],
pkg-config
You can see the results in my OpenVPN-backports PPA.