Date Series Part 3 of Building RPMs Tags rpm / collectd

Introduction

Now that we have an rpm build environment , we can build RPM packages.

In this guide, we will use the collectd monitoring application and walk through three different ways of building RPMs for this package: using a spec file, rebuilding a source rpm, and using mock.


Prerequisites

We already installed the necessary packages in Part 1. These packages provide the utilities that we will use to build RPMs, namely rpmbuild, spectool, and yum-builddep.

The steps in this guide will be carried out by the makerpm user.

Additionally, we will need the source for collectd:

cd $HOME
mkdir src && cd src
curl --progress https://collectd.org/files/collectd-5.5.0.tar.gz | tar xz


Building RPMs from a SPEC file

A spec file includes all the information and steps the rpmbuild command needs to build an RPM package. Among other things, it lists the name and version of your application, all package dependencies and how the application should be configured and built. A spec file is written in plain text and, by convention, its name is formed by appending a .spec suffix to the application name, for example, collectd.spec.

The collectd spec file is relatively complex compared to other spec files. It contains custom macros for the many built-in plugins as well as subpackage definitions for each configured plugin.

Have a look at the spec file and then copy it to the SPECS directory in your rpmbuild directory:

cd $HOME/src/collectd-5.5.0/contrib/redhat
cp collectd.spec $HOME/rpmbuild/SPECS

Change to the SPECS directory inside your rpmbuild directory and use the spectool command to download the sources listed inside the spec file.

cd $HOME/rpmbuild/SPECS
spectool -g -R collectd.spec

The -g flag tells spectool to download the Source listed in the spec file.

Name:           collectd
Version:        5.5.0
Source:         http://collectd.org/files/%{name}-%{version}.tar.bz2

The -R flag downloads the Source to the rpm's %{_sourcedir}, which translates to $HOME/rpmbuild/SOURCES directory:

$ rpm --eval "%{_sourcedir}"
/home/makerpm/rpmbuild/SOURCES

Now, run the rpmbuild command to build the RPM from the spec file:

rpmbuild -bb collectd.spec 
error: Failed build dependencies:
    libtool-ltdl-devel is needed by collectd-5.5.0-1.el6.x86_64
    libcap-devel is needed by collectd-5.5.0-1.el6.x86_64
    librabbitmq-devel is needed by collectd-5.5.0-1.el6.x86_64
    curl-devel is needed by collectd-5.5.0-1.el6.x86_64
    yajl-devel is needed by collectd-5.5.0-1.el6.x86_64
    libdbi-devel is needed by collectd-5.5.0-1.el6.x86_64
    libudev-devel is needed by collectd-5.5.0-1.el6.x86_64
    libpcap-devel >= 1.0 is needed by collectd-5.5.0-1.el6.x86_64
    ganglia-devel is needed by collectd-5.5.0-1.el6.x86_64
    OpenIPMI-devel is needed by collectd-5.5.0-1.el6.x86_64
    iptables-devel is needed by collectd-5.5.0-1.el6.x86_64
    java-devel is needed by collectd-5.5.0-1.el6.x86_64
    lvm2-devel is needed by collectd-5.5.0-1.el6.x86_64
    libmemcached-devel is needed by collectd-5.5.0-1.el6.x86_64
    libmodbus-devel is needed by collectd-5.5.0-1.el6.x86_64
    libmnl-devel is needed by collectd-5.5.0-1.el6.x86_64
    iproute-devel is needed by collectd-5.5.0-1.el6.x86_64
    libnotify-devel is needed by collectd-5.5.0-1.el6.x86_64
    libesmtp-devel is needed by collectd-5.5.0-1.el6.x86_64
    nut-devel is needed by collectd-5.5.0-1.el6.x86_64
    openldap-devel is needed by collectd-5.5.0-1.el6.x86_64
    perl-ExtUtils-Embed is needed by collectd-5.5.0-1.el6.x86_64
    protobuf-c-devel is needed by collectd-5.5.0-1.el6.x86_64
    liboping-devel is needed by collectd-5.5.0-1.el6.x86_64
    hiredis-devel is needed by collectd-5.5.0-1.el6.x86_64
    libatasmart-devel is needed by collectd-5.5.0-1.el6.x86_64
    varnish-libs-devel is needed by collectd-5.5.0-1.el6.x86_64
    libvirt-devel is needed by collectd-5.5.0-1.el6.x86_64

The rpmbuild process checks package dependencies based on the packages listed in the Requires and BuildRequires tags in spec file.

Thankfully, these dependencies are easy to satisfy with the yum-builddep command found in the yum-utils package. Run the following to download the necessary dependencies:

sudo yum-builddep collectd.spec

Now that all dependencies are installed, re-run rpmbuild:

rpmbuild -bb collectd.spec

This gets to the very end and throws an error when checking rpaths:

+ /usr/lib/rpm/check-rpaths /usr/lib/rpm/check-buildroot
*******************************************************************************
*
* WARNING: 'check-rpaths' detected a broken RPATH and will cause 'rpmbuild'
*          to fail. To ignore these errors, you can set the '$QA_RPATHS'
*          environment variable which is a bitmask allowing the values
*          below. The current value of QA_RPATHS is 0x0000.
*
*    0x0001 ... standard RPATHs (e.g. /usr/lib); such RPATHs are a minor
*               issue but are introducing redundant searchpaths without
*               providing a benefit. They can also cause errors in multilib
*               environments.
*    0x0002 ... invalid RPATHs; these are RPATHs which are neither absolute
*               nor relative filenames and can therefore be a SECURITY risk
*    0x0004 ... insecure RPATHs; these are relative RPATHs which are a
*               SECURITY risk
*    0x0008 ... the special '$ORIGIN' RPATHs are appearing after other
*               RPATHs; this is just a minor issue but usually unwanted
*    0x0010 ... the RPATH is empty; there is no reason for such RPATHs
*               and they cause unneeded work while loading libraries
*    0x0020 ... an RPATH references '..' of an absolute path; this will break
*               the functionality when the path before '..' is a symlink
*          
*
* Examples:
* - to ignore standard and empty RPATHs, execute 'rpmbuild' like
*   $ QA_RPATHS=$[ 0x0001|0x0010 ] rpmbuild my-package.src.rpm
* - to check existing files, set $RPM_BUILD_ROOT and execute check-rpaths like
*   $ RPM_BUILD_ROOT=<top-dir> /usr/lib/rpm/check-rpaths
*  
*******************************************************************************
ERROR   0001: file '/usr/bin/collectd-nagios' contains a standard rpath
'/usr/lib64' in [/usr/lib64]
ERROR   0001: file '/usr/bin/collectdctl' contains a standard rpath
'/usr/lib64' in [/usr/lib64]
ERROR   0001: file '/usr/bin/collectd-tg' contains a standard rpath
'/usr/lib64' in [/usr/lib64]
error: Bad exit status from /var/tmp/rpm-tmp.HwWMnY (%install)


RPM build errors:
    Bad exit status from /var/tmp/rpm-tmp.HwWMnY (%install)

The reason it fails is because collectd hardcodes the /usr/lib64 path into the collectd-nagios, collectdctl and collectd-tg binaries when the Linux dynamic loader will automatically load libraries from this path, given that it is a system default library path. Therefore, hardcoding this particular path in the library is redundant.

There are two ways to get around this. The first is to comment out the /usr/lib/rpm/check-rpaths line from the $HOME/.rpmmacros and then rerun the rpmbuild command. The second option is to tell rpmbuild to ignore standard RPATHs by running rpmbuild as follows:

QA_RPATHS=0x0001 rpmbuild -bb collectd.spec

After you rerun rpmbuild and it finishes successfully, you will see the last line as:

 **+ exit 0**

The built RPMs will be in the $HOME/rpmbuild/RPMS/x86_64 directory:

ls $HOME/rpmbuild/RPMS/x86_64/
collectd-5.5.0-1.el6.x86_64.rpm
collectd-netlink-5.5.0-1.el6.x86_64.rpm
collectd-amqp-5.5.0-1.el6.x86_64.rpm
collectd-nginx-5.5.0-1.el6.x86_64.rpm
collectd-apache-5.5.0-1.el6.x86_64.rpm
collectd-notify_desktop-5.5.0-1.el6.x86_64.rpm
collectd-ascent-5.5.0-1.el6.x86_64.rpm
collectd-notify_email-5.5.0-1.el6.x86_64.rpm
collectd-bind-5.5.0-1.el6.x86_64.rpm
collectd-nut-5.5.0-1.el6.x86_64.rpm
collectd-ceph-5.5.0-1.el6.x86_64.rpm
collectd-openldap-5.5.0-1.el6.x86_64.rpm
collectd-collection3-5.5.0-1.el6.x86_64.rpm
collectd-perl-5.5.0-1.el6.x86_64.rpm
collectd-contrib-5.5.0-1.el6.x86_64.rpm
collectd-php-collection-5.5.0-1.el6.x86_64.rpm
collectd-curl-5.5.0-1.el6.x86_64.rpm
collectd-pinba-5.5.0-1.el6.x86_64.rpm
collectd-curl_json-5.5.0-1.el6.x86_64.rpm
collectd-ping-5.5.0-1.el6.x86_64.rpm
collectd-curl_xml-5.5.0-1.el6.x86_64.rpm
collectd-postgresql-5.5.0-1.el6.x86_64.rpm
collectd-dbi-5.5.0-1.el6.x86_64.rpm
collectd-python-5.5.0-1.el6.x86_64.rpm
collectd-debuginfo-5.5.0-1.el6.x86_64.rpm
collectd-redis-5.5.0-1.el6.x86_64.rpm
collectd-disk-5.5.0-1.el6.x86_64.rpm
collectd-rrdtool-5.5.0-1.el6.x86_64.rpm
collectd-dns-5.5.0-1.el6.x86_64.rpm
collectd-sensors-5.5.0-1.el6.x86_64.rpm
collectd-email-5.5.0-1.el6.x86_64.rpm
collectd-smart-5.5.0-1.el6.x86_64.rpm
collectd-gmond-5.5.0-1.el6.x86_64.rpm
collectd-snmp-5.5.0-1.el6.x86_64.rpm
collectd-hddtemp-5.5.0-1.el6.x86_64.rpm
collectd-utils-5.5.0-1.el6.x86_64.rpm
collectd-ipmi-5.5.0-1.el6.x86_64.rpm
collectd-varnish-5.5.0-1.el6.x86_64.rpm
collectd-iptables-5.5.0-1.el6.x86_64.rpm
collectd-virt-5.5.0-1.el6.x86_64.rpm
collectd-java-5.5.0-1.el6.x86_64.rpm
collectd-write_http-5.5.0-1.el6.x86_64.rpm
collectd-log_logstash-5.5.0-1.el6.x86_64.rpm
collectd-write_redis-5.5.0-1.el6.x86_64.rpm
collectd-lvm-5.5.0-1.el6.x86_64.rpm
collectd-write_riemann-5.5.0-1.el6.x86_64.rpm
collectd-memcachec-5.5.0-1.el6.x86_64.rpm
libcollectdclient-5.5.0-1.el6.x86_64.rpm
collectd-modbus-5.5.0-1.el6.x86_64.rpm
libcollectdclient-devel-5.5.0-1.el6.x86_64.rpm
collectd-mysql-5.5.0-1.el6.x86_64.rpm


Rebuilding Source RPMs (SRPMs)

Building from a src rpm is admittedly easier than from a spec file. A source RPM, or SRPM, is the corresponding source code to a ready-to-install, compiled RPM. It includes a spec file, source files and possibly patch files. It can be compiled for a particular architecture and even modified before compiling into an RPM.

There are several places where you can find SRPMs:

Grab the src rpm for collectd from the EPEL repository. It is an outdated version of collectd but serves the purpose of this guide. Download and save the file somewhere in your home directory:

cd $HOME
mkdir local && cd local
wget http://dl.fedoraproject.org/pub/epel/6/SRPMS/collectd-4.10.9-1.el6.src.rpm

Before using rpmbuild, run yum-builddep on the src rpm to download any missing dependencies.

sudo yum-builddep collectd-4.10.9-1.el6.src.rpm

Now, use rpmbuild to rebuild the src rpm:

QA_RPATHS=0x0001 rpmbuild --rebuild collectd-4.10.9-1.el6.src.rpm

Again, you can choose to either prepend the rpmbuild command with QA_RPATHS or comment out the /usr/lib/rpm/check-rpaths line in your $HOME/.rpmmacros to avoid issues with rpaths during the build.

The built RPMs are again in the $HOME/rpmbuild/RPMS/x86_64/ directory.


Building RPMs using Mock

Mock is a tool that creates chroot environments and builds RPM packages in them. It comes with default configurations, but it is easy to adapt and customize a configuration.

First, download mock:

sudo yum install mock

Mock's configuration files are stored in /etc/mock. Change to that directory and copy the epel-6-x86_64.cfg, since that configuration file includes the repositories for CentOS and EPEL, which is exactly the repos needed for satisfying dependencies for collectd.

cd /etc/mock
sudo cp epel-6-x86_64.cfg centos-6-x86_64.cfg

Now, we can customize the centos-6-x86_64.cfg configuration file. You can make the following changes:

  • change config_opts['root'] from epel-6-x86_64 to centos-6-x86_64 to match the name of our configuration
  • remove unneeded yum repositories, such as testing and local
  • optionally, you could add the config_opts['macros']['%dist'] option and set it to something like .org.el6, which would include the name of your org into the RPM names to help distinguish them from RPMs in the CentOS and EPEL repositories.

The mock utility should be run as non-root. Any user using mock should be a member of the mock group:

sudo usermod -a -G mock makerpm

Next, use the newgrp command to change groups during the current login session:

newgrp mock

Mock needs the spec file to first build the SRPM. Copy the spec file to the SPECS directory:

cp collectd-5.5.0/contrib/redhat/collectd.spec $HOME/rpmbuild/SPECS/

As before, use spectool to download the sources specified in the spec file:

spectool -R -g $HOME/rpmbuild/SPECS/collectd.spec

Now, instruct mock to use our custom centos-6-x86_64 configuration and build a source rpm from the spec file and sources:

mock -r centos-6-x86_64 --buildsrpm \
--spec $HOME/rpmbuild/SPECS/collectd.spec \
--sources $HOME/rpmbuild/SOURCES

By default, all builds are written in the /var/lib/mock directory. After building the SRPM in the previous command, the output shows where the results are written:

INFO: Results and/or logs in: /var/lib/mock/centos-6-x86_64/result

Mock will build the RPMs in a chroot environment and install all the necessary packages. Therefore, there is no need to run yum-builddep. Mock will take care of them.

Lastly, instruct mock to build the collectd RPMs using the source RPM built in the previous step:

mock -r centos-6-x86_64 --no-clean --rebuild /var/lib/mock/centos-6-x86_64/result/collectd-5.5.0-1.el6.src.rpm

Again, the output will show where the built RPMs are written:

INFO: Results and/or logs in: /var/lib/mock/centos-6-x86_64/result
cd /var/lib/mock/centos-6-x86_64/result/
ls *rpm
collectd-netlink-5.5.0-1.el6.x86_64.rpm
collectd-5.5.0-1.el6.src.rpm
collectd-nginx-5.5.0-1.el6.x86_64.rpm
collectd-5.5.0-1.el6.x86_64.rpm
collectd-notify_desktop-5.5.0-1.el6.x86_64.rpm
collectd-amqp-5.5.0-1.el6.x86_64.rpm
collectd-notify_email-5.5.0-1.el6.x86_64.rpm
collectd-apache-5.5.0-1.el6.x86_64.rpm
collectd-nut-5.5.0-1.el6.x86_64.rpm
collectd-ascent-5.5.0-1.el6.x86_64.rpm
collectd-openldap-5.5.0-1.el6.x86_64.rpm
collectd-bind-5.5.0-1.el6.x86_64.rpm
collectd-perl-5.5.0-1.el6.x86_64.rpm
collectd-ceph-5.5.0-1.el6.x86_64.rpm
collectd-php-collection-5.5.0-1.el6.x86_64.rpm
collectd-collection3-5.5.0-1.el6.x86_64.rpm
collectd-pinba-5.5.0-1.el6.x86_64.rpm
collectd-contrib-5.5.0-1.el6.x86_64.rpm
collectd-ping-5.5.0-1.el6.x86_64.rpm
collectd-curl-5.5.0-1.el6.x86_64.rpm
collectd-postgresql-5.5.0-1.el6.x86_64.rpm
collectd-curl_json-5.5.0-1.el6.x86_64.rpm
collectd-python-5.5.0-1.el6.x86_64.rpm
collectd-curl_xml-5.5.0-1.el6.x86_64.rpm
collectd-redis-5.5.0-1.el6.x86_64.rpm
collectd-dbi-5.5.0-1.el6.x86_64.rpm
collectd-rrdtool-5.5.0-1.el6.x86_64.rpm
collectd-debuginfo-5.5.0-1.el6.x86_64.rpm
collectd-sensors-5.5.0-1.el6.x86_64.rpm
collectd-disk-5.5.0-1.el6.x86_64.rpm
collectd-smart-5.5.0-1.el6.x86_64.rpm
collectd-dns-5.5.0-1.el6.x86_64.rpm
collectd-snmp-5.5.0-1.el6.x86_64.rpm
collectd-email-5.5.0-1.el6.x86_64.rpm
collectd-utils-5.5.0-1.el6.x86_64.rpm
collectd-gmond-5.5.0-1.el6.x86_64.rpm
collectd-varnish-5.5.0-1.el6.x86_64.rpm
collectd-hddtemp-5.5.0-1.el6.x86_64.rpm
collectd-virt-5.5.0-1.el6.x86_64.rpm
collectd-ipmi-5.5.0-1.el6.x86_64.rpm
collectd-write_http-5.5.0-1.el6.x86_64.rpm
collectd-iptables-5.5.0-1.el6.x86_64.rpm
collectd-write_redis-5.5.0-1.el6.x86_64.rpm
collectd-java-5.5.0-1.el6.x86_64.rpm
collectd-write_riemann-5.5.0-1.el6.x86_64.rpm
collectd-log_logstash-5.5.0-1.el6.x86_64.rpm
libcollectdclient-5.5.0-1.el6.x86_64.rpm
collectd-lvm-5.5.0-1.el6.x86_64.rpm
libcollectdclient-devel-5.5.0-1.el6.x86_64.rpm
collectd-memcachec-5.5.0-1.el6.x86_64.rpm
collectd-modbus-5.5.0-1.el6.x86_64.rpm
collectd-mysql-5.5.0-1.el6.x86_64.rpm


Conclusion

Mock is a great tool for building RPMs. You get a clean chroot environment in which to build RPMs. Building from a source rpm or a spec file is slightly more involved. In the last part of this guide, Part 4, we will look at signing our RPMs and distributing the GPG key.


Comments

comments powered by Disqus