Chapter 10 The package collection

Table of Contents
10.1 Installing the package collection
10.2 Updating the package collection
10.3 Example: installing a program from source
10.4 Example: installing a binary package
10.5 Package management commands
10.6 Quick Start Packaging Guide

The NetBSD package collection is a collection of tools that greatly simplify the compilation and installation of a huge quantity of free software for Unix systems. Only one or two commands are required to install a perfectly configured and working package.

The first contact with the NetBSD package system can generate a little confusion: apparently there are different commands that do the same thing. The question is actually very simple: here are two ways to install a program. You can

The two aforementioned methods both require that someone else has "created a package", i.e. has ported and configured a program for NetBSD. Although the package collection comprises more that 3500 programs, it is possible that the one that you want is still not included. In this case you can compile it without using the package system and, once you get it working, create a package that can be added to the collection: other users will benefit from your work.

10.1 Installing the package collection

Before installing a program from source, you should download and install the package collection from the NetBSD site or from the mirror of your choice. This is described in the following steps.

  1. Download the latest version of the package system sources, which include all the necessary makefiles and configuration files, from ftp://ftp.NetBSD.org/pub/NetBSD/NetBSD-current/tar_files/. The file to be downloaded is pkgsrc.tar.gz.

  2. Remove the existing collection from your hard disk (if you already have installed it) with the following command:

    # cd /usr
    # rm -rf pkgsrc
    
  3. Install the collection that you have downloaded:

    # tar -xzvpf pkgsrc.tar.gz -C /usr
    

    The execution of the previous command can last many minutes, because of the huge number of (small) files that are extracted. At the end, the framework required for the installation of new programs is ready and you can start installing them.

    Note: by now it is probably clear that the previous commands installed the configuration files required for the automatic installation of programs on your system: you have not yet installed the programs! Basically, the system now has a list of the available packages and the instructions to fetch, compile and install them.



When you have installed the package collection you can browse it with Lynx or Netscape and read the details and the descriptions of all the available packages and package categories. For example:

$ cd /usr/pkgsrc
$ lynx README.html

10.2 Updating the package collection

The package collection is frequently updated: you can find a new version on the ftp site almost weekly. To update the collection on your system follow the same instructions as for first time installation.

Sometimes, when updating the package collection, it will be necessary to update the "pkgtools" utilities too. You can easily spot if you need to do it: when you try to install a program the package system complains that your pkgtools are outdated.

# make
===> Validating dependencies for gqmpeg-0.6.3
Your package tools need to be updated to 2000/02/02 versions.
The installed package tools were last updated on 1999/01/01.
Please make and install the pkgsrc/pkgtools/pkg_install package.
*** Error code 1

The easiest way to update is:

# cd /usr/pkgsrc/pkgtools/pkg_install
# make install

After this you can resume the installation of the package which issued the original error message.

Note: You can determine the required version examing the pkgsrc/mk/bsd.pkg.mk file: look for the line with:

PKGTOOLS_REQD = 20000202

(the 20000202 date is only an example.) This means that the version of the programs that you need is in the pkg_install-20000202.tar.gz file, that you can find on the NetBSD ftp site under packages/distfiles/LOCAL_PORTS. pkg_install can be installed like any other package.

10.3 Example: installing a program from source

This section describes the installation of an example program: the cdrecord application, which simplifies the creation of new accounts on your system. First, cd to the /usr/pkgsrc/sysutils/cdrecord directory.

10.3.1 Downloading the sources

If you are connected to the Internet, the Makefile will automatically fetch the required sources and you don't need to read this section.

Otherwise you should take care of getting the tarballs yourself. In this case you need to know the name(s) of the tarball(s); look in the Makefile for the line

DISTNAME = cdrtools-2.0

The full name of the package is cdrtools-2.0.tar.gz.

You can achieve the same result in an easier way with the following commands:

# cd /usr/pkgsrc/sysutils/cdrecord
# make fetch-list

which also show a list of sites from where the package can be downloaded.

10.3.2 Compiling and installing

To compile the package write

# cd /usr/pkgsrc/sysutils/cdrecord
# make

The previous command fetches the source archive (if it is not already present in the distfiles directory), extracts the sources, applies the patches necessary to compile it on NetBSD and then builds the package.

To install it

# make install

The installation of the new program is recorded on your system: you can check with the pkg_info -a command.

Then cdrecord package is ready for use; you can make some room removing the intermediate files created by the compiler:

# make clean
# make clean-depends

The second command is needed if some dependent packages have been installed. The same result can be achieved with one command:

# make clean CLEANDEPENDS=1

10.4 Example: installing a binary package

I have already explained in the first part of this chapter that the package system can install program from source but can also install binary packages, prepared by someone else for NetBSD. This second form of installation is faster because the compilation of the package is not needed and the tarballs for the binary package are usually smaller and faster to fetch. To install a binary package the package collection is not needed: only the "pkgtools" utilities are needed.

The tarballs for the binary programs usually have the extension .tgz while the source tarballs usually end with .tar.gz.

Note: not all the source tarballs end with .tar.gz. The package system can handle other types of packages, for example .zip, .bz2, etc.

It is not strictly necessary to download binary packages prior to installation: you can also use ftp://-URLs. For example:

ftp://ftp.NetBSD.org/pub/NetBSD/packages/1.4.2/i386/All/tcsh-6.09.00.tgz

If you don't know what version of the package is available on the FTP site you can even leave out the version inormation and pkg_add will pick the latest version on the FTP server. For example:

# pkg_add ftp://ftp.NetBSD.org/pub/NetBSD/packages/1.4.2/i386/All/tcsh

It is also possible to set PKG_PATH to a ;-separated list of path and URLs and then omit that part for pkg_add:

# PKG_PATH="/cdrom;/usr/pkgsrc/packages/All;ftp://ftp.NetBSD.org/pub/NetBSD/packages/1.4.2/i386/All/"
export PKG_PATH
# pkg_add tcsh

The previous command installs the first tcsh binary package that it finds.

As an example, let's install the texinfo program in precompiled form.

  1. Copy gtexinfo-3.12.tgz in a temporary directory.

  2. Give the following command

    # pkg_add -v gtexinfo-3.12.tgz
    
  3. Check that the package has been installed with the command

    # pkg_info
    
  4. Remove the file gtexinfo-3.12.tgz from the temporary directory.

Precompiled packages are very practical to use because they require a minimal effort and time for installation, but source packages give more control because their compilation options can be customized. The installation is somewhat longer, because of the compilation, and this could be critical on some (older) platforms.

Before installing a precompiled package with pkg_add, it is better to examine it with the pkg_info command. For example:

# pkg_info -f jpeg-6b.tgz

It is worth examining the first CWD command, to check where the package is installed (base directory.) The most common base directories are /usr/pkg and /usr/X11R6. If the base directory is not what you want, you can change it with the -p of the pkg_add command. For example, the jpeg-6b.tgz package is installed in /usr/pkg by default, but you can install it in /usr/X11R6 if you extract it with the following command:

# pkg_add -p /usr/X11R6 -v jpeg-6b.tgz

10.5 Package management commands

The most important commands for package management are:

pkg_add

adds precompiled packages.

pkg_delete

removes installed packages. Package names can be given with or without version; if no version is given, pkg_delete will find out which version is installed. Wildcards can be used (but must be escaped for the shell); for example:

# pkg_delete "*emacs*"

The -r option is very powerful: it removes all the packages that require the package in question and then removes the package itself. For example:

# pkg_delete -r jpeg

will remove jpeg and all the packages that used it; this allows upgrading the jpeg package.

pkg_info

shows information on packages, installed and not installed.

pkg_create

creates new packages for the package collection. This program is used to create new precompiled packages. It's called automatically by the build system and there's no need to call it by hand.

pkg_admin

executes various administrative functions on the package system.

10.6 Quick Start Packaging Guide

This section details a quick start method for building relatively small Packages for the NetBSD packaging system. For more details on some of the intricacies on the NetBSD packaging system see pkgsrc documentation.

10.6.1 Tools

There are three primary tools for rapidly building a small package addition to NetBSD:

url2pkg
a template package
pkglint

10.6.1.1 url2pkg

The url2pkg utility can be installed from the pkgsrc tree. This tool helps the package builder quickly fill in and test rudimentry aspects of package building.

10.6.1.2 Template package

A template package example, bison, is provided in the appendices of the pkgsrc documentation.

10.6.1.3 pkglint

The pkglint utility can be installed from the pkgsrc tree. This tool basically checks for package correctness.

10.6.2 Getting Started

Starting the process is fairly simple. It is important that the builder (e.g. you) have already tested building the package from the sources on a NetBSD system. Otherwise setting up a new package could be problematic if if fails to build. It should be noted that most often, a patch can be written for the sources and included in the package to fix any build problems. That is beyond the scope of this quick start guide (see the pkgsrc documentation for details).

10.6.2.1 Using url2pkg

The next step is to use url2pkg.

Following are the steps for using url2pkg to create some of the initial files needed for a new package:

  1. Make the directory under the appropiate pkgsrc directory for the new package. Do not put anything in it yet.

  2. cd into the new directory.

  3. type

    $ url2pkg
    
  4. You will be prompted to enter a url at this point, enter the url and hit <Return>.

  5. A vi session will begin

    Note: this uses the default location of vi on NetBSD, by default it is nvi. If you normally use another vi clone such as vim you may get .exrc errors.

    The vi session is for the Makefile of the new package. You must enter the package name, package maintainer email and the category that the package will fall under.

  6. Save the file and exit.

  7. url2pkg will automatically fetch the package and put it in the work subdirectory.

  8. Next, url2pkg will generate md5 files.

That ends a url2pkg session, please note url2pkg does not fill in any other files except the Makefile. It will generate an empty PLIST file.

10.6.3 Filling in the Rest

Now that the Makefile has been generated, the remaining files must be created. Using your template package, copy over the following files from the template package's pkg subdirectory:

DESCR

A multi-line description of the piece of software. This should include credits as well.

COMMENTS

A one-line description of the piece of software. There is no need to mention the package's name - this will automatically be added by the pkg_* tools when they are invoked.

PLIST

This file contains the location of files to be installed on the system, for a small package (e.g. one binary and one or two man pages) peeking at the distribution's Makefile, install script etc. should easily illustrate where to put these.

10.6.4 Checking with pkglint

With all of the files ready, it is time to check the package with the pkglint tool. Often the Makefile needs a section moved, changed or added, however, for the first time around it is helpful just to run pkglint before hand so you know exactly what you may need to change, following is some sample output taken from the pkgsrc documentation pkglint session:

$ pkglint
OK: checking pkg/COMMENT.
OK: checking pkg/DESCR.
OK: checking Makefile.
OK: checking files/md5.
OK: checking patches/patch-aa.
looks fine.

If an error occured, it is normally pretty straightforward, here is a sample error I got while building a package:

extract suffix not required

I did not need to define an extract suffix in the Makefile.

10.6.5 Running and Checking Build/Installs

At this point if pkglint has passed, I normally run a complete check of the fetch, build and installation. To do this properly I must delete the work subdirectory and the distfile(s) from /usr/pkgsrc/distfiles. This way I can ensure I will be doing a full and complete test.

10.6.6 Submitting a Package Using send-pr

First make an archive of the package tree itself (including the pkg/work subdirectory) like so:

$ tar -czf packagename.tgz package_dir

Next, upload the archive to a location that NetBSD package maintainers can access it from, if you cannot upload the archive, contact NetBSD to see if there is some other method you might try to make your archive available to package maintainers.

The preferred method of notifying NetBSD package maintainers is to use the send-pr utility with a category of "pkg", a synopsis which includes the package name and version number, a short description of the package and the URL of the tar file.

You can use either the send-pr utility on your NetBSD system or the online form at http://www.NetBSD.org/cgi-bin/sendpr.cgi?gndb=netbsd if for some reason you cannot get send-pr to work locally.

10.6.7 Final Notes

Again this little guide is for small packages that contain only a few files to be installed on a NetBSD system. It makes the assumption that the package does not require any patches and can build with no dependancies.

For more advanced issues, be sure to read the full pkgsrc documentation.