Wednesday, November 29, 2006

Binpatch Updates Coming Soon

OpenBSDI know, I know, I know... I've fallen behind with posting binpatch updates.

Previously, I had been maintaining several binpatch build environments to support multiple kernels (GENERIC, GENERIC.MP, GENERIC with RAIDFrame support, GENERIC.MP with RAIDFrame support, GENERIC.MP with pcibios disabled and GENERIC.MP with RAIDFrame support with pcibios disabled) and sendmail with SASL support.

This was becoming too much to handle. Plus, I wanted to add some features. Here's a (probably incomplete) list of enhancements coming soon (some changes were already in my binpatch files):

  • Create a /var/db/binpatch directory to track which binpatches have been installed. Each file in that directory will contain a list of files modified by the patch.

  • Add the ability to include a LICENSE file for the binpatch tarballs in /var/db/binpatch/

  • Compile multiple kernels by setting "KERNEL=GENERIC GENERIC.MP CUSTOM_RAID ..." in Makefile. Each kernel will be compiled. In the example given, the following kernel files would be created: bsd,, bsd.GENERIC_RAID.

  • Ability to sign binpatch tarballs using gzsig (suggested by Gerardo Santana, creator of Binpatch).

  • Ability to ALSO compile a SASL-ized version of sendmail.

Right now, everything is done except the SASL version of sendmail. I'm still working this part out, but my vision is that when setting the patch information in the Makefile, you'll add a "_sasl" build dependency for sendmail patches like this:

001_sendmail 003_sendmail2 005_sendmail3: _sasl
          cd ${WRKSRC}/gnu/usr.sbin/sendmail && \
          (${_obj}; ${_depend}; ${_build})

The result of this will be:

  • Existence of cyrus-sasl2 will be checked.

  • If a work-sasl (or something) directory does not exist, sendmail specific sources and /usr/lib/include are extracted into the work-sasl directory.

  • If a "base" SASL-ized sendmail has not been built, an unpatched SASL sendmail binpatch will be created.

  • The patch specified in PATCH will be applied to the work-sasl source and sendmail will be built with WANT_SMTPAUTH=1.

  • Then, the main work source directory patched and an un-SASL-ized version of sendmail will be built.

Hopefully this will be done sometime next week.


  1. Hi Mike,

    I appreciate your work on binpatch. I've just discovered how useful it can be. Since I've just began to work with binpatch, maybe you could send me a copy of your Makefile for OpenBSD 4.0 so I can compare it to the one I've just built.

    This is what I've put in it:

    cd ${WRKSRC}/usr.sbin/httpd && \
    (${_obj_wrp}; ${_cleandir_wrp}; ${_depend_wrp}; \

    cd ${WRKSRC}/lib/libssl && \
    (${_obj}; ${_depend}; ${_includes}; ${_build})

    003_systrace 004_arc: _kernel

    cd ${WRKSRC}/libexec/ && \
    (${_obj}; ${_cleandir}; ${_depend}; ${_build})

    I created a new shortcut for "make includes" like the others.

    I have a few questions you should be able to answer:

    If there is more than one patch that concerns the kernel, do I have to apply them in any order? Or is it enough to just apply the last one published and all the other ones before it will be applied as well? I guess not, but maybe you can explain a little bit.

    My build host does not have Internet access (it's a quemu machine). Can I just put the patch files into some directory so binpatch won't try to download them?

    Why does the shortcut ${_build} include the shortcut ${_install}? And why did you use both "build" and "install" shortcut in your Makefile like this:

    cd ${WRKSRC}/usr.sbin/httpd && \
    (${_obj_wrp}; ${_cleandir_wrp}; ${_depend_wrp}; \
    ${_build_wrp}; ${_install_wrp})

    You've got another entry in your Makefile like this:

    _authpf: .USE
    cd ${WRKSRC}/usr.sbin/authpf && \
    (${_obj}; ${_build})

    What is it for, I couldn't understand how it's connected to the actual patches.

    I'm taking about this Makefile of yours:

    thanks for your information,

    Tobias W.

  2. Hi Mike,

    I guess I have solved most of the issues by playing a little bit with it and reading the Makefile.

    A manpage would be a nice addition anyway ;-).


  3. Tobias,

    Here's the 4.0 Makefile I've used:

    A couple of things:
    1. I didn't create binpatch. I'm only trying to tailor it to my needs. The official site for binpatch is
    2. You're right. $_build does include $_install. I don't know why. My use of both was redundant.
    3. There was a patch (I think 3.8) where you patch the kernel and then rebuild authpf. I used make(1)'s .USE feature to make an authpf macro. It allowed me to specify "_kernel _authpf" to massage the order to build the way I wanted.
    4. Download the patches manually to ${BINPATCH_HOME}/patches and either common or the ${ARCH} you're using.

  4. Hi Mike,

    thanks for your reply.

    1.) I know ;-) But you seem to be the only one who's actually using binpatch and writing about it, when I look stuff up with Google. :-)

    3.) OK, that explains.

    4.) Actually, I didn't get that my QEMU installation can access the Internet without making any changes to it or the way I run QEMU, so it's OK.

    I noticed some other stuff about binpatch you might know the answer to.

    The binpatch makefile only gets the set tarballs for a non-X system. If there's a patch to the sources in XF4.tar.gz this isn't covered by binpatch. Any suggestions? As soon as I've got some extra time on my hands I'll try to add this but maybe you already have something up your sleeves?

    When I run make 'make PATCH="001" plist' a list of the changed files is created. When I later run make 'PATCH="002" plist' after building 002, will the new list include the changed files from 001 or will it have created a fresh list? I don't have to delete the plist entries in between patches, do I?

    My binary patch for 002 on i386 is 18MB big. I guess this is because the build did a "make includes", correct?

    If I run md5 on my binary patches, I ought to get the same checksums you get, correct? Can we compare as soon as I've built 001 to 005?

    thanks a bunch ;-)
    Tobias W.

  5. Tobias

    Currently, binpatch does not support building X patches. I've been thinking about adding that functionality. It's low on my priority list as I don't run X on any of my boxes. I'd imagine that after I finish the code to build SASL-ized sendmail, porting that code to also build X shouldn't be too hard.

    The plist files are created in this way:
    When the build starts, a file is created (a cookie). When the build completes, another file is created. "make plist" finds files in the "fake" directory that are newer than the first cookie but not newer than the second cookie.
    So, only those files that have changed while compiling for 002 will be in the plist.

    My binpach for 002 is also 18 MB. A bunch of man pages were compiled and installed though I doubt they changed. But they might have. You _could_ go in and delete the man page lines in the plist file before running "make package" and see if it's smaller.

    I think there's a chance that our checksums would match. Here are mine:
    MD5 (binpatch-4.0-i386-001.tgz) = 8f116c405162b475f1dd1aa6515c6c55
    MD5 (binpatch-4.0-i386-002.tgz) = 811aee8b42eb2bf83187846a33d5169c
    MD5 (binpatch-4.0-i386-003.tgz) = aadb8000bb421cfc14b676067c581eb6
    MD5 (binpatch-4.0-i386-004.tgz) = 852547ae71ee134f32c8fae493730986
    MD5 (binpatch-4.0-i386-005.tgz) = c94f35b173fd90312a8464d01021129f

    Note that kernel based patches have site specific strings in them (build time/date):
    OpenBSD 4.0 (GENERIC) #1107: Sat Sep 16 19:15:58 MDT 2006
    OpenBSD 4.0 (GENERIC) #1: Thu Nov 16 18:34:30 EST 2006

  6. Hi Mike,

    The checksums do not match at all:

    MD5 (binpatch-4.0-i386-001.tgz) = 3349061bed72417c82eb81ad8cda23d9

    Weird, isn't it?

    binpatch-4.0-i386-001.tgz is 224073 Bytes large.

    I guess there must be things that are not deterministic when building binary patches. Any ideas?


  7. It's not surprising at all that the checksums don't match, actually. Now that I think about it, I already do some of the things I mentioned in the post above:
    * Create a /var/db/binpatch directory to track which binpatches have been installed
    * Compile multiple kernels by setting “KERNEL=GENERIC GENERIC.MP" in Makefile

  8. Hey Mike,

    As you're working on binpatch, here's a small patch that I sent to Gerardo recently so that binpatch works correctly on architectures that have different kernel and application architecture names like on a Mac, you have macppc for kernel and powerpc for application. I also added make includes:

    --- Sun Nov 12 15:10:43 2006
    +++ Sun Nov 12 15:16:51 2006
    @@ -27,7 +27,7 @@

    # ================================================================ VARIABLES
    # Architecture we are building on/for

    # You'll need to copy your kernel configuration file into
    # ${WRKDIR}/sys/arch/${ARCH}/conf/ if you want to compile it.
    @@ -85,6 +85,7 @@
    _obj=${MAKE_ENV} make obj
    _cleandir=${MAKE_ENV} make cleandir
    _depend=${MAKE_ENV} make depend
    +_includes=${MAKE_ENV} make includes
    _build=${MAKE_ENV} make && ${_install}
    _install=${MAKE_ENV} make install

    Now I'm running OpenBSD on my iBook I can build binpatches for macppc too. Just a shame my internet is down as they're all sitting on my server atm :(

    Cheers z0mbix

  9. and...I'm sure you have something like this already, but just incase you don't, here are patch_add and patch_info:


    # This is just a wrapper script that installs binpatch patches and logs the
    # date and time and a list of what files were installed. See the binpatch
    # website for more information on binary patching your OpenBSD system:
    # Author: z0mbix (zombie | at | zombix | dot | org)
    # Date: 19/01/06

    RELEASE=`uname -r`
    DATE=`date '+%b %d %T'`

    if [ $# != 1 ]; then
    echo "usage: patch_add 001"
    exit 1

    if [ -f $PATCHFILE ]; then
    if [ ! -d $PATCHDBDIR/$PATCHNO ]; then
    if grep "^./bsd$" $PATCHDBDIR/$PATCHNO/${PATCHNO}_files > /dev/null; then
    echo "Kernel Update!"
    echo " Backing up /bsd to /bsd.old"
    cp /bsd /bsd.old
    tar xzpvf "$PATCHFILE" -C /
    echo "$RELEASE-$ARCH-$PATCHNO installed: $DATE" | tee -a $DBFILE
    echo "Directory $PATCHDBDIR/$PATCHNO already exists!"
    echo "If it's not required just delete it and re-run patch_add"


    # This is just a wrapper script that lists installed binpatch patches
    # See the binpatch website for more information on binary patching your OpenBSD system:
    # Author: z0mbix (zombie | at | zombix | dot | org)
    # Date: 30/08/06

    RELEASE=`uname -r`
    DATE=`date '+%b %d %T'`

    usage() {
    echo "usage: patch_info"
    echo " patch_info 001"
    exit 1

    getPatchInfo() {
    if [ -d $PATCHDBDIR/$PATCHNO ]; then
    INSTALLDATE=`grep $RELEASE-$ARCH-$PATCHNO $DBFILE | cut -f3-5 -d' '`
    if [ -f $PLIST ]; then
    echo "Installed file(s) for patch $PATCHNO ($INSTALLDATE):"
    sed s/^\.//g $PLIST
    echo "Patch $PATCHNO not installed!"

    # Check arguments
    if [ $# -gt 2 ]; then

    if [ $# -eq 0 ]; then
    grep "^$RELEASE-$ARCH" $DBFILE
    elif [ $# -eq 1 ]; then

  10. Looks like somebody already some binpatch enhancements :
    Maybe this could help you to speed up your works.