Thomas’ Lab Notes

Stuff worth not forgetting

HP Laserjet M1217nfw Setup With CUPS on FreeBSD 10

This is an entry level network-connected multi function printer. It does not have a built-in Postscript interpreter. Instead, it receives raster data through a proprietary network protocol implemented as a closed source binary plugin to the CUPS filtering system.

In addition to CUPS, the following ports must be installed:

  • print/hplip
  • print/hplip-plugin

Once this is done, the printer can be added to CUPS. The standard socket connection options cannot be used. Instead, the “HPLIP” transport must be selected. The printer URI must be set manually from the output of hp-makeuri <IP-address>. (The plugin requires an URI starting with “hp:”, and will reject any other device URI with an error message saying “Error: This module is designed to work with HP Printers only”).

GDM Keyboard Layout Revisited

So, I wanted to upgrade Firefox on my FreeBSD 10 workstation, and this in turn caused some supporting libraries to be upgraded, and this broke all sorts of things again.

Initially gdm just segfaulted. After more manual upgrades, it turned out to work again, except that GDM had lost all localization, and in particular got the wrong keymap for the login screen.

It appears that gdm_lang is no longer honored (despite still being mentioned in /usr/local/etc/rc.d/gdm): you now need to set gdm’s locale in /usr/local/etc/gdm/locale.conf. Also note that unlike other user-editable configuration files, this one is overwritten each time gdm is reinstalled.

Setting Up GRUB/GPT/LVM

Notes on setting up a machine to use GPT partitioning, LVM for all filesystems (including root), and GRUB2 to boot.

Starting with a vanilla Debian 7.8 setup. Here we assume that /dev/sdb is the disk that will ultimately contain the system.

GPT setup

(parted) mklabel gpt
(parted) mkpart primary 2048s 4095s                                       
(parted) set 1 bios_grub on                                               
(parted) name 1 "BIOS Boot Partition"                                     
(parted) mkpart primary 4096s 100%                                        
(parted) set 2 lvm on                                                     
(parted) name 2 "LVM"

Do we want a swap partition there??? If we don’t provision one now, we’ll have to swap to an LVM LV.

LVM setup

pvcreate /dev/sdb1
# Format given disk for LVM

vgcreate tank /dev/sdb2
# Create a volume group with that disk as the underlying storage

lvcreate -n rootfs -L 10G tank
lvcreate -n home -l 100%FREE tank

Filesystem

mkfs.ext4 /dev/mapper/tank-rootfs
mkfs.ext4 /dev/mapper/tank-home
mount -t ext4 /dev/mapper/tank-rootfs /mnt

Set up root filesystem (including /boot subdirectory) in /mnt.

Make sure that /etc/fstab on tank-rootfs points to the proper root fs.

GRUB2

for i in /dev /dev/pts /proc /sys /run; do mount -B $i /mnt$i; done
chroot /mnt
rm -f /boot/grub/device.map
grub-mkconfig -o /boot/grub/grub.cfg
grub-install /dev/sdb

FreeBSD Unicode Symbols Support

The font packages available on a desktop environment with a default FreeBSD installation do not support the Miscellaneous Symbols and Pictographs Unicode range (U+1F300..U+1F5FF), which contains various dingbats and emoji.

A nice vector font providing these symbols is however available from ports: x11-fonts/symbola.

(Note: x11-fonts/gnu-unifont and x11-fonts/gnu-unifont-ttf are not nearly as exhaustive.)

Update 2017-01-23 Similar problem on Debian (CIRCLED LATIN CAPITAL LETTER V (U+24CB), Ⓥ was missing). Resolved by installing fonts-linuxlibertine.

Digikam, Dependencies, and Building KDE Libraries

Context

I have a very basic KDE environment, just enough to be able to run Digikam. Anytime I try to delete a photo, I get an error message:

1
2
Could not start process
Unable to create io-slave: klauncher said: Unknown protocol 'trash'.

Oh, well. So I guess the FreeBSD port for Digikam fails to list a required dependency. Fixing this is a long (ongoing) journey, with lots of interesting adventures all along. This is not a step by step buid, but a series of notes about various traps I fell on the way.

KDE libraries versioning

Executive summary: you cannot build KDE libraries (such as sysutils/kfilemetadata) of a given version if it does not match exactly the installed version of kdelibs:

Build log
1
2
3
4
[...]
===>   Registering installation for kfilemetadata-4.14.3_2 as automatic
pkg-static: Unable to access file /var/ports/work/usr/ports/sysutils/kfilemetadata/work/stage/usr/local/lib/libkfilemetadata.so.4.14.3: No such file or directory
*** Error code 74

kfilemetadata fails to install. That file is indeed missing; the staging area does however contain a libkfilemetadata.so.4.14.2. So why does the source package of 4.14.3 generate a 4.14.2 library?

Answer: the library version is not set by the package itself, it comes from a default value from: o

/usr/local/share/apps/cmake/modules/KDE4Defaults.cmake
1
set(GENERIC_LIB_VERSION "4.14.2")

Where does this come from?

1
2
$ pkg which /usr/local/share/apps/cmake/modules/KDE4Defaults.cmake
/usr/local/share/apps/cmake/modules/KDE4Defaults.cmake was installed by package kdelibs-4.14.2_5

Conclusion: the build dependency for kfilemetadata should list the exact same version of kdelibs, or the port won’t build.

Upgrading kdelibs

Let’s instead upgrade kdelibs from binary package, and hope for the best:

1
# pkg install -f kdelibs

This breaks because the binary package depends on a newer libpng, so let’s upgrade this one, keeping the old shared lib intact just in case.

1
2
$ digikam 
/usr/local/lib/libpng16.so.16: version PNG16_0 required by /usr/local/lib/libkhtml.so.5 not defined

Strange that libpng 1.6.16 does not have version 16… Sigh… OK, upgrading from png-1.6.16 to png-1.6.18 appears to fix the problem. Back on track…

Now Digikam displays its splash screen and starts initializing, then segfaults. Hell, I’ll have to bite the bullet and upgrade a few hundred packages from ports. :-(

VLC ports variants

The vlc port by defaults depends on QT5, whereas the rest of the KDE system depends on QT4. You can rebuild vlc with the QT4 option, but that’s not quite sufficient: actually phonon (part of KDE) depends explicitly on slave port vlc-qt4 (so you can’t just install vlc with QT4 option, you have to go through the separate slave port).

OpenSLP

Digikam does not segault anymore, CUPS is repaired (I had to reinstall it somewhere in the process, as it would silently fail to startup due to a missing symbol) but I still cannot delete photos. On second guess, the missing item might be kde4-runtime, not kde4-workspace.

Here the dead end is quickly reached: x11/kde4-runtime depends on net/openslp, which won’t build because of a security vulnerability… Oh well, let’s build with DISABLE_VULNERABILITIES=yes

Epilogue

At long last, small victory: the missing piece was indeed x11/kde4-runtime. The problem has been reported. I must admit I’m getting sick and tired of the amount of breakage I need to investigate and fix most times I want to install something using the ports system. Desktop work nowadays requires humongous dependency closures that are extremely fragile, and I’m very much tempted these days to switch back to Debian for that.

Subsonic, FreeBSD 10, and UTF-8

In the context of upgrading to FreeBSD 10, I reinstalled the Subsonic media server from ports.

Servlet container

It turns out that using Jetty as the underlying servlet container would not work: I would get an obscure Java exception during various operations:

1
Message  /WEB-INF/jsp/settingsHeader.jsp(12,0) PWC6340: According to the TLD, rtexprvalue is true, and deferred-value is specified for the attribute items of the tag handler org.apache.taglibs.standard.tag.rt.core.ForTokensTag, but the argument for the setter method is not a java.lang.Object

Switching to Tomcat 8 worked.

Changing filesystem charset to UTF-8

I had been using ISO-8859-15 filenames for ever. As part of the OS ugprade, I decided it was more than time to switch the whole system to UTF-8. (One specific issue that prompted this was the fact that GDM now seems to not support ISO 8859-15 GECOS user names anymore).

In order to have Subsonic properly handle file and directory names encoded in UTF-8, I had to set LANG for it:

/etc/rc.conf.d/tomcat8
1
export LANG=fr_FR.UTF-8

and to re-create the database from scratch (remove everything from /var/subsonic/db/ except subsonic.script).

GnuPG 2.1.2 Doesn’t Work With Caff

Today I signed a GnuPG key using my air-gapped master private key, and then tried to send the signature to the key owner from my network-connected workstation using caff. This failed miserably, with caff unable to find a valid signature, and gpg --list-secret-keys missing the (stub) private key.

It turns out that I had inadvertently upgraded GnuPG on this workstation to version 2.1.2, which has a completely revamped secret keys handling: secret key material is now entirely handled by gpg-agent, and the --secret-keyring command line option for gpg (which caff depends on) is now obsolete.

GnuPG 2.1 apparently also chokes on some legacy keys, and the work-around is to reimport the keyring manually.

caff has been fixed to support GnuPG 2.1. However this depends on GnuPG 2.1.3 or newer, which is not in the ports tree yet, so for the time being I have reverted to the “stable” 2.0 release: portmaster -o security/gnupg20 gnupg.

GPhoto2 Fails to Connect to Canon EOS 20D

For the past few months, Digikam (and the underlying GPhoto2 library) failed to connect to my Canon EOS 20D camera:

1
2
3
4
5
6
7
8
9
10
11
12
13
canon/canon/usb.c           (2): Initializing the (USB) camera.
canon/canon/usb.c           (2): canon_usb_camera_init()
canon/canon/usb.c           (2): canon_usb_identify: USB ID match 0x04a9:0x30eb (model name "Canon:EOS 20D (normal mode)")
gp_context_status           (2): Detected a 'Canon:EOS 20D (normal mode)'.
Detected a 'Canon:EOS 20D (normal mode)'.
gp_port_usb_msg_read        (3): Reading message (request=0xc value=0x55 index=0x0 size=1=0x1)...
gp_port_usb_msg_read        (3): Read 0 = 0x0 out of 1 bytes USB message (request=0xc value=0x55 index=0x0 size=1=0x1) (empty hexdump of empty buffer)
gp_context_error            (0): Could not establish initial contact with camera

*** Error ***              
Could not establish initial contact with camera
gp_port_close               (2): Closing port...
gp_context_error            (0): An error occurred in the io-library ('Unknown error'): No error description available

Upgrading various components of my system did not help, so I ended up suspecting a possible issue with the USB stack, which prompted a major OS upgrade from which I am still slowly recovering.

It turned out that the upgrade did not help. Further investigation and debugging finally allowed me to zero in on the cause of the problem, which turned out to be a bug in libgphoto2… which coincidentally got fixed hours before I identified it on my own.

Bizarre Packet Filtering on OVH Kimsufi Server

Context

I am leasing a Kimsufi dedicated server from OVH, ks3269175.kimsufi.com aka 5.39.82.72. Since early January 2015, TCP connections to that machine (and in particular SSH connections) are sporadically hanging.

Analysis of the issue

This machine is on a network whose default router is 5.39.82.254 (vss-gw-6k.fr.eu).

As far as I was able to determine, this is a virtual router, load balanced using GLBP . The two actual routers are:

  • vss-9b-6k.fr.eu, MAC address 00:07:b4:00:01:01
  • vss-9a-6k.fr.eu, MAC address 00:07:b4:00:01:02

When attempting to establish an SSH connection from the outside to that machine, the first data packet in the connection appears to be dropped if sent through 00:07:b4:00:01:01.

This does not appear to be related to any kind of stateful firewalling system. As an experiment, I wrote a simple Scapy script that loops sending identical TCP segments, one per second, through both of the above MAC addresses, to a remote address outside OVH.

A tcpdump on the dedicated server shows the stream of outgoing packets:

1
2
3
4
5
6
23:04:15.390421 00:22:4d:83:36:80 > 00:07:b4:00:01:01, ethertype IPv4 (0x0800), length 115: 5.39.82.72.2122 > 194.98.77.4.60347: Flags [P.], seq 0:49, ack 0, win 8192, options [TS val 123 ecr 456,eol], length 49
23:04:15.406734 00:22:4d:83:36:80 > 00:07:b4:00:01:02, ethertype IPv4 (0x0800), length 115: 5.39.82.72.2222 > 194.98.77.4.60347: Flags [P.], seq 0:49, ack 0, win 8192, options [TS val 123 ecr 456,eol], length 49
23:04:16.424437 00:22:4d:83:36:80 > 00:07:b4:00:01:01, ethertype IPv4 (0x0800), length 115: 5.39.82.72.2122 > 194.98.77.4.60348: Flags [P.], seq 0:49, ack 0, win 8192, options [TS val 123 ecr 456,eol], length 49
23:04:16.441460 00:22:4d:83:36:80 > 00:07:b4:00:01:02, ethertype IPv4 (0x0800), length 115: 5.39.82.72.2222 > 194.98.77.4.60348: Flags [P.], seq 0:49, ack 0, win 8192, options [TS val 123 ecr 456,eol], length 49
23:04:17.459641 00:22:4d:83:36:80 > 00:07:b4:00:01:01, ethertype IPv4 (0x0800), length 115: 5.39.82.72.2122 > 194.98.77.4.60349: Flags [P.], seq 0:49, ack 0, win 8192, options [TS val 123 ecr 456,eol], length 49
23:04:17.476966 00:22:4d:83:36:80 > 00:07:b4:00:01:02, ethertype IPv4 (0x0800), length 115: 5.39.82.72.2222 > 194.98.77.4.60349: Flags [P.], seq 0:49, ack 0, win 8192, options [TS val 123 ecr 456,eol], length 49

Now observing traffic on the remote machine, we see only those packets that went through 00:07:b4:00:01:02:

1
2
3
4
5
6
23:05:13.322004 IP 5.39.82.72.2222 > 194.98.77.4.60403: Flags [P.], ack 0, win 8192, options [TS val 123 ecr 456,eol], length 49
23:05:14.355176 IP 5.39.82.72.2222 > 194.98.77.4.60404: Flags [P.], ack 0, win 8192, options [TS val 123 ecr 456,eol], length 49
23:05:15.390245 IP 5.39.82.72.2222 > 194.98.77.4.60405: Flags [P.], ack 0, win 8192, options [TS val 123 ecr 456,eol], length 49
23:05:16.426968 IP 5.39.82.72.2222 > 194.98.77.4.60406: Flags [P.], ack 0, win 8192, options [TS val 123 ecr 456,eol], length 49
23:05:17.456869 IP 5.39.82.72.2222 > 194.98.77.4.60407: Flags [P.], ack 0, win 8192, options [TS val 123 ecr 456,eol], length 49
23:05:18.494918 IP 5.39.82.72.2222 > 194.98.77.4.60408: Flags [P.], ack 0, win 8192, options [TS val 123 ecr 456,eol], length 49

Resolution

None so far. OVH has been notified of the problem (TICKET#2015010719008317) and all analysis elements in my possession have been conveyed to them, to no avail so far: the machine has been essentially unusable for the past two months and counting.

Update 2015-03-20 OVH say they identified a problem and are working on a fix. No more SSH failure observed since 2015-03-18 in the afternoon, so apparently the fix did work. Still waiting for a post-mortem explanation as to what went wrong, and why it took them so long to ackonwledge, investigate, and resolve the problem.

Update 2015-04-01 Service remains stable, in that failures are not observed anymore. OVH indicates they are still discussing the underlying issue with Cisco, and the fix is not completed yet.

Update 2015-06-17 Service remains stable. OVH indicates that they have identified the origin of the issue, a fix is available, and they have scheduled its deployment.

Update 2015-09-17 At long last, OVH confirmed that the problem is indeed resolved on their side, and agreed to extend my subscription by 3 months at no cost in compensation, which is decent of them.

Acknowledgements

Thanks to Pierre Beyssac for hinting at GLBP, and to Fabian at OVH support for following up internally on this issue.

Upgrading to FreeBSD 10.1-RELEASE

I endeavoured to upgrade my old FreeBSD 9.3-RELEASE system to brand new 10.1. As expected, this was a quite bumpy process, and below are a few things I had to find out the hard way.

Ports

Neither pkg update nor portupgrade can update all ports. I had to dump all origins, remove all ports, then reinstall everything manually.

Linuxulator

New CentOS-6 based linux_base requires sysctl:

1
compat.linux.osrelease=2.6.18

X11

New Xorg

As of Apr. 16, 2014, the X server has been upgraded to a new release. From https://wiki.freebsd.org/Graphics/WITH_NEW_XORG:

linenos:false
1
2
3
4
5
6
7
Note that there's a know regression with syscons and kernel video
drivers: you can't switch back to a console once an X.Org session is
started. A new console driver called vt(4) fixes this issue while
bringing nice features. It's available in FreeBSD 9.3-RELEASE and
10.1-RELEASE but isn't enabled by default. To enable it, put the
following line in your /boot/loader.conf:
    kern.vty=vt

It is a real shame that users essentially have no choice but switching from the default syscons to the “new” (unfinished, far from functionally complete) vt console driver.

The X mouse cursor occasionnally disappears for some unidentified reason. Alt-Tab brings it back.

DRI

Both GDM and the GNOME desktop now require DRI access. At least for ATI video cards, this means that user gdm, as well as anyone logging in to a GNOME session, must have access to /dev/dri/card0:

/etc/devfs.rules linenos:false
1
add path 'dri/card0 mode 0666

GDM

Gdm won’t work out of the box (black screen): gdm_lang cannot be set to a non-UTF-8 locale anymore (if the month name in the current date contains an accent, the greeter will abort). Time to bite the UTF-8 bullet, then.

Oh, and I can’t just remove the variable altogether, see below.

Keymap

Interesting issue for GNOME users. I found out that the GDM login screen would always revert to US layout, no matter what. Initially I thought the X server had an incorrect keymap due to HAL device enumeration, so I added the following to my setup:

/usr/local/share/hal/fdi/policy/99local/10-x11-keymap.fdi
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
<?xml version="1.0" encoding="ISO-8859-1"?>

<deviceinfo version="0.2">
  <device>
    <match key="info.capabilities" contains="input.keyboard">
      <merge key="input.x11_options.XkbLayout" type="string">fr</merge>
      <merge key="input.x11_options.XkbOptions" type="string">terminate:ctrl_alt_bksp,compose:rctrl</merge>
      <merge key="input.xkb.layout" type="string">fr</merge>
      <merge key="input.xkb.options" type="string">terminate:ctrl_alt_bksp,compose:rctrl</merge>
    </match>
  </device>
</deviceinfo>

<!-- Legacy X11 options:
  Option  "XkbRules"    "xorg"
  Option  "XkbModel"    "pc105"
  Option  "XkbLayout"   "fr"
  Option  "XkbOptions"  "terminate:ctrl_alt_bksp" 
  Option  "XkbOptions"  "compose:rctrl" 
  -->

However this happened to be a total red herring, as by default the port configures Xorg to use devd, not HAL. For devd, I found out this is achieved using xorg.conf options:

/usr/local/share/hal/fdi/policy/99local/10-x11-keymap.fdi
1
2
3
4
5
6
Section "InputClass"
        Identifier "Keyboard Defauls"
        Driver "keyboard"
        MatchIsKeyboard "on"
        Option "XkbLayout" "fr"
EndSection

But all of this was mostly irrelevant for my setup since I add AutoAddDevices turned off in the X server setup, and the correct layout was hardcoded in xorg.conf. And indeed, starting it with startx yields the expected French layout.

However, it appears that gnome-shell considers that whatever keymap is configured in the X server probably must be unsuitable, and changes it on its own to a better default based on the current locale (or “us” if no locale is set for gdm).

Printing

I am using an HP MFP1217nfw network printer, which requires the proprietary print/hplip and print/hplip-plugin packages. These install print/cups as a dependency. print/cups-filters is not installed as a dependency, but is required anyway, or all print operations will fail with:

/usr/local/share/hal/fdi/policy/99local/10-x11-keymap.fdi
1
D [02/Mar/2015:22:24:01 +0100] Print-Job client-error-document-format-not-supported: Unsupported format "application/pdf".