RFID reader for IETF meeting

I attend the IETF meetings whenever I can and I follow the proceedings remotely when I am not able to travel. But following an IETF session remotely is not simple: You need one application to listen to the audio stream (I use XMMS), another application to connect to the Jabber room (Pidgin) and a PDF or Powerpoint viewer to look at the slides (Evince and OpenOffice.org). On top of this complex setup, there is a lot of annoying problems: The audio stream is not very good, the audio servers crash from time to time, and people forget to speak on the microphone; very often the slides are available few minutes before the beginning of a session - it even happened that the slides are not available at all. The Jabber room is in fact used to state the name of the person talking in the microphone, because people forget very often to do this. Another usage for the Jabber room is to state the current slide on the screen.

Obviously all of this does not make for a good experience. There was multiple attempts to improve this, and I did work at a time on one of this attempts. The project was canceled after I left my job at 8x8, but I still continued to work on it on my spare time. One of the first component I designed was to capture the name of an attendee speaking at a microphone. There was some similar experiences, one during IETF 74 by Columbia University, and another during IETF 76 by the host, but my goal was a little bit different: I wanted the system to protect privacy and to be as non-intrusive as possible, so people had just to stand in from of the microphone to have their RFID tag scanned. The following picture shows how the system works:


The RFID antenna needed to be big enough so the speaker just stand naturally in front of the microphone. The RW310 has an 11" range but unfortunately only has a TTL interface which cannot be directly connected to most PC so I used a TTL to USB converter, with the whole circuit fitting inside the USB connector. The box itself was made from plexiglass - I used VariCAD to design the box and an Epilog laser cutter at Techshop to cut the plexiglass.

Finding a way to attach the RFID reader to the microphone stand was the most challenging part - I had to go through 3 prototypes until I found something that permit to change the distance between the reader and the stand and that does not unbalance the whole thing. I finally found a microphone holder that was perfect for this, with a microphone surface-mount attached on the box so the RFID reader can be easily removed at the end of the day.

Privacy is very important for the IETF meeting attendees, so there is some concerns with the introduction of RFID tags - and in my opinion, rightly so. Having an RFID tag without a way to shielding it is unwise, so I also designed a badge holder that would permit to shield the RFID tag also in a non-intrusive way, as shown in the following picture:


The idea is simple: Either you both show your name and give access to your RFID tag or you hide both. I think that this idea is key here - if you accept that people can read your name on your badge then there is no reason why you would not let an RFID reader do the same thing on your tag. The top half of the badge holder contains a metallic shield, but the tag itself in in the bottom half. When the holder is open your name is readable and your tag is not shielded; when closed both your name and your tag are invisible to the full electromagnetic spectrum.

Having the shield only on one side works because this RFID tag works at 13.56MHz (with 125 Khz tags, two layers of aluminum need to enclose the tag and this on both sides). Another reason for the 13.56Mhz tag is that it permit to store the user information directly in the tag, instead of simply retrieving a serial number and match it with a central database which, in my book, is a no-no as I want my personal data to be destroyed at the same time I destroy by badge.

Unfortunately the software is not ready at this time, but I'll try to find some time to finish it if there is enough interest in this project.

Jarc: Junit integration

Unit testing is, in my opinion, one of the most important tool that a developer wants to use to write software. I am not fond of all the techniques proposed by the eXtreme Programming (to say the least) but having popularized unit testing is one of the things that the Agile methodologies should be credited for. Instead of having various tests scattered on multiple systems and prone to bit rotting, Junit permits to package all this tests together and to run them each time a product is built. I generally integrate the unit tests in the build, so a final package cannot be built without passing all the tests. The proprietary version of jarc had a switch to build and run the unit tests in one command line.

The version 0.2.5 of jarc continues in this spirit, but instead of having to build first a jar file and then compile and run the unit tests, it automatically build and run unit tests if they are available. Here's an example of manifest file with unit tests:

Manifest-Version: 1.0
Main-Class: org.implementers.apps.turnuri.Main
X-Jarc-Target: 1.6

Name: org/implementers/apps/turnuri/Main.class

Name: org/implementers/apps/turnuri/TurnUriTest.class
X-Jarc-Test: true

With this manifest file, jarc will build a jar file with Main as root of the dependencies, then build a second, internal, jar file with TurnUriTest as root of the dependencies and use it as Junit test. The X-Jarc-Test attribute can also take the value "false" to deactivate a test temporarily.

Note that the jar file is written on the disk even if the unit tests fail.

RFC2629 tools

Now that I have a java build that is redistributable, I can finally release the code source of the two tools I wrote to convert I-Ds and RFCs to a format that can be used on a Kindle. The Debian package containing this two tools is named "rfc2629-tools" and can be found at the usual place. In addition to this two tools, I also put in this package an additional tool that I use to process my RFC2629 XML sources.

Some quick explanation for the people new to the IETF process. All IETF documents, standards and others, are published as RFC (which is no longer an acronym) which are immutable documents. Before been published as RFC, documents are published as Internet-Drafts (I-D). The canonical form of RFCs and I-Ds is a line printer-ready format that is often called text format. There is other formats possible like PDF or HTML, but only the text format must be used as reference. Writing a well-formed text I-D or RFC is difficult, so authors generally write it in another format, then use a conversion tool to generate the text, PDF or HTML file. Some people use Winword, other use nroff but the most interesting format is an XML application defined in RFC 2629. There is available tools to convert an RFC 2629 source to text, PDF, nroff and HTML, but nothing for the Kindle platform.

One of the tools in the rfc2629-tools package takes as input a RFC 2629 source and generate an intermediary file that can then be processed to create a .mobi file that can be loaded in a Kindle. Here's the command lines to do this:

$ xml2mobi draft-ietf-behave-turn-uri-09.xml
$ kindlegen draft-ietf-behave-turn-uri-09.htm

Note that the xml2mobi program is very much a work in progress, as it is incomplete and really need more work to be able to process with any RFC2629 source. I released the code source because I do not have much time currently for this, so anybody interested can work on a fork. For example, the file generated is named from the content of the docName attribute in the rfc element in the input file.
The kindlegen program can be found on Amazon Web site.

The IETF Secretariat archives the RFC 2629 source if the author provides it when uploading the text file but few people do this so another solution is needed to be able to read I-Ds and RFCs that are only available in text form. This is what the text2kindle program do:

$ txt2kindle draft-ietf-behave-turn-uri-09.txt

This command creates a file named draft-ietf-behave-turn-uri-09.zip that can be installed directly in a directory named /pictures on the Kindle 2. On the Kindle 1, the file must be unzipped in the /pictures directory.
Note that the document is displayed as a series of pictures so you cannot annotate them.

Even if the RFC2629 source of an I-D is available on the IETF web site, that does not mean that you will be able to generate a mobi file for your Kindle that is identical in content to the canonical version. The first reason is that the full date is generally not set, which means that running a conversion tool on this source will generate a document with the current date instead of the date that is used in the canonical document. The second reason is that people generally uses the <?rfc include> statement to insert references. The problem with this is that the text included will change when a new version of the I-D referenced will released, so again it is not possible to generate a content identical to the canonical document. The solution to this problem is to generate an RFC2629 document that always contains a complete date and where all references are already included. With this what the third tool in the package provides:

$ rfc2629 draft-ietf-behave-turn-uri.xml

This command will copy the content of the file passed as input with the following transformations:

• The destination file is named from the content of the docuName attribute (this helps when the document is stored in a source control system).


• The date in the destination file is filled with the current date.


• All the xinclude statements are replaced by their content


• All comments are removed.

The resulting XML file can be uploaded to the IETF site with a guarantee that other formats can be generated from this file as identical as the files generated by the author.

The same script also implements a feature that permit to rename a reference. A reference to an I-D generally looks like this is the result text:

The TURN specification [I-D.ietf-behave-turn] defines a process for a

The map element can be used to change the name displayed:

<map anchor="TURN">
<xi:include xi:href="http://xml.resource.org/public/rfc/bibxml3/reference.I-D.ietf-behave-turn.xml" />
</map>

Which, after been processed by the script and xml2rfc will generate this instead:

The TURN specification [TURN] defines a process for a TURN client to

The last feature of this script (for now) was added when I tried to release two I-Ds at the same time, with each I-D containing a reference to the other. Because the text that will be included is generated after the upload to the IETF, there was no way to automatically use the correct reference. What the script do is to generate a reference file automatically in /tmp for all the files processed. It is then easy to include the correctly generated reference is the document. For example draft-ietf-behave-turn-uri.xml and draft-petithuguenin-behave-turn-uri-bis.xml have a reference on each other. So the first step is to change the reference inside draft-ietf-behave-turn-uri.xml like this:

<xi:include xi:href="/tmp/reference.I-D.petithuguenin-behave-turn-uri-bis.xml" />

and to change the reference inside draft-petithuguenin-behave-turn-uri-bis.xml like this:

<xi:include xi:href="/tmp/reference.I-D.ietf-behave-turn-uri.xml" />

Then running the script on the two XML files at the same time generates the correct XML file, ready to be uploaded to the IETF:

$ rfc2629 ../src/share/docs/draft-ietf-behave-turn-uri.xml ../src/share/docs/draft-petithuguenin-behave-turn-uri-bis.xml


Jarc: Cross-compilation

The new version of jarc now supports cross-compilation, i.e. the possibility to build jar files that works on a version of the JRE lower than the the version of the JRE used to run the jarc program. The jarc program itself will always run under the most recent JRE and by default will build a jar file targeted to the same version. But adding an X-Jarc-Target option will request jarc to build for a lower version. For example this manifest file means that the resulting jar file will work with a 1.5 JRE:

Manifest-Version: 1.0
X-Jarc-Target: 1.5

Without this line, the jar file would have been created for the 1.6 JRE so executing it on a 1.5 JRE would fail.

It is also possible to request a specific JRE vendor:

Manifest-Version: 1.0
X-Jarc-Target: 1.6 openjdk

Note that there is no need to install the JDK for all versions of the JRE. Only the JDK for the latest version (i.e. the one that is used to run jarc) is needed. But at least one installed JRE is needed for each potential X-Jarc-Target value, because the rt.jar file is needed for the cross-compilation.

It is even possible to configure jarc to use the JDK 1.7 by modifying the configuration in /etc/jarc.conf (see comments at the end of the file). The jarc package itself is built by jarc running on 1.7 but targeted to 1.6.

The current version of jarc (0.2.2) can build for the JDKs 1.6 and 1.5 and supports sun and openjdk JVMs. A future version will also support the JVMs generated by java-package and perhaps the gcj JVM.

Fireproof backups

After the last three earthquakes in the Bay Area, I started thinking that developing code without backup in an house made entirely of wood was probably not the best idea in the world. I work from home since a long time but until last year I always committed my code in the central repository of my employer, so it was not really an issue. I use RAID10 for my development computer so losing a disk is not a problem, but I would still loose all the code I wrote during the last year if the house burns down.

I basically never delete anything - when my disks are full I just buy bigger disks and copy everything on them. The consequence is that I have currently 572Gb to backup, so that excludes doing remote backup - I used rsync.net in the past and they have a terrific service, but uploading 300Gb of compressed data is not an option here - and I do not want to save only a subset of the disks.

Doing a backup on an external disk was the only solution remaining, but the problem is still that if the house burn then the backup disk will also burn. I could have rented a safe box at my bank but having to exchange the disks each week was too much of a burden. The solution I choose was to put the backup disk in a fireproof safe installed in the house. Now the next problem is that the house can burn when I am doing the backup - which will be done each night, so that's more or less eight hours each day with an unprotected backup. One neat solution I found was a disk that was directly encased in a fireproof safe. The big advantage of this disk is that it is protected even during the backup. The problem is that I also wanted to be able to store my passport and other documents in it. So I found the perfect solution: A USB fireproof safe. I can now put a 1TB disk inside the safe, and use it without having to open and close the safe.

The next step was the backup itself. I wanted to use good old dump/restore but with only one partition containing everything I was not able to unmount it to do the backup and doing a backup on a live partition - even during the night - is not a good idea. And anyway with a full backup taking ten hours to complete that was not an option. The best solution I found was to use a LVM snapshot - you just create a snapshot of the partition to backup, then you backup the snapshot and can continue to use the main partition. Unfortunately I did not had LVM installed so I had to copy the whole partition on an external disk, change the partitions to support LVM and then copy back the whole external disk - it took twenty-four hours to do this, but it was worth it.

The last step was to install dump and configure it to do a full backup each Sunday and an incremental backup each other night. I used some of the scripts delivered in the dump package, with some modifications to adjust them to my needs.

Now I receive an email each morning with the result of the backup. The last thing remaining to do will be to try a full restore.

Jarc: a jar file compiler

Using the "make" tool to build a jar file was never a good option, mostly because Java files are different from C/C++ files. Today most people use Ant but I never liked it so around 2002 I started writing a tool that would permit to build a jar file in one step. I had two requirements in mind: First the tool had to be faster than running javac and jar - at the time the JVM was really slow, so I used zip and jikes, at least until IBM stopped updating jikes.

The second requirement was to compile only the Java files that were necessary to build the jar file, the idea been to give to the jar tool only the name of the "root" Java classes, and let it figure out the dependencies (I am not an Ant specialist but it seems to me that Ant is still not capable of doing this correctly). What was missing was a convenient way of passing the list of "root" classes to the tools, and this is when I thought of using the manifest file for this.

A manifest file is a text file stored inside the META-INF directory of the jar file. This file contains the meta-information about the jar file and is generated by the jar tool. The idea was to invert the process - instead of having the jar tool generating this manifest file from the command line parameters, why not use this file as the source of the information needed to build a jar file?

The jarc tool (jar compiler) takes as input one or more manifest files, parses the content and tries to build a jar file of same name. The tool will use the class names found in the manifest file as the "root" classes to use as dependencies. A manifest file looks like this:

Manifest-Version: 1.0
Class-Path: /usr/share/java/jspeex.jar
Main-Class: org.implementers.apps.Main

Name: org/implementers/apps/Main.class

Name: org/implementers/apps/resources/schema.xsd

If this text is stored in a file named apps.mf, then running "jarc apps" will build a jar file named apps.jar by executing the following steps:

• The org.implementers.apps.Main class will be compiled together with all the classes that depends on it. The content of the Class-Path attribute is passed to the compiler as if a -classpath option was used.


• The org/implementers/apps/resources/schema.xsd file will be copied from the disk to the jar file.


• The manifest file itself will be filtered and copied inside the jar file.
  
• A short bash program will be generated to call the class described in the Main-Class attribute

There is also some attributes that can be added in the manifest to drive the build. One example is the X-Jarc-Service that can be added to an entry to declare it as a service as shown in the following example:

Name: org/implementers/nio/channels/spi/CompositeProvider.class
X-Jarc-Service: java.nio.channels.spi.SelectorProvider

When jarc reads this line, it generates in the jar file a service file in META-INF/services.

The tools is accessible on the following Debian repository (see a previous post on this blog for the explanation on how to configure Ubuntu or Debian):

deb http://debian.implementers.org/ testing/$(ARCH)/
deb http://debian.implementers.org/ testing/all/
deb-src http://debian.implementers.org/ testing/source/

The tools is contained in the "jarc" package and is distributed under the GPL v3 license.

I wrote the previous version of jarc when I was working for my former employer, so to be sure to not infringe on their rights, I rewrote the tool from scratch in Java (the original version was a bash script). Actually the new version implements only the minimum necessary to build itself, so expect updates in the future.

Update 01/31/2010:
jarc was put in the unstable repository by mistake. The new versions (starting with 0.2.3) will be in testing. The best is to have both testing and stable in your /etc/apt/sources.list file and remove unstable completely:

deb http://debian.implementers.org/ testing/$(ARCH)/
deb http://debian.implementers.org/ testing/all/
deb-src http://debian.implementers.org/ testing/source/
deb http://debian.implementers.org/ stable/$(ARCH)/
deb http://debian.implementers.org/ stable/all/
deb-src http://debian.implementers.org/ stable/source/


Debian package for shadok

Shadok is a tool written by Stéphane Bortzmeyer to verify and create graphs for state machines written in a very simple language named Cosmogol. Everything is explained at the following URL:

http://www.cosmogol.fr/shadok.html

The language was proposed some time ago to the IETF as a way to describe a state machine inside a text document like an RFC. I still think that this is a great idea and in the hope that more people use it I packaged it for Debian and Ubuntu and uploaded it on a public Debian repository.

To access this repository, add the following lines in your /etc/apt/sources.list file:

deb http://debian.implementers.org/ testing/$(ARCH)/
deb http://debian.implementers.org/ testing/all/
deb-src http://debian.implementers.org/ testing/source/

You will also need to install a GPG key to verify the repository:

$ wget http://debian.implementers.org/keys/key.asc
$ sudo apt-key add key.asc

The fingerprint of the key is as follow:

CDB3 BC52 DFDC 8A58 35F1 714D 0309 943D 9AD0 8F3F

You can then run the following commands to install the shadok package:

$ sudo apt-get update
$ sudo install shadok

The repository contains only the binary packages for the i386 and amd64 architectures. You can download the source and recompile it for other architectures with the following commands:

$ sudo apt-get build-dep shadok
$ apt-get source shadok
$ cd shadok-0.0.0
$ dpkg-buildpackage -D -b -us -uc

Enjoy!