The Bamboo Feeder – Automating Continuous ARM Image Tests

To minimize the need for manual image testing of our ARM images, the canonical QA team invited me to their team sprint this week in the Lexington test lab to help with setting up a fully automated infrastructure on a bunch of pandaboards.

If you know the pandaboard you might also know that it can boot from its mini USB port as well as from an SD card. As long as the SD is empty it will listen on USB for the first stage bootloader file (which then pulls u-boot via the USB connection).

The central design idea was to have a single machine with USB hub serving as an initial bootloader dispenser for all the other pandas (indeed we could have taken any kind of machine but there were some discontinued panda models around that we don’t use anymore so we picked one of these as the central server).

Once the bootloader is completely recieved and executed on the client pandas they default to do PXE booting and pull their kernel and initrd from a tftp machine over the network.

For this initrd i developed a small initramfs script (and matching initramfs-tools hook indeed) that simply streams the most recent Ubuntu image (location and name are configurable via a kernel cmdline parameter) from http directly to the SD card, mounts the first partition and dumps a bootloader configuration (similar configurable via a cmdline parameter) in place which has references to a remote debian-installer preseed file before it reboots the board into a fully automated installation.

In case there are issues all the pandas can be power cycled through a web-controlled power strip and indeed all of them are hooked up to a serial console server so you can access their serial console remotely in case your preseed file didn’t tell it to install sshd.

At the end of the install the beginning of the SD card gets zeroed so the panda thinks the card is empty and will boot via USB again (in case the kernel hangs on boot or something similar fatal happens u-boot luckily provides an erase command for MMC’s that can be executed via the serial control server).

Indeed we hit the first issue with the first step we tried to implement, USB booting only worked with one model of the pandas we had around, all the others simply didn’t pull their u-boot after the first stage bootloader was executed. Thanks to an impressing and tireless overnight debug and hack effort from John Rigby and Ricardo Salveti from linaro this issue was fixed on Tuesday. You can read Johns blog post about how to USB boot a panda on his blog, all fixes that were worked out should land in the quantal packages soon.

The next obstacle we hit was that TI apparently decided the panda should (like the beagleboard) be capable to be powered via the mini USB port (even though the current provided here is not sufficient to actually run the board unless you make sure that your kernel disables most of the power hungry devices on board).

Read: we could not power cycle the boards remotely as long as the USB connection was in place.

I sat down and read up the USB specs. Theoretically the data transfer should not need the 5V connection to transfer its bits and bytes through the cable so i opened an USB cable and just cut the power connection. Transferring data to my tegra2 netbook seemed to work this way (seems the port is always powered from the board side here), but sadly the pandas did not enable the mini port at all if there was no power applied to it from the other end.

Luckily there was a solution … the panda actually operates with 5V mains power so we spent our Tuesday with roaming around the nearby radioshack shops and buying barrel connectors (and sockets), shrink tube, a soldering iron and cable pieces. I then connected the power of the mini USB directly to the pandas main power connection so that if you powercycle remotely the USB connection would be powered off as well.


Popular posts from this blog

What is Class I Division 2?


7/8 16UN Connectors that Provide 600 Volts and 15 Amps