Automated hardware, OS, patch level and application testing

Contributions
1

Before I go down the rabbit hole. I’ve started work on creating a gitlab integrated hardware/OS/Application testing platform to specifically test combinations of hardware, OS, patching and with a Application test (although the gitlab already has a test suite for applications using gitlab runners).

This looks achievable using netboot for pi 3 onwards, but as the pi2 doesn’t have network boot, an idea would be to use a multi-boot SD card with the variations of installs and testing for the pi2.

The architecture would have a pi actings as the management/network attached storage node. The various versions of hardware would then boot off this, go through the install, patching of the SD images to the defined patch levels, then verify the application functions using either a version of the existing tests (start, stop, etc) from the branch that’s used for the installation.

The image below is a logical view, and currently being updated (note the test pis are in the VLAN box):

Screenshot 2026-09-24 at 16.54.33
Screenshot 2026-09-24 at 16.54.33738×553 41 KB

Essentially it would be:

  1. Gitlab runner triggered → triggers a test with a configuration
  2. Management PI then creates and images, packages etc needed, and stores the required images, etc.
  3. Management PI then works through all the hardware, checking each version defined and image.
  4. Each PI then installs via netboot, patches as required, executes the tests, provides the log files and output of the tests to the current test run location (on the management pi).
  5. Management PI then takes the output and packages back for gitlab for each variation.
  6. Management PI makes a summary with links and uploads to gitlab.

An alternative could be running with net runner to build the images (more scalable) then download the image to the management pi to update it’s image repository.

Only issue could be the size of the repository but the PI4 could run with external storage.

  1. Would this be of use?
  2. The architecture could operate from a non-RP as a management server.
  3. The mechanism should support new hardware being released.
  4. The power reset I think would be best through a simple ‘smart’ relay bank that controls power switching (default off). If the RP was more advanced you could use a power over ethernet and then use the switch todo a power reset.