How a small development team handles more supported operating systems than engineers
The last few Prelude Operator releases were long, drawn out events.
Going into each, we thought we had all bells-and-whistles tested, all components ready to go. But in each, we found gaps in our testing which delayed our deployments by hours and made them less than enjoyable events.
After a few missed deployments, we noticed a trend: we support multiple operating systems but we haven’t been reliably testing each before a deployment.
As an ElectronJS desktop app, Operator is compiled for:
Each major OS above has minor versions that introduce variations that require testing, such as the differences between Ubuntu 20.04 and Kali Linux. With three developers, this was creating a testing headache. Doubly so because we all prefer developing on our MacOS laptops, meaning Windows and Linux were only getting pulled out for pre-deployment checks.
After an 8-hour 0.9.24 deployment, we decided it was time to make a change. Here’s our story.
The day after the 0.9.24 deployment, our CEO jumped on the problem and immediately got the necessary QA box delivered:
After a quick Best Buy trip to pick up the new Dell desktop, we now have the necessary hardware. Next step: installation.
Hearing about the side project, Alex - being the hardware enthusiast he is - wanted in. So swapping a lazy Sunday afternoon for tinkering on the desktop, we started the unboxing process.
Taking the cover off, we wanted to know what our expansion options were.
For starting specs, our Dell desktop has:
An Intel i5 processor
12GB of RAM (8GB and 4GB sticks)
256GB solid state drive
There is room (and extra SATA cables) for additional drives, if we need them.
Putting the cover back on, Alex starts the Windows 10 installation, using the operating system the desktop shipped with. While he’s doing that, I prop up on a couch in the basement and burn a Kali Linux ISO to a spare USB, so we can dual-boot the machine after the first OS is in place.
While the ISO is burning (about a 30 minute process) - and while Alex is installing Windows updates - my home internet cuts out.
Have you ever done a “computer project” without hiccups?
The internet continues to cut in and out over the course of the next hour, while we get sporadic network connections in bursts. Simultaneously, my wife is trying to hop on a Zoom call with friends, adding to the mix of devices desperately seeking modem time.
So Alex and I hop off WiFi and switch locations, forming an ad-hoc computer lab on the living room floor. With the internet still borked, we abandon the Windows updates and focus on the dual-booting.
Our first attempt to install Kali went like this:
Enter the bios on a reboot and disable Safe Boot
Restart and select the USB as the bootable drive
Partition the 256GB drive into 2 parts, one for Windows and one for Linux
It was on the last point we encountered our next issue: Kali was only giving us the option to use the entire drive (which would wipe the Windows installation) and wasn’t giving any partitioning options, like is usual during this process.
Booting back into Windows, Alex partitions the drive from the GUI and retrys. Kali is still not finding the partitions, giving us the same option of overwriting the entire thing.
Remembering the extra SATA cables inside the box, Alex has a new idea: let’s get a second drive, install Linux on that and boot from either drive.
Given that we’ve encountered a few hiccups so far, we opt to postpone the Linux installation and focus on getting Windows online. Booting back there, we find the internet has stabilized and we can complete the updates.
With everything in place, it’s now time to download Operator.
Pulling the latest Windows version, we start it up:
Voila! Latest Operator up and running.
For good measure, we install the Jetbrains toolkit, Node and Git so we can use the box for general development, as well as production testing.
So what’s the plan moving forward?
For the time being, I’ll start using this Windows desktop for general development and testing purposes. Once we get the Linux dual-boot in place, I’ll alternate between the two operating systems.
Then we’ll write an automated test suite which runs Operator through its paces on both Windows and Linux. This test suite will run the application through normal use cases and flush out any obvious bugs from operating system variations.
There are two ways to approach the test suite: internal or external. An internal suite would leverage an Electron library such as Spectron and run unit/integration level tests. An external suite would be Selenium-like and execute tests from outside the code base, treating the application like a normal user.
But, despite the hardships of spotty internet and restrictions on dual-booting the drive, we are one step closer to stable testing across our core supported operating systems. So we’ll chalk that up as a win for stability.