Verhaert: video over 4G in 300 ms, and a medical device on BLE

In this story: What it does · How it works · Lessons learned · How it ended

What it does

Verhaert Masters in Innovation is a product development company: clients come with an idea and leave with a product. From September 2019 to September 2020 I worked there as a freelance senior embedded software engineer, responsible for several projects end to end, from the system architecture to deployment in the field.

Two of them stood out: safety-critical video sent over 4G that had to be on screen within 300 ms, and a medical device communicating over Bluetooth Low Energy and 6LoWPAN. Alongside them I helped build the CI/CD pipeline for the embedded firmware and coached two junior engineers. Most of that work is under NDA, so this story stays with the engineering problems and leaves out the names.

How it works

Video over 4G in 300 ms

An industrial environment and a safety case: live video sent over a 4G link had to be on the screen within 300 ms. For safety, a late image is worse than no image. Someone acting on what happened half a second ago is acting on the wrong picture.

What makes that hard:

• The budget is end to end. Capture, encoding, sending, receiving, decoding and drawing all share the same 300 ms. Every stage has to be measured, and every buffer is a suspect.
• 4G is not a cable. Latency and bandwidth change with the signal, the load on the cell and every handover, and the uplink is usually the weak side. A stream tuned for a good moment stalls in a bad one.
• Smooth video is late video. Network stacks and players buffer to hide jitter, and each buffer adds delay. The buffers had to be as small as the link allowed, and a late frame dropped rather than shown.
• The embedded processor cannot keep up on its own. Decoding in software is too slow, and copying every frame between the decoder, the CPU and the GPU eats the budget.
• The information on top costs time too. Drawing overlays on the CPU means one more pass over every frame.

So the pipeline was built in GStreamer around the hardware decoders, with the frames handed to OpenGL and the overlay drawn by shaders on the GPU, keeping the image off the CPU on its way to the screen. The link itself ran over OpenVPN, with Linux traffic control (tc) to keep the video stream first in line.

A medical device on BLE and 6LoWPAN

A medical device communicating over Bluetooth Low Energy 4.0 and 6LoWPAN, IPv6 over low-power radio, in a government-subsidised project. My part ran from the compliance documentation to the implementation of a sensor. In medical work the documentation is not paperwork after the fact: it is part of the product, and it has to hold up as well as the code does.

Releases you can trust

I helped design and implement the CI/CD pipeline for the embedded firmware, built with Yocto and delivered to devices with SWUpdate and Eclipse hawkBit. It made releases more reliable and repeatable: a firmware image that is built, tested and rolled out the same way every time, instead of by hand on someone’s laptop.

Lessons learned

Teach until they can carry it alone

I coached two newly graduated engineers from the consultancy Nalys. The goal was not to hand them tasks but to get them to the point where they could carry a project independently. That takes more time at first and frees far more later.

Documentation is part of the product

On the medical device my part ran from the compliance documentation to the implementation of a sensor, and I learned to treat both as the same job. In medical work, a sensor nobody can show to be compliant is not finished, however well it works. Owning a deliverable means owning the evidence that it does what it should, not only the code.

How it ended

In September 2020 I left Verhaert to help build Herd-itt, where I became co-founder and CTO.