BLE Serial 2.0 Update

This is a detailed description of the major 2.0.0 release of BLE Serial. It is a tool to connect Bluetooth low energy 4. and 5.x (BLE) UART modules to virtual serial ports on Linux and now also COM ports on Windows.

Backend Change

In the first BLE Serial versions (1.0 up to 1.3) it used bluepy as bluetooth backend library. Problem with it was an open issue that required the manual installation of a fork, like described in the previous post about it. Now over the course of 2020 some PRs got merged, but unfortunately there are many open issues and the version on PyPI is still more than 2 years old.

Also the recommended interface to use BlueZ on Linux is D-Bus, while bluepy uses a custom bluepy-helper written in C, that required additional permissions for some actions.

So as discussed in this issue a backend change was needed. The descision was made to switch to bleak which is pure Python and uses D-Bus on Linux. In addition it has support for Windows and macOS.

Installation is simply done with one command now:

$ pip install ble-serial

Windows Support

As already mentioned BLE Serial has now cross platform support. Following is the Windows setup described (excerpt from README).

Windows does not have a builtin feature to create virtual serial ports (like Linux does), so it is required to install a additional driver. I decided to use the open source com0com Null-modem emulator, downloaded from here as signed version. This is required because unsigned drivers can not be installed anymore. Note that on latest Windows 10 you likely still have to disable secure boot for it to work.

ble-serial includes the ble-com-setup script to make the com0com configuration easier:

> ble-com-setup.exe -h
usage: ble-com-setup [-h] [--install-path INSTALL_PATH]

Setup required COM port pair

optional arguments:
  -h, --help            show this help message and exit
  --install-path INSTALL_PATH
                        Installation directory of the null modem emulator (default: C:/Program Files (x86)/com0com/)

It will request administrator privileges (if it does not already have it) and setup the port in another CMD window:

Changing into C:/Program Files (x86)/com0com/

> Checking port list for BLE
       CNCA0 PortName=-
       CNCB0 PortName=-

BLE port does not exist

> Checking port list for COM9
       CNCA0 PortName=-
       CNCB0 PortName=-

> Trying to create port pair
       CNCA1 PortName=COM9
       CNCB1 PortName=BLE
ComDB: COM9 - logged as "in use"

Setup done!

Hit any key to close

As you can see it created the BLE<->COM9 pair. ble-serial will internally connect to BLE, users can then send/receive the data on COM9.

Otherwise there exist multiple proprietary serial port emulators, these should work too. Just manually create a pair that includes a port named BLE.

Performance

To test the performance and compare it with the older implementation I added automated transfer tests in the repo tests directory. The idea is to connect a USB <-> serial adapter to the BLE module. Then run ble-serial on the same host, pass data between the adapter and virtual port and check if it is still the same, to identify configurations with packet loss and other issues.

My test setup consists of a HM-10 module and CP2102 based USB/UART adapter. The scripts uses AT commands to automatically switch through the baud rates, for example AT+BAUD0 for 9600 bit/s. Then it sends packets with different intervall/delays to either the BLE or UART side, while checking (reading) on the other side and comparing how much was actually received.

Note that I have also tested different packet sizes (which has the same effect as changing the --mtu option), but did not find a significant difference between the 4, 16 and 64 bytes, so I am not including it in the graphs here. The following was all tested with 32 bytes packets. You can see the raw CSV data in the results/ subdirectory, if you are interested anyway.

I have grouped the results in graphs by the transmission direction. First writing to the virtual BLE port and receiving on hardware UART:

As you can see it is getting better with higher baud rates, which makes sense, because the BLE chip is receiving data at a constant rate. The module will just drop data, if the configured UART is too slow to keep up with outputting it.

The maximum real throughput is at at about 21 kbit/s, this explain what we saw above: 19200 is not enough, but 57600+ can receive 100%.

Now to the tests in the opposite direction, writing to hardware UART:

Losses show the opposite behavior here, 9600 is completely fine, higher ones only if some delay is used. From 19200 only about 80% is received.

Apparently the maximum throughput is lower in this direction with about 17 kbit/s, this explains why 19200 shows already some loss.

An always safe option is to use 1 ms delay per byte (= max. 10000 baud). Note that that I calculate 10 bit per byte, because UART uses 8-N-1 encoding, so: 1 start + 8 data + 1 stop = 10 bits.

Compared with v1.3

Here it tested with 0 delay, so writing as fast as possible, because this shows differences most clearly.

In the case of writing to UART both implementations are almost equal, but there is a massive difference when writing to the virtual BLE serial port. This is not throttled, so you can multiple MB/s to it, of course bluetooth and UART are not that fast. 1.3 dropped about 98% in that case, it only worked reliably if the application wrote small amounts of data continuously.

On 2.0 this is buffered, that makes writing continuously into BLE safe. There is still the hardware limitation though, the UART output baud rate should be above ~20000, for example it can receive all data with 57600 baudrate.

Colored Logs

Another improvement in this release are that log levels are now indicated also by colors: This should make it easier to spot important infos, even with the noisy verbose (-v) log level.

I hope this update is helpful, let me know how it works.