With the release of Android 8.0 (“Oreo”), I as well as others have noticed that they cannot find some HM-10 clone BLE modules when scanning for them with an Android 8.0 phone. Why is that happening and what can you do about it?
tl;dr, the issue is due to a bug in the firmware of those modules, typically referred to as CC41 firmware. Fixing these modules is too much of a hassle and you would probably be better off getting a new module for communicating with Android 8.0 and perhaps use the problematic ones for iOS and pre Android 8.0 devices only.
update: it has been reported that Android 9 went back to the “relaxed” behavior. You might want to check your problematic modules with other versions of Android if you are having this issue. See more details in the comments.
I have several projects which involve BLE, most noticeably Fochica, a system that alerts parents if they have unintentionally left their children in a vehicle. As a result I have been dealing with all sort of BLE modules in the last couple of years. Many of these modules are in a category called “HM-10 clones”. I have written an Arduino sketch, arduino-ble-ident-n-set, to help identify the specific module type and to help configure it easily.
Typically listed or sold as just “HM-10” these modules are a group of modules that share a form factor, pin layout and a micro-controller (the TI CC2541). While trying to mimic the original HM-10 by “Jinan Huamao technology Co” they differ by firmware, breakout board and daughter board. When ordering online it is very difficult to know which exact combination you will get. Only the original HM-10 modules have some kind of support, while other variations are mostly made by anonymous clone manufactures. I personally have dozens of modules with 6 different unique combinations, including 2 variants of CC41, 2 variants of MLT-BT05, an original HM-10 and a botched CC41 that I will describe in more detail later in the article.
When these modules fail, it could be due to an issue in the daughter board, the breakout board or in the firmware. This case is due to a bug in the CC41 firmware. When a BLE device receives a scan request it responds with a scan response. The response includes information about the device such as device name, services offered, etc. The length of the response depends on the amount of bytes in the response fields. The CC41 firmware always sends a response with length of 31 bytes. It basically performs a “read beyond bound” from the variable that stores the scan response. The first part of the 31 bytes is a valid scan response and the second part is some padding containing either uninitialized memory or data of the next variable in memory.
iOS and pre Android 8.0 systems ignored this corrupt data but starting with Android 8.0 Google decided to perform stricter validation of this data. This was probably done to prevent potential security issues, crashes and to follow the standard more closely. The malformed response is ignored and the resulting behavior is as if the CC41 is not responding at all.
I was able to reproduce and observe this behavior with the following BLE sniffing setup:
- TI SmartRF Protocol Packet Sniffer software – http://www.ti.com/tool/PACKET-SNIFFER
- CC2540 USB device – http://www.ti.com/tool/cc2540emk-usb – I got mine from eBay, probably a clone.
- SmartRF Flash Programmer software – http://www.ti.com/tool/flash-programmer
- TI SmartRF04EB programmer device – http://www.ti.com/tool/CC-DEBUGGER – I got mine from eBay, probably a clone. For a more DIY approach, there are articles showing how you can use an Arduino as a TI programmer.
The steps are:
- Install the right software versions for your matching devices.
- Install a driver for the programmer. Program the “CC2540 USB” device with “Texas Instruments\SmartRF Tools\Packet Sniffer\bin\general\firmware\sniffer_fw_cc2540_usb.hex” firmware.
- Connect the CC2540 USB to your computer and run the packet sniffer software.
I found the following guide from TI useful.
We can now observe BLE traffic. Setting a filter on scan responses will make it easier to see the bug. Here are some responses from different BLE devices:
CC41: 0D 09 46 6F 63 68 69 63 61 5F 33 31 41 44 00 00 81 00 10 00 FF FF FF FF FF FF FF FF FF FF
ESP32: 02 01 06 0F 09 45 53 50 5F 47 41 54 54 53 5F 44 45 4D 4F 02 0A EB 05 03 EE 00 FF 00
HM-10: 07 09 48 4D 53 6F 66 74
MLT-BT05-V4.0: 09 09 4D 4C 54 2D 42 54 30 35
Local name, Field type, Length, Junk.
I have color-coded the various parts to easily convey the issue in the response. For exact protocol details see the specifications.
- A spontaneous idea might be to change the name of the CC41 device to something longer such that we have a valid response or at least a zero-padded one. However, the CC41 firmware doesn’t allow to set a name longer than 18 characters.
- Most modules send only their name in response to a scan, while the ESP32 device in this example was programmed to send 3 other fields as well.
It is unlikely that Android’s new behavior will change so the typical resolution would be to update the firmware of the device to a bug free version. Unfortunately Bolutek, who seem to be the company behind the CC41, doesn’t provide updated CC41 firmware or any easy way to upload the firmware to the device. We are left with the following options:
- Use another module
- Painful process to flash a different working available firmware
- Trying to hack the device
Use another module
This is probably the easiest option in terms of invested time. HM-10 clone modules are sold at $2 a piece so if you need to replace just a few it might be easier to order another HM-10 (clone) module. New modules will most likely come with a different firmware such as a MLT-BT05 variant, or even better an original HM-10 which has means for over-the-serial flash updates. If you still get a CC41 module nowadays I feel it is a valid reason to ask for a refund.
Personally I am a bit tired of dealing with issues of the HM-10 ecosystem. You never know what you will get as the variability is so high. Most modules have an anonymous entity behind them and no way to get updates. The original model lost its value as a unique identifier of a specific model and became a generic identifier. Pairing/bonding is poorly implemented in the clones compared to alternatives. Due to these reasons and more you too might want to stay away from HM-10 all together.
Nowadays I prefer using an ESP32 instead of an Arduino + HM-10 combination. The main reasons are:
- It is a bit more complex but you have full control of the firmware, you can define your own services/characteristics and use this for more than just wireless UART.
- There is a company behind the device and you don’t need to do any wiring as the BLE (+BT+WiFi) is built into to the micro-controller.
- The ESP32 is a much more powerful chip than a typical Uno/Nano but still can be programmed the same way from an Arduino IDE.
- The ESP32 now has a decent BLE library.
The big difference between the two approaches is that with the ESP32 you don’t have a separate firmware for the external module, instead the code of the BLE is compiled together with the code of your program. When you upload your code you upload both your program and any BLE related code/libraries.
Re-flashing to another firmware
Those looking to “fix” their CC41 flashed modules can flash them with a different firmware. The only other firmware I am aware of that is available in a file that can be flashed is the HM-10 original firmware. Any modules with other firmware that I have tested always came “chip locked” so you can’t read the firmware from the module. I guess the HM-10 firmware once leaked or was distributed without a “chip lock”, which resulted in its availability. If you are aware of another firmware that exists for this purpose, please mention that in the comments.
There are advantages and disadvantages to flashing the HM-10 original firmware to a clone device.
- Pairing/bonding works better and with more phones models and OS versions
- Ability to do over-the-serial updates
- Works with Android 8.0
- Works better as a beacon (iBeacon)
- Less bugs in general
- You will need to cut the shrink warp of your module
- You will need to do some soldering
- You will probably need to buy/build some tools (clips, programmer, etc)
- You might have to fix issues and apply workarounds to make the STATE pin work as expected
PSA: The jnhuamao.cn site contained downloads with trojans and was classified as a malicious site on several instances so if you are downloading executables from there you have a chance of getting infected. You can do the process without going there which is what I would recommend, otherwise be careful.
The instructions below are not going to be a complete tutorial for re-flashing the CC41, for detailed instructions, firmware files and discussion see the following resources:
- JDY-08 : bluetooth 4.0 module
- How to flash genuine HM-10 firmware on CC2541 (make genuine HM-10 from CC41) – Video
I used the following tools to re-flash the module:
- SmartRF Flash Programmer software – http://www.ti.com/tool/flash-programmer
- TI SmartRF04EB programmer device – http://www.ti.com/tool/CC-DEBUGGER – I got mine from eBay, probably a clone. See CCLoader for a DIY approach.
- DIP16 programming clip/clamp – an option to consider instead of soldering wires to the module
The steps I took for flashing
- Cutting the transparent shrink wrap that protects the module. You should consider taping it back when you are done flashing.
- Identifying the pins needed for programming: debug clock, debug data, reset, power 3.3V and ground.
- Adding solder to the pins/pads where connection are to be made.
- Bending the connectors of the DIP16 clip so they align to the pads of the castellated daughter board. They are not the same pitch by default.
- Running the SmartRF programmer software and setting the settings for writing the desired firmware.
- Connecting Gnd to the 2.54mm module pin and the other programmer connections via the clip.
- Connecting the programmer to the USB.
- Performing the program.
- Verifying the result through a BLE app and a serial connection.
This worked surprisingly well. The difficult part was holding the clip in place steady for a good connection to the pads. If the connection is not good, then the device would not be detected in the programmer when it is plugged in.
Fixing the STATE pin
All works well with the new firmware except for the STATE pin. In the clones this pin is supposed to be HIGH when the module is connected and LOW when the module is disconnected. To accomplish that the clones use pin P1_1 (PIO2) which is connected directly to STATE while also having a separate pin P1_2 (PIO1) to drive the on board LED. The default behavior for the LED is to blink when the module is not connected and HIGH when the module is connected.
In the original module, P1_1 is not used and is configured as OUTPUT LOW. Instead P1_2 is used for both the STATE and the LED. This means that by default the STATE will blink when the module is not connected, which is a weird behavior for a digital output signal. To compensate for that, the original HM-10 allows to reconfigure the behavior of P1_2 (PIO1) through AT commands to have it as LOW when the device is not connected. This is one aspect where I believe the clones are actually better than the original. It makes much more sense to have separate digital signal and visual LED indicator pins as each has its own expected logic.
If, like me, you rely on the proper operation of the STATE pin for your application logic, then you will have to make another alteration to the module to reconnect the STATE pin to P1_2 instead of P1_1. Note that it is not enough to bridge the two, you have to actually cut the previous connection, otherwise there will be a short circuit between P1_2 and P1_1. Identify the trace that connects between STATE and pin P1_1 and cut it using a hobby knife. Verify that there is no conductivity. Then solder a bridge between STATE and the P1_2 pin or the LED or the LED resistor (depending on what is convenient on your breakout board). Verify that the two pins are not connected and that STATE is now properly connected to pin P1_2. Using AT commands or arduino-ble-ident-n-set configure the module to drive the LED in a boolean fashion. You should now have a proper STATE pin.
Note, some HM-10 clone modules come with no solder bridge between pin P1_1 and the matching pad on the breakout board. If you intend to re-flash those, this can actually be an advantage. Don’t fix this connection, re-flash the module and then bridge the two pads on the breakout board without bridging the matching pads on the daughter board. This should connect STATE to P1_2 while leaving P1_1 disconnected.
Here you can see a CC41 module which was previously fixed to support STATE, now re-worked to support STATE in a HM-10 configuration:
Perhaps there are additional ways to hack the CC41 module. I am really interested to know if you have any ideas. We know that there is at least one bug in the firmware and perhaps there are others, which can allow us to take control of the device, alter its behavior or dump its firmware. Perhaps somebody has an updated firmware or an original extracted firmware that can be patched…
It is possible to fix misbehaving modules by re-flashing them, but it is not a quick or easy process. Might be better to get new HM-10 modules or switch to alternatives.
It would be nice to have more options.
The importance of software updates is demonstrated once again.
For the benefit of the community and other buyers, you are encouraged to leave reviews for any modules you get from eBay/aliexpress and describe their parameters such as type, version, STATE pin implementation, etc. This is the only way we can beat the huge fragmentation in the market and educate the sellers about the items they sell.