ChibiOS Free Embedded RTOS

Hi everyone,

There is an application based on ChibiOS/RT v2.6 that works fine without any known issues. It runs on STM32F105, and it uses the following ChibiOS drivers: USB, USB Serial, Serial (2x), I2C (master), and PAL; also it includes a custom CAN driver for both on-chip CAN interfaces. The I2C bus in this application is used to retrieve sensor measurements from two on-board sensors at the rate about 80 measurements per second. OS is mostly configured with default settings (max clock rate, default IRQ priorities, IRQ preemption enabled, etc).

Recently I had to port the application to ChibiOS/RT v3.0. The new OS has also been configured mostly with default settings: tickless mode, 16-bit ST counter at 2 kHz. When porting was finished, everything seemed to work fine except one problem with I2C described below.

The application seems to work fine as long as there is no activity on the CAN bus; but if either or both of the CAN bus interfaces are subjected to any traffic, the I2C driver would occasionally report failed transactions with return value MSG_RESET and error code I2C_ACK_FAILURE. The errors would appear approximately once-twice per second at highly irregular intervals. I suspected that there could be a cross-talk between CAN and I2C and inspected the board with a scope, but it turned out that the signals were perfectly well-formed and the hardware was performing well. Furthermore, it worked perfectly with Ch/RT v2.6, so it cannot be a hardware-only issue.

Also, I discovered that the frequency of I2C errors would depend on optimization levels in a very inconsistent way. Changing optimization between O1, O2, Os and switching between release and debug builds would randomly increase or decrease the frequency of errors.

Scope inspections revealed that there are random spikes at SDA that are not synchronized with SCL, which only occur while CAN is active:



(legend: trace 3 - SCL, trace 2 - SDA; trace 1 is irrelevant)

This observation made me suspect that there could be a problem in the new I2C driver that only appears while CAN is active. The first idea was to check whether CAN IRQ may preempt I2C IRQ, and it turned out that the priority of the CAN IRQ was higher. I raised priority of the I2C interrupt to the same level as CAN (2), and the problem seemingly disappeared (30 min run time under high CAN traffic without issues).

So, my question is, what am I doing wrong? Does it look like a bug in the I2C driver?

For reference, the custom CAN driver I'm using is available here: https://github.com/UAVCAN/libuavcan/blo ... 32_can.cpp

Thanks in advance,
Pavel.

Hi,

Could you make sure that I2C interrupts have higher priority than CAN interrupts?

Giovanni

Is it a requirement, or do you want me to test whether the problem is reproducible when I2C has higher IRQ priority?

Hi,

The I2Cv1 peripheral is quirky, preempting its ISRs is one of the problems we identified. Setting its priority above other peripherals is a workaround.

Try 3 for example (0 and 1 are reserved, 2 is left usually empty).

Giovanni

Okay, as I understand, priority of I2C interrupts does not necessarily have to be higher than all other peripherals, but not lower than any other peripheral, correct? Because IRQ at identical priority levels do not preempt each other, so it should be safe for me to keep both CAN and I2C at level 2.

Is this issue valid only for 3.x, or does it also exist for 2.6?

Is it documented somewhere? I couldn't find any documentation for this bug, but probably I was just looking in the wrong places.

Thanks,
Pavel.

Equal priority should not be OK because CAN IRQs could still delay I2C IRQ servicing.

I thought it was documented here: http://wiki.chibios.org/dokuwiki/doku.p ... e_shooting
But apparently this bit is missing.

The issue is with HW, not specific of a ChibiOS version. Only F1, F2, F4, L1 are affected, newer devices use a different I2C peripheral.

Giovanni

Huh, I didn't know about that thing with I2C. I will update the priority levels then.

Thanks Giovanni!

Try 3 for example (0 and 1 are reserved, 2 is left usually empty).

What is the reason to keep level 2 empty?

It is the same level used internally for the SVC vector. It could be used but there is no real need for that usually.

Giovanni