Fast IRQ causes memory corruption

[Spym]

The application runs on STM32F103CB & ChibiOS 2.6 (latest stable branch).
ChibiOS config can be found here: https://github.com/pavel-kirienko/px4es ... s/chconf.h, https://github.com/pavel-kirienko/px4es ... /halconf.h.

There are two fast interrupts that shall be processed with (virtually) zero latency: ADC1 and TIM2.
Their handlers: https://github.com/pavel-kirienko/px4es ... mer.c#L105 and https://github.com/pavel-kirienko/px4es ... _adc.c#L50.
As can be seen, no OS API are used from these interrupt handlers.

Other interrupt sources are UART1 (ChibiOS driver, low priority) and the ChibiOS core itself, nothing else.

The problem is that the memory got corrupted only if the fast IRQ priority exceeds CORTEX_MAX_KERNEL_PRIORITY (i.e. if the priority equals 0 or 1). It appears as stack overflow of the main thread (no other threads ever created); checking the system state in the debugger I can see that the otp structure got spoiled:

Code: Select all

otp Thread *    0x20001120
    p_next  Thread *    0x20001228
    p_prev  Thread *    0x1
    p_prio  tprio_t 1
    p_ctx   struct context  {...}
    p_newer Thread *    0x20001138
    p_older Thread *    0x0
    p_name  const char *    0x0
    p_stklimit  stkalign_t *    0x0
        *p_stklimit stkalign_t
    p_state tstate_t    72 'H'
    p_flags tmode_t 17 '\\021'
    p_preempt   tslices_t   0 '\\000'
    p_time  volatile systime_t  134239287
    p_u union {...} {...}
    p_waiting   ThreadsList {...}
    p_epending  eventmask_t 536875352
    p_mtxlist   Mutex * 0x800ca71
    p_realprio  tprio_t 536875360

For reference, there is the same structure dumped during normal operation:

Code: Select all

otp Thread *    0x20001120
    p_next  Thread *    0x20001228
    p_prev  Thread *    0x20001228
    p_prio  tprio_t 1
    p_ctx   struct context  {...}
    p_newer Thread *    0x20001228
    p_older Thread *    0x200011d0
    p_name  const char *    0x800d070 "idle"
    p_stklimit  stkalign_t *    0x200011d0
    p_state tstate_t    1 '\\001'
    p_flags tmode_t 17 '\\021'
    p_preempt   tslices_t   20 '\\024'
    p_time  volatile systime_t  536875296
    p_u union {...} {...}
    p_waiting   ThreadsList {...}
    p_epending  eventmask_t 536875560
    p_mtxlist   Mutex * 0x20001228
    p_realprio  tprio_t 0

The stack overflow panic is always invoked from the UART1 interrupt (remember this is the only low priority interrupt source besides the core IRQs), so that makes me think that there is something wrong with IRQ preemption. Note that I checked both main stack and the process stack in the debugger after the failure, and they both appear to be not overflowed (default fill pattern 0x55 can be seen). Regardless, I even tried to increase both stacks up to 4kb each, didn't help.

What am I doing wrong?

Thanks!

[Giovanni]

Hi,

Those fast handlers call a callback function. Is the callback function clean of OS calls?

Giovanni

[Spym]

Those fast handlers call a callback function. Is the callback function clean of OS calls?

Yes of course.
The fast IRQ control flow never leaves the code from the motor/ directory, so it can be inspected easily.

[Giovanni]

Fast interrupts are something the OS does not interact with so I have nothing to verify in the code.

The stacks you have to consider are: exceptions stack and process stack (in the .ld file) and the idle thread stack (PORT_IDLE_THREAD_STACK_SIZE, this may need an increase if the compiler is in -O0 mode and uses more stack than expected, inspect this one too).

Also, do you have arrays or other things that can overflow and overwrite the structures?

Giovanni

[Spym]

The stacks you have to consider are: exceptions stack and process stack (in the .ld file) and the idle thread stack (PORT_IDLE_THREAD_STACK_SIZE, this may need an increase if the compiler is in -O0 mode and uses more stack than expected, inspect this one too).

Fixed:

Code: Select all

#define PORT_IDLE_THREAD_STACK_SIZE    64
#define PORT_INT_REQUIRED_STACK        128

Doesn't crash anymore. Though I need to perform more extensive testing to adjust the stack size, but the primary cause is clear now.

Thanks Giovanni!