HardFaultVector, INVSTATE?

[russian]

I am sorry if that's not exactly a ChibiOS question but a general ARM question, but the people here are so nice and helpful...

I have sporadic HardFaultVector errors - sometimes it takes 3 seconds to get it, sometimes 30 minutes. I know it's HardFaultVector because I have explicit handler defined.
I also know that INVSTATE is 1.
ARM tells us

Code: Select all

Invalid state UsageFault:

0 = no invalid state UsageFault

1 = the processor has attempted to execute an instruction that makes illegal use of the EPSR.

When this bit is set to 1, the PC value stacked for the exception return points to the instruction that attempted the illegal use of the EPSR.

This bit is not set to 1 if an undefined instruction uses the EPSR.

and that's where I am lost.

Code: Select all

Execution Program Status Register

The EPSR contains the Thumb state bit, and the execution state bits for either the:

    If-Then (IT) instruction

    Interruptible-Continuable Instruction (ICI) field for an interrupted load multiple or store multiple instruction.

See the register summary in Table 2.2 for the EPSR attributes. The bit assignments are:

Table 2.6. EPSR bit assignments

wat?

So I wonder what this diagnostics is supposed to tell me.

[Giovanni]

You need to find the instruction pointed by the PC when the exception occurs, that could give us some info. Illegal of EPSR could be trying to put there an invalid value. Normally it is written only by the IT instructions, multiple move instructions or on exception return.

One possible cause could be a corrupted exception stack frame (extctx in chibios) on exception return.

Giovanni

[russian]

These pictures above show PC = 0x75660064 which seems to be an invalid address outside of the expected ranges, does not it?

So far I've added a bunch of

Code: Select all

   if (dbg_isr_cnt > 0) {
      remainingStack = (__get_SP() - sizeof(struct intctx)) - (int)&IRQSTACK$$Base;
   } else {
      remainingStack = (stkalign_t *)(__get_SP() - sizeof(struct intctx)) - otp->p_stklimit;
   }

efiAssertVoid(getRemainingStack(chThdSelf()) > 16, "lowstck#9c");

but these asserts do not seem to be triggered. Also added

Code: Select all

void dbg_check_enter_isr(void) {
  if (dbg_isr_cnt > 2)
    firmwareError("nesting3");

also not being trigged

Also got another kind of hard fault (?):


Note how this one does not show INVSTATE bit but shows FORCED and IBUSERR bits.

Code: Select all

   

Indicates a forced hard fault, generated by escalation of a fault with configurable priority that cannot be handles, either because of priority or because it is disabled:

0 = no forced HardFault

1 = forced HardFault.

When this bit is set to 1, the HardFault handler must read the other fault status registers to find the cause of the fault.

[russian]

IBUSERR

Instruction bus error flag:

1 = instruction bus error

0 = no instruction bus error.

The IBUSERR flag is set by a prefetch error. The fault stops on the instruction, so if the error occurs under a branch shadow, no fault occurs. The BFAR is not written.

Oh my god. What is this about?

[Giovanni]

The application is jumping to an invalid address for some reason, the cause for this could be a corruption of a return context, is it possible there is some kind of automatic buffer that overflows? the point is, how the application jumps there, the nature of the exception is not much important because it is a consequence of a previous error.

Could it be a string? that address looks suspiciously like ASCII data.

Giovanni

[russian]

I've reduces stack usage yesterday and today I am getting IBUSERR flag instead of INVSTATE. Does this change anything?

I hope I am monitoring all my buffers. Obviously unless I am not doing a good job 0x75660064 is "uf" <zero terminator> "d" - I'll check the binary if I have it somewhere, thank you for the great hint!

[russian]

Two times in a row I can find the address of my ADCConversionGroup->end_cb on the stack while I am in HardFaultVector with INVSTATE

ADCConversionGroup->end_cb is always invoked by adc_lld_serve_rx_interrupt into which I've just injected my

Code: Select all

static void adc_lld_serve_rx_interrupt(ADCDriver *adcp, uint32_t flags) {
   efiAssertVoid(getRemainingStack(chThdSelf()) > 32, "sys_adc");
   ...
   ...
}

int getRemainingStack(Thread *otp) {
   int remainingStack;
   if (dbg_isr_cnt > 0) {
      remainingStack = (__get_SP() - sizeof(struct intctx)) - (int)&IRQSTACK$$Base;
   } else {
      remainingStack = (stkalign_t *)(__get_SP() - sizeof(struct intctx)) - otp->p_stklimit;
   }
   otp->remainingStack = remainingStack;
   return remainingStack;
}

I guess my getRemainingStack does not return the real remaining stack amount since if it would I would get my assert fail and not the HardFaultVector

chThdSelf()->remainingStack = 116

[Giovanni]

Note that measuring the remaining stack using the filler is not necessarily reliable, you could allocate a large frame and write the lower parts of it without touching the filler.
You could try placing stack assertions in the lowest parts of your threads (using GCC you can access the register SP as a variable).

Giovanni

[russian]

remainingStack = (stkalign_t *)(__get_SP() - sizeof(struct intctx)) - otp->p_stklimit;

I am an idiot and I should not be programming in C, I cannot even subsctract one value from another.

(stkalign_t *)(__get_SP() - sizeof(struct intctx)) = 0x2001FEF4
otp->p_stklimit = 0x2001FC00

0x2001FEF4-0x2001FC00=0x2F4

(stkalign_t *)(__get_SP() - sizeof(struct intctx)) - otp->p_stklimit != 0x2F4

(stkalign_t *)(__get_SP() - sizeof(struct intctx)) - otp->p_stklimit = 94

What I need for my remainingStack math? I need

remainingStack = (stkalign_t *)(__get_SP() - sizeof(struct intctx)) - (int)otp->p_stklimit;

[russian]

I start that IRQ nesting is the issue - looks like I have up to 4 exceptions nested: ADC_DMA nested within PWM nested within SysTick nested within USB

I am thinking to reduce the number of different IRQ priorities used, maybe even put all IRQ on the same priority - in the end of the day all the handles are pretty fast and a tiny delay is fine in my case. How bad does it sound?

By default UART and OTG irq had low priorities, is there a reason these need to be low priority?

(ADC_DMA nested within PWM is especially funny - PWM irq start the adc conversion and the dma irq happens before PWM irq is done)