Page 1 of 1
Thread priorities...
Posted: Tue Sep 17, 2019 8:02 pm
by rew
In a project I'm working on it looks as if one of my threads is not running as often as it is supposed to.
It is as if there is a high priority thread that is blocking my worker-thread. But each time I stop the system with the debugger, I find that execution stopped in idle_thread.
If my analysis of the following gdb stuff is correct, chibios is running "idle_tread" at prio 1, while my DMX_IO thread is runnable (it is on the rlist).
There is also a blinky thread: It blinks a led. Each time "blinky" toggles the led, it seems that my DMX_IO thread gets a chance to run.
When my DMX_IO thread is waiting for the IO to finish It sleeps for half a milisecond before checking a few global variables, and going to sleep again. But with blinky doing 350 and 700 ms on/off periods that 5 ticks sleep seems to sleep for hundreds of ms (350 or 700 in fact).
Any suggestions to get to the bottom of this?
Code: Select all
(gdb) print ch.rlist
$2 = {queue = {next = 0x20001a60 <waDMX_IO+512>, prev = 0x20001a60 <waDMX_IO+512>},
prio = 0, ctx = {sp = 0x0}, current = 0x20001e00 <ch_idle_thread_wa+144>}
(gdb) print * (thread_t *) 0x20001a60
$4 = {queue = {next = 0x20002368 <ch>, prev = 0x20002368 <ch>}, prio = 130, ctx = {
sp = 0x200019cc <waDMX_IO+364>}, wabase = 0x20001860 <waDMX_IO>, state = 0 '\000',
flags = 0 '\000', u = {rdymsg = -1, exitcode = -1, wtobjp = 0xffffffff,
wttrp = 0xffffffff, wtsemp = 0xffffffff}}
(gdb) print * (thread_t *) 0x20001e00
$5 = {queue = {next = 0x20002368 <ch>, prev = 0x20002368 <ch>}, prio = 1, ctx = {
sp = 0x20001dac <ch_idle_thread_wa+60>}, wabase = 0x20001d70 <ch_idle_thread_wa>,
state = 1 '\001', flags = 0 '\000', u = {rdymsg = 0, exitcode = 0, wtobjp = 0x0,
wttrp = 0x0, wtsemp = 0x0}}
(gdb)
Re: Thread priorities...
Posted: Tue Sep 17, 2019 8:23 pm
by Giovanni
Hi,
You should enable statistics and verify the worst case execution path for each thread assuming it is not an M0, in that case no statistics, you may try the profiler but it is way less accurate.
Also make sure that the state checker and assertions are enabled, those would reveal if there are problems with the scheduler caused by invalid API usage.
Giovanni
Re: Thread priorities...
Posted: Tue Sep 17, 2019 9:16 pm
by rew
Alas, I have an M0.
(stm32F072 is my favorite processor: has USB, has DAC but is still quite economical to purchase). (I feel weird when I buy a processor with 200k RAM and 1M flash for EUR 10 (F405), while I could also buy a processor (H2+), PCB and 256Mb RAM for that amount of money (orange pi zero)).
I put a "toggle pad" in the idle thread. That toggles at 2MHz all the time, with gaps for when interrupts happen or other threads run. In the logic analyser I can clearly see the idle thread running.
The thread that I think blocks when it should not has:
Code: Select all
set_bit (PB11);
chThdSleep (5);
clr_bit (PB11);
and I see the PB11 high while the idle thread is happily toggling away....
Re: Thread priorities...
Posted: Tue Sep 17, 2019 11:19 pm
by mikeprotts
I've tended to avoid use of any sleeps, I prefer to suspend and resume threads so I can be more certain of the current state when code runs. I try to only use sleep when there's a problem (e.g. to blink an LED at a set rate).
It might be worth developing code on a processor that allows more diagnostics, but only using features that will work on a cheaper processor, to make diagnosing problems easier.
The debugger can also be confusing unless you've also looked at how the compiler has optimised the code (the .list file) - this is often quite revealing. I've found a logic analyser (with a few additional pins used as GPIO to track state) to be very helpful in diagnosing problems with the order or timing of specific code sections.
Mike
Re: Thread priorities...
Posted: Wed Sep 18, 2019 7:36 am
by rew
If I make the interrupt that handles the IO do a resume of the thread, the interrupt code would need to lock and unlock. I don't like that.
One of the reasons is that I like to keep the interrupt routines small and simple. For a processor like the M0, that helps keeping the save-state at the beginning of the interrupt small.
It pays to optimize the code that runs thousands of times (or takes a long time) as opposed the code that runs much less often or doesn't take that long. So in this case, the interrupt runs tens of thousands of times per second while the thread needs to wake up less than 50 times per second. If the latter is slow: so be it.
Re: Thread priorities...
Posted: Sun Oct 20, 2019 5:57 pm
by rew
Ok. Guys...
Different project, same issue: A thread that does "sleep" sometimes never wakes up again.
Context: CPU: an STM32F030F4.
In my current project, I use the UART, but I've written my own special purpose driver. I need a few things that the chibios drivers don't support.
The "sleep never returns" issue suddenly went away when I initialized the UART interrrupt with priority 2 instead of priority 3.
I have considered an "interrupt storm" situation. But then when I'd stop my CPU from GDB, I would expect it to show it executing the interrupt routine, not sleeping in _idle_thread.
Also, as far as I can tell, when my main program is mostly
Code: Select all
while (1) {
chThdSleepMilliseconds (500);
// test some pins and variables, set two GPIO pins in response.
}
with the "test some pins" being very short, I would expect to catch it 99% of the time in the sleep, right?
And when it's sleeping I would expect *ch.vtlist->next to have that delta of 500. But when it crashes ch.vtlist->next points back to vtlist and the delta is -1.
Hmm.... What is the units of that "delta" variable? Wouldn't it be ten-thousandths of a second? so when I sleepMilliseconds I'd expect it to be 5000, and not 500. What am I missing?
Does anybody have an idea about how this could happen?
Re: Thread priorities...
Posted: Sun Oct 20, 2019 6:12 pm
by Giovanni
Questions:
1) Do your ISRs have the prologue/epilogue macros?
2) What compiler are you using?
3) What ChibiOS questions?
Giovanni
Re: Thread priorities...
Posted: Sun Oct 20, 2019 6:25 pm
by rew
prologue/epilogue: Ooops!
I've apparently started from an interrupt routine from before that was standard. Now added. Now testing. .... Update: That helped!
Now it also works with my USART interrupt running at the 3 level. Thanks for the suggestion!
Compiler is:
arm-none-eabi-gcc (15:4.9.3+svn231177-1) 4.9.3 20150529 (prerelease)
The Chibios questions are:
How could it be that "sleepMilliseconds" doesn't return?
and:
Howcome I see "500" in the "delta" of the first real vtlist entry when I'm doing 500ms sleeps?