[NEWS] The ChibiOS Book

[Ceco]

Thanks for the book access, Giovanni.

I have read it in a fast manner and I can give some sort of feedback now. HAL is more or less well documented in doxygen. My main problems are to understand kernel usage logic.

What is missing in Chibios documentation (or what is miss me) is a simple kernel usage tutorial. But not in the way of doxygen format, more like a guide with examples. Your system, more or less has an unique kernel conception. Or let's say unique in user interface. For examlpe:

chSYSLock();
chMBPostS();
chSysUnlock();

I have an experience with few other OS and such a code structure looks strange for me on first view. For most of the people the scheduler/interrupts are blocked for entire section and this code is wrong. After a deep digging in the forum, I found the explanation. In my opinion it should be written clearly some where in documentation that context switch is happens inside. Because immediately the next question arrives - OK, where the "protection" of the section finishes? At the exit of wait function or after sys unlock? What about if I have second wait in this section, is it also protected or not? And why I need to lock kernel before posting a message? I understand that such an interface give us a simple and fast code, but it must be explained in deeper details to be used correctly, because to produce a good code, the user needs to know the kernel operation strategy.

I have similar doubts with mailboxes and memory pools. In most of the OS I have touch, mailbox and memory pool are combined in an abstraction level to automate usage. So for the user all memory pool operation is hidden. I understand your implementation - abstraction always reduces efficiency of the code. But again - it should be documented well, since for the 90% of new users this is an unknown mailbox interface (even they have experience with other OS).
The same with memory pool - for most of the users, memory pool is exactly what you have implement in one of the trunk revisions - mail pool. This is the common memory pool interface known from the other OS. I don't know why you decide to kill this part of the code, for me this is something really useful and well organized from user point of view.

All these features of the OS makes the user interface of Chibios more or less unique. And exactly these features need to be documented more detailed and with a simple and straightway examples. I found a lot of examples in demos and test, but most of them don't give a clear view, since they are specified and optimized for a specific usage which not covers 90% of the user needs in real development. For me this way of education is not correct - the example must give us the simplest and clear way to use a given primitive. The user will use this for a base and built what he/she needs. The examples in test code cannot be used for basic education. The code there is already "deformed" for specific usage.

Another section which miss documentation is the interrupts handling. In my opinion this is the part where most of the beginners has the biggest problems. Comparing with other OS - Chibios is really well designed for reaction time. For example MQX and Nuttx has a common interrupt vector which makes a critical time reaction more or less impossible. Nuttx don't support fast interrupts at all (or not support them on usable level). To have a Cortex on 180Mhz and to have a reaction times of 10 us looks crazy

You are on the right way - the efficient code. But the lack of abstractions needs a better documentation for usage. Because we have more freedom to make a mistakes also

In fact Chibios is more or less unique in the following - we have freedom and efficiency with a maximum possible level of abstraction (without efficiency reduction). This is why I like it.

A part of these questions will be resolved with CMSIS-RTOS abstraction layer. But I don't like this conception at all, so I'm not sure this is the right way to find the answers.

To be absolutely clear - this is not a critic! You are making a great job! This is a proposition of a person which has an outside view of the system. These are the problems/questions that I have. I think in the book you need to focus on this point a bit more - kernel/synchronization usage tutorial with simple examples coming from real life development. Where we need to lock? Where we don't need to lock? How to create an usable mailbox for an arbitrary structure and thread to thread communication (interrupt to thread)? And etc....

Regards
Ceco

[RoccoMarco]

Hi Ceco,
I have read the Preliminary book and it seems like in this book Giovanni would not explain how ChibiOS works but how an RTOS should be.

Most likely this is just a preface to another book, something like "what you should know to understand the idea behind ChibiOS".

You are talking about mailboxes, pool, irq handlers. Don't forget that some users don't even know about the existence of these mechanisms.

Anyway I will be much interested about demos on kernel

regards,
RM

[Giovanni]

Hi Ceco,

A looong answer but I appreciate feedback, so far you are the only one who gave back info out of a dozen of guests.

About your points:

I have read it in a fast manner and I can give some sort of feedback now. HAL is more or less well documented in doxygen. My main problems are to understand kernel usage logic.

The plan is to have the HAL covered in a second book with a much more "hands on" approach: experiments, interfacing HW etc.

What is missing in Chibios documentation (or what is miss me) is a simple kernel usage tutorial. But not in the way of doxygen format, more like a guide with examples. Your system, more or less has an unique kernel conception.

Correct, it is a different approach used in order to satisfy performance requirements. I hope to expand on this in the concepts chapter.

Or let's say unique in user interface. For examlpe:

chSYSLock();
chMBPostS();
chSysUnlock();

I have an experience with few other OS and such a code structure looks strange for me on first view. For most of the people the scheduler/interrupts are blocked for entire section and this code is wrong. After a deep digging in the forum, I found the explanation. In my opinion it should be written clearly some where in documentation that context switch is happens inside. Because immediately the next question arrives - OK, where the "protection" of the section finishes? At the exit of wait function or after sys unlock? What about if I have second wait in this section, is it also protected or not? And why I need to lock kernel before posting a message? I understand that such an interface give us a simple and fast code, but it must be explained in deeper details to be used correctly, because to produce a good code, the user needs to know the kernel operation strategy.

The system states diagram is supposed to make clear how the "lock zones" work, S-class functions can perform a context switch inside so a lock...unlock sequence is not necessarily atomic because an unlock can be performed on the "other side". You could have a thread entirely within a lock only calling S-class functions but that would not mean that the interrupts are disabled forever.

I have similar doubts with mailboxes and memory pools. In most of the OS I have touch, mailbox and memory pool are combined in an abstraction level to automate usage. So for the user all memory pool operation is hidden. I understand your implementation - abstraction always reduces efficiency of the code. But again - it should be documented well, since for the 90% of new users this is an unknown mailbox interface (even they have experience with other OS).
The same with memory pool - for most of the users, memory pool is exactly what you have implement in one of the trunk revisions - mail pool. This is the common memory pool interface known from the other OS. I don't know why you decide to kill this part of the code, for me this is something really useful and well organized from user point of view.

Mailboxes+pools are only a single use case for mailboxes and pools, this is why we have two distinct mechanisms. I decided that something that could be build entirely on top of existing functionalities goes against to a minimalist approach. I didn't like it much for some reason but I wanted to return on it at some point.

All these features of the OS makes the user interface of Chibios more or less unique. And exactly these features need to be documented more detailed and with a simple and straightway examples. I found a lot of examples in demos and test, but most of them don't give a clear view, since they are specified and optimized for a specific usage which not covers 90% of the user needs in real development. For me this way of education is not correct - the example must give us the simplest and clear way to use a given primitive. The user will use this for a base and built what he/she needs. The examples in test code cannot be used for basic education. The code there is already "deformed" for specific usage.

Unlike the system states and function classes the primitives like semaphores, mutexes, condvars, mailboxes are pretty standard. The exception could be things unique to ChibiOS like synchronous messages and events. I plan to expand examples on those.

Another section which miss documentation is the interrupts handling. In my opinion this is the part where most of the beginners has the biggest problems. Comparing with other OS - Chibios is really well designed for reaction time. For example MQX and Nuttx has a common interrupt vector which makes a critical time reaction more or less impossible. Nuttx don't support fast interrupts at all (or not support them on usable level). To have a Cortex on 180Mhz and to have a reaction times of 10 us looks crazy

You are on the right way - the efficient code. But the lack of abstractions needs a better documentation for usage. Because we have more freedom to make a mistakes also

I thought interrupts handling was well covered already, what would you add? most examples include ISRs.

In fact Chibios is more or less unique in the following - we have freedom and efficiency with a maximum possible level of abstraction (without efficiency reduction). This is why I like it.

Definitely a strong point.

A part of these questions will be resolved with CMSIS-RTOS abstraction layer. But I don't like this conception at all, so I'm not sure this is the right way to find the answers.

CMSIS RTOS is there just because ARM decided to create that thing, personally I don't like it much but since ChibiOS can do that without effort then why not.

To be absolutely clear - this is not a critic! You are making a great job! This is a proposition of a person which has an outside view of the system. These are the problems/questions that I have. I think in the book you need to focus on this point a bit more - kernel/synchronization usage tutorial with simple examples coming from real life development. Where we need to lock? Where we don't need to lock? How to create an usable mailbox for an arbitrary structure and thread to thread communication (interrupt to thread)? And etc....

Regards
Ceco

I can handle criticism please feel free to suggest things or examples to add, the book is far from complete.

I have read the Preliminary book and I think that in this book Giovanni would not explain how ChibiOS works but how an RTOS should be.

Not correct, I am trying to explain how ChibiOS/RT is, not going into comparisons, I'll leave that to users.

Giovanni

[Ceco]

Hello Giovanni.

For me, the biggest miss in documentation is examples according OS states. For example it's really helpful to have an example showing exactly how the kernel lock/unlock, interrupts and context switching deals together. I'm not an OS expert, but I have a good experience like a OS user. Even of this I have to read a lot of forum messages and dig deeply in the code, to understand how it works. And I'm still not sure that I understand it absolutely correctly. But like I told before - Chibios gives the freedom of efficient and simple code but it also gives a freedom to make a mistake if you don't understand correctly how the kernel works. Most of the other OS has an abstraction of this part which prevents many mistakes (in a price of efficiency) but all these mistakes are possible with Chibios. I think one time diagram showing a case like previously mentioned (lock, context switch, unlock) which shows exactly where the kernel is locked/unlocked, will help a lot to the beginners. Some examples which explains where we need to work in locked zone, and where we don't need will also be helpful for beginners.

It's the same for state switching during interrupts (I state) and context switch at interrupts finish. The state diagrams don't give a clear view what exactly happens - they just shows possible transitions. Maybe your idea was to not document things which are not important for usage - just to give APIs to the user. But exactly in this case it's extremely important to know how the things happen, since the possibility to make an error is bigger.

Fast interrupts is something which need to be documented in a deeper details also. It's sure that 90% of the users, will never use it. But it's extremely important when microseconds reaction time is needed. And this is a part in which Chibios has an advantages over many other OS. But to use it - you need to understand clearly how it works - how to manipulate the NVIC levels for example. Of course this topics covers mainly Cortex M users, but In my filling this is the biggest part of Chibios users at all.

According mailboxes and memory pools - I understand why you made them on separate modules. But since this is a something unusual for other OS-es it must be documented with clear examples how to use them. For example for most of the people is strange that Memory pool must be Filled before usage, because most of other OS-es hides this operation. They need to dig inside the code to see that this "fill" operation just makes a chain of pointers. But not all of them has a knowledge to dig in the code. 90% of mailbox usages in practice is to transfer an arbitrary structure between threads or between thread and ISR. A clear example how to organize this with Chibios will be very helpful, I believe.

For me each synchronization primitive should have one simple and clear example how to use it. This is also valid for thread APIs and Kernel APIs. If you look in commercial OS - this is the biggest part of documentation and most of the users just search for such an examples in Chibios also. Doxygen is perfect way to document but for an advanced user, which just need an reference for APIs.

One interesting addition on HAL level will be SWO debug output. Again this is important for Cortex M users, but normally this means most of the users.

Maybe some of my suggestions looks stupid to you or more advanced users, but this is because you have the entire conception in your head. For a user which looks from the outside some parts looks confusing and a couple of clear and simple examples will help a lot.

[jcw]

If you look in commercial OS

At the risk of stating the obvious... ChibiOS is open source, free, and sent into the world as a gift. The driving force is generosity, not commerce.

Wouldn't it be more effective to write examples, and post them here or send them to Giovanni for review and discussion? As it is, this thread is starting to read like a long list of wishes, which may serve to explain the need but does very little to help the next person along.

I too struggle with concepts and details in ChibiOS, but I'm not sure this discussion is filling in the rough spots. It's mostly pointing them out.

How about steering all the recent energy in this thread into a slightly different direction, by writing some examples and reviewing / discussing those instead? I'm sure I'd learn from it, and I'll be happy to try and participate where I can.

I haven't read Giovanni's book yet, BTW, so I'll stop here.

-jcw

[Ceco]

Yes, you are right - it starts to look like a list of wishes. But for the moment I don't feel myself strong enough in Chibios to propose the "right way to do the things", i.e. examples. That's why I'm just marking the points in which I feel a lack of information.

[Giovanni]

Yes, you are right - it starts to look like a list of wishes. But for the moment I don't feel myself strong enough in Chibios to propose the "right way to do the things", i.e. examples. That's why I'm just marking the points in which I feel a lack of information.

Feel free to propose things you would like to see in the book or enhancements to existing parts. The work is just half done so far.

At the risk of stating the obvious... ChibiOS is open source, free, and sent into the world as a gift. The driving force is generosity, not commerce.

It is open source but it is also a commercial product with customers and it is a good thing, because the project is, at least, self sustaining. The fun thing is that we don't do advertising, all customers are people that tried ChibiOS and decided to invest on it, some of them are surprisingly important companies too.

Wouldn't it be more effective to write examples, and post them here or send them to Giovanni for review and discussion? As it is, this thread is starting to read like a long list of wishes, which may serve to explain the need but does very little to help the next person along.

I too struggle with concepts and details in ChibiOS, but I'm not sure this discussion is filling in the rough spots. It's mostly pointing them out.

How about steering all the recent energy in this thread into a slightly different direction, by writing some examples and reviewing / discussing those instead? I'm sure I'd learn from it, and I'll be happy to try and participate where I can.

This could be a good approach. A couple years ago I wanted to start working on an "open source" book but the idea didn't get momentum and died. This is why I am going for a more traditional approach now.

Giovanni

[barthess]

some of them are surprisingly important companies too

Could you write some company names? I want to have ability to say "Dude, %COMPANY% uses ChibiOS.", when he says "What the ChibiOS? Never heard."

Some ideas about simple demos:
Discovery board. 1th thread polls button. 2nd blinks (or not blinks) LED. Data from 1th to 2nd passes using different OS's mechanisms (semaphores, mailboxes, events, etc).

[Giovanni]

Could you write some company names? I want to have ability to say "Dude, %COMPANY% uses ChibiOS.", when he says "What the ChibiOS? Never heard."

Some ideas about simple demos:
Discovery board. 1th thread polls button. 2nd blinks (or not blinks) LED. Data from 1th to 2nd passes using different OS's mechanisms (semaphores, mailboxes, events, etc).

I cannot disclose most of them or I would have created a page

About your suggestion, it will be material for the 2nd book, which will be about the HAL with lots of applications and hands-on experiments using some boards (discovery or nucleo most likely), the problem is that without introducing the HAL even accessing a LED is problematic, you can see that from the existing examples. The alternative would be to handle the HAL in the same book.

Giovanni

[reportingsjr]

Do you still want feedback on the book?

I will happily read over it and try to give advice.