Example of memory pool + struct + mailbox.

[kermitas]

Hello Giovanni and community.

Can you provide few simple lines of code demonstrating how to:

- initialize memory pool
- allocate some memory (from memory pool), sizeof( BasicMessage )
- create new instance of BasicMessage struct
- put this instance into this allocated memory
- then free memory (released it into memory pool)
?

Here is my struct:

Code: Select all

typedef struct {
int code;
void* ptr;
} BasicMessage;

BasicMessage structure is message that will be send to Mailbox and received by the thread. After procesing this message thread should free allocated memory (release it back to the memory pool).

Thank you
Arthur.

[Giovanni]

It looks like what you described could be useful in a lot of situations, I will add this as a library module and let you have fun debugging it

Giovanni

[Badger]

Here's a quick run down of how I'd do it:

Code: Select all

MemoryPool pool;

// align data, this depends on your compiler. this works for GCC
BasicMessage data[N] __attribute__((aligned(sizeof(stkalign_t))));

// initialise pool
chPoolInit(&pool, sizeof(BasicMessage), NULL);

// add each element to the pool
// this is the least obvious part of the process, as far as my experience went
for(int i=0; i < N; i++)
    chPoolFree(&pool, &data[i]);

while(true) {
    // get an object
    BasicMessage *msg = (BasicMessage *)chPoolAlloc(&pool);
    // check we succeeded in allocation
    if(msg != NULL) {
        // do some stuff
        // release it
        chPoolFree(&pool, msg);
    }
}

I haven't tested that code, but it is loosely copied from my C++ Notifications Class.

[kermitas]

It looks like what you described could be useful in a lot of situations, I will add this as a library module and let you have fun debugging it

Giovanni

"Library module"? Where will I find it? In SVN?
When?

I can't wait!

(Now I have a lot of free time so that is why I am fighting with ChibiOS everyday, my board is a STM32F4Discovery.)

[kermitas]

Here's a quick run down of how I'd do it:

Code: Select all

MemoryPool pool;

// align data, this depends on your compiler. this works for GCC
BasicMessage data[N] __attribute__((aligned(sizeof(stkalign_t))));

// initialise pool
chPoolInit(&pool, sizeof(BasicMessage), NULL);

// add each element to the pool
// this is the least obvious part of the process, as far as my experience went
for(int i=0; i < N; i++)
    chPoolFree(&pool, &data[i]);

while(true) {
    // get an object
    BasicMessage *msg = (BasicMessage *)chPoolAlloc(&pool);
    // check we succeeded in allocation
    if(msg != NULL) {
        // do some stuff
        // release it
        chPoolFree(&pool, msg);
    }
}

I haven't tested that code, but it is loosely copied from my C++ Notifications Class.

What a nice piece of code! Tank you Bagder! This is what I need .

I have few questions:

Code: Select all

// align data, this depends on your compiler. this works for GCC
BasicMessage data[N] __attribute__((aligned(sizeof(stkalign_t))));

1) why we need to do "align data" ?

Code: Select all

// initialise pool
chPoolInit(&pool, sizeof(BasicMessage), NULL);

2) is this second argument a size of each block in the pool? if yes then what maximum count of blocks can hold this pool?
3) if third (last) argument is NULL then this list cannot grow - am I right? if yes then how to write grovable memory pool?

Code: Select all

typedef struct
{
int code;
void* ptr;
} BasicMessage;

4) if BasicMessage contains integer and pointer then I can't see any place where we pass this number and some pointer into a new "instance" , where to do that?

5) if I want to send to mailbox a task that will be executed only once (for example: in the start of system write "hello") then should it be BasicMessage allocated staticaly or by memory poll/malloc?

Code: Select all

// add each element to the pool
// this is the least obvious part of the process, as far as my experience went
for(int i=0; i < N; i++)
    chPoolFree(&pool, &data[i]);

6)hmmm I am starting to thinking that I am completly doesn't undesrstand memory pools , here is what I want to do:
- thread receive messages from it's Mailbox
- messages in Mailbox are BasicMessage
- it task (task code is in "int code" in BasicMessage) to execute by the thread is to execute many times then I think it should be alocated statically
- if task to execute by the thread is one-time task then I think it should by allocated dynamically -> for this purpose I thought I should use memory pools but maybe it should be malloc() and after execution free()?

Thank you very much for your time,
Arthur.

PS:

I haven't tested that code, but it is loosely copied from my C++ Notifications Class.

OMG! You are those guy from this example ! I study it every day! I also considered to write in C++ style (my brain is object oriented after many years in Java and Scala) but then Giovanni wrote that C++ wrappers are old and small subset of ChibiOS so I leave this idea . What is your experience? Did you have to write all wrappers by yourself? Why did you wrote in C++ when everybodoy will tell you that in embeddes systems you have to use pure C??

What is your board? My is STM32F4Discovery, great, powerful, fast and cheap.

And the most important question. Can whole of your code be publictly viewable (like on Github? Bitbucket? ect) ??

[Badger]

1) why we need to do "align data" ?

I'm not entirely sure on the reasoning for this, but if you have debug enabled then the asserts in the memory pool code will halt the system. Maybe Giovanni can explain better?

2) is this second argument a size of each block in the pool? if yes then what maximum count of blocks can hold this pool?

yes it is the size of the memory objects in the pool. I think the upper limit is probably your mcu memory..

3) if third (last) argument is NULL then this list cannot grow - am I right? if yes then how to write grovable memory pool?

It can still grow by adding more with chPoolFree, but there is no function called whenever data is allocated. I think you only need this if you're doing some advanced allocation, e.g using several memory pools from one block of data

4) if BasicMessage contains integer and pointer then I can't see any place where we pass this number and some pointer into a new "instance" , where to do that?

I'm not sure I understand this question; are you asking how to put data into the structure? you can of course expand BasicMessage to be as long as you want.

5) if I want to send to mailbox a task that will be executed only once (for example: in the start of system write "hello") then should it be BasicMessage allocated staticaly or by memory poll/malloc?

I'd avoid the use of malloc unless you really need it. In the case of using memory pools in the example I provided, the memory is statically allocated as at the beginning it defines an array of objects of length N.

6)hmmm I am starting to thinking that I am completly doesn't undesrstand memory pools , here is what I want to do:
- thread receive messages from it's Mailbox
- messages in Mailbox are BasicMessage
- it task (task code is in "int code" in BasicMessage) to execute by the thread is to execute many times then I think it should be alocated statically
- if task to execute by the thread is one-time task then I think it should by allocated dynamically -> for this purpose I thought I should use memory pools but maybe it should be malloc() and after execution free()?

You can achieve this easily using memory pools. Your server thread uses chPoolAlloc to get a new BasicMessage pointer, and checks it isn't NULL. It adds in the data and posts it to your mailbox. Your client thread waits on that mailbox, receives a mail and gets out the pointer by casting the message to BasicMessage*. Once it is done with the data the client thread calls chPoolFree, not the server. This is an important part of the mechanism!

That's the basic structure used in the C++ class I posted, you should be able to get the relevant parts from it.

OMG! You are those guy from this example ! I study it every day! I also considered to write in C++ style (my brain is object oriented after many years in Java and Scala) but then Giovanni wrote that C++ wrappers are old and small subset of ChibiOS so I leave this idea . What is your experience? Did you have to write all wrappers by yourself? Why did you wrote in C++ when everybodoy will tell you that in embeddes systems you have to use pure C??

It is true that the C++ wrappers for Chibios represent a small subset, but you can still call all the C functions, so you don't lose any functionality, but gain the ability to use object oriented classes which may be of benefit in your application! You should note that it is not easy (or probably recommended) to use some features of C++ on an embedded system, e.g new and delete.

What is your board? My is STM32F4Discovery, great, powerful, fast and cheap.

My board is a custom PCB with some inertial sensors and an RF modem (RFM22B).

And the most important question. Can whole of your code be publictly viewable (like on Github? Bitbucket? ect) ??

Not yet, it is mostly specific to a research project I'm working on and is not ready for release, sorry!

[kermitas]

...

Great answer .
Thank you that you tryed to answer all my questions.

[Giovanni]

I am thinking to create a generic "dynamic mail" mechanism built on top of pools, counting semaphores and mailboxes. The "mail" would be a structure defined by the user, there could be more mail types.

it would have 4 main functions:

On the sender side:
mailAlloc() - Allocates a mail object from the pool, if the pool is empty waits on a semaphore.
mailSend() - Sends a mail object using the mailbox.

On the receiver side:
mailReceive() - Waits for a mail to arrive from the mailbox.
mailRelease() - Returns the mail to the pool and eventually signals a sender.

This would allow the exchange of any kind of objects without having to worry about allocation and synchronization.

This could be part of the generic mailboxes but would create a dependency between them and sems+pools, I want to keep modularity so it has to be a generic module on top of those.

Giovanni

[kermitas]

I am thinking to create a generic "dynamic mail" mechanism built on top of pools, counting semaphores and mailboxes. The "mail" would be a structure defined by the user, there could be more mail types.

it would have 4 main functions:

On the sender side:
mailAlloc() - Allocates a mail object from the pool, if the pool is empty waits on a semaphore.
mailSend() - Sends a mail object using the mailbox.

On the receiver side:
mailReceive() - Waits for a mail to arrive from the mailbox.
mailRelease() - Returns the mail to the pool and eventually signals a sender.

This would allow the exchange of any kind of objects without having to worry about allocation and synchronization.

This could be part of the generic mailboxes but would create a dependency between them and sems+pools, I want to keep modularity so it has to be a generic module on top of those.

Giovanni

Great tool to exchange data between any parts of the system (including threads).
Does SVN repository is best place to see your latest works (including this described upper)?

In future we can provide something similar to "actors" where you have thread pool and many mailboxes. You can process messages from mailboxes (sometimes hundreds or thousands) having only 1-or-2 threads in the pool or having 10 threads. When mailbox receive message look to the thread pool if there is free thread. If yes use it to process all messages. Each actor have its own context (set of variables). Actors described on Akka pages (Akka=actors for Scala and Java) http://doc.akka.io/docs/akka/2.0/general/actors.html.

Thanks,
Arthur.

[Giovanni]

"Thread pools" is something we already have in the to-do list, they will eventually appear at some point.

Giovanni