Giovanni,
After sleeping over it, I honestly think that the MAC low level driver receive buffer front padding is as of now, by far, the BEST option to implement zero-copy reception.
Rationale:
1- It is
dead easy to implement receive buffer front padding in a low level driver which is already capable of zero-copy.
2- Allows use of ETH_PAD_SIZE, which would be impossible otherwise (see note 1).
3- Makes the lwipthread zero-copy code SUPER simple:
3.1- You do not need a separate pool for the struct pbuf, just need to initialize it in the padding area of the buffer that mac_lld_get_next_receive_buffer returns.
3.2- You do not need to worry about the separate pool having the same number of elements as MAC low level driver buffers (less will result in dropped packets, more will waste memory).
4- Requires a trivial, minimal patch to lwIP (see note 2). This is good because (a) one can be pretty much sure it does not break anything and (b) the patch is dead easy to maintain and port to future lwIP versions (see note 3).
Note 1: remember that the buffers are owned by the low level driver, there is no way you can place additional ETH_PAD_SIZE space in front of it because the buffer is given to you already filled with received data.
Note 2: the patch is trivial because the mechanism allows receive pbufs to work EXACTLY as normal PBUF_RAM/PBUF_POOL where the struct pbuf is contiguous with the actual buffer. Separating them (as is done in PBUF_ROM/PBUF_REF) would require a more elaborated patch and an INSANE amount of testing to make sure no lwIP system is broken because it makes idiotic assumptions.
Note 3: we could sit and wait forever for lwIP developers to implement and test the required pbuf management changes. From what I have read, users have been requesting exactly that at least from 2003 (see
http://lists.gnu.org/archive/html/lwip-users/2003-03/msg00085.html).
BTW, I though that now that I'm down deep into this I may as well implement zero copy transmission, however, I think it's not going to be very useful without huge, deep and inpractical changes in lwIP because of the usual special requirements of MAC periphreals regarding the DMA buffers (alignment, size, memory areas, etc). I believe the STM32 MAC is an exception regarding size and alignment, but still can read only from SRAM, and not all SRAM areas. I could make the driver "reject" a buffer located in ROM and force lwipthread to make a copy of the pbuf in SRAM, or have an internal scratch area and do the copying internally, but the code would be very complex.
That said, I think there is some room for easy improvement of the transmit path: instead of N transmit buffers of 1522 bytes, allocate just a monolithic single buffer and use it as a ring buffer. That way, if frames are small you can place many more in it and return immediately.