STM32F407 + USB + ULPI Interface

[chradev]

Hi Giovanni,

In my opinion the queue usage is the main problem.
I can share the project (STM32Cube based) used to measure throughput if it can help.
In addition I will make respective tests if you add ability to use linear buffers.
It is important the size of the buffers to be controlled by the user.

Best regards
Chris

[Giovanni]

Hi,

Linear buffers are already usable on the USB driver, just not used by the CDC driver.

Rewriting the CDC driver to use linear buffers is complex and it would take more RAM, you would need 4 large buffers, 2 for TX and 2 for RX. The queued CDC driver is very simple in comparison.

Giovanni

[chradev]

Hi Giovanni,

As I note before current CDC driver has a throughput much less than 1 MBps which is unacceptable even for USB FS.
If rewriting the CDC driver to use linear buffers is complex for you it is definitely beyond the strengths for regular users like me.
On the other hand Ethernet over USB driver (RNDIS for example) with LWiP is more appropriate scenario for higher speeds.

About the need of 4 large buffers as I can see in STM32Cube only 2 buffers are used and it works quite well.
The only side effect is that the application has to care about data transmission progress to use them all again.
This side effect can be overcame in case of callbacks at both half and full buffer transmission events.

And finally, the usage of 4 buffers is acceptable price if one needs of speeds higher than 100 Mbps LAN.
As you can see from the graphics throughput more than 24 MBps is achievable at 16 kB transmit buffer.
Even for buffers as small as 2 kB the throughput is more than theoretical one for 100 Mbps LAN.

It is another story about large receive buffers.
Most of the cases requiring large transmit speeds really has a small receive once and vice versa.
That is why the user having buffer size control can balance memory usage vs required speeds.

Best regards
Chris

[Giovanni]

The problem is CDC has to be efficient for both small read/writes and large blocks. it strongly depends on the application behavior and how data is handled.

In applications requiring high bandwdth probably the best solution is to not use the Serial_USB driver because it intrinsically requires queues (being an emulation of a serial port). The USB API for linear buffers can already be used for large data transfers.

Giovanni

[Giovanni]

Hi,

Small update. I decided to try some changes in the USB/CDC driver in order to address the performance issue.

Some points:

1) The fastest way to use USB is through linear API, this will not change.
2) The CDC driver is convenient because the serial-like API.
3) Queues are too slow for USB because data is copied twice and it is done one byte at time.

The idea is to improve the speed of CDC driver and get it closer in performance to the USB linear API. This is how I am proceeding:

1) Introduce a new object type called "buffers queue". It is a queue of large buffers that is written/read in blocks from one side and presents an interface similar to queues on the other side. It is a general enough object so it has its own place in the HAL.
2) Modify the CDC driver to use the new buffered queue.
3) Remove the queued API from the USB driver because: a) driver simplicity, b) the linear API allows for use of DMAs, this is currently prevented by the queued API. It was a sorry point in the driver model.

We will see if there is a consistent advantage.

Giovanni

[chradev]

Hi Giovanni,

It is good news.Proposed solution is very interesting because it can be applied at other cases as well.
It will be big advantage if both linear and circular buffers can be supported in the new object.

I am ready to help in testing and assessment phases of the implementation so please keep me informed about your plans and progress.

Currently I have a project in which big amounts of data have to be transferred from ADC to remote network client.
Intermediate digital processing will be chained in different threads so proposed solution is perfect in such a case.

BTW this project is the candidate for networking via HS USB with external ULPI PHY.

Best regards
Chris

[Giovanni]

Hi,

I committed some code about the new object, basically it is a circular queue of static buffers, depth of the queue and (maximum) size of the buffers can be defined. It can be used in any scenario requiring moving/buffering blocks of data from ISR to thread context and/or vice versa.

It is patterned after the byte-oriented queue. Data can be fetched/written in the queue both using the usual get/put/read/write paradigm or by pointer in copy-less mode.

Giovanni

[DeusExMachina]

It's nice that finally Chibios has ULPI HS support!
BTW I have a question: how about 2.6.x branch? Do you plan to backport new ULPI driver there?
Thanks

[Giovanni]

Hi,

It is not planned, 2.6 is in maintenance mode. I recommend a switch to 3.0.

Giovanni

[Giovanni]

Hi,

I committed the changes related to the new buffering mechanism. Only the STM32F4xx-USB_CDC demo has beed updated for it.

The only change at application level is that the SOF callback must now call sduSOFHookI(), it is used for flushing partially filled buffers.

There could be bugs, the code is not even final but it can be used for a first assessment of performance.

There are two configurable parameters: buffers size (same as before, must be a packet multiple) and number of buffers per queue, defaulted to 2, it is not clear if deeper queues can give better performance.

Giovanni