[STM32F7] Performance Reports....

This forum is dedicated to feedback, discussions about ongoing or future developments, ideas and suggestions regarding the ChibiOS projects are welcome. This forum is NOT for support.
Post Reply
User avatar
Giovanni
Site Admin
Posts: 14894
Joined: Wed May 27, 2009 8:48 am
Has thanked: 1202 times
Been thanked: 997 times

[STM32F7] Performance Reports....

Post by Giovanni »

Hi,

Just FYI, I have preliminary code working now....

Code: Select all


*** ChibiOS/RT test suite
***
*** Kernel:       3.0.0
*** Compiled:     Aug  3 2015 - 15:30:46
*** Compiler:     GCC 4.7.4 20140401 (release) [ARM/embedded-4_7-branch revision 209195]
*** Architecture: ARMv7E-M
*** Core Variant: Cortex-M7
*** Port Info:    Advanced kernel mode
*** Platform:     STM32F746 Very High Performance with DSP and FPU
*** Test Board:   STMicroelectronics STM32F746G-Discovery

----------------------------------------------------------------------------
--- Test Case 1.1 (System, critical zones)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 1.2 (System, interrupts handling)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 1.3 (System, integrity)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 2.1 (Threads, enqueuing test #1)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 2.2 (Threads, enqueuing test #2)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 2.3 (Threads, priority change)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 2.4 (Threads, delays)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 3.1 (Semaphores, enqueuing)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 3.2 (Semaphores, timeout)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 3.3 (Semaphores, atomic signal-wait)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 3.4 (Binary Semaphores, functionality)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 4.1 (Mutexes, priority enqueuing test)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 4.2 (Mutexes, priority return)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 4.3 (Mutexes, status)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 4.4 (CondVar, signal test)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 4.5 (CondVar, broadcast test)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 4.6 (CondVar, boost test)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 5.1 (Messages, loop)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 6.1 (Mailboxes, queuing and timeouts)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 7.1 (Events, registration and dispatch)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 7.2 (Events, wait and broadcast)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 7.3 (Events, timeouts)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 8.1 (Heap, allocation and fragmentation test)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 9.1 (Memory Pools, queue/dequeue)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 10.1 (Dynamic APIs, threads creation from heap)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 10.2 (Dynamic APIs, threads creation from memory pool)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 10.3 (Dynamic APIs, registry and references)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 11.1 (Queues, input queues)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 11.2 (Queues, output queues)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.1 (Benchmark, messages #1)
--- Score : 1145926 msgs/S, 2291852 ctxswc/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.2 (Benchmark, messages #2)
--- Score : 886360 msgs/S, 1772720 ctxswc/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.3 (Benchmark, messages #3)
--- Score : 747815 msgs/S, 1495630 ctxswc/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.4 (Benchmark, context switch)
--- Score : 4592984 ctxswc/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.5 (Benchmark, threads, full cycle)
--- Score : 633882 threads/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.6 (Benchmark, threads, create only)
--- Score : 1112681 threads/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.7 (Benchmark, mass reschedule, 5 threads)
--- Score : 334679 reschedules/S, 2008074 ctxswc/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.8 (Benchmark, round robin context switching)
--- Score : 2468520 ctxswc/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.9 (Benchmark, I/O Queues throughput)
--- Score : 3488828 bytes/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.10 (Benchmark, virtual timers set/reset)
--- Score : 3397730 timers/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.11 (Benchmark, semaphores wait/signal)
--- Score : 2399808 wait+signal/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.12 (Benchmark, mutexes lock/unlock)
--- Score : 2949572 lock+unlock/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.13 (Benchmark, RAM footprint)
--- System: 336 bytes
--- Thread: 68 bytes
--- Timer : 20 bytes
--- Semaph: 12 bytes
--- EventS: 4 bytes
--- EventL: 20 bytes
--- Mutex : 16 bytes
--- CondV.: 8 bytes
--- Queue : 36 bytes
--- MailB.: 40 bytes
--- Result: SUCCESS
----------------------------------------------------------------------------

Final result: SUCCESS


Compare with F4...

Code: Select all


*** ChibiOS/RT test suite
***
*** Kernel:       3.0.0p3
*** Compiled:     Apr 13 2015 - 14:21:33
*** Compiler:     GCC 4.7.4 20140401 (release) [ARM/embedded-4_7-branch revision 209195]
*** Architecture: ARMv7-ME
*** Core Variant: Cortex-M4
*** Port Info:    Advanced kernel mode
*** Platform:     STM32F407 High Performance with DSP and FPU
*** Test Board:   STMicroelectronics STM32F4-Discovery

----------------------------------------------------------------------------
--- Test Case 1.1 (System, critical zones)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 1.2 (System, interrupts handling)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 1.3 (System, integrity)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 2.1 (Threads, enqueuing test #1)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 2.2 (Threads, enqueuing test #2)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 2.3 (Threads, priority change)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 2.4 (Threads, delays)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 3.1 (Semaphores, enqueuing)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 3.2 (Semaphores, timeout)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 3.3 (Semaphores, atomic signal-wait)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 3.4 (Binary Semaphores, functionality)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 4.1 (Mutexes, priority enqueuing test)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 4.2 (Mutexes, priority return)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 4.3 (Mutexes, status)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 4.4 (CondVar, signal test)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 4.5 (CondVar, broadcast test)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 4.6 (CondVar, boost test)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 5.1 (Messages, loop)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 6.1 (Mailboxes, queuing and timeouts)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 7.1 (Events, registration and dispatch)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 7.2 (Events, wait and broadcast)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 7.3 (Events, timeouts)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 8.1 (Heap, allocation and fragmentation test)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 9.1 (Memory Pools, queue/dequeue)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 10.1 (Dynamic APIs, threads creation from heap)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 10.2 (Dynamic APIs, threads creation from memory pool)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 10.3 (Dynamic APIs, registry and references)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 11.1 (Queues, input queues)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 11.2 (Queues, output queues)
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.1 (Benchmark, messages #1)
--- Score : 799993 msgs/S, 1599986 ctxswc/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.2 (Benchmark, messages #2)
--- Score : 648644 msgs/S, 1297288 ctxswc/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.3 (Benchmark, messages #3)
--- Score : 648644 msgs/S, 1297288 ctxswc/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.4 (Benchmark, context switch)
--- Score : 2559992 ctxswc/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.5 (Benchmark, threads, full cycle)
--- Score : 471906 threads/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.6 (Benchmark, threads, create only)
--- Score : 699995 threads/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.7 (Benchmark, mass reschedule, 5 threads)
--- Score : 208178 reschedules/S, 1249068 ctxswc/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.8 (Benchmark, round robin context switching)
--- Score : 1581160 ctxswc/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.9 (Benchmark, I/O Queues throughput)
--- Score : 1873164 bytes/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.10 (Benchmark, virtual timers set/reset)
--- Score : 1366044 timers/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.11 (Benchmark, semaphores wait/signal)
--- Score : 3199988 wait+signal/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.12 (Benchmark, mutexes lock/unlock)
--- Score : 1856348 lock+unlock/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.13 (Benchmark, RAM footprint)
--- System: 336 bytes
--- Thread: 68 bytes
--- Timer : 20 bytes
--- Semaph: 12 bytes
--- EventS: 4 bytes
--- EventL: 20 bytes
--- Mutex : 16 bytes
--- CondV.: 8 bytes
--- Queue : 36 bytes
--- MailB.: 40 bytes
--- Result: SUCCESS
----------------------------------------------------------------------------

Final result: SUCCESS


Context switch in 217nS vs 390nS.

The difference in performance is well above the mere difference in frequency (216 vs 168 MHz).

On the other hand the results fluctuate, even 1% differences in each runs while the F4 is clock-cycle accurate, effect of the deeper pipeline and I/D caches I think. With caches disabled it appears to be slower than the F4 (strange because it should still have the ART). Anyway, probably it is early silicon.

Giovanni
steved
Posts: 834
Joined: Fri Nov 09, 2012 2:22 pm
Has thanked: 12 times
Been thanked: 138 times

Re: [STM32F7] Performance Reports....

Post by steved »

Looks good, although interesting anomaly on test 12.11 - F7 is actually slower!
User avatar
barthess
Posts: 861
Joined: Wed Dec 08, 2010 7:55 pm
Been thanked: 7 times

Re: [STM32F7] Performance Reports....

Post by barthess »

Giovanni wrote:With caches disabled it appears to be slower than the F4 (strange because it should still have the ART)

I remember some evil bugs with ART in early F4x. May be the same situation here?
User avatar
Giovanni
Site Admin
Posts: 14894
Joined: Wed May 27, 2009 8:48 am
Has thanked: 1202 times
Been thanked: 997 times

Re: [STM32F7] Performance Reports....

Post by Giovanni »

I contacted people in ST about this but a lot of people is in vacation right now.

Probably the variations are caused by cache iterations with IRQ sources. I am still curious about ART, apparently enabling or disabling it makes no difference.

Giovanni
User avatar
Giovanni
Site Admin
Posts: 14894
Joined: Wed May 27, 2009 8:48 am
Has thanked: 1202 times
Been thanked: 997 times

Re: [STM32F7] Performance Reports....

Post by Giovanni »

Well, got answers and I like what I heard :)

The F7 has two ways to access flash:

1) Through the AXIM/AHB bridge using the M7 I-Cache. This is the flash mapped at 0x08000000.
2) Through the ITCM bus (uncached) that reaches the ST's ART accelerator. This is the flash mapped at 0x00200000.

The benchmarks I posted before used the solution 1. The cache makes the system not very deterministic.

Now I re-run benchmarks using solution 2:

Code: Select all

----------------------------------------------------------------------------
--- Test Case 12.1 (Benchmark, messages #1)
--- Score : 1270579 msgs/S, 2541158 ctxswc/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.2 (Benchmark, messages #2)
--- Score : 1053652 msgs/S, 2107304 ctxswc/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.3 (Benchmark, messages #3)
--- Score : 1053651 msgs/S, 2107302 ctxswc/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.4 (Benchmark, context switch)
--- Score : 4595720 ctxswc/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.5 (Benchmark, threads, full cycle)
--- Score : 765952 threads/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.6 (Benchmark, threads, create only)
--- Score : 1167560 threads/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.7 (Benchmark, mass reschedule, 5 threads)
--- Score : 342025 reschedules/S, 2052150 ctxswc/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.8 (Benchmark, round robin context switching)
--- Score : 2478464 ctxswc/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.9 (Benchmark, I/O Queues throughput)
--- Score : 3438796 bytes/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.10 (Benchmark, virtual timers set/reset)
--- Score : 2238280 timers/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.11 (Benchmark, semaphores wait/signal)
--- Score : 3724128 wait+signal/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.12 (Benchmark, mutexes lock/unlock)
--- Score : 2969064 lock+unlock/S
--- Result: SUCCESS
----------------------------------------------------------------------------
--- Test Case 12.13 (Benchmark, RAM footprint)
--- System: 336 bytes
--- Thread: 68 bytes
--- Timer : 20 bytes
--- Semaph: 12 bytes
--- EventS: 4 bytes
--- EventL: 20 bytes
--- Mutex : 16 bytes
--- CondV.: 8 bytes
--- Queue : 36 bytes
--- MailB.: 40 bytes
--- Result: SUCCESS
----------------------------------------------------------------------------

Final result: SUCCESS


Now the results are fully deterministic and the performance went UP significantly in most benchmarks. Good work with ART ST!! Looking at results the M7 I-Cache is pretty much wasted space.

The problem now is in ChibiOS, the centralized rules.ld is not sufficient because the flash can be read at 0x00200000 but can only be written at 0x08000000. This requires special rules... the DMA handling also requires a special region... probably custom rules_xxx.ld files will be required.

Giovanni
User avatar
Giovanni
Site Admin
Posts: 14894
Joined: Wed May 27, 2009 8:48 am
Has thanked: 1202 times
Been thanked: 997 times

Re: [STM32F7] Performance Reports....

Post by Giovanni »

OK news.

All the above is now totally transparent to the user. I introduced an STM32F7-specific ld rules file that organizes things as follow:

1) Code is accessed through ITCM for performance and determinism.
2) Constants are accessed through AXI because "const" buffers should be DMA-accessible.
3) A new section ".dma" has been added. It is possible to put there all the DMA-accessible buffers. Using the MPU is possible to make the region non-cacheable, this way it is not necessary to do buffers flushing/invalidation. This still requires add __attribute__((section ".dma")) before buffers declaration or, in alternative, prefix the buffer name with "__dma_".

Examples:

Code: Select all

__attribute__((section ".dma")) uint8_t rxbuf[128];


or

Code: Select all

uint8_t __dma_rxbuf[128];


Not entirely transparent but better than manual flush/invalidation.

Giovanni
User avatar
Giovanni
Site Admin
Posts: 14894
Joined: Wed May 27, 2009 8:48 am
Has thanked: 1202 times
Been thanked: 997 times

Re: [STM32F7] Performance Reports....

Post by Giovanni »

More news.

I altered the normal memory layout, now by default the BSS is placed in the DTCM RAM which is not cached, this means that buffers management is no more necessary by default. The limitation is that the BSS is now limited to 64kB. The large RAM area is used by DATA and Heap.

Buffers management is still necessary only if buffers are placed in a cache-able area, for example allocating them from the heap.

There is also a new scatter file STM32F746xG_MAX.ld that maximizes contiguous RAM but requires management or explicit placement of buffers in the "nocache" section (not used by default).

I think all problems have been solved now, F7 is a "normal" STM32.

Giovanni
steved
Posts: 834
Joined: Fri Nov 09, 2012 2:22 pm
Has thanked: 12 times
Been thanked: 138 times

Re: [STM32F7] Performance Reports....

Post by steved »

Small suggestion which might save someone some grief.
I declared an initialised RAM-based array without thought of cache considerations. It turned out that cache usage was low enough that cache was never flushed to RAM - so the DMA controller transferred junk. Explicit cache flush instruction solved the problem.
So suggest that cache is flushed after initialisation of RAM to avoid this scenario (maybe a total cache flush just before main() is entered?)
User avatar
Giovanni
Site Admin
Posts: 14894
Joined: Wed May 27, 2009 8:48 am
Has thanked: 1202 times
Been thanked: 997 times

Re: [STM32F7] Performance Reports....

Post by Giovanni »

Hi,

It could be done but you shouldn't rely on this kind of solutions, it could work in some cases and not in others potentially raising even more hidden problems.

Cache has to be handled knowingly or buffers have to be placed in the non-cacheable area.

Giovanni
Post Reply