Using OV9655 camera by STM32F429 (DCMI)

[A319]

Hi!

At the moment I'm trying to get a demo to work with ChibiOS having a camera (OV9655) connected to a STM32F429 demo board.
My goal is it, to sample images from the camera by DCMI through DMA to RAM. The demo board has actually connected SDRAM and
I could sample larger images, but so far I didn't manage it to tell the linker it has more memory.

Finally after ~3 days I was happy to get everything to work.

data.png (19.9 KiB) Viewed 16983 times

This is the largest image I can sample without external SDRAM. (It's actually BMP but the forum-software doesn't allow BMP)

I've added the code to a Git repository:
https://github.com/DL7AD/STM32F429-DCMI

Just my code is really really crude and I want to improve it, before I continue with something else.
I've got the code from a webpage http://mikrocontroller.bplaced.net/word ... ge_id=1115

This code uses the stm32f4xx.h file which is needed by the sample but unfortunately incompatible with ChibiOS.
So what I'm doing: Putting the stm32f4xx.h somewhere else, compiling ChibiOS (until the error), then putting the stm32f4xx.h to it's original location and compile it again.
What I want to do, is converting the code (from the main.c), that it matches to the ChibiOS's stm32f429.h Doing this for the GPIO's is quiet easy. But I'm missing the DCMI support in ChibiOS. So I can't convert the code, or did I miss something?

So what's the best way to continue to make the code compatible?

Regards
Sven

[Giovanni]

Hi Sven,

Nice work!!

There is no DCMI support in ChibiOS currently, DCMI is very STM32-specific so it cannot be part of the HAL.

What you could do is to create a low level ChibiOS-style platform driver, much like the DMA driver. This kind of drivers are architecture-specific but still part of the ChibiOS structure.

Like other similar drivers, it would be named as /os/hal/ports/STM32/LLD/DCMIv1/stm32_dcmi.c/.h and would use the DMA and RCC drivers services for transfers and initializations.

I am not sure about the API such a driver should have, never used video peripherals, probably you have a better idea about that than me.

Giovanni

[Jacon]

Perhaps this topic would be useful:

http://forum.chibios.org/phpbb/viewtopic.php?f=2&t=2542&p=20838&hilit=camera#p20838

[A319]

Hi,

Thank's for the answers. I finally made the code compatible by adding the code from the other files directly into the main.c so far. I've already pushed it into the Git repository. It should now work without any external libraries. Atm. the code is still a mess which I will tidy up the next days.

Also I've written a simple Python program which detects the BMP header out of the serial stream and saves it into a file.

Code: Select all

import serial,os
from subprocess import call

ser = serial.Serial(
    port='/dev/ttyUSB0',
    baudrate=38400,
)

while 1:
   data = ser.read()
   if data != 'B':
      continue
   data = ser.read()
   if data != 'M':
      continue
   data = ser.read()
   if data != '6':
      continue

   print 'Detected BMP'

   file = open('data.bmp', 'w')
   file.write('BM6')
   i = 0
   while i<57650: # BMP size of 160*120px
      if i % 576 == 0:
         print str(i/576) + '%'
      data = ser.read()
      file.write(data)
      i += 1

   print 'Done'

The serial comes out of pin PD5 with the code, which is available on Github.

Regards
Sven

[A319]

Update

I do not want to end up with 160x120. But there's already not enough memory to store a raw image of 320x240. It would take 225kB (320px [width] x 240px [height] x 3bytes [RGB]).
So I've tried if I would be able to move the data into it's 2MB flash. But unfortunately the flash is much too slow. I haven't tried this before. So I was unaware of it's speed.

I finally solved the problem by encoding the raw data into JPEG in real time. I've found a JPEG encoding library specialized for microcontrollers named JPEGANT:
JPEGant: http://sourceforge.net/projects/jpegant.berlios/

The JPEG library encodes JPEG in 16x16px blocks. So I would need 16 lines of the image in the SRAM to get 16x16px blocks which I need for JPEG encoding.
For this purpose I've added an interrupt which is generated every 16 lines of the image. So the amount of memory which I need for conversion at every time is 10kB (320px [width] x 16px [height] x 2bytes [per pixel]).
I need additional 10kB because the DMA continuously samples data into the DMA buffer. So I have to copy the buffer first somewhere else, otherwise the data would get overwritten by the DMA. At the moment I'm solving this problem by copying the data by memcpy, but it would be better to change the destination location of the DMA due to performance. Somehow this didn't work on an easy way, so I'm still working on memcpy.

All in All: 10kB (DMA memory) + 10kB (copy memory) + 10...20kB (for final JPEG image) = 30...40kB for everything

Finally the encoding algorithm is using about 80% of the available interrupt time (until the next interrupt occurs). Therefore I adjusted the camera input clock to 8Mhz. All in all the picture is encoded/taken in 94ms which is good enough even if the camera is moved slightly.

BTW. The serial interface speed also improved because the image size is roughly 10kB. Much less than 160x120 as BMP format (56kB).

I've updated all the sources on the Github repository: https://github.com/DL7AD/STM32F429-DCMI

Regards
Sven

[Giovanni]

Very interesting.

Probably you could use a thread for encoding, do that into the ISR should degrade performance for the rest of the system.

Giovanni

[A319]

Hi Giovanni,

That's what I've considered before (but didn't mention on the forum). I copy the DMA buffer into a ring buffer, from where it's encoded then. The encoder just has to take care about the DMA's speed, to avoid getting overtaken by the DMA.
Finally the code is able to sample a picture within 260ms. So I think VGA is the maximum you should do, because otherwise the images will get too blurred at the slightest movement. So finally I just want to improve the performance of the algorithm. So far I had no optimization activated (-O0). But when I tried to activate optimization, the code got not only larger, it also had a loss in speed

Code: Select all

# Compiler options here.
ifeq ($(USE_OPT),yes)
  USE_OPT = -O2 -ggdb -fomit-frame-pointer -falign-functions=16
endif

The second minor improvement I wanted to do, is increasing the speed of the CPU from 168MHz to is maximum capable speed (180MHz). But somehow the program didn't startup anymore.

Code: Select all

#define STM32_SW                            STM32_SW_PLL
#define STM32_PLLSRC                        STM32_PLLSRC_HSE
#define STM32_PLLM_VALUE                    8
#define STM32_PLLN_VALUE                    330 // <= changed this from 336
#define STM32_PLLP_VALUE                    2
#define STM32_PLLQ_VALUE                    7

BTW. The DMA is sampling all lines except of the last 16 lines. The DMA should buffer 16 lines and thereafter generate an interrupt. But somehow it doesn't occur. That's why the last 16 lines are missing in the sample image. :/

Regards Sven

[Giovanni]

Hi,

-O2 should definitely be MUCH better than -O0, there is something wrong if you see something different.

About DMA, I would try to implement a circular buffer of two lines and have a callback at half buffer and at end buffer (see how ADC driver works, the same concept). This way you could process one half buffer WHILE the DMA is filling the other half.

About frequency, you just need to change the multiplier value, last time I tried it worked. Could you try that on a "naked" test application? it is better to try things one at time.

Giovanni

[A319]

Unfortunately the code was already optimized. When I added yes, it what instead unoptimized.

However I added DBM (Double buffer mode) which switches between two buffers. So I can handle one buffer while the other is filled. The microcontroller thereafter switches the buffer by itself. Quiet handy.

Finally a friend made me aware, that the STM32F4 has a CCM (core coupled memory). So I added an attribute to one of the function which is called quiet often:

Code: Select all

void huffman_encode(huffman_t *const ctx, const short data[64]) __attribute__ ((section(".ram4")));

But it ended up in:

Code: Select all

Compiling main.c
Compiling jpegenc.c
Linking build/ch.elf
/home/sven/gcc-arm-none-eabi-4_8-2014q2/bin/../lib/gcc/arm-none-eabi/4.8.4/../../../../arm-none-eabi/bin/ld: final link failed: Section has no contents
collect2: error: ld returned 1 exit status
make: *** [build/ch.elf] Error 1

Unfortunately I couldn't find anything on the Internet.

[Giovanni]

Hi,

Note that assigning code to the RAM sections is not sufficient, you need to actually copy the code there because the current startup files don't do that for you. I need to implement a general solution for this.

Giovanni