STM32(F7) SPI+DMA stuck byte Topic is solved

[Dron_Gus]

Hi,

ChibiOS 21.11.x
I have a custom hw with STM32F767 and W25Q128 connected over SPI.
W25Q driver is based on serial_nor. Nothing special, just reworked micron_n25q.
I'm using MFS on this flash. Two banks of 512Kb.
After few write of big blob of data (25Kb) I got failure in verification code of mfs_flash_write(). Data read back is not equal to what was written.

I had to attach logic analyzer to find real cause of this issue.
Here is "good" write:

1. 0x05 poll for status - reply is 0x00, busy bit is zero. code starts writing next page
2. 0x06 - write enable command. No arguments, no reply.
3. 0x02 - write command with 3 bytes of address and 256 bytes of data.
Here is "bad" write:

1. 0x05 poll for status - reply is 0x00
2. 0x06 - write enable command
3. write command, should be 0x02 with three bytes of address. But 0x06 is transmitted again. And some random bytes. I checked previous and next writes and this three bytes of address is not in a line with previous and next writes.

So this command is ignored by W25Q chip, no data is actually written and MFS later fails on validation.
Every other write commands (before and after) looks like correct. This corruption observed only once during write of 25K of data.

My current theory is that Tx FIFO or some internal DMA buffer is not fully flushed after previous transaction. And when new transfer is initiated last byte of previous transfer is transmitted.
I already checked STM32F7 errata - did not found anything related. I also reading DS and checking init code of spi and dma. But everything looks good.

Any thoughts?

[Giovanni]

Hi,

I think it is a good hypothesis, it would be possible to place temporary code and check the state of FIFOs before starting a transfer.

Giovanni

[Dron_Gus]

Hi,

I think it is a good hypothesis, it would be possible to place temporary code and check the state of FIFOs before starting a transfer.

Giovanni

Hi,

Thanks, good idea!
I try adding following code to spi_lld_ignore, spi_lld_exchange and spi_lld_send right before enabling DMAs:

Code: Select all

  osalDbgAssert((spip->spi->SR & (SPI_SR_FRLVL_Msk | SPI_SR_FTLVL_Msk)) == 0, "FIFO not empty!");
  osalDbgAssert((spip->spi->SR & SPI_SR_BSY) == 0, "Busy!");

But none of these assertions have fired.

I checked DMA irq handling and found that current code does not enable and handle "FEIE: FIFO error interrupt". While it is still applicable to Direct mode (with FIFO disabled).
After enabling it and adding some code to spi_lld_serve_rx_interrupt and spi_lld_serve_tx_interrupt I start getting a LOT of FIFO errors from Tx DMA channels.
It seems correct as SPI module is not disabled between transactions: SPI_CR1_SPE set in CR1 register only on spi_lld_enable and never cleared.
So according to documentation SPI keeps generating DMA request to DMA while DMA is disabled.
Meanwhile I did not receive any FIFO errors on Rx DMA channel.

Do you think I should try adding code that clears SPI_CR1_SPE bit in SPI CR1 after transaction ends? And setting it back after setuping new transaction and DMAs?

[Dron_Gus]

Ok. I was looking at wrong place.

So this is totally my fault. I'm using STM32F7 which is actually Cortex M7 and it has caches.
We have big code-base and some of developers. So to avoid any cache-related issue I decided to add two wrapper to hal_serial_nor.c that are copying data to/from devp->nocache buffer before/after Tx/Rx. So users can write any data, not thinking about caches.
Something like this:

Code: Select all

static void spiSendNoCache(SNORDriver *devp, size_t n, const void *txbuf)
{
  BUSDriver *busp = devp->config->busp;
  while (n) {
    size_t chunk = (n > SNOR_BUFFER_SIZE) ? SNOR_BUFFER_SIZE : n;
    memcpy(devp->nocache->buf, txbuf, chunk);
    osalDbgAssert(spiSend(busp, chunk, devp->nocache->buf) == MSG_OK, "spiSendNoCache");
    txbuf += chunk;
    n -= chunk;
  }
}

static void spiReceiveNoCache(SNORDriver *devp, size_t n, void *rxbuf)
{
  BUSDriver *busp = devp->config->busp;
  while (n) {
    size_t chunk = (n > SNOR_BUFFER_SIZE) ? SNOR_BUFFER_SIZE : n;
    osalDbgAssert(spiReceive(busp, chunk, devp->nocache->buf) == MSG_OK, "spiReceiveNoCache");
    memcpy(rxbuf, devp->nocache->buf, chunk);
    rxbuf += chunk;
    n -= chunk;
  }
}

Even nocache buffer is located in no-cached area (thanks to STM32_NOCACHE_ENABLE) I still need a memory barrier after memcpy to dev->nocache->buf.

So adding __DSB() after memcpy solved my issue:

Code: Select all

static void spiSendNoCache(SNORDriver *devp, size_t n, const void *txbuf)
{
  BUSDriver *busp = devp->config->busp;
  while (n) {
    size_t chunk = (n > SNOR_BUFFER_SIZE) ? SNOR_BUFFER_SIZE : n;
    memcpy(devp->nocache->buf, txbuf, chunk);
    /* ensures that all memory accesses are completed */
    __DSB();
    osalDbgAssert(spiSend(busp, chunk, devp->nocache->buf) == MSG_OK, "spiSendNoCache");
    txbuf += chunk;
    n -= chunk;
  }

Sorry for bothering you. There is no issue in MFS/SPI/DMA code.

I'll run some more tests, but it seems to me that problem is solved.

Andrey.

[Giovanni]

Good point, need to check demos, probably there is the same error and necessity for DSB is not explained at all.

Giovanni