UDP receive missing messages

[hybridA]

I have two threads, one receive, one transmit. The microcontroller responds to an incoming UDP packet at 10Hz:

Code: Select all

void EthernetTxThread(void*) {
    chRegSetThreadName("ethtx");
    systime_t prev = chVTGetSystemTime();
    while(1) {
        chSysLock();
        chThdSuspendS(&trp);
        chSysUnlock();
        sendResponse();
    }
}

void EthernetRxThread(void*) {
    chRegSetThreadName("ethrx");
    systime_t prev = chVTGetSystemTime();
    while(1) {
        if (parseMessage() != -1)
            chThdResume(&trp, MSG_OK);
    }
}

Inside parseMessage() is a blocking recvfrom() :

Code: Select all

int len = recvfrom(sock, recv_buf, sizeof(recv_buf), 0, (struct sockaddr*) &eth_rx_addr, (socklen_t *)sizeof(eth_rx_addr));

Wireshark shows the PC is sending out packets at 10Hz as expected, and around 30% of the time the microcontroller responds in-time. However i see that sometimes it takes up to 1 second to respond. By capturing timestamps I saw that the delay is actually in the rx thread. recvfrom() is blocking for much longer than 100ms. I went a little lower and found that

Code: Select all

sys_arch_mbox_fetch(&conn->recvmbox, &buf, 0);

in api_lib.c is the source of the delay.
Are there LWIP options needed to improve the UDP rx performance?

[Giovanni]

Hi,

It is possible this is related to this old bug:

https://sourceforge.net/p/chibios/bugs/973/

Could you try that proposed fix?

Giovanni

[hybridA]

Thank you Giovanni! I tried the proposed fix, but unfortunately it didn't fix the issue.

[Giovanni]

Hi,

Do you have threads slowing down lwip threads? check priorities order.

Giovanni

[hybridA]

I stripped down my application to nothing but the ethernet and lwip threads + a main loop blinking an LED. recvfrom() is still behaving badly.

[Giovanni]

You should try to identify which layer causes the delay, there are several, is the MAC driver delaying frames ? does it happen in lwip ?

Giovanni

[hybridA]

Can you point me to some areas of interest that would be indicative of where the delay comes from? So far I only verified that the recvfrom() is truly blocking for much longer than the message interval in wireshark.

[Giovanni]

You should try to verify if it is the MAC driver delaying the frame or if it is a problem within lwIP or its integration layer.

One possible approach is to flip a GPIO from the MAC ISR, send periodic packets and look with a scope if the ISR occurs immediately.

Giovanni

[hybridA]

Thanks Giovanni. I went ahead and got some captures of a toggling pin inside the irq handler:

Code: Select all

OSAL_IRQ_HANDLER(STM32_ETH_HANDLER) {
  MACDriver *macp = &ETHD1;
  uint32_t dmacsr;

  OSAL_IRQ_PROLOGUE();

  dmacsr = ETH->DMACSR;
  ETH->DMACSR = dmacsr; /* Clear status bits.*/

  if ((dmacsr & (ETH_DMACSR_RI | ETH_DMACSR_TI)) != 0U) {
    if ((dmacsr & ETH_DMACSR_TI) != 0U) {
      /* Data Transmitted.*/
      __mac_tx_wakeup(macp);
    }

    if ((dmacsr & ETH_DMACSR_RI) != 0U) {
      palClearLine(LINE_A);
      /* Data Received.*/
      __mac_rx_wakeup(macp);
      palSetLine(LINE_A);
    }

    __mac_callback(macp);
  }

The results are interesting, as they do reflect the delays as well. You can see some 50ms deltas but a lot of them are much longer. The longer durations are all multiples of 50ms. Does this mean the issue is even lower level (phy?)? It's a DP83848I over RMII.

[Giovanni]

Next step is to understand if there is a driver problem or an HW issue. Are you using MACv1 or MACv2?

Giovanni