Page 1 of 2

UDP receive missing messages

Posted: Sat Mar 16, 2024 2:29 am
by hybridA
I have two threads, one receive, one transmit. The microcontroller responds to an incoming UDP packet at 10Hz:

Code: Select all

void EthernetTxThread(void*) {
    chRegSetThreadName("ethtx");
    systime_t prev = chVTGetSystemTime();
    while(1) {
        chSysLock();
        chThdSuspendS(&trp);
        chSysUnlock();
        sendResponse();
    }
}

void EthernetRxThread(void*) {
    chRegSetThreadName("ethrx");
    systime_t prev = chVTGetSystemTime();
    while(1) {
        if (parseMessage() != -1)
            chThdResume(&trp, MSG_OK);
    }
}


Inside parseMessage() is a blocking recvfrom() :

Code: Select all

int len = recvfrom(sock, recv_buf, sizeof(recv_buf), 0, (struct sockaddr*) &eth_rx_addr, (socklen_t *)sizeof(eth_rx_addr));


Wireshark shows the PC is sending out packets at 10Hz as expected, and around 30% of the time the microcontroller responds in-time. However i see that sometimes it takes up to 1 second to respond. By capturing timestamps I saw that the delay is actually in the rx thread. recvfrom() is blocking for much longer than 100ms. I went a little lower and found that

Code: Select all

sys_arch_mbox_fetch(&conn->recvmbox, &buf, 0);
in api_lib.c is the source of the delay.
Are there LWIP options needed to improve the UDP rx performance?

Re: UDP receive missing messages

Posted: Sat Mar 16, 2024 8:53 am
by Giovanni
Hi,

It is possible this is related to this old bug:

https://sourceforge.net/p/chibios/bugs/973/

Could you try that proposed fix?

Giovanni

Re: UDP receive missing messages

Posted: Sat Mar 16, 2024 4:13 pm
by hybridA
Thank you Giovanni! I tried the proposed fix, but unfortunately it didn't fix the issue.

Re: UDP receive missing messages

Posted: Sun Mar 17, 2024 9:27 am
by Giovanni
Hi,

Do you have threads slowing down lwip threads? check priorities order.

Giovanni

Re: UDP receive missing messages

Posted: Sun Mar 17, 2024 5:45 pm
by hybridA
I stripped down my application to nothing but the ethernet and lwip threads + a main loop blinking an LED. recvfrom() is still behaving badly.
Screenshot from 2024-03-17 09-41-19.png

Re: UDP receive missing messages

Posted: Mon Mar 18, 2024 1:40 pm
by Giovanni
You should try to identify which layer causes the delay, there are several, is the MAC driver delaying frames ? does it happen in lwip ?

Giovanni

Re: UDP receive missing messages

Posted: Tue Apr 02, 2024 7:25 am
by hybridA
Can you point me to some areas of interest that would be indicative of where the delay comes from? So far I only verified that the recvfrom() is truly blocking for much longer than the message interval in wireshark.

Re: UDP receive missing messages

Posted: Tue Apr 02, 2024 7:59 am
by Giovanni
You should try to verify if it is the MAC driver delaying the frame or if it is a problem within lwIP or its integration layer.

One possible approach is to flip a GPIO from the MAC ISR, send periodic packets and look with a scope if the ISR occurs immediately.

Giovanni

Re: UDP receive missing messages

Posted: Wed Apr 03, 2024 5:43 am
by hybridA
Thanks Giovanni. I went ahead and got some captures of a toggling pin inside the irq handler:

Code: Select all

OSAL_IRQ_HANDLER(STM32_ETH_HANDLER) {
  MACDriver *macp = &ETHD1;
  uint32_t dmacsr;

  OSAL_IRQ_PROLOGUE();

  dmacsr = ETH->DMACSR;
  ETH->DMACSR = dmacsr; /* Clear status bits.*/

  if ((dmacsr & (ETH_DMACSR_RI | ETH_DMACSR_TI)) != 0U) {
    if ((dmacsr & ETH_DMACSR_TI) != 0U) {
      /* Data Transmitted.*/
      __mac_tx_wakeup(macp);
    }

    if ((dmacsr & ETH_DMACSR_RI) != 0U) {
      palClearLine(LINE_A);
      /* Data Received.*/
      __mac_rx_wakeup(macp);
      palSetLine(LINE_A);
    }

    __mac_callback(macp);
  }


The results are interesting, as they do reflect the delays as well. You can see some 50ms deltas but a lot of them are much longer. The longer durations are all multiples of 50ms. Does this mean the issue is even lower level (phy?)? It's a DP83848I over RMII.

Re: UDP receive missing messages

Posted: Wed Apr 03, 2024 6:41 am
by Giovanni
Next step is to understand if there is a driver problem or an HW issue. Are you using MACv1 or MACv2?

Giovanni