[LWIP] How to get rid of heap allocation

[Rado1]

HI

Working with lwip, tcp , installed using CubeMx. Trying to remove the heap usage in TX.

As the RX implemented as a memory pool, the TX is kept using the heap.

TX: __attribute__((section(".LwipHeapSection"), aligned(32))) u8_t lwip_ccm_heap[MEM_SIZE];

(RX: is fine, uses the stack fixed size queue.)

1. Any reason for that? Is it in some way better?

note: the flag that causes usage of heap: TCP_WRITE_FLAG_COPY 

err_t err = tcp_write(client_pcb_, data, static_cast<u16_t>(length), TCP_WRITE_FLAG_COPY);

Thank you for your attention!

[Andrew Neil]

LwIP Heap and memory pool controls:

https://www.nongnu.org/lwip/2_0_x/group__lwip__opts__mem.html 

[Rado1]

Thanks for that! 
@Adam BERLINGER @Andrew Neil 

Made some changes to: 
How to create a project for STM32H7 with Ethernet ... - STMicroelectronics Community

-RX (fixed size buffered) and TX(allcoated) are both placed in D2_RAM.

Please help to create same RX behavior in TX.

Issue is how to make LWIP fixed size buffers reside in D2_RAM??

 
My line of work:

 RX in the RTOS example is pre-buffered already! so half work is done.

//code
extern u8_t __attribute__((section(".Rx_PoolSection"))) memp_memory_RX_POOL_base[];
// linker
    . = ABSOLUTE(0x30000400);
    *(.Rx_PoolSection) 

  } >RAM_D2

 
But the TX in NOT!

#define LWIP_RAM_HEAP_POINTER 0x30004000

So unfortunately, the LWIP will be using dynamic allocation in TX using that address.

To fix that, I enabled the flags in CubeMX:

/*----- Default Value for MEM_USE_POOLS: 0 ---*/
#define MEM_USE_POOLS 1
/*----- Default Value for MEMP_USE_CUSTOM_POOLS: 0 ---*/
#define MEMP_USE_CUSTOM_POOLS 1

CubeMX then generated a file called: 'lwippools.h'.

I enabled the code as it is commented.

/* USER CODE BEGIN 0 */
/* Warning 1: The following code is only given as an example */
/* You can use this example code by uncommenting it */
/* -------------- EXAMPLE of CODE ------------------------*/
/* Define three pools with sizes 256, 512, and 1512 bytes */

LWIP_MALLOC_MEMPOOL_START
LWIP_MALLOC_MEMPOOL(5, 1536)
LWIP_MALLOC_MEMPOOL_END
/* -------------- END of EXAMPLE of CODE -----------------*/

Now finally, I need to place those MEMPOOL in a D2_RAM , how to do that without changing LWIP?

[Adam BERLINGER]

Hello,

I'm not sure I understand you question.
If your goal is to get rid of the dynamic allocation inside LwIP in general, I'm not sure if that is possible and I can't help with that. My understanding is that the heap dynamic allocation is baked into the LwIP core.
The size of the LwIP heap is controlled by MEM_SIZE macro.

For UDP it might be possible to use API and allocate the TX packets yourself, but with TCP protocol it is more complicated and it might not be possible.

If the issue is that the LwIP heap is placed incorrectly, I could help with that. However, I don't see any error the code snippets you provided. In that case, maybe you can provide where the heap is actually placed?

P.S.: I plan to update the linked FAQ as well. One of the points that is missing there is that the MEM_ALIGNMENT should be set to 32, so it matches the Cache line size in Cortex-M7. I also assume you are using STM32H7.

Best regards,
Adam Berlinger

[Rado1]

@Adam BERLINGER 
Hi, 

It took some effort, but I got it working.

Yes, I am using STM32H7, I use RAW mode (no sockets etc). 

# I don't/can't use heap, only pools, so I entirely removed it.  I validated that by setting:

MEM_SIZE 0
LWIP_RAM_HEAP_POINTER 0

#Note: In some cases, depending on the flag you use to store TX, PBUF is entirely irrelevant and can save a bunch of space, unfortunately though CubeMX does not let me zero it entirely, and had to hardcode it.

 /* PBUF_POOL_SIZE: the number of buffers in the pbuf pool.
 Setting this to zero as Rx_PoolSection is used. */
#undef PBUF_POOL_SIZE
#define PBUF_POOL_SIZE 0U

The flag that makes PBUF redundant:

/* TCP_WRITE_FLAG_COPY - flag used for TX */
tcp_write(client_pcb_, data, static_cast<u16_t>(length), TCP_WRITE_FLAG_COPY);

# I am only using the pre-allocated buffers instead of heap. That allows no dynamic allocation in runtime.

/* USER CODE BEGIN 0 */
/* Warning 1: The following code is only given as an example */
/* You can use this example code by uncommenting it */
/* -------------- EXAMPLE of CODE ------------------------*/
/* Define three pools with sizes 256, 512, and 1512 bytes */

LWIP_MALLOC_MEMPOOL_START
LWIP_MALLOC_MEMPOOL(5, 1536)
LWIP_MALLOC_MEMPOOL_END
/* -------------- END of EXAMPLE of CODE -----------------*/

For the Stm32H7, placed those buffers in linker, in D2 RAM. (as instructed in the example)

LWIP names them according to their size. 

/* must place this before .bss */
..
    . = ALIGN(0x80);            /* ensure alignment */
    *(.bss.memp_memory_POOL_*_base)

  } >RAM_D2 AT> FLASH

The difference now, is that LWIP uses a different mem_malloc function, one that doesn't use a heap allocator. And that PBUF is not needed (note: in RAW mode usage!).

An FAQ will be great, as all the information is scattered around. Support for bare metal RAW users will be much appreciated!

As well as a suggested tool to maybe test it will help a lot!

[Pavel A.]

@Rado1 Example  how to place a custom pool in specific linker section is given in comments in LwIP/src/include/lwip/memp.h  

 * To relocate a pool, declare it as extern in cc.h. Example for GCC:
 *   extern u8_t \_\_attribute\_\_((section(".onchip_mem"))) memp_memory_my_private_pool_base[];
 */
#define LWIP_MEMPOOL_DECLARE(name,num,size,desc) \
  LWIP_DECLARE_MEMORY_ALIGNED(memp_memory_ ## name ## _base, ((num) * (MEMP_SIZE + MEMP_ALIGN_SIZE(size)))); 
.............

Also, review the definitions of flags for pbuf_type in pbuf.h : there is a possibility to indicate that a pbuf comes from a custom pool.

> As well as a suggested tool to maybe test it

Simply check, when you allocate from a custom pool, that the pointer is within the pool memory?

[Rado1]

@Pavel A. Thanks for that!

Did check in .map file, looking fine. 

Looking for a more of external test tool, to test configurations.

Something that can verify correct overall behavior.

[Rado1]

Ok, thanks

did some quick sanity check.

Installed pv and nc in a linux shell. 

@Echo off
wsl bash -c "pv -L 10m /dev/urandom | nc 192.168.0.XX XXXX"
pause

Speed looks fine.