mellanox connect x 5 drops when cache full

mellanox connect x 5 drops when cache full

[Yaron Illouz]

[-- Attachment #1: Type: text/plain, Size: 650 bytes --]

Hi

I am using multiqueue with RSS to read from MT27800 Family [ConnectX-5] 1017

My application receive traffic, and write some data to disk. As a result I/O write is cached in linux memory.
When the server memory is completely cache ( I know it is still available) I start seeing drops at nic.
If I delete the data from disk every minute the cached memory is released to free and and no drops at nic.
How can I avoid the drops at nic?

        total        used        free      shared  buff/cache   available
Mem:           125G         77G        325M         29M         47G         47G
Swap:          8.0G        256K        8.0G


[-- Attachment #2: Type: text/html, Size: 3486 bytes --]

RE: mellanox connect x 5 drops when cache full

[Matan Azrad]

[-- Attachment #1: Type: text/plain, Size: 967 bytes --]

Hi Yaron

Maybe the caching takes PCI bandwidth?

Matan


From: Yaron Illouz <yaroni@radcom.com>
Sent: Wednesday, January 19, 2022 4:27 PM
To: dev@dpdk.org
Cc: Matan Azrad <matan@nvidia.com>
Subject: mellanox connect x 5 drops when cache full

External email: Use caution opening links or attachments

Hi

I am using multiqueue with RSS to read from MT27800 Family [ConnectX-5] 1017

My application receive traffic, and write some data to disk. As a result I/O write is cached in linux memory.
When the server memory is completely cache ( I know it is still available) I start seeing drops at nic.
If I delete the data from disk every minute the cached memory is released to free and and no drops at nic.
How can I avoid the drops at nic?

        total        used        free      shared  buff/cache   available
Mem:           125G         77G        325M         29M         47G         47G
Swap:          8.0G        256K        8.0G


[-- Attachment #2: Type: text/html, Size: 4546 bytes --]