Audio buffer delay increasing in Catalina

Hi everyone, I succesfully booted an Hackintosh with Catalina 10.15.7, here's the specs, then I'll come to my problem:

- CPU: AMD Ryzen 7 2700X

- MOBO: MSI X470 Gaming PRO

- GPU: Sapphire Nitro AMD RX 5700XT

- PSU: ThermalTake Smart 630W 80+

- RAM: Corsair Vengeance 3000MHz 2*8GB

- Main disk: Silicon Power SATA SSD 128GB

- Network Interface: TP-Link Archer T6E

I had almost zero problems since first boot, just some little optimizations, except for this one: I've connected a USB audio interface (M-Audio Mtrack plus), which configures and connects perfectly, but while it stays on and connected its delay increases drastically over time, over the entire machine, while the specific program I'm using (Logic Pro X) doesn't recognize an increasing I/O delay. I tried with other interfaces but the results are the same.

This sucks because using Logic is one of the main reasons for which I built this machine, so having .5 seconds delay makes the card unusable to record stuff and working. Does anyone had a similar problem?

Thank you for your time!