Big Sur, booting but unstable with GPU artifacts

Hardware here: https://pcpartpicker.com/list/pBgyqp

Followed dortania guide.

example of artifacts: https://imgur.com/c0zJDof

in this case, everything froze except for the cursor. sometimes there's some red artifact rectangle on cursor. sometimes screen is yellow with or without artifacts.

Was happening right after boot -- sometimes I could just unplug/plug back in the HDMI cable and it would recover itself and perform normally. Now (at least so far, after enabling multi monitor on my iGPU which I'm not using) it seems to boot but can lock up with those artifacts randomly.

Any advice? Would this sort of thing show up in logs anywhere?

What format is preferred for posting EFI/config info? OC 0.6.4. Lilu 1.5.0 and WhateverGreen 1.4.5 seem to be loading. Geekbench scores seem correct.

Thanks,