Replacing original GPU

How do you physically replace a MacPro 5,1 GPU hd5770 with a RX580? RX580 does not fit in the slot. I tried 2 different “brands”. I noticie “the let say heatsink” or the metal below the card does not let me to insert in the slot. There is more gap between the card and the metal, as you can see. Thx