Had the same conversation with my wife about it. Who's going to arrest him? The local cops (who's chief Malvo murdered)? The FBI (who had 2 dead agents outside that Malvo murdered)?
There are dead FBI agents, the local cops would have to explain who killed Malvo to the Feds, Gus didn't make it a secret it was him. I kept waiting for some kind of explanation, but it just ended.