Agent Milton actually gave the gang a lot of chances, just a man doing his job at the end of the day.

Who was the Agent, Jack killed there's the answer

They’re both as bad as each other

Ross is a genuinely bad person who happened to work on the side of the law. Milton for all his theatrics was just doing his job