Second-placed Milan staggeringly ended a five-match league winning streak upon slumping to an embarrassing loss at home.
Fabrizio Miccoli broke the deadlock after 49 minutes with a powerful shot, and helped Mark Bresciano double the visiting team's lead seven minutes from time.
Milan are now five points adrift of Inter, who are taking the lead on 36 points.
At Atalanta, Inter were put upfront after 15 minutes when Argentina striker Diego Milito dribbled past two challengers to slot home.
Wesley Sneijder could have scored the second for the away side, but his free-kick hit the bar, before he was red carded for a second booking after the interval, which gave the hosts a numerical advantage.
Simone Tiribocchi brought the score level for Atalanta on 81 minutes.