After drawing with Zambia and losing to Algeria, the six-time African champions needed to win all their remaining four games to reach their group's leadership.
Superstar Mohamed Abou-Treika opened the score line on 65 minutes before wrapping up the win deep in injury time.
Hosni Abd-Rabou hit his team's second from the penalty spot in the 76th minute.
The win lifts The Pharaohs to second place in Group C with four points from three games, three behind leaders Algeria.
Egypt looked keen on scoring early and their first wasted chance came in the fifth minute.
Mohamed Shawky headed Hosni Abd-Rabou's ball towards Wael Gomaa but the latter blazed over from point blank.
In the 16th minute, Abd-Rabou failed to capitalise on the Rwandan keeper Jean-Claude Ndori's blunder after the latter dropped Ahmed Al-Muhammadi's cross inside the box.
Mohamed Zidan was unlucky to see his left-footed shot go over after the Borussia Dortmund man collected another Muhammadi cross from the right flank.
The Pharaohs were not successful in creating much more chances as Rwanda's pressure from midfield and a tight offside trap managed to keep Hassan Shehata's men at bay.
Time was ticking out for Shehata, who hoped to clinch a lead by bringing on debutant striker Ahmed Raouf in Shawky's place on 40 minutes.
The best chance of the half fell to Rwanda when Harouna Niyonzima was clear through on Essam Al-Hadari before the midfielder's chip went inches over the bar.
In the second half, Zidan could have broken the deadlock after another delightful Muhammadi cross found him unmarked, but the forward's effort was kept out by Ndori.
Abd-Rabou had another long-range shot tipped away by the Rwanda guardian before Abou-Treika finally ended his team's jinx by heading the former's cross in the back of the net.
Egypt were awarded a penalty 15 minutes from time when Raouf was brought down in the area by Ndori. Abd-Rabou stepped up to send the keeper the wrong way.
Abou-Treika completed his brace after a successful counter lead by Abd-Rabou and substitute Ahmed Eid was squared to the Ahli man for an easy tap in.