The win, which comes right after Algeria's 3-1 win over Egypt, lifts Rabeh' Saadane's team to first spot in qualifications Group C with seven points from three games.
Zambia are second with four points, followed by Rwanda and Egypt, who are level on one points as the Pharaohs have a game in hand.
Madjid Bougherra and Rafik Saifi scored on 21 and 66 minutes to secure the crucial win for the Desert Foxes.
Lounes Gaouaoui was the hero of the day for the visitors as the keeper managed to block a wave of Zambian successive attacks in both halves.
Algeria took advantage of their opponent's sloppy defence when Bougherra looped his header past the advancing Zambian guardian.
The second goal came courtesy of a counter attack led by Karim Ziani, who squared to Saifi to tap in from close range.
Algeria will next meet both Zambia and Rwanda at home, before their final group game against Egypt in Cairo.