Fabio Grosso curled a brilliant left-footer home in the 119th minute before Alessandro Del Piero sealed the victory one minute later through a fast counter-attack.
Italy will face the winners from France and Portugal who clash on Wednesday in the other semi-finals tie.
The Azzurri, whom their last bid for the world title was against Brazil in the 1994, will be playing their sixth title in all-time and seeking their fourth title.
Hosts Germany began in a defensive style, leaving the Italians to take the attacking initiative.
Francesco Totti unleashed a lethal free-kick in the fourth minute but Germany's Jens Lehmann caught it easily before the Arsenal keeper advanced just in the right time to block a free break-in by Simone Perrotta 12 minutes later.
Bernd Schneider had the first chance for Germany in the 33rd minute when he was fed by Miroslav Klose but fired over the crossbar.
Italy continued their domination over the second period but failed to create more chances thanks to a sublime performance from Germany's back four and goalkeeper.
Germany coach Jurgen Klinsmann made his usual substitutions throwing David Odonkor, Bastian Schweinsteiger and Oliver Neuville but they didn't make the expected impact.
On the other hand, Marcello Lippi opted for strikers when he called Vincenzo Iaquinta, Alberto Gilardino and Alessandro Del Piero.
These substitutions put Italy in full charge and they could have score twice just after the start of the first extra time.
Gilardino darted from the right, rounded Michael Ballack inside the area and pinned a low shot that beat Lehmann but hit the left post.
Just one minute later, Gianluca Zambrotta hammered a long-range bullet but again the woodwork stood for Lehmann and Germany.
With the penalty shootout looming, Grosso received a deft through pass from Andrea Pirlo and played a first-timer into the side netting to beat the stretching Lehmann with one minute to go.
Germany attacked in numbers in a desperate bid for an equalizer but they left huge spaces in the defense and Italy never wished for more to seal the win.
Gilardino ran from the left flank, looked one way and slipped the ball into the other for the darting Del Piero who curled the ball into the upper top-corner of the net.