The Nigerian striker notched the winner eight minutes before the break after capitalizing on an error by Cardiff keeper Peter Enckelman.
The Finish guardian could only push John Utaka's right-wing cross into the path of Kanu, who slotted into an empty net.
The winning goal atoned for the former Arsenal man's dreadful miss in the first half.
In the 22nd minute, Kanu should have put Portsmouth ahead when he outpaced defender Roger Johnson before rounding Enckelman.
But his shot from an acute angle was denied by the post, prompting an emotional reaction from coach Harry Redknapp on the touchline.
Redknapp was over the moon following the final whistle, hailing the 'dream' victory.
"It's a fantastic feeling. To win the FA Cup is a dream come true," Redknapp told Sky Sports.
"I'm a family man and it is all for my family. I'm just so pleased for them and so pleased for everyone."
Portsmouth have now qualified for the UEFA Cup. "It's fantastic. I'm looking forward to it (the UEFA Cup)," Redknapp added.