Centre-back Yasuyuki Konno netted the decisive penalty, placing the ball into the right-hand side of the goal, after Koo Ja-Cheol, Lee Yong-Rae and Hong Jeong-Ho had all failed from 12 yards for South Korea.
Japan led going into the final minute of extra-time but Hwang Jae-Won slammed in a sensational equaliser following a scramble inside the Japanese box to make the score 2-2 and send the game to penalties.
It was a redemptive moment for Hwang, who had conceded the controversial penalty from which substitute Hajime Hosogai had given Japan the lead in the seventh minute of extra-time, but his joy proved short-lived.
Ki Sung-Yueng had given South Korea the lead via another disputed penalty mid-way through the first half, with Ryoichi Maeda drawing Japan level nine minutes before half-time.
Japan will play the victors of Tuesday's second semi-final between Australia and Uzbekistan in the final on Saturday.
In front of just over 16,000 fans at Doha's 22,000-capacity Al Gharafa Stadium, South Korea withstood early Japanese pressure before coming close to making the breakthrough themselves in the 16th minute.
Ki's precise free-kick from a tight angle on the left drew a superb, athletic save from Eiji Kawashima and Lee Chung-Yong's goal-bound follow-up effort was headed away from danger by Konno.
A minute later it was Japan's turn to threaten and the South Korean goal was fortunate to survive intact as Shinji Okazaki's downward header was scrambled onto the left-hand post by Jung Sung-Ryong.
Park Ji-Sung, making his 100th international appearance, made an inauspicious start to the match when he was booked for elbowing Makoto Hasebe, but he redeemed himself by winning the penalty from which Korea took the lead.
The Manchester United midfielder was barged to the floor by Konno as he chased a ball into the box and, following a patient run-up, Ki swept the resulting spot-kick into the bottom-left corner.
The referee rejected Korean appeals for a second penalty, after Park tumbled in the box again under a challenge by Maeda, and in the 36th minute Japan drew level.
Enterprising left-back Yuto Nagatomo got in behind Cha Du-Ri on the Japan left and toed the ball square for Maeda to tuck away the equaliser from six yards.
Korea came desperately close to establishing a 2-1 lead in the 72nd minute, but Lee Yong-Rae's gentle dead-ball effort from the edge of the Japanese 'D' missed the right-hand upright by a whisker.
A lack of clear-cut goalscoring opportunities saw the game drift into extra-time and the refereee stunned Cho Kwang-Rae's side when he awarded a penalty for Hwang's block on Okazaki on the periphery of the penalty area.
Jung dropped to his right to repel Keisuke Honda's soft spot-kick, but the in-rushing Hosogai crashed the rebound into the roof of the net from close range to put Japan on the brink of the final.
Hwang's dramatic equaliser prolonged the drama even further, but Korea's nerve faltered in the shoot-out, with Kawashima saving from Koo and Lee Yong-Rae, while Hong missed the target completely.