After Japan midfielder Shinji Kagawa had put his side ahead, Dortmund were less that three minutes away from victory at Hanover on Sunday before goals in the 87th and 89th minute condemned them to a 2-1 defeat.
Having also been beaten by Hertha Berlin at home the week before, Dortmund have now suffered three defeats in six league games this season after also losing to Hoffenheim last month.
The 1-1 home draw with Arsenal last Tuesday in their opening Champions League group game can not disguise the fact Dortmund are showing little of the composure and patience which brought them so much success last season.
Having won last season's Bundesliga title by seven points, Dortmund are already eight adrift of league leaders Bayern with six games gone, but Klopp rejects the 'false start' notion.
"It's not a false start, but we are not satisfied and we have too few points," said Klopp with his side to face mid-table Mainz on Saturday.
"Now we are certainly under pressure at Mainz.
"Statistically, we were leading in every part of Sunday's match, apart from the result.
"From the 70th minute against Hanover, we let a good win escape from our hands, we were passive.
"A defeat like this doesn't make much sense, unless you learn from it.
"Our performance was okay, but the result was not."
His players were equally unimpressed with the lack of focus which allowed Hanover captain Karim Haggui and Ivory Coast striker Didier Ya Konan to score in the 87th and 89th minutes respectively.
"In the end, we did not defend properly, but in the Bundesliga, you can only succeed if we are fully focused for 90 minutes," said Dortmund's Germany defender Marcel Schmelzer.
Dortmund are expected to have teenage star Mario Goetze, captain Sebastian Kehl and possibly Germany midfielder Sven Bender back for Saturday's trip to Mainz as the trio are expected to return from either suspension or injury.
"We will found our way back to form," said Kagawa, who was one of the few players to come through Sunday's defeat with credit.