City failed to produce an attempt on target at St Mary's on Wednesday as they slipped to a 2-0 quarter-final defeat, courtesy of goals from Sekou Mara and Moussa Djenepo.
Not even the second-half introductions of substitutes Kevin De Bruyne and Erling Haaland could spark a fightback for the visitors, who have won the competition eight times.
The Premier League champions return to action against Manchester United at Old Trafford on Saturday and Gundogan said there was no margin for error.
"There's no room for any mistakes, or for any poor performances," said the Germany international, whose team are five points behind pacesetters Arsenal.
"Hopefully, if there is something good we can take out of this game, it was something like a wake-up call at the right time.
"Even though it is sad and disappointing to be out of this cup, hopefully, at the end, in a few weeks we can at least look back and take something good out of this game."
City boss Pep Guardiola said City had "no chance" of beating in-form United if they repeated their dismal display against Southampton.
"Every game is different, different competition," he said. "But of course, if we perform in this way, we will not have a chance.
"We know exactly what we have to do to play better, so that's what we're going to try to do."
Under-pressure Southampton manager Nathan Jones celebrated the best result of his short tenure.
The Welshman believes the victory, which set up a two-legged semi-final with Newcastle, provides optimism for the future as he turns his attention back to the top-flight relegation battle.
"City are one of the best sides in the world, with the most innovative manager of all time," said Jones, whose side are bottom of the table ahead of Saturday's visit to fellow strugglers Everton.
"We knew we had to be at our best but we showed from minute one to 96 and a half that we can be a very good Premier League side."
(For more sports news and updates, follow Ahram Online Sports on Twitter at @AO_Sports and on Facebook at AhramOnlineSports.)
Short link: