Uruguay, who finished the game with ten men after Nahitan Nandez was sent off in the 74th minute, will face Colombia in Charlotte, North Carolina, in their semi-final on Wednesday.
Colombia, now unbeaten in 27 games, beat Panama 5-0 in Saturday's other quarter-final earlier in Arizona.
The other semi-final, in New Jersey on Tuesday, will see world champions Argentina face surprise package Canada.
For five-times world champions Brazil it was a disappointing early end to a tournament in which they never truly clicked, and a lot of work remains to be done if Dorival Junior's team are to be in shape to compete for the title in the 2026 World Cup.
In a city better known for boxing than for the beautiful game, it was a bruising contest with little quality play and a tournament high 41 fouls.
A poor quality playing field hardly helped with both teams struggling to produce their best football on an uneven surface.
The first - and best - chance of a game of few opportunities came in the 35th minute when Uruguay striker Darwin Nunez had a clear header in front of goal but mis-timed his effort which flew wide off his shoulder.
Within moments, Brazil created an opening of their own with Raphinha breaking clear, but Uruguay keeper Sergio Rochet stayed tall and made a vital save.
Marcelo Bielsa's Uruguay were as tenacious as always, harrying Brazil in midfield and never afraid to interrupt their flow with a foul.
Brazil resorted too often to long balls forward but with only their 17-year-old talent Endrick, in for the suspended Vinicius Junior, as a central striker they lacked the physical presence to make that approach effective.
The game deteriorated the longer it went on with foul after foul, not deterred by lenient refereeing.
But Uruguay's hopes of wearing Brazil down were dealt a blow when Nandez hacked down Rodrygo with a dangerous slide into his ankle and after a VAR review the defender was sent off.
From then on it was clear that Uruguay were simply trying to make it to full-time and penalties and with no extra-time in Copa America, they were able to achieve their aim.
Tight game
After Federico Valverde scored with the first spot kick, Eder Militao saw his effort saved by the diving Rochet.
When Douglas Luiz hit the post for Brazil, they trailed 3-1 and Jose Gimenez had the chance to clinch the game for Uruguay. His effort was superbly saved by Alisson Becker.
Substitute Gabriel Martinelli scored to keep Brazil alive but midfielder Manuel Ugarte kept his cool to drive home the decisive kick and send the 15-times Copa champions into the last four.
Bielsa praised his team's calmness in the shoot-out and their desire during the 90 minutes.
"I'm more seduced by attacking than defending, but I have to appreciate that in a tight game we created one more situation than our opponents, we defended well and we played 15 minutes with one less player, which at this level is a real factor," he said.
"If you ask me if I'm happy with having created three goal situations in 90 minutes, no, I'm not. But our opponents had two," he said.
"Today we outplayed Brazil in segments of the game and were outplayed in others," added the Argentine.
For Brazil the thoughts turn to the need to ensure they make it to the next World Cup after a poor start to CONMEBOL qualifying.
"We leave the tournament undefeated but not satisfied," said Dorival Junior whose team won once and drew twice in the group stage.
"We didn't play at a high level from a technical point of view, but I don't dismiss any of the games. I think there was commitment, fighting spirit. At no time did the team stop going for the result," he added.
"This process needs patience. We have two years to work before the World Cup. The first thing is to qualify for the World Cup because we are sixth in the (South American qualifiers) and it is a position that makes us uncomfortable," he said.
The tournament's final will be held in Miami on July 14.
(For more sports news and updates, follow Ahram Online Sports on Twitter at @AO_Sports and on Facebook at AhramOnlineSports.)
Short link: