A total of 57.5 percent of more than 17 million accounts voted for him to step down. Musk, who also runs car maker Tesla and rocket firm SpaceX, has not yet reacted publicly to the results.
"The question is not finding a CEO, the question is finding a CEO who can keep Twitter alive," the South African-born billionaire tweeted before the vote closed.
In a response to another tweet, he added: "No one wants the job who can actually keep Twitter alive. There is no successor."
Musk has fully owned Twitter since October 27 and has repeatedly courted controversy as CEO, sacking half of its staff, readmitting far-right figures to the platform, suspending journalists and trying to charge for previously free services.
Analysts have also pointed out that the stock price of Tesla has slumped by one-third since Musk's Twitter takeover. The share price briefly rallied by 3.3 percent on Monday before fading.
"It's hard to ignore the numbers since [the Twitter] deal closed," tweeted investment expert Gary Black, saying he reckoned Tesla's board was putting pressure on Musk to quit his Twitter role.
In discussions with users after posting his latest poll, Musk renewed his warnings that the platform could be heading for bankruptcy.
"Won't happen again"
Resorting to Twitter's polling feature has been a favorite strategy of Musk's to push through policy decisions, including the reinstatement of the account of former president Donald Trump.
Paris-based Reporters Without Borders, which defends press freedom around the world, said the polls were a "crude and cynical" ploy.
"These methods appear to be democratic procedures, but in reality they are... the opposite of democracy," said the group's head, Christophe Deloire.
Unpredictable entrepreneur Musk posted his latest poll shortly after trying to extricate himself from yet another controversy.
On Sunday, Twitter users were told they would no longer be able to promote content from other social media sites.
But Musk seemed to reverse course a few hours later, writing that the policy would be limited to "suspending accounts only when that account's *primary* purpose is promotion of competitors."
"Going forward, there will be a vote for major policy changes. My apologies. Won't happen again," he tweeted.
The attempted ban had prompted howls of disapproval and even bemused Twitter cofounder Jack Dorsey, who had backed Musk's takeover.
Dorsey questioned the new policy with a one-word tweet: "Why?"
Musk has generated a series of controversies in his short reign, one which analyst Dan Ives from Wedbush described as a "perfect storm."
He noted that "advertisers have run for the hills and left Twitter squarely in the red ink potentially on track to lose roughly $4 billion per year."
Shortly after taking over the platform, Musk announced it would charge $8 per month to verify account holders' identities, but had to suspend the "Twitter Blue" plan after an embarrassing rash of fake accounts. It has since been relaunched.
On November 4, with Musk saying the company was losing $4 million a day, Twitter laid off half of its 7,500-strong staff.
Musk also reinstated Trump's account -- though the former US president indicated he had no interest in the platform -- and said Twitter would no longer work to combat Covid-19 disinformation.
In recent days, he suspended the accounts of several journalists after complaining some had published details about the movements of his private jet, which he claimed could endanger his family.
Employees of CNN, The New York Times and The Washington Post were among those affected in a move that drew sharp criticism, including from the European Union and the United Nations.
Washington Post executive editor Sally Buzbee said the suspension of journalist Taylor Lorenz's account "further undermines Elon Musk's claim that he intends to run Twitter as a platform dedicated to free speech."
Some of the suspended accounts have since been reactivated.
On Monday, the head of the European Parliament, speaker Roberta Metsola, sent a letter to Musk inviting him to testify before the legislature, her spokesman said.
The parliament has no power to compel Musk to turn up, and his response was not immediately known.