But the US government had not given up its pursuit of one of the planners of the September 11, 2001 attacks and the heir to Osama bin Laden.
After years of tracking him down, US armed forces fired two Hellfire missiles from a drone flying above the Afghan capital, striking Zawahiri's safe house and killing him, President Joe Biden announced on Monday.
US officials described an operation as meticulously planned as that which killed bin Laden in his Pakistan hideout in 2011.
That the leader of the violent jihadist group was in Afghanistan was not surprising: since the hard-line Islamist Taliban regained control in August, Al-Qaeda has felt more at home, analysts say.
But finding him was still hard.
"For several years the US government has been aware of a network that we assessed supported Zawahiri," a senior administration official told reporters.
But it was only this year that US intelligence learned that his family, his wife, his daughter and her children, had moved to the Afghan capital.
They were careful, the official said, exercising "longstanding terrorist tradecraft" to prevent anyone tracking them to the Qaeda leader.
Still, eventually Zawahiri showed up, and never left.
"We identified Zawahiri on multiple occasions for sustained periods of time on the balcony," the official said.
An attack plan developed over May and June. The United States constantly monitored the multi-story residence -- just how the official would not say -- to understand the family's pattern of life.
Detailed Safe-House Model
They studied the construction of the home, aiming to hit Zawahiri without threatening the building's structural integrity, to minimize the risk to civilians.
Defense and intelligence officials finalized the plan in June and presented it to Biden in the White House on July 1, using a detailed model of the residence, as was done before the bin Laden raid.
Biden asked detailed questions on the structure, weather issues, and the risk to civilians, the official said.
Finally, on July 25, Biden, still ill with a bout of Covid-19, made the decision.
It took place with key cabinet officials joining the final briefing, echoing the April 28, 2011 White House meeting where president Barack Obama decided to deploy US special operations troops to enter Pakistan and get bin Laden.
At that time Biden was vice president, and he expressed doubts. The risks of things going wrong were high, bin Laden had not been clearly identified, and relations with Pakistan could suffer, he later recalled.
With Zawahiri, however, no US troops would enter the country; Zawahiri was clearly identified; and relations with the Taliban were next to nil.
At the end of the discussion on the 25th, Biden -- as Obama had done 11 years earlier -- asked each participant for their view.
"All strongly recommended approval of this target," and Biden gave the go-ahead, the official said.
The strike involved a US drone, armed with two precision-guided Hellfire missiles, which were launched at 6:18 am Sunday, Kabul time.
Zawahiri was "killed on the balcony," the official said.
The missiles appear not to have been ordinary Hellfires, whose high explosives could have destroyed the home.
'Significant Blow To Al-Qaeda'
Apparent photographs of the building show only a few windows on one floor blown out, and the rest intact.
That points to the likely use of a non-explosive version of the Hellfire, the R9X, which deploys a series of knife-like blades from its fuselage and shreds its target but leaves nearby people and objects intact.
The so-called "flying ginsu" missile has been used a half-dozen or more times by US forces to kill other jihadist group leaders without hurting bystanders.
The official did not provide details, but expressed strong confidence that Zawahiri had been killed and no others were hurt.
"Zawahiri's family members were present in other parts of the safe house at the time of the strike and were purposely not targeted and were not harmed," he said.
The official said the strike "deals a significant blow to Al-Qaeda and will degrade the group's ability to operate."
"As President Biden has consistently said, we will not allow Afghanistan to become a safe haven for terrorists who might bring harm to Americans," the official said.