The estimated 500 demonstrators danced, chanted slogans and marched around Narayanghat, a town about 100 miles (160 kilometres) south of the capital of Katmandu on Sunday, which is also Nepal's traditional festival of Gaijatra honouring the dead.
The gay community has been holding demonstrations on the festival day in the capital, but it was the first time that activists have organised a rally outside the capital.
Gay rights activists and parliament member Sunil Pant said their main aim was to spread their campaign outside the capital city.
Nepal, a traditionally conservative, mostly Hindu nation, has only recently begun accepting homosexuality. The country is also in the process of writing a new constitution which could include provisions to guarantee rights for the sexual minorities.
Pant said the major political parties have agreed to include the rights of the sexual minorities in the new constitution. However, the new constitution that was due to be promulgated last year has been delayed following differences among the political parties on various other issues.
In the town of Narayanghat, the rally was led by two decorated elephants which was followed by musical bands drawing the local people to the streets. Many of them wore saris that were bright red, pink, yellow.
They danced to the music, and holding banners that said: "Sexual rights for all. Lesbian, gay, bisexual and transgender right are human rights." "We want spread awareness among the people in the rural areas too. People in the capital mostly accept the idea but we need to spread the awareness," Pant said.
Anee Lama, who had traveled all night on a bus from her hometown in east Nepal with her friends, said it there were positive changes being made in the country but there was still more that needs to be done.
"In the past three years we have seen a lot of changes for people like us. We could not have imagined taking part in a rally like this just few years ago or being accepted by family and friends for who we are," Lama said. "But now we are campaigning for the people in the villages and rural parts to come out and join us." Until a few years ago, gays and lesbian and transgenders were not able to declare their sexual choice in the open.
According to the Blue Diamond Society that advocates for sexual rights in Nepal, transvestites were beaten by both the public and police and were discriminated.
Pant said more people are coming out in the open to declare their sexual preference in the urban areas, but people in rural villages are still afraid.