He was born on 30th/06/1944, in Bumbaire Village, Igara sub-county, in present-day Bushenyi District in the Western Region of Uganda. He attended pre-primary school in the church building at Bweeza, Bushenyi District. He went on to attend Ibaare Primary School before he joined Ibanda Preparatory Seminary from 1958 until 1960. In 1961, he joined Kitabi Seminary, where he graduated with a High School Diploma. He was admitted to Bukalasa Minor Seminary, in present-day Kalungu District for his A-Level studies but he did not complete. He was expelled, along with others, for “indiscipline”.
Bakyenga was ordained a priest on 11th July 1971 at Mbarara at the age of 27. He served as a priest in the Roman Catholic Archdiocese of Mbarara, until 6 March 1989. Bakyenga was appointed Coadjutor Bishop of Mbarara, Uganda on 6 March 1989. He was consecrated as bishop on 24th June 1989 at Mbarara by Bishop John Baptist Kakubi, Bishop of Mbarara.
On 23th November 1991, he succeeded as bishop of Mbarara, replacing the late Bishop John Baptist Kakubi, who resigned. He was appointed Archbishop of Mbarara on 2 January 1999.
He retired as archbishop officially on 20th June 2020 and handed over to Archbishop Lambert Beinomugisha .
As an Archbishop, he made annual visits to Mbarara University of Science and Technology every Divine Mercy Sunday (the first Sunday after Easter. The Late was at the forefront of helping the Corpus Christi Catholic Community of the University acquire land where the chaplaincy has a home and a hostel. He will forever be missed by the MUST community. Bakyenga died in on 18th July 2023, at Nsambya Hospital in Kampala at the age of 79.