Arriving from England in 1920 the young Olivia Spencer Bower was a pupil of Rangi-ruru Girls School, Christchurch. Encouraged by her mother Rosa Bower, also an artist, she quickly established a reputation as one of New Zealand's most gifted practitioners of watercolour. She was one of few women artists of her time who supported herself financially to pursue a professional painting career.  Spencer Bower studied at the Canterbury College School of Art with fellow artists Rita Angus and Rata Lovell-Smith and under tutors Cecil Fletcher Kelly, Richard Wallwork, Leonard Booth and Archibald Frank Nicoll.

She returned to England in 1929 where she studied at the Slade School, London under tutor Henry Tonks . From there she made several extended trips to Europe before she returned to Christchurch two years later. On her return to Christchurch she began to exhibit with 'The Group', a collection of artists who reacted against the way the Canterbury Society of Arts catered for popular taste and excluded younger, more adventurous painters. 'The Group' included Louise Henderson, Doris Lusk, Ngaio Marsh and Evelyn Page. In 1943 she travelled to Northland to paint with Sydney Lough Thompson whom had just returned from Brittany. Later that year she moved to Auckland to take lessons at the Elam School Art under John Weeks , Anna Lois White  and A J C Fisher.  The years that followed saw Spencer Bower visit Queenstown and Kaikoura whilst minding her aged mother. After her mothers death in 1960, she explored the Pacific Islands for three months visiting Fiji, Samoa and Tahiti. On her return she exhibited fifty seven island paintings at the Canterbury Society of Arts. She made several return trips to Europe in the mid 1960s. In 1980, at the age of 75, she received the Canterbury Society of Arts' silver medal for services to visual arts.

A legend in New Zealand art circles, she was easily recognisable at gallery openings by the picture hats she favoured and the cheroots she habitually smoked. She never married, and before her death in 1982 she established a foundation to finance artists' scholarships now funded by her estate. Spencer-Bower worked in numerous mediums including acrylic and oil, and produced a number of linocut and woodblock prints.