Here's Bill Wyman's view on Stu having to step down:
Andrew Loog Oldham insisted that our pianist, Stu, should step down from the band line-up on stage: his tidy image, he said, was 'completely wrong' for the band. Stu was vital to the band's roots, musically and socially. But to Andrew, his looks - something the band had never cared about - were a major problem. Stu had a particular prominent facial characteristic: an attack of measles at the age of eight had left him with a calcium deficiency that caused his jaw to grow very large. At eighteen he had an operation to correct it, but it remained very visible and he was highly conscious of the abnormality.
Stu also felt that Andrew thought "my hair wasn't long enough. But there was a good reason for this. Bill and I were the only ones working and we just couldn't go around with long hair, or we would have got the sack". Andrew probably knew what a close unit we had become, and was aware of the shock waves his decree had caused, because he offered an olive branch: Stu should remain with the Stones, playing with us on recording sessions and also becoming our road manager.
Fortunately for us, Stu accepted the new role, although reluctantly, and he became incredibly important to us. Outwardly Stu took it well. Brian said to him: "Don't worry about it. You're part of the Stones. You'll always have a sixth of everything". Stu reflected later: "Brian told me all sorts of rubbish but I ignored it". His attitude to Brian, who had so willingly accepted Oldham's order, now became bitter and the tensions between group members began to increase.
Brian's relationship with Mick blossomed temporarily, but there was an underlying feeling that ruthless determination was replacing idealism. I thought that the 'sacking' was a strange way to repay Stu's incredible loyalism.
Source: Bill Wyman, Stone Alone, Penguin Books, 1990.