Hi Alain,
It sounds like you might be using this button outside of a Ribbon or ToolBar? If so, the focus is probably remaining on the button after you click it and the button template does have it show a "state" background when it's focused. To change this you could either move focus off the control after it's clicked or put it in a container that is a focus scope (FocusManager.IsFocusScope="true").