SplitButton changes behavior depending on VariantSize

Hello,

I am adding to my ribbon a control that like Powerpoint lets you change the fill/border etc color of a shape.

For it I am using the ribbon  SplitButton.

It has an image and a label.

Now If I set the VariantSize to Large (which I do not want really)  I see that the label is selected together with the dropdown arrow(which is what I want).

If I set the VarianSize to Medium or Small (which I want) I see that the label is NOT selected together with the dropdown arrow(which I do not want).

How can I achieve the Powerpoint type behavior without having to set the VariantSize to Large?

Thanks.