Columns get unhidden upon changes to managed columns

I realize that whenever I make changes to a managed column on a board, it automatically unhides all the columns that I have hidden on the same board. Edits include adding/deleting a label, adding a label description, changing color of the labels, etc.

I find this quite frustrating as I have to hide those columns again every time I make an edit to the managed columns.

Is anyone else having similar experience? Is there a way to get around this? Thanks!