Hey Christine,
I am afraid if that setting is configured to show weekends then they will continue to display in the timeline column. It isn’t currently possible to customise the setting per board. I encourage you to cast your vote on the existing feature request here → Control timeline weekends per board or workspace (which I am currently waiting on an update for!).
In the meantime, does this formula work for you?
WORKDAYS({Timeline#End} , {Timeline#Start})
It will count the number of working days in the timeline column - let me know!