Formula fields with Mirror Columns I'm not permitted to see

Hi Guys, I got a question here.
So I have a board for our Controlling that is a collection of BudgetIDs each of our three teams can link to. So that we can say some activity with x€ spend is associated to BudgetID 1.
We use the mirror column to mirror those spends back to the Controlling board.

Now we got three teams whose spends we want to track here, and each one of them can only see their own board. In the Controlling board however they should be able to see the sum of all boards and how much is left of that budget.

Now as you can see from that Screenshot everything looks fine for me as an admin as I’m able to see all boards. But if you work in one of those teams they just receive an error message in all formula columns (orange) as they can’t see all those linked columns.

Is there any way to have the formula column always show the correct value of all fields even if you’re not permitted to see all of them?

A dashboard wouldn’t work in this case as we got a lot of (50+) different BudgetIDs here that should be editable very easily. Therefore the linked column would work perfectly if there wouldn’t be this limitation with the fields you’re not permitted to see.