As far as we know there’s currently no option to fetch all files added to the board, only those added to items. If a file is added directly to the file gallery, we can’t get it via API. It would be really useful if there was an endpoint for that!
(or if it’s possible, please let me know how to do it 
)