We reported this issue to monday.com support a few months ago. The macOS notifications in Monterey from the momday.com app disappear whether they are set to be a banner (in which case they are supposed to disappear) or an alert (in which case they are supposed to remain in the upper right until acknowledged. It is as if the alerts dismiss themselves, since the banners retreat to the notification center while the alerts disappear completely.
Monday.com support at first reported that they cannot reproduce the issue, then claimed it was a macOS issue (couldn’t be since it’s the only app that behaves that way) then declared it wasn’t a priority.