I would like to invite people to certain boards but only let them see specific groups. You can currently do this for particular boards in folders, boards or folders in workspaces, even columns on particular boards, but there is no option to set visibility for particular groups.
Yes, please add in group level permissions/restrictions!