"rules" will depend on your regulatory/certification/customer requirements. Are you with SQF? For BRC at least, there is a requirement that:
"7.4.2: Protective clothing shall be available that ... is of suitable design to prevent contamination of the product". It will also depend on your product/process/risk and the interpretation guidelines state "suitable for the production processes".
In my place (cooked meat products), we determine that the overalls buttons are fully fastened. This is the rule from the start and enforced on every person going into the factory, no exceptions. We would record it as an issue during internal audits, and no-one wants to be the guy that put audit points on the department score, and a good reason for the area managers to keep on top of it. It probably also helps that if they didn't have the overalls fastened up, they'd likely have meat and gravy on their personal clothes by the end of the day
Are the 'people' factory operatives? If so I'd bring it up in your next management meeting. I'm all up for all members of staff being able to have their say and raise a concern, but I don't think that you as a QA should repeatedly have to justify the hygiene rules to individual staff who refuse, and it certainly shouldn't be optional once set. If it's senior staff, I'd still insist they comply but may be more difficult enforcing it unfortunately.
That said, it may be worth reviewing your clothing requirements if you are unsure whether fully fastened outer shirts are suitable for your production.