Good question. I would agree that the non-conformance, at least based on how you have described it, sounds a bit odd.
Are you able to share the exact wording with us?
Clause 4.11.8.1 simply requires that the environmental monitoring programme is risk-based and includes the documented sampling procedures, locations, frequency, target organisms, methods, and recording/evaluation of results. No where does it actually specify when sampling must occur (e.g. pre-operational, during production, or post-production) nor does it specify a frequency.
The key question should really be whether the risk assessment supports your chosen approach, rather than whether the sampling is conducted at a particular point in time.
In my experience pre-operational swabbing is commonly used to verify the effectiveness of cleaning and sanitation before start-up, and this can be perfectly reasonable depending on the product and process risk. Sampling during production tends to be more in situations where the objective is to detect contamination introduced during processing (for example personnel activity, product contact, or environmental transfer), particularly in high-risk/high-care ready-to-eat environments. Again there is no specific requirement, it's based on the output from the risk assessment.
For the non-conformance to be justified the auditor would need to demonstrate that your risk assessment is insufficient and/or does not adequately justify why sampling is limited to pre-operational inspections, rather than simply stating that swabbing should be conducted during production.
If they have actually raised this as a non-conformance, then depending on the wording I'd be exploring the appeal route and highlighting that it is not possible to raise a non-conformance against something that is not a requirement.