Jump to content

  • Quick Navigation
Photo
- - - - -

Audit Query on validation of metal detector


  • You cannot start a new topic
  • Please log in to reply
13 replies to this topic

#1 James Dough

James Dough

    Grade - AIFSQN

  • IFSQN Associate
  • 30 posts
  • 1 thanks
1
Neutral

  • Canada
    Canada

Posted 03 December 2018 - 07:06 PM

I was just audited 2 weeks ago and encountered questions regarding validation.

I presented the auditor with our procedures and calibration and monitoring frequency, as well as a metal detector contractor (expert) annually checking our detectors and proving they are working as they should. The auditor would not accept this as validation, only verification. For Validation I had to present fact based evidence of why we have the threshold we do and the background how we came to that decision.

based on some of the previous posts I see what seem to be verification activities used as validation activities.

food for thought?


Edited by Charles.C, 04 December 2018 - 06:09 AM.
assumed SQF, split from 33665. which related USDA


#2 Scampi

Scampi

    Fellow

  • IFSQN Fellow
  • 3,123 posts
  • 848 thanks
441
Excellent

  • Canada
    Canada
  • Gender:Not Telling

Posted 03 December 2018 - 07:16 PM

A validation should be completed prior to running the product (or when changes are proposed)

 

I agree James, the GFSI's have muddied the water to the point that no one is really understanding the difference between validate and verify


Please stop referring to me as Sir/sirs


#3 012117

012117

    Grade - MIFSQN

  • IFSQN Member
  • 227 posts
  • 68 thanks
33
Excellent

  • Earth
    Earth
  • Gender:Male
  • Location:Philippines
  • Interests:Validation, basketball, chocolatier

Posted 03 December 2018 - 11:41 PM

Hi, James.

 

Question though.

 

1. Between any of the activity conducted such as calibration, monitoring activity and the activity conducted by the metal detector expert, is there any instance that for a particular (or group) of product that the metal detector settings have been changed? If yes, what could be  the cause of the change in settings? 

 

2. For the settings to be established, what is the procedure? Also between the 3 activities, how is it different, the number of testing the test pieces?

 

The reason I asked the question is that, if everything still stands for the 1st time since it was commissioned and no interference from the environment (such as heavy vibration, good grounding etc) that the setting should not change as you have established it based on the worst case scenario. If the settings changes from time to time for a specific product then it could not be "validated".

 

The scheme of what we do, the difference is that for:

 

Validation (checked initially with 3rd party expert) - the installation for MD is 1st check for any possible interference and wand test is performed. Setting is established step considering nature of product and its orientation (including line speed). The efficiency of the MD is established. The efficiency in terms of it should only alarm in the presence of contaminant. Then with the settings established, statistically checked for efficiency of metal detection (with test piece) and have a conclusion.

 

Verification - months data is review in terms rejection rate without contaminant being found. Check settings, if found the same, pass test piece on a lesser repetition.

 

3rd party contractor (the MD expert) check motherboard and if still ok, do wand test to check interference,if ok, will be left as found and just test certain number of pieces as "handshake". If found with defective then, rectify issue, if it changes setting, "revalidation" would be necessary.



#4 James Dough

James Dough

    Grade - AIFSQN

  • IFSQN Associate
  • 30 posts
  • 1 thanks
1
Neutral

  • Canada
    Canada

Posted 03 December 2018 - 11:49 PM

Hello,

 

No the settings have never changed, the threshold is based on initial setup and test wands used based on CFIA/Health Canada stating that over 2.0 mm size is severe risk. That statement is ultimately the basis of our validation, all settings, monitoring, calibrating and verification activities are based on that 2.0 mm rule. 



#5 Charles.C

Charles.C

    Grade - FIFSQN

  • IFSQN Moderator
  • 18,042 posts
  • 5052 thanks
1,071
Excellent

  • Earth
    Earth
  • Gender:Male
  • Interests:SF
    TV
    Movies

Posted 04 December 2018 - 06:18 AM

Hi James,

 

Some context necessary, eg Standard, clause. eg validation of  what ?

 

i assumed the post no.1 above was  SQF-related ?. If not please clarify.

 

If SQF, the cause/explanation is likely simple, eg -

 

As detailed in several older threads here, SQF have a long-running problem due their official / practical  interpretation(s) of validation.

 

This culminated in an attempted clarifying statement on their website (ca 2015-6) which however appears to sometimes be inconsistently implemented in the field.

 

PS - If this actually related to CFIA, looks like they hv a similar problem to SQF.


Kind Regards,

 

Charles.C


#6 Scampi

Scampi

    Fellow

  • IFSQN Fellow
  • 3,123 posts
  • 848 thanks
441
Excellent

  • Canada
    Canada
  • Gender:Not Telling

Posted 04 December 2018 - 02:43 PM

To validate a metal detector you need to run trials in worst case scenarios with all 3 types of metal to ensure that the detector will reject any/all pieces that come through above the size limits that have been established. 

 

What you gave your auditor is the BACKGROUND on your parameters for the metal detector limits...........but you did not mention actual data where the detector was tested and tested again VALIDATING that is will function as intended under the worst possible conditions (2 units stacked on top of each other/ check weigher not working and food was larger than normal/packaging had double thickness/line speed faster than normal etc etc)  The information you provided is a PART of the validation but you need to also do the testing. 

 

Generally speaking to VALIDATE you need a large number of data points to prove a CCP functions as intended

 

FSEP validation (cfia) 3.3 Validation

Every establishment shall demonstrate that the critical control points are capable, on a consistent basis, of achieving the intended level of hazard control.

Validation is performed at the time the CCP is designed, or when changes indicate the need for re-validation. Validation of a CCP is, whenever possible, performed before it is fully implemented.

Depending on the CCP that is being validated, the validation documentation may include:

  • Scientific, technical or regulatory support to demonstrate that the selected critical limit is effective for control of the hazard.
  • Commissioned testing data specific for a piece of equipment (e.g. pasteurizer) to demonstrate that the equipment is capable of meeting the selected critical limit.
  • Supporting data to demonstrate that the monitoring procedures are effective enough to detect loss of control at a CCP before the finished product leaves the control of the producing establishment.

CFIA may request validation documentation for any control measures within the HACCP system that have an immediate impact on food safety.


Please stop referring to me as Sir/sirs


#7 Charles.C

Charles.C

    Grade - FIFSQN

  • IFSQN Moderator
  • 18,042 posts
  • 5052 thanks
1,071
Excellent

  • Earth
    Earth
  • Gender:Male
  • Interests:SF
    TV
    Movies

Posted 04 December 2018 - 06:10 PM

To validate a metal detector you need to run trials in worst case scenarios with all 3 types of metal to ensure that the detector will reject any/all pieces that come through above the size limits that have been established. 

 

What you gave your auditor is the BACKGROUND on your parameters for the metal detector limits...........but you did not mention actual data where the detector was tested and tested again VALIDATING that is will function as intended under the worst possible conditions (2 units stacked on top of each other/ check weigher not working and food was larger than normal/packaging had double thickness/line speed faster than normal etc etc)  The information you provided is a PART of the validation but you need to also do the testing. 

 

Generally speaking to VALIDATE you need a large number of data points to prove a CCP functions as intended

 

FSEP validation (cfia) 3.3 Validation

Every establishment shall demonstrate that the critical control points are capable, on a consistent basis, of achieving the intended level of hazard control.

Validation is performed at the time the CCP is designed, or when changes indicate the need for re-validation. Validation of a CCP is, whenever possible, performed before it is fully implemented.

Depending on the CCP that is being validated, the validation documentation may include:

  • Scientific, technical or regulatory support to demonstrate that the selected critical limit is effective for control of the hazard.
  • Commissioned testing data specific for a piece of equipment (e.g. pasteurizer) to demonstrate that the equipment is capable of meeting the selected critical limit.
  • Supporting data to demonstrate that the monitoring procedures are effective enough to detect loss of control at a CCP before the finished product leaves the control of the producing establishment.

CFIA may request validation documentation for any control measures within the HACCP system that have an immediate impact on food safety.

 

Hi Scampi,

 

The FSEP Validation definition is chronologically  commendable but IMO undermined by words like  "whenever possible" (it is unclear what the preceding word "consistent" refers to, Validation + Verification?). Auditors (especially SQF) may interpret the "whenever .."  as a  comingling of Validation with Verification. Hence  confusion.

 

IMO it is "preferable" to document a specific one-off  Validation Event  for the relevant Limit/Range  etc. despite potential chronological difficulties.

 

FSIS attempted to dilute the confusion by inventing "on-going" Validation. Debatable.

 

Auditees may be obliged to take one's "Pick" from the above options (or Others) as per Auditor's individual preference/interpretation.


Kind Regards,

 

Charles.C


#8 Scampi

Scampi

    Fellow

  • IFSQN Fellow
  • 3,123 posts
  • 848 thanks
441
Excellent

  • Canada
    Canada
  • Gender:Not Telling

Posted 04 December 2018 - 06:27 PM

Validate "whenever possible"  before it's implemented.......CFIA will grant permission to run a regulated commodity PRIOR to validating a CCP (usually if you agree to put all product on hold pending the study).  I had to VALIDATE my blast freezer to meet time/temp requirements...........I couldn't do that without running product and I couldn't establish my CCP limits until we had............so if they had forced me to VALIDATE prior to running.........we never would have started.  Charles, remember this is federal regulation and as such, the language has to be very particular.

 

Consistent---pretty straight forward.........it means you will almost always operation within your CCP range....if your record show CONSISTENT deviations than the CCP is not functioning as intended and you would have to perform a VALIDATION again as the CCP would have to change 

 

I personally think that GFSI are the ones who have muddy the water........CFIA FSEP program has consistently met international standards for being "best in class" for all encompassing HACCP + program

 

 

I do agree about "on-going" validation...........that is simply verification

 

If you look to SQF definitions.........the same the same d*** thing for both verification and validation and they wonder why people are confused never mind that Codex does NOT list them this way???  Why GFSI aren't working?????

 

Validation

As defined in the Food and Agriculture Organization’s CODEX Alimentarius Commission. Hazard Analysis and Critical Control Point (HACCP) System and Guidelines for its Application – Annex to CAC/RCP 1 – 1969, Rev. 4-2003), – “A system, which identifies, evaluates and controls hazards which are significant for food safety. Essentially validation as applied to control limits seeks to prove that the intended result was achieved and that it actually worked.

 

Verification

As defined in the Food and Agriculture Organization’s CODEX Alimentarius Commission. Hazard Analysis and Critical Control Point (HACCP) System and Guidelines for its Application – Annex to CAC/RCP 1 – 1969, Rev. 4-2003), – “A system, which identifies, evaluates and controls hazards which are significant for food safety. Essentially verification as applied to control measures seeks to prove that the control measure was done according to its design


Please stop referring to me as Sir/sirs


#9 Charles.C

Charles.C

    Grade - FIFSQN

  • IFSQN Moderator
  • 18,042 posts
  • 5052 thanks
1,071
Excellent

  • Earth
    Earth
  • Gender:Male
  • Interests:SF
    TV
    Movies

Posted 04 December 2018 - 06:57 PM

hi Scampi

 

I deduce you anticipate that the OP was referring to a Federal Audit..

 

Awaiting James input.

 

and, ahem, Codex 2008 -

 

Validation:    Obtaining  evidence  that  a  control  measure  or  combination  of  control  measures,  if  properly implemented, is capable of controlling the hazard to a specified outcome.

Verification: The application of methods, procedures, tests and other evaluations, in addition to monitoring, to determine whether a control measure is or has been operating as intended

 

--------------------------

 

NACMCF 1998 -

 

Validation – that element of Verification focused on collecting and evaluating scientific and technical information to determine whether the HACCP plan, when properly implemented, will effectively control the hazards.

--------------------------------

 

I doubt that GFSI express any specific preference ?. They usually stay neutral. Playing it Safe.

 

Regardless, i applaud yr Patriotism. :thumbup:


Kind Regards,

 

Charles.C


#10 dfreund

dfreund

    Grade - MIFSQN

  • IFSQN Member
  • 68 posts
  • 15 thanks
9
Neutral

  • United States
    United States
  • Gender:Male

Posted 05 December 2018 - 08:13 PM

This may be oversimplifying but does your program documentation have scientific/regulatory backup to prove the size of metal you are detecting will eliminate "hazardous" foreign metals.  There are allowable sizes where there is no expected injury.  No possible injury means control is established. 

 

I bet your federal/ local regulatory codes will be sufficient for establishing the limit of detection.  That may be what the Auditor is looking for.



Thanked by 1 Member:

#11 mgourley

mgourley

    Grade - FIFSQN

  • IFSQN Fellow
  • 1,253 posts
  • 927 thanks
202
Excellent

  • United States
    United States
  • Gender:Male
  • Location:Plant City, FL
  • Interests:Cooking, golf, firearms, food safety and sanitation.

Posted 05 December 2018 - 09:39 PM

It may be a bit of an over simplification, but I have been under the impression that "validation" is something that is done PRIOR to installing a piece of equipment (or whatever) to ensure that equipment (or whatever) will eliminate or reduce to an acceptable level some hazard at the process step.

In the case of a production line that may have been running for years, it's pretty much a given that no actual "validation study" was ever done. About the best that can be done is take your current critical limits and pass through test pieces smaller than the current limits and see if the metal detector will reject these in ALL products that run through this detector.

I guess it's not "technically" validation, but it will show that your current critical limits are the lowest  they can go.

 

And count me in on the bastardization of validation and verification terms. If your critical limits are at or below whatever requirement there is, and your monitoring records show the detector is properly working when challenged, what more really is needed?

 

Marshall



#12 James Dough

James Dough

    Grade - AIFSQN

  • IFSQN Associate
  • 30 posts
  • 1 thanks
1
Neutral

  • Canada
    Canada

Posted 25 March 2019 - 03:49 PM

Hi Charles and Scampi,

 

Apologies for not replying, I was not getting notice of addtional posts.

I was referring to SQF, and thank you both for the information you gave.

The initial commisioning of the units was quite a while ago and we have no "real documentation" of how the test strip sizes were decided. I know the CFIA references Health Canada and the statement that anything over 2.0 MM is a severe risk.

Our test strips are 2.0 mm and under in size. We run them at the start of each product, check every hour, and at the end of production.

Annually we have the metal detector "expert" in to check that the machines are doing what they are supposed to.

we run unpackaged frozen product through, never double stacked or in packaging, the belt speeds have never changed.

 

So now for Validation, we have the CFIA standard that they reference (2.0 mm), we have all the documentation that we check it when we say we do, and we have the "expert" verifying that the detector is operating as it should.

will this be enough moving forward?

 

Also, another quick question, for the annual visit from the expert, if the anniversary date is in April from his last visit, and he does not come until July, is this still considered "annual"?

 

James



#13 Scampi

Scampi

    Fellow

  • IFSQN Fellow
  • 3,123 posts
  • 848 thanks
441
Excellent

  • Canada
    Canada
  • Gender:Not Telling

Posted 25 March 2019 - 04:32 PM

Nope, July is not annual as you have now gone 15 months between checks. 

 

I would also try and correlate at least a month (assuming you run 5/week) of data to prove control. So you'll need to create some sort of excel sheet that shows when the checks were done, what product you were running and the pass/fail rate.  Then you should have everything you need to satisfy CFIA.

 

VI. Validation documentation/Evidence of effectiveness

You validate, prior to implementing your HACCP plan, the effectiveness of the critical limits established for each CCP in controlling the hazard(s) identified. You have evidence showing that all the control measures you apply, to prevent, reduce or eliminate hazards, are effective.

The information used to conduct a validation includes:

  • scientific or technical publications, regulatory guidelines and standards on the critical limits or control measures applied
  • commissioned testing data specific for a piece of equipment (for example, pasteurizer) to demonstrate that the equipment is capable of meeting the selected critical limit
  • records you generate and maintain during the implementation of your FSEP
  • results of your monitoring and verification activities

http://inspection.gc...25869759693#a66


Please stop referring to me as Sir/sirs


Thanked by 1 Member:

#14 James Dough

James Dough

    Grade - AIFSQN

  • IFSQN Associate
  • 30 posts
  • 1 thanks
1
Neutral

  • Canada
    Canada

Posted 25 March 2019 - 05:06 PM

Hi Scampi,

 

Great, thanks so much for your input!!

The knowledge/experience you and Charles and others share is fantastic.

 

James






0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users