19 replies to this topic

[jfrey123]

Hello everyone,

 

Long time lurker, and figured I might tap some of the village knowledge for insights here.

 

I'm currently working as an FSQA Analyst, and I get to compile lab and sanitation data for multiple fruit and vegetable processing plants.  Long story short, I've begun to suspect one of our plants is "too clean" on paper.  We have some additional circumstances calling them into question, and we've sent someone from our corporate office to oversee an onsite investigation. However, I have a specific question for all when reviewing their ATP history:

 

How often are you receiving a reading of 0 RLU in your plants?

 

I'm used to working with ATP readers, I have experience using them to verify cleanliness, but this plant stands out in my data reporting.  They stand out as the only plant that consistently scores 0 RLU on initial cleanings.  In the 4 months of this year, they score a 0 RLU 82.4%, 97.3%, 96.8%, and 100% of the time (an average of 200 swabs taken per month).  Their outstanding looking results do not align with data from the other 6 plants.

 

We are using Hygiena Ultra-Snaps across the entire company.  Machines show proper calibration through the online portal.  I've wanted to give them credit for simply being excellent at sanitation, but other factors are leading us to question the circumstances.

 

Any thoughts or insights would be greatly appreciated.

[Scampi]

A zero means 1 of 2 things

 

A) they are not actually swabbing

 

B) they are swabbing post sanitizer - but to do that they'd have to wait until it's dry see A)

 

I just checked my raw data---since Oct of last year, the lowest achieved is a 2, and that is very rare AND i know this piece of equipment is A) easy to clean and B) not prone to buildup throughout the production day

On the same night, they also got a result of 126 on a different area, so clearly doing the work

[SQFconsultant]

Had a client that had a similar situation.

They choae to have an independent lab conduct a surprise visit along the qa durector of the parent conpany and a seceuity agent (considering this is potential fraud they felt it pudent and what was found was that it was not being done - lots of people got fired as there were other issues as well.

[Charles.C]

Hello everyone,

 

Long time lurker, and figured I might tap some of the village knowledge for insights here.

 

I'm currently working as an FSQA Analyst, and I get to compile lab and sanitation data for multiple fruit and vegetable processing plants.  Long story short, I've begun to suspect one of our plants is "too clean" on paper.  We have some additional circumstances calling them into question, and we've sent someone from our corporate office to oversee an onsite investigation. However, I have a specific question for all when reviewing their ATP history:

 

How often are you receiving a reading of 0 RLU in your plants?

 

I'm used to working with ATP readers, I have experience using them to verify cleanliness, but this plant stands out in my data reporting.  They stand out as the only plant that consistently scores 0 RLU on initial cleanings.  In the 4 months of this year, they score a 0 RLU 82.4%, 97.3%, 96.8%, and 100% of the time (an average of 200 swabs taken per month).  Their outstanding looking results do not align with data from the other 6 plants.

 

We are using Hygiena Ultra-Snaps across the entire company.  Machines show proper calibration through the online portal.  I've wanted to give them credit for simply being excellent at sanitation, but other factors are leading us to question the circumstances.

 

Any thoughts or insights would be greatly appreciated.

Hi jfrey,

 

I presume by "initial Cleaning" you mean at the start of Production after Cleaning was previously carried out ?

 

How is the measurement Procedure / Baseline Validated in this / other Plants? And the Results ?

 

How about the results for the Hygiena Positive Control Test ?

 

https://help.hygiena...ve-control-kit/

 

I deduce your opinion is that either data is simply fictitious or instrument/Procedure (only one machine in use ?) is (grossly) incorrect. Or both.

 

Either should be fairly obvious IMO to an appropriately trained independent evaluator.

[jfrey123]

Hi jfrey,

 

I presume by "initial Cleaning" you mean at the start of Production after Cleaning was previously carried out ?

 

How is the measurement Procedure / Baseline Validated in this / other Plants? And the Results ?

 

How about the results for the Hygiena Positive Control Test ?

 

https://help.hygiena...ve-control-kit/

 

I deduce your opinion is that either data is simply fictitious or instrument/Procedure (only one machine in use ?) is (grossly) incorrect. Or both.

 

Either should be fairly obvious IMO to an appropriately trained independent evaluator.

 

Fair question regarding my "initial cleanings" comment.  The results at this plant are consistent between post-sanitation swabbing as well as pre-op swabbing.  The percentages for 0 RLU I show is for their entire history, tests performed in the AM and PM and for all activities where ATP swabs are taken.

 

I'm not privy to the validation procedures across all the plants, not my role.  But it appears the company's validation procedures are sufficient as all our plants are SQF repeat certified.  In this role, I merely run the reports and trend data to assist the QA function at each plant.  But my role is new, and includes corporate reporting to monitor the plant PRP's.  In a past job, I was a QA Supervisor and ran a sanitation program at a plant, so I've got experience using ATP but can fully admit not being intimately familiar with the science behind it all.  It was based on my past experience, along with noticing the pattern at this plant does not match the other 6, that led me to start investigating further (including asking for input here).  

 

Thanks for the info regarding the Positive Control Test.  I'll include it when I raise the flag of concern through my official channels.

 

Decided to share my charting of the ATP results.  It's unique in that I've never been able to chart monthly results in such a manner, usually because of a high number of data points.  Values shown are actual ATP results.

 

 ATP Graph.jpg   52.62KB   2 downloads

 

[Charles.C]

Fair question regarding my "initial cleanings" comment.  The results at this plant are consistent between post-sanitation swabbing as well as pre-op swabbing.  The percentages for 0 RLU I show is for their entire history, tests performed in the AM and PM and for all activities where ATP swabs are taken.

 

I'm not privy to the validation procedures across all the plants, not my role.  But it appears the company's validation procedures are sufficient as all our plants are SQF repeat certified.  In this role, I merely run the reports and trend data to assist the QA function at each plant.  But my role is new, and includes corporate reporting to monitor the plant PRP's.  In a past job, I was a QA Supervisor and ran a sanitation program at a plant, so I've got experience using ATP but can fully admit not being intimately familiar with the science behind it all.  It was based on my past experience, along with noticing the pattern at this plant does not match the other 6, that led me to start investigating further (including asking for input here).  

 

Thanks for the info regarding the Positive Control Test.  I'll include it when I raise the flag of concern through my official channels.

 

Decided to share my charting of the ATP results.  It's unique in that I've never been able to chart monthly results in such a manner, usually because of a high number of data points.  Values shown are actual ATP results.

 

ATP Graph.jpg

Hi jfrey,

 

thks for graph.  I assume numbers which are in cells (eg 150) are a monthly average of an unknown number of points. Would be interesting to know the max-min values.

Statistically IIRC it is possible to assess data sets such as these for evidence of "improbability of occurrence".

 

Personally, only  have experience with micro. data however most of your results look "amazingly" good compared to the typical pass-fail limits in product instruction document (see attached). Also note the caveat about overloading the swab.

 

 ultrasnap-insert.pdf   691.55KB   33 downloads

[jfrey123]

Hi jfrey,

 

thks for graph.  I assume numbers which are in cells (eg 150) are a monthly average of an unknown number of points. Would be interesting to know the max-min values.

 

 

Thanks for the info (again!).  That UltraSnap insert is exactly what we have in plant manuals across the country.

 

No averages in the graph, no min-max averaging.  Numbers in the cells are the total count of individual RLU readings: the color coated legend to the far left is the RLU reading, cells contain a count of the readings.  So taking March for example: 243 tests were 0 RLU, 3 tests were 2 RLU, 4 tests were 3 RLU, and 1 test scored 45 RLU.  I've never, ever, in my life, been able to graph ATP readings in such a fashion because normally there would be far too many results to be able to list them as such in a data table.

[Charles.C]

Thanks for the info (again!).  That UltraSnap insert is exactly what we have in plant manuals across the country.

 

No averages in the graph, no min-max averaging.  Numbers in the cells are the total count of individual RLU readings: the color coated legend to the far left is the RLU reading, cells contain a count of the readings.  So taking March for example: 243 tests were 0 RLU, 3 tests were 2 RLU, 4 tests were 3 RLU, and 1 test scored 45 RLU.  I've never, ever, in my life, been able to graph ATP readings in such a fashion because normally there would be far too many results to be able to list them as such in a data table.

Hi jfrey,

 

Ok, thks for clarification.

 

I assume (hope) various validated detectors were utilised in the generation of the data.

 

Assuming the measurement values are operationally comparable, in addition to data in post 5, I suggest to trend the average RLU/month versus month for each plant in addition to the raw data  shown in Post 5. From a v. quick calculation this would yield Y-coordinates of 0.64, 0.082, 0.25, 0.00.

 

PS - how about results in 2021 ?

 

PPS- Is also a possible comment on SQF auditing if previous History displays a similar pattern.

[jfrey123]

Yep.  Before posting here, I did most of the trending and comparing you suggested.  My main goal was to confirm my preconception that the high number of 0 RLU readings are uncommon, and that does appear to be the case.  We have a VP dedicated to food safety on site this week, and I'm providing support documentation as he investigates this and some other inconsistencies we've uncovered through trend analysis.

 

For 2021, results were pretty similar.  They utilize two ATP readers that show proper calibration status.

 

As to whether this is a commentary on SQF auditing...  I don't think I will venture a comment as to the past auditor's prowess in reviewing the records that the plant would have made available during their past audits 

[Charles.C]

Yep.  Before posting here, I did most of the trending and comparing you suggested.  My main goal was to confirm my preconception that the high number of 0 RLU readings are uncommon, and that does appear to be the case.  We have a VP dedicated to food safety on site this week, and I'm providing support documentation as he investigates this and some other inconsistencies we've uncovered through trend analysis.

 

For 2021, results were pretty similar.  They utilize two ATP readers that show proper calibration status.

 

As to whether this is a commentary on SQF auditing...  I don't think I will venture a comment as to the past auditor's prowess in reviewing the records that the plant would have made available during their past audits 

Re ^^^^(red) - Sadly the "Name Game" is typically followed by the "Blame Game". Be prepared.

[jfrey123]

Posting an update for the folks who might've had a genuine curiosity.  Investigation revealed a few unsatisfactory actions by the tech taking the swabs (though well intentioned), and corrective actions are being implemented.  Root cause of the consistent 0RLU score, affirmed by a Hygiena support rep, is "overloading" of the swab resulting in false negatives.

 

-Technician demonstrated swabbing too large of an area.  Tech described swabbing a 14"x14" instead of the manufacturer recommended 4"x4".

-Technician demonstrated swabs would be collected, activated, and then taken to a desk for later insertion into the reader.  This exceeded the 30 second maximum exposure to solution, resulting in discoloration of the sample liquid, and a false 0RLU.

 

Random samples were taken to challenge the readers (things like dirty boots, a visibly dirty sink, etc), which resulted in 0RLU readings.  Hygeina rep stated directly that ATP swabbing is to occur *only* on visibly clean surfaces, and that a test tube with visibly cloudy solution:

"...will cause a low reading because it is blocking the light from penetrating, and ALL ATP testing is based off of RLUs (relative light units). If that light is unable to penetrate, you get a false low value, which is what you are observing."

 

Corrective actions included review of SOP to verify it's accuracy against manufacturer instructions and retraining of the involved technicians.  Results coming forth now more closely resemble the expected results in comparison to similar facilities.

 

 

The description of overloading from Hygeina was interesting to me.  I would have (incorrectly) assumed a tube so visibly dirty would result in a high RLU reading, but their explanation of reader's operation is telling me otherwise.  If the light cannot penetrate sufficiently, the machine will return a 0RLU value.  This can come from swabbing visibly dirty spots as well as allowing the sample to sit in the solution for too much time.

 

Thanks for all the input from folks here.

[QAGB]

Posting an update for the folks who might've had a genuine curiosity.  Investigation revealed a few unsatisfactory actions by the tech taking the swabs (though well intentioned), and corrective actions are being implemented.  Root cause of the consistent 0RLU score, affirmed by a Hygiena support rep, is "overloading" of the swab resulting in false negatives.

 

-Technician demonstrated swabbing too large of an area.  Tech described swabbing a 14"x14" instead of the manufacturer recommended 4"x4".

-Technician demonstrated swabs would be collected, activated, and then taken to a desk for later insertion into the reader.  This exceeded the 30 second maximum exposure to solution, resulting in discoloration of the sample liquid, and a false 0RLU.

 

Random samples were taken to challenge the readers (things like dirty boots, a visibly dirty sink, etc), which resulted in 0RLU readings.  Hygeina rep stated directly that ATP swabbing is to occur *only* on visibly clean surfaces, and that a test tube with visibly cloudy solution:

"...will cause a low reading because it is blocking the light from penetrating, and ALL ATP testing is based off of RLUs (relative light units). If that light is unable to penetrate, you get a false low value, which is what you are observing."

 

Corrective actions included review of SOP to verify it's accuracy against manufacturer instructions and retraining of the involved technicians.  Results coming forth now more closely resemble the expected results in comparison to similar facilities.

 

 

The description of overloading from Hygeina was interesting to me.  I would have (incorrectly) assumed a tube so visibly dirty would result in a high RLU reading, but their explanation of reader's operation is telling me otherwise.  If the light cannot penetrate sufficiently, the machine will return a 0RLU value.  This can come from swabbing visibly dirty spots as well as allowing the sample to sit in the solution for too much time.

 

Thanks for all the input from folks here.

 

 

I'm just now seeing this post - and can totally relate with this. When I worked in food mfg, we also had the Hygiena Ultra and Super Snaps. We found that equipment that was visibly dirty would test at 0RLU or in single digits because the dirt will overwhelm the luminometer. As Scampi mentioned too, the equipment needs to be dry. Was told during this process too that our quat sanitizer could potentially inactivate so we had to use the more robust swabs (I can't remember for sure, but I think we upgraded from Ultra to Super) because we needed a more sensitive swab.

 

We also learned that the testing has to be pretty much immediate. As soon as you swab, you need to mix the swab with the liquid and test on the spot.  

 

These are things they need to do better about explaining to companies that newly implement their luminometer program. I only found out after I kept noting inconsistencies in the results. Once we understood the test method and process, it was much easier to do swabbing and the results made more sense.

[Spidey]

The Hook Effect is the formal name for the overloading phenomenon.  FYI, this can also happen with immunoassay allergen tests.

[Charles.C]

Re Posts 11, 12. - See the last comment/attachment in Post 6. This potential error is emphasized in Hygiena's Procedural document.

[QAGB]

Re Posts 11, 12. - See the last comment/attachment in Post 6. This potential error is emphasized in Hygiena's Procedural document.

Yes Charles, it is noted (not necessarily emphasized) in that document. Also, that procedural document leaves a lot to be desired in defining "overload". I'm not even sure if that I know that we received instructions, but I can barely remember what happened yesterday vs. implementation of the luminometer 7-8 years ago. I just know that I had to do a lot of exploration and question asking to Hygiena after the fact vs. getting clear instructions/training from Hygiena from the start. It seems like this still might be the case in present-day according to the OPs analysis. I remember the Sales Rep did a quick demo with us, we bought the luminometer and starter materials and that was pretty much it. It is a super simple tool to use, but you can easily get inaccurate results from not correctly using the equipment...which is why I state that the training should be a little more robust at implementation. Having great equipment and an awesome trend reporting tool means nothing if the testing is inaccurate.

 

Despite the above, the luminometer is pretty awesome and having the trend reporting tool and data analytics module always impressed our auditors. It was also really great to be able to share those trends with our operations team so they knew how to improve their cleaning efforts.

[Charles.C]

Yes Charles, it is noted (not necessarily emphasized) in that document. Also, that procedural document leaves a lot to be desired in defining "overload". I'm not even sure if that I know that we received instructions, but I can barely remember what happened yesterday vs. implementation of the luminometer 7-8 years ago. I just know that I had to do a lot of exploration and question asking to Hygiena after the fact vs. getting clear instructions/training from Hygiena from the start. It seems like this still might be the case in present-day according to the OPs analysis. I remember the Sales Rep did a quick demo with us, we bought the luminometer and starter materials and that was pretty much it. It is a super simple tool to use, but you can easily get inaccurate results from not correctly using the equipment...which is why I state that the training should be a little more robust at implementation. Having great equipment and an awesome trend reporting tool means nothing if the testing is inaccurate.

 

Despite the above, the luminometer is pretty awesome and having the trend reporting tool and data analytics module always impressed our auditors. It was also really great to be able to share those trends with our operations team so they knew how to improve their cleaning efforts.

Hi QAGB,

 

TBH when I see an introductory phrase like  - "It is important..." it acts like a red flag for me. However it's true that the time for this restriction to be originally known/noticed is unknown. Also interesting to note that only one facility of many had this problem suggesting a fundamental, localized QA problem.

The effect is now apparently of sufficient occurrence (or ignorance)  to justify its own page on the website -

https://help.hygiena...y-contaminated/

 

I'm equally curious as to how a baseline was obtained.

 

Just as an afterthought, it would be interesting to know if competing instruments (are there any ?) suffer from the same "malaise".

[jfrey123]

I'm equally curious as to how a baseline was obtained.

 

Just as an afterthought, it would be interesting to know if competing instruments (are there any ?) suffer from the same "malaise".

 

Not sure if you'll find my response satisfactory here, but I can explain the process my company is developing that led me down this path.  My analyst position is a new corporate role for my company, overseeing data from 7 manufacturing plants and compiling it into an executive report for the corporate leadership to participate in a food safety culture.  The report is a continuing work in progress, but includes trends of environmental and sanitation monitoring, hold and foreign material trending, audit monitoring (deficiency tracking primarily), adding in pest control monitoring this month, and a few other markers.  We don't (yet) have an established baseline agreed upon for the corporate level, but in 6 months of ATP monitoring, the plant in question showed a near 100% pass ratio in their ATP readings.  This stood out against the additional 6 plants, where graphing of pass/caution/fail results is very similar between them.  This led me to look more closely at the actual ATP values, where my experience in the QA field led me to believe hundreds of 0 RLU results were uncommon, hence my original question coming to this forum.  I compiled that information separately and presented it to our VP of Food Safety, and it was deemed sufficient to warrant an on-site investigation.

 

To your question of instruments, two are in use at the plant.  Both show proper passing of calibrations, and the history of results are nearly identical between the two.  This pointed us to the user errors when the VP of Food Safety visited the plant to conduct the investigation.

 

At any rate, improving the corporate oversight of the plants was the reason my position was created, and I think this is a win for our system as we're developing it.

[Scampi]

The data never lies-ever

[Miss Frankie]

I used ATP swabs at my former job and I questioned their accuracy quite often.   
We ran several tests and found the numbers to be all  over.

I had stainless steel counters in my lab.  We cleaned a section of counter with regular dish soap and water. 
Tested each side, got very different results.
Sprayed the area with Sanite 75, dried and retested, again, very different results. 

Cleaned again with dish soap, got even higher, and again, different results.

The ONLY time we got consistent results was when we spray the area with bleach, then we got 0 on both sides.

We tried showing our data to higher-ups, but they insisted we continue to use them.  
One of our larger customers didn't have any faith in them either.

 

Just my experience.

[Scampi]

People often expect ATP to be a miracle swab

 

They are not

 

If you are using them to verify the cleaning step, then they can be a great tool, provided you allow enough time in your program to perform baseline testing---ATP data without a baseline is useless

 

We are currently performing baseline---this allows me to see what they are currently achieving without the #'s actually meaning anything

 

Once we feel we have enough data points, then we will set pass/fail values 

 

ATP only measures the organic matter left behind on a surface and doesn't measure bacteria or viruses. It is primarily a tool used to help improve cleaning methodologies.