Hi, and welcome to the forum!
Good question and as someone who has a child with a Brazil nut allergy, I'm gladdened to hear people are taking it so seriously and wanting to improve without being pushed. I think you're right to consider staff movements in your nut risk assessment. I'm a little unclear if non nut staff are operating at the same time as nut staff as you say it's before a clean down? So would the tables and other lines not be cleaned at the same time? In any case one thing which would be really good to do is get a factory plan and draw on it where staff need to move during their shift while they're handling nuts and also draw on it where staff handling non nut products would walk during the same time period. Even if there are no "non nut" staff, this is still a useful exercise. That will show you where the cross over points are because as you've rightly indicated, it's not necessarily going to be immediate but it could be someone touching a table or a trolley then another person touching it an hour later. Once you've done that think about the areas you want to restrict movement. If they're collecting paperwork, can you move the paperwork closer to them or have it ready in the area it will be used before they start the production run (that's better from an efficiency point of view anyway)? If the trolley is wrapped product, can a different or dedicated member of staff take it to the warehouse? Or can the trolley be fully washed after it's been used for the nut products?
You've rightly assessed I think that it is going to be difficult in your plant to exclude all traces. You do need to be aware though that unless you have a very severe allergy, some allergy specialists (including the one I see with my son) tell you not to bother avoiding "may contain" as it's virtually impossible when it comes to nuts. I have no idea why people put "may contain" nuts but very little other "alibi" labelling when other allergies can be just as severe? But they do. In any case, if you had a product which caused an allergic reaction in a consumer and that ingredient wasn't on the ingredient list, you're probably liable under the FIC / FIR in any case despite the "may contains" statement which is almost certainly on your packs. So it's really great to see you guys taking it seriously.
That all said, while cross contamination is a valid risk, the biggest risk is gross contamination and one of the worst is putting the wrong label on the pack so don't forget to make sure those controls are absolutely robust.
A last comment and a plea... So many sites control "nuts" as one allergen. They are not one allergen (if you're allergic to pecans you are probably allergic to walnuts but you needn't be allergic to hazelnuts for example) and nowadays, especially kids are told to eat other nuts to avoid contracting an allergy to them. So please ensure that as much as you can you control the nuts as separate species if this is possible? I always say this to people on this forum because unless you have a relative who has been recently diagnosed with a nut allergy you may not be aware of the changed advice from medical practitioners. It's also really helpful to consumers that if you feel the need to put "may contains" on a pack you specify which nut species may be contained; i.e. in your case walnuts and almonds. In that case for my son with his brazil nut allergy I know that product is absolutely fine.