Hi Apollo,
Because your packaging is not for direct food contact there really is very little risk of physical contamination of the end food product e.g. nut debris being transferred from person / clothing to your packaging product and then to food via some direct food contact packaging. Especially if you have some controls such as hand washing before entering production, only eating and drinking in canteens, maybe being a nut free site and perhaps even removing work clothes prior to eating (overkill in your case I think).
Your biggest risk with allergens could be down a different track completely for example mixing packaging designs, printing defects on ingredients etc., which ultimately could cause a consumer to mistakenly eat a food product containing an allergen they are allergic to.
Regards,
Simon