Short answer: ask for a signed statement from the supplier that they are acting as the FSVP importer.
Importer of record for CBP purposes is different from FSVP importer in the eyes of the FDA, and they are not the same things. Is the NJ "office" just a business office or are they an actual distribution center? One requirement to be a FSVP importer is they must be based in the US, which is met if your supplier has the US office, but they must be willing to be the FSVP importer and perform the required tasks. Now, if they're importing it and warehousing it before distributing to you, that's pretty clear cut and dry.
However, if they're importing it specifically to fulfill your PO/order, and especially if they're importing as a direct shipment to you, that means you are likely the "US owner or consignee" with a vested interest (you either own the food, purchased the food, or have agreed in writing to purchase the food). If your importer of record fails to fulfill the FSVP requirements, FDA will come to you has a consignee. This happens a lot with brokers, who will fill out the CBP paperwork as importer of record listing you as a consignee, but will shrug their shoulders when asked about the FSVP requirements.