Of course you can get away with it, provided that your auditor doesn't write you up for it :)
You didn't say what standard you're using so is this just a GMP audit or is it a GFSI level audit? Look at the audit criteria closely and see what it says and be able to answer to an auditor about it. I would test different cleaning methods on the wall, find one that makes it look pretty clean, and then put that SOP in writing with a note along the lines of:
Cleaning procedure is designed to leave wall clean and dry. Although wall is porous, water does not penetrate it or cause an additional food safety concern and the wall is cleanable and washable by following the steps in this procedure.
If you wanted to go one step further, do some APC or ATP tests on the wall to validate that your sop is effective in making it a clean surface.