If the surveyor (who is your agent) causes damage through negligence while a-surveying, he is liable for his actions, and that's why he has, or should have, professional indemnity insurance. But he probably won't be taking the boat out of the water. If the yard who are taking the boat out of the water cause damage through negligence, they would be liable, and should be suitably insured as well. They are taking the boat out of the water on the instructions of the owner (or his agent, the broker). not you; you are in no position to give instructions regarding what is still someone else's property. If the boat really does fall to bits when being pulled out, I expect there will be an almighty row between the owner, their insurer and the yard as to whether this was negligent, or foreseeable, and who is to blame, but that's not you.
if the owner is reluctant to allow a survey, or claims the boat is too frail to come out of the water for a survey, then either it it really is, or he has something to hide that he doesn't want you or your surveyor to see. Either way, don't walk away, run!