So as an #actuallyAutistic person (I think), I do have an issue with touch - I am especially sensitive to it. Which is something of a pain.
But also, I think it is just that touch is very intense. I know that, when visiting somewhere, I like to touch the buildings, because I sense so much more about the place through that. I know that if I touch someone on their bare skin (say, in summer on the tube), it is powerful.
So it is not that I don't like touch. It is more that it is incredibly intense for me. So has to be just right.
Incidentally, in Quakers, we shake hands. I find this touch of skin to be very powerful.