That depends on the person; in many cases, it helps to propose kindly another activity or subject, to ask for a favour, to say if you are tired, if something hurts, or if you need help or not, or some privacy, etc.
Here's an example of what you shouldn't do in case of an annoying neighbour:
(Why didn't Mr. Bean just leave silently this train compartment and look for a seat in another compartment?)