【Annoy】 , 【vex】 , 【irk】 and 【bother】 all mean to disturb and nervously upset a person.

【Annoy】 stresses loss of equanimity or patience as a result of being forced to endure something that one finds obnoxious or offensive or sometimes merely displeasing or distasteful. It seldom implies more than a temporary disturbance or display of irritation.

• Richard’s absence annoyed him. The youth was vivacious, and his enthusiasm good fun
  —Meredith
• it was . . . his lack of the ghost of a notion what anyone else was feeling that annoyed her, had always annoyed her
  —Woolf

【Vex】 usually implies greater provocation and a stronger disturbance than 【annoy】 ; it often connotes a degree of anger but at other times it suggests deep perplexity or some worry.

• faulty translation that so vexes teachers
  —Grandgent
• pointlessly vexing their minds with insoluble problems
• Mr. Darcy’s behavior astonished and vexed her. "Why, if he came only to be silent, grave, and indifferent," said she, "did he come at all?"
  —Austen
• such petty details as now vexed the brooding soul of the old gentlewoman
  —Hawthorne

【Irk】 emphasizes difficulty in enduring and resulting weariness of spirit; it is most often used in reference to something that persists or recurs annoyingly.

• the speed and the clatter 【irk】 me
  —Kipling
• the overiterated becomes the monotonous, and the monotonous irks and bores
  —Lowes

【Bother】 implies a usually mild interference with one’s comfort or peace of mind such as may arise on the one hand from something that calls for activity or effort or on the other from something that excites, puzzles, worries, concerns, or confuses.

• the sight of him bothered her and set her heart beating faster
• he would be too accessible and excessively bothered with details and complaints
  —Crozier
• let dozens of little matters go, rather than 【bother】 myself
  —Bennett
• I am not really bothered by these questions—the hoary old puzzles of ethics and philosophy
  —L. P. Smith