Idioms: Humans or machines, it’s all about context

Expressions can be ambiguous between idiomatic and literal interpretation depending on the context they occur in (“sales hit the roof” vs “hit the roof of the car”). Previous studies suggest that idiomaticity is not a binary property, but rather a continuum or the so-called “scalar phenomenon” ranging from completely literal to highly idiomatic. This paper reports the results of an experiment in which human annotators rank idiomatic expressions in context on a scale from 1 (literal) to 4 (highly idiomatic). Our experiment supports the hypothesis that idioms fall on a continuum and that one might differentiate between highly idiomatic, mildly idiomatic and weakly idiomatic expressions. In addition, we measure the relative idiomaticity of 11 idiomatic types and compute the correlation between the relative idiomaticity of an expression and the performance of various automatic models for idiom detection. We show that our model, based on the distributional semantics ideas, not only outperforms the previous models, but also positively correlates with the human judgements, which suggests that we are moving in the right direction toward automatic idiom detection.