Question

Explain the difficulties in processing of natural language

Answer 1

Siddartha Reddy mentioned the ambiguity problem, which is a one of the biggest challenges in NLP (I'm also struggling with it, trying to recognize lexical inference). I would like to add another big challenge, which is the complementary challenge as I see it: language variability. Human language is rich and there are various ways to express a certain meaning, for example, you may say: "where is the nearest sushi restaurant?" and "can you please give me addresses of sushi places nearby?", both meaning the same. You would like NLP algorithms to figure out that both these sentences mean the same. So there are many attempts to represent the semantics of a text (Sent2vec, etc.), or to compare two texts (Textual entailment, paraphrasing).

Answer 2

Answer:

The main challenge of NLP is the understanding and modeling of elements within a variable context. In a natural language, words are unique but can have different meanings depending on the context resulting in ambiguity on the lexical, syntactic, and semantic levels.