Abstract
Intent classification in Natural Language Processing involves identifying the intention of the user based on their input/interaction with an interface. This can be in a natural usage setting (voice assistants) or an interaction between users, customer service personnel, or agents (in a large organization). This paper aims to study the problem of intent classification using language models (like DistilBERT) and Large Language Models (Phi2 and LLAMA3) on open-source datasets available in banking, travel, small talk, office, etc. This is a challenging problem as there is a need for more available data that spans various domains. Identifying the user’s intention is sometimes tricky as the classification changes based on the context (and ambiguity in natural language). Apart from the given intents, the data poses a challenge when a user presents the interface with out-of-scope queries (for which the models aren’t trained). We test this on transformer-based approaches like DistilBERT and LLMs (like Phi2 and LLAMA3). We find that DistilBERT, with fewer parameters, trains faster and runs inference faster than LLAMA3 and Phi2 (with PEFT-LoRA). We also find that the results from DistilBERT are much better than those of the language model-based approaches for the Banking-77 and CLINC-150 datasets. These two datasets cover various domains (as mentioned above).
Showing the abstract — retrieve the full paper via the Exa API.