Skip to content

Latest commit

 

History

History
43 lines (28 loc) · 2.36 KB

README.md

File metadata and controls

43 lines (28 loc) · 2.36 KB

Consumer-Complaint-Classification-OPEN-AI

This repository is made in lieu of submission towards the solution of problem statement 2 of the OPEN AI NLP hackathon. The objective here is to classify the voice recordings of a call center proceeding by treating them as consumer complaints into the said categories of the automotive industry.

Dataset

https://www.kaggle.com/dushyantv/consumer_complaints

Methodology

Pre-Training

  • Batching complaints from the common pool as a pair with SQUAD like format.
  • Evaluation model for pre-training used is: SQUAD 2.0 dev

2D Feature Space Generation

  • For each pair from the pre-trained pool, a 2d vector is plotted on the feature space.
  • The resultant magnitude of that vector is taken as a sentence embeddinng.
  • This embedding is validated using the gensim wordmodel, to create the sentence level representations.
  • These representations are further clustered by biased c-means to find the average number of classes.
  • Further, the number of classes is mapped against the existing classes of consumer complaints.

Model Training

  • A 6 layer, 32 node wide CNN is created with the node weights initialized with the sentence level representations.
  • This network is trained on the basis of end-member uniqueness for each class, to define the boundary of each class of complaints.

Model Evaluation / Testing

  • The results are evaluated using a reverse TF-IDF matrix, mapping it with the original class matrix.
  • The dot product of these matrices determines the correctness of the model:
    1. a dense matrix represents high variance and hence low accuracy
    2. a sparse matrix represents a low variance and high accuracy.

References

  • SQUAD 2.0 pre-training was referenced from the sequel paper on SQUAD 2.0 dataset and training strategy.
Rajpurkar, Pranav, Robin Jia, and Percy Liang. “Know What You Don’t Know: Unanswerable Questions for SQuAD.” Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 2018. https://doi.org/10.18653/v1/p18-2124.
  • Also, a shoutout to supreetkt for providing a foundation for consumer complaints classification on text based responses!