Search Results for author:
Found 107 papers, 41 papers with code
Analyzing the Domain Robustness of Pretrained Language Models, Layer by Layer
The robustness of pretrained language models(PLMs) is generally measured using performance drops on two or more domains.
A Time-Aware Transformer Based Model for Suicide Ideation Detection on Social Media
Understanding the build-up of such ideation is critical for the identification of at-risk users and suicide prevention.
GupShup: Summarizing Open-Domain Code-Switched Conversations
Code-switching is the communication phenomenon where the speakers switch between different languages during a conversation.
Deep Attentive Learning for Stock Movement Prediction From Social Media Text and Company Correlations
In the financial domain, risk modeling and profit generation heavily rely on the sophisticated and intricate stock movement prediction task.
Ranked #1 on
Stock Market Prediction
on stocknet
(using extra training data)
A Preliminary Exploration of GANs for Keyphrase Generation
We observed that our model achieves state-of-the-art performance in the generation of abstractive keyphrases and is comparable to the best performing extractive techniques.
VolTAGE: Volatility Forecasting via Text Audio Fusion with Graph Convolution Networks for Earnings Calls
Natural language processing has recently made stock movement forecasting and volatility forecasting advances, leading to improved financial forecasting.
Context-Enhanced Language Models for Generating Multi-Paper Citations
This research underscores the potential of harnessing LLMs for citation generation, opening a compelling avenue for exploring the intricate connections between scientific documents.
MM-PhyRLHF: Reinforcement Learning Framework for Multimodal Physics Question-Answering
We employ the LLaVA open-source model to answer multimodal physics MCQs and compare the performance with and without using RLHF.
Mathify: Evaluating Large Language Models on Mathematical Problem Solving Tasks
Our experiments reveal that among the three models, MAmmoTH-13B emerges as the most proficient, achieving the highest level of competence in solving the presented mathematical problems.
TC-OCR: TableCraft OCR for Efficient Detection & Recognition of Table Structure & Content
Our proposed approach achieves an IOU of 0. 96 and an OCR Accuracy of 78%, showcasing a remarkable improvement of approximately 25% in the OCR Accuracy compared to the previous Table Transformer approach.
RanLayNet: A Dataset for Document Layout Detection used for Domain Adaptation and Generalization
To solve this problem, domain adaptation approaches have been developed that use a small quantity of labeled data to adjust the model to the target domain.
KG-CTG: Citation Generation through Knowledge Graph-guided Large Language Models
Citation Text Generation (CTG) is a task in natural language processing (NLP) that aims to produce text that accurately cites or references a cited document within a source document.
Isometric Neural Machine Translation using Phoneme Count Ratio Reward-based Reinforcement Learning
In this paper, we present the development of an isometric NMT system using Reinforcement Learning (RL), with a focus on optimizing the alignment of phoneme counts in the source and target language sentence pairs.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+6
Multilingual Coreference Resolution in Low-resource South Asian Languages
We introduce a Translated dataset for Multilingual Coreference Resolution (TransMuCoRes) in 31 South Asian languages using off-the-shelf tools for translation and word-alignment.
Behavior Optimized Image Generation
We show that BoigLLM outperforms 13x larger models such as GPT-3. 5 and GPT-4 in this task, demonstrating that while these state-of-the-art models can understand images, they lack information on how these images perform in the real world.
Exploring Graph Neural Networks for Indian Legal Judgment Prediction
We explored various embeddings as model features, while nodes such as time nodes and judicial acts were added and pruned to evaluate the model's performance.
Partial Rank Similarity Minimization Method for Quality MOS Prediction of Unseen Speech Synthesis Systems in Zero-Shot and Semi-supervised setting
That is the partial rank similarity is measured (PRS) rather than the individual MOS values as with the L1 loss.
Large Content And Behavior Models To Understand, Simulate, And Optimize Content And Behavior
We call these models Large Content and Behavior Models (LCBMs).
Long-Term Ad Memorability: Understanding and Generating Memorable Ads
Finally, with the intent of memorable ad generation, we present a scalable method to build a high-quality memorable ad generation model by leveraging automatically annotated data.
A Video Is Worth 4096 Tokens: Verbalize Videos To Understand Them In Zero Shot
Multimedia content, such as advertisements and story videos, exhibit a rich blend of creativity and multiple modalities.
CiteCaseLAW: Citation Worthiness Detection in Caselaw for Legal Assistive Writing
In legal document writing, one of the key elements is properly citing the case laws and other sources to substantiate claims and arguments.
Certified Zeroth-order Black-Box Defense with Robust UNet Denoiser
However, these methods suffer from high model variance with low performance on high-dimensional datasets due to the ineffective design of the denoiser and are limited in their utilization of ZO techniques.
Emotionally Enhanced Talking Face Generation
To mitigate this, we build a talking face generation framework conditioned on a categorical emotion to generate videos with appropriate expressions, making them more realistic and convincing.
Ranked #1 on
Talking Face Generation
on CREMA-D
H-AES: Towards Automated Essay Scoring for Hindi
In this study, we reproduce and compare state-of-the-art methods for AES in the Hindi domain.
On Comparing Fair Classifiers under Data Bias
In this paper, we consider a theoretical model for injecting data bias, namely, under-representation and label bias (Blum & Stangl, 2019).
Emotional Talking Faces: Making Videos More Expressive and Realistic
Lip synchronization and talking face generation have gained a specific interest from the research community with the advent and need of digital communication in different fields.
A novel multimodal dynamic fusion network for disfluency detection in spoken utterances
Based on early-fusion and self-attention-based multimodal interaction between text and acoustic modalities, in this paper, we propose a novel multimodal architecture for disfluency detection from individual utterances.
GANTouch: An Attack-Resilient Framework for Touch-based Continuous Authentication System
Zero-effort, Population, and Random-vector.
Persuasion Strategies in Advertisements
Motivated by persuasion literature in social psychology and marketing, we introduce an extensive vocabulary of persuasion strategies and build the first ad image corpus annotated with persuasion strategies.
FaIRCoP: Facial Image Retrieval using Contrastive Personalization
Retrieving facial images from attributes plays a vital role in various systems such as face recognition and suspect identification.
Span Classification with Structured Information for Disfluency Detection in Spoken Utterances
Existing approaches in disfluency detection focus on solving a token-level classification task for identifying and removing disfluencies in text.
LDKP: A Dataset for Identifying Keyphrases from Long Scientific Documents
Identifying keyphrases (KPs) from text documents is a fundamental task in natural language processing and information retrieval.
Leveraging Transformers for Hate Speech Detection in Conversational Code-Mixed Tweets
In the current era of the internet, where social media platforms are easily accessible for everyone, people often have to deal with threats, identity attacks, hate, and bullying due to their association with a cast, creed, gender, religion, or even acceptance or rejection of a notion.
Automated Speech Scoring System Under The Lens: Evaluating and interpreting the linguistic cues for language proficiency
English proficiency assessments have become a necessary metric for filtering and selecting prospective candidates for both academia and industry.
Using Sampling to Estimate and Improve Performance of Automated Scoring Systems with Guarantees
Automated Scoring (AS), the natural language processing task of scoring essays and speeches in an educational testing setting, is growing in popularity and being deployed across contexts from government examinations to companies providing language proficiency services.
Intent Classification Using Pre-trained Language Agnostic Embeddings For Low Resource Languages
We perform experiments across three different languages: English, Sinhala, and Tamil each with different data sizes to simulate high, medium, and low resource scenarios.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
DeToxy: A Large-Scale Multimodal Dataset for Toxicity Classification in Spoken Utterances
We believe that our dataset would act as a benchmark for the relatively new and un-explored Spoken Language Processing task of detecting toxicity from spoken utterances and boost further research in this space.
Perception Point: Identifying Critical Learning Periods in Speech for Bilingual Networks
Recent studies in speech perception have been closely linked to fields of cognitive psychology, phonology, and phonetics in linguistics.
NoisyActions2M: A Multimedia Dataset for Video Understanding from Noisy Labels
We present this as a benchmark dataset in noisy learning for video understanding.
MINIMAL: Mining Models for Data Free Universal Adversarial Triggers
It is well known that natural language models are vulnerable to adversarial attacks, which are mostly input-specific in nature.
AES Systems Are Both Overstable And Oversensitive: Explaining Why And Proposing Defenses
This is in stark contrast to recent probing studies on pre-trained representation learning models, which show that rich linguistic features such as parts-of-speech and morphology are encoded by them.
Speaker-Conditioned Hierarchical Modeling for Automated Speech Scoring
In our technique, we take advantage of the fact that oral proficiency tests rate multiple responses for a candidate.
Multimodal Multi-Speaker Merger \& Acquisition Financial Modeling: A New Task, Dataset, and Neural Baselines
We introduce M3ANet, a baseline architecture that takes advantage of the multimodal multi-speaker input to forecast the financial risk associated with the M{\&}A calls.
Defending Touch-based Continuous Authentication Systems from Active Adversaries Using Generative Adversarial Networks
Previous studies have demonstrated that commonly studied (vanilla) touch-based continuous authentication systems (V-TCAS) are susceptible to population attack.
Multitask Learning for Emotionally Analyzing Sexual Abuse Disclosures
We show how for more domain-specific tasks related to sexual abuse disclosures such as sarcasm identification and dialogue act (refutation, justification, allegation) classification, homogeneous multitask learning is helpful, whereas for more general tasks such as stance and hate speech detection, heterogeneous multitask learning with emotion classification works better.
Suicide Ideation Detection via Social and Temporal User Representations using Hyperbolic Learning
Recent psychological studies indicate that individuals exhibiting suicidal ideation increasingly turn to social media rather than mental health practitioners.
An Empirical Investigation of Bias in the Multimodal Analysis of Financial Earnings Calls
In this work, we present the first study to discover the gender bias in multimodal volatility prediction due to gender-sensitive audio features and fewer female executives in earnings calls of one of the world{'}s biggest stock indexes, the S{\&}P 500 index.
Quantitative Day Trading from Natural Language using Reinforcement Learning
It is challenging to design profitable and practical trading strategies, as stock price movements are highly stochastic, and the market is heavily influenced by chaotic data across sources like news and social media.
GupShup: An Annotated Corpus for Abstractive Summarization of Open-Domain Code-Switched Conversations
Towards this objective, we introduce abstractive summarization of Hindi-English code-switched conversations and develop the first code-switched conversation summarization dataset - GupShup, which contains over 6, 831 conversations in Hindi-English and their corresponding human-annotated summaries in English and Hindi-English.
PHASE: Learning Emotional Phase-aware Representations for Suicide Ideation Detection on Social Media
Building on clinical studies, PHASE learns phase-like progressions in users{'} historical Plutchik-wheel-based emotions to contextualize suicidal intent.
FAST: Financial News and Tweet Based Time Aware Network for Stock Trading
Designing profitable trading strategies is complex as stock movements are highly stochastic; the market is influenced by large volumes of noisy data across diverse information sources like news and social media.
Factorization of Fact-Checks for Low Resource Indian Languages
The majority of studies on automatic fact-checking and fake news detection is restricted to English only.
Cisco at AAAI-CAD21 shared task: Predicting Emphasis in Presentation Slides using Contextualized Embeddings
This paper describes our proposed system for the AAAI-CAD21 shared task: Predicting Emphasis in Presentation Slides.
Exploring Semi-Supervised Learning for Predicting Listener Backchannels
To analyze our identification module's feasibility, we compared the backchannel prediction models trained on (a) manually-annotated and (b) semi-supervised labels.
What all do audio transformer models hear? Probing Acoustic Representations for Language Delivery and its Structure
In recent times, BERT based transformer models have become an inseparable part of the 'tech stack' of text processing models.
Towards Modelling Coherence in Spoken Discourse
While there has been significant progress towards modelling coherence in written discourse, the work in modelling spoken discourse coherence has been quite limited.
My Teacher Thinks The World Is Flat! Interpreting Automatic Essay Scoring Mechanism
However, little research has been put to understand and interpret the black-box nature of these deep-learning based scoring models.
Get It Scored Using AutoSAS -- An Automated System for Scoring Short Answers
In this paper, we present a fast, scalable, and accurate approach towards automated Short Answer Scoring (SAS).
GPolS: A Contextual Graph-Based Language Model for Analyzing Parliamentary Debates and Political Cohesion
Parliamentary debates present a valuable language resource for analyzing comprehensive options in electing representatives under a functional, free society.
Two-Step Classification using Recasted Data for Low Resource Settings
We further improve the performance by using a joint-objective for classification and textual entailment.
Augmenting NLP models using Latent Feature Interpolations
Models with a large number of parameters are prone to over-fitting and often fail to capture the underlying input distribution.
mask-Net: Learning Context Aware Invariant Features using Adversarial Forgetting (Student Abstract)
Training a robust system, e. g., Speech to Text (STT), requires large datasets.
LIFI: Towards Linguistically Informed Frame Interpolation
We also release several datasets to test computer vision video generation models of their speech understanding.
Conditional Hybrid GAN for Sequence Generation
Conditional sequence generation aims to instruct the generation procedure by conditioning the model with additional context information, which is a self-supervised learning issue (a form of unsupervised learning with supervision information from data itself).
MIDAS at SemEval-2020 Task 10: Emphasis Selection using Label Distribution Learning and Contextual Embeddings
This paper presents our submission to the SemEval 2020 - Task 10 on emphasis selection in written text.
Vyaktitv: A Multimodal Peer-to-Peer Hindi Conversations based Dataset for Personality Assessment
Automatically detecting personality traits can aid several applications, such as mental health recognition and human resource management.
PAI-BPR: Personalized Outfit Recommendation Scheme with Attribute-wise Interpretability
The rise in the fashion industry and its effect on social influencing have made outfit compatibility a need.
Ranked #1 on
Preference Mapping
on IQOON3000
Addressing the Cold-Start Problem in Outfit Recommendation Using Visual Preference Modelling
Despite various cutting-edge solutions proposed in the past for personalising fashion recommendation, the technology is still limited by its poor performance on new entities, i. e. the cold-start problem.
Evaluation Toolkit For Robustness Testing Of Automatic Essay Scoring Systems
This number is increasing further due to COVID-19 and the associated automation of education and testing.
Unsupervised Paraphasia Classification in Aphasic Speech
Aphasia is a speech and language disorder which results from brain damage, often characterized by word retrieval deficit (anomia) resulting in naming errors (paraphasia).
Semi-Supervised Iterative Approach for Domain-Specific Complaint Detection in Social Media
These indicators are then used to expand the dataset.
On the Inference of Soft Biometrics from Typing Patterns Collected in a Multi-device Environment
On the other hand, for regression tasks, we evaluated three ML and four DL-based regressors.
"Notic My Speech" -- Blending Speech Patterns With Multimedia
To close the gap between speech understanding and multimedia video applications, in this paper, we show the initial experiments by modelling the perception on visual speech and showing its use case on video compression.
audino: A Modern Annotation Tool for Audio and Speech
The tool allows audio data and their corresponding annotations to be uploaded and assigned to a user through a key-based API.
End-to-end Named Entity Recognition from English Speech
Named entity recognition (NER) from text has been a widely studied problem and usually extracts semantic information from text.
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+4
Multi-modal Automated Speech Scoring using Attention Fusion
In this study, we propose a novel multi-modal end-to-end neural approach for automated assessment of non-native English speakers' spontaneous speech using attention fusion.
C3VQG: Category Consistent Cyclic Visual Question Generation
In this paper, we try to exploit the different visual cues and concepts in an image to generate questions using a variational autoencoder (VAE) without ground-truth answers.
COBRA: Contrastive Bi-Modal Representation Algorithm
In this paper, we present a novel framework COBRA that aims to train two modalities (image and text) in a joint fashion inspired by the Contrastive Predictive Coding (CPC) and Noise Contrastive Estimation (NCE) paradigms which preserve both inter and intra-class relationships.
An Annotated Dataset of Discourse Modes in Hindi Stories
In this paper, we present a new corpus consisting of sentences from Hindi short stories annotated for five different discourse modes argumentative, narrative, descriptive, dialogic and informative.
Touchless Typing using Head Movement-based Gestures
The modified GRU-based model outperforms the standard CNN-RNN and Conv3D models for three of the four scenarios.
Human-Computer Interaction I.2.7
An Iterative Approach for Identifying Complaint Based Tweets in Social Media Platforms
Twitter is a social media platform where users express opinions over a variety of issues.
#MeTooMA: Multi-Aspect Annotations of Tweets Related to the MeToo Movement
In this paper, we present a dataset containing 9, 973 tweets related to the MeToo movement that were manually annotated for five different linguistic aspects: relevance, stance, hate speech, sarcasm, and dialogue acts.
Text2FaceGAN: Face Generation from Fine Grained Textual Descriptions
We then model the highly multi-modal problem of text to face generation as learning the conditional distribution of faces (conditioned on text) in same latent space.
Universal EEG Encoder for Learning Diverse Intelligent Tasks
Brain Computer Interfaces (BCI) have become very popular with Electroencephalography (EEG) being one of the most commonly used signal acquisition techniques.
Learning based Methods for Code Runtime Complexity Prediction
Predicting the runtime complexity of a programming code is an arduous task.
Keyphrase Extraction from Scholarly Articles as Sequence Labeling using Contextualized Embeddings
In this paper, we formulate keyphrase extraction from scholarly articles as a sequence labeling task solved using a BiLSTM-CRF, where the words in the input text are represented using deep contextualized embeddings.
BHAAV- A Text Corpus for Emotion Analysis from Hindi Stories
In this paper, we introduce the first and largest Hindi text corpus, named BHAAV, which means emotions in Hindi, for analyzing emotions that a writer expresses through his characters in a story, as perceived by a narrator/reader.
End-to-End Resume Parsing and Finding Candidates for a Job Description using BERT
We present an end-to-end solution for ranking candidates based on their suitability to a job description.
Keyphrase Generation for Scientific Articles using GANs
In this paper, we present a keyphrase generation approach using conditional Generative Adversarial Networks (GAN).
Investigating Political Herd Mentality: A Community Sentiment Based Approach
Analyzing polarities and sentiments inherent in political speeches and debates poses an important problem today.
ARHNet - Leveraging Community Interaction for Detection of Religious Hate Speech in Arabic
The rapid widespread of social media has lead to some undesirable consequences like the rapid increase of hateful content and offensive language.
\#YouToo? Detection of Personal Recollections of Sexual Harassment on Social Media
The availability of large-scale online social data, coupled with computational methods can help us answer fundamental questions relat- ing to our social lives, particularly our health and well-being.
Lipper: Synthesizing Thy Speech using Multi-View Lipreading
The model takes silent videos as input and produces speech as the output.
Speak up, Fight Back! Detection of Social Media Disclosures of Sexual Harassment
The {\#}MeToo movement is an ongoing prevalent phenomenon on social media aiming to demonstrate the frequency and widespread of sexual harassment by providing a platform to speak narrate personal experiences of such harassment.
SNAP-BATNET: Cascading Author Profiling and Social Network Graphs for Suicide Ideation Detection on Social Media
Suicide is a leading cause of death among youth and the use of social media to detect suicidal ideation is an active line of research.
MobiVSR: A Visual Speech Recognition Solution for Mobile Devices
Visual speech recognition (VSR) is the task of recognizing spoken language from video input only, without any audio.
Suggestion Mining from Online Reviews using ULMFiT
In this paper we present our approach and the system description for Sub Task A of SemEval 2019 Task 9: Suggestion Mining from Online Reviews and Forums.
Identifying Offensive Posts and Targeted Offense from Twitter
In this paper we present our approach and the system description for Sub-task A and Sub Task B of SemEval 2019 Task 6: Identifying and Categorizing Offensive Language in Social Media.
Harnessing GANs for Zero-shot Learning of New Classes in Visual Speech Recognition
To solve this problem, we present a novel approach to zero-shot learning by generating new classes using Generative Adversarial Networks (GANs), and show how the addition of unseen class samples increases the accuracy of a VSR system by a significant margin of 27% and allows it to handle speaker-independent out-of-vocabulary phrases.
Kiki Kills: Identifying Dangerous Challenge Videos from Social Media
There has been upsurge in the number of people participating in challenges made popular through social media channels.
Mind Your Language: Abuse and Offense Detection for Code-Switched Languages
In multilingual societies like the Indian subcontinent, use of code-switched languages is much popular and convenient for the users.
Did you take the pill? - Detecting Personal Intake of Medicine from Twitter
We believe that the developed classifier has direct uses in the areas of psychology, health informatics, pharmacovigilance and affective computing for tracking moods, emotions and sentiments of patients expressing intake of medicine in social media.
A Multimodal Approach to Predict Social Media Popularity
In this work, we propose a multimodal dataset consisiting of content, context, and social information for popularity prediction.
Theme-weighted Ranking of Keywords from Text Documents using Phrase Embeddings
Keyword extraction is a fundamental task in natural language processing that facilitates mapping of documents to a concise set of representative single and multi-word phrases.
Harnessing AI for Speech Reconstruction using Multi-view Silent Video Feed
Recently, research has started venturing into generating (audio) speech from silent video sequences but there have been no developments thus far in dealing with divergent views and poses of a speaker.
Sound Audio and Speech Processing
Key2Vec: Automatic Ranked Keyphrase Extraction from Scientific Articles using Phrase Embeddings
Keyphrase extraction is a fundamental task in natural language processing that facilitates mapping of documents to a set of representative phrases.
#phramacovigilance - Exploring Deep Learning Techniques for Identifying Mentions of Medication Intake from Twitter
Mining social media messages for health and drug related information has received significant interest in pharmacovigilance research.
