Hi, I'm Ralph.


I'm a senior research scientist who's passionate about machine learning, natural language processing, statistics, and probability theory. Research-wise, I've driven the creative vision and technical direction for my 10+ first-author publications, winning a best paper award at ACL 2023. Engineering-wise, I've led and implemented language technology systems serving 20 million+ queries per day, receiving a companywide innovation award at Comcast.

I've served as a reviewer for AAAI, AACL-IJCNLP, ACL, EACL, EMNLP Industry, NAACL, and workshops at ACL, CVPR, ICML, and NeurIPS. My curriculum vitae has more details.

Papers and Talks

2023
Methods and Systems for Voice Control
Raphael Tang, Karun Kumar, ..., G. Craig Murray
US Patent App. 17/566,036
What the DAAM: Interpreting Stable Diffusion Using Cross Attention
Raphael Tang*, Linqing Liu*, Akshat Pandey, ..., Karun Kumar, Pontus Stenetorp, Jimmy Lin, Ferhan Ture
ACL 2023 (Best Paper Award)
*Equal contribution.
Operator Selection and Ordering in a Pipeline Approach to Efficiency Optimizations for Transformers
Ji Xin, Raphael Tang, Zhiying Jiang, Yaoliang Yu, Jimmy Lin
ACL 2023 Findings
"Low-Resource" Text Classification: A Parameter-Free Classification Method with Compressors
Zhiying Jiang, Matthew Y. R. Yang, Mikhail Tsirlin, Raphael Tang, Yiqin Dai, Jimmy Lin
ACL 2023 Findings
Comcast SpeechNet: Weakly Supervised, End-to-End Speech Recognition at Industrial Scale
Raphael Tang
Snorkel FDCAI 2023
Predictive Query Execution
Raphael Tang, Karun Kumar, ..., G. Craig Murray
US Patent App. 17/454,234
2022
Systems and Methods for Improved Automatic Speech Recognition Systems
Raphael Tang, Karun Kumar, ..., Craig Murray
Filed US Patent
SpeechNet: Weakly Supervised, End-to-End Speech Recognition at Industrial Scale
Raphael Tang, Karun Kumar, Gefei Yang, Akshat Pandey, ..., Craig Murray, Ferhan Ture, Jimmy Lin
EMNLP 2022 Industry
Temporal Early Exiting for Streaming Speech Commands Recognition
Raphael Tang, Karun Kumar, ..., G. Craig Murray, Jimmy Lin
ICASSP 2022
2021
How Does BERT Rerank Passages? An Attribution Analysis with Information Bottlenecks
Zhiying Jiang, Raphael Tang, Ji Xin, Jimmy Lin
EMNLP 2021 BlackboxNLP Workshop
Voice Query Auto Completion
Raphael Tang, Karun Kumar, Kendra Chalkley, Ji Xin, ..., G. Craig Murray, Jimmy Lin
EMNLP 2021 (Oral)
The Art of Abstention: Selective Prediction and Error Regularization for Natural Language Processing
Ji Xin, Raphael Tang, Yaoliang Yu, Jimmy Lin
ACL-IJCNLP 2021
BERxiT: Early Exiting for *BERT with Better Fine-Tuning and Extension to Regression
Ji Xin, Raphael Tang, Yaoliang Yu, Jimmy Lin
EACL 2021
2020
Inserting Information Bottlenecks for Attribution in Transformers
Zhiying Jiang, Raphael Tang, Ji Xin, Jimmy Lin
EMNLP 2020 Findings
Howl: A Deployed, Open-Source Wake Word Detection System
Raphael Tang*, Jaejun Lee*, Afsaneh Razi, Julia Cambre, Ian Bicking, Jofish Kaye, Jimmy Lin
EMNLP 2020 NLPOSS Workshop
*Equal contribution.
Covidex: Neural Ranking Models and Keyword Search Infrastructure for the COVID-19 Open Research Dataset
Edwin Zhang, Nikhil Gupta, Raphael Tang, ..., Kyunghyun Cho, Hui Fang, Jimmy Lin
EMNLP 2020 SDP Workshop
Exploring the Limits of Simple Learners in Knowledge Distillation for Document Classification
Ashutosh Adhikari, Achyudh Ram, Raphael Tang, Jimmy Lin
ACL 2020 Repl4NLP Workshop
Showing Your Work Doesn't Always Work
Raphael Tang, Jaejun Lee, Ji Xin, Xinyu Liu, Yaoliang Yu, Jimmy Lin
ACL 2020
DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference
Ji Xin, Raphael Tang, Jaejun Lee, Yaoliang Yu, Jimmy Lin
ACL 2020
Rapidly Bootstrapping a Question Answering Dataset for COVID-19
Raphael Tang, ..., Phuong Cam, Kyunghyun Cho, Jimmy Lin
arXiv
2019
Machine Learning for Streamflow Prediction: Current Status and Future Prospects
Martin Gauch, Raphael Tang, Juliane Mai, Bryan Tolson, Shervan Gharari, Jimmy Lin
AGU Fall Meeting 2019
The Runoff Model-Intercomparison Project over Lake Erie and the Great Lakes
Juliane Mai, ..., Jimmy Lin, Raphael Tang
AGU Fall Meeting 2019
What Would Elsa Do? Freezing Layers During Transformer Fine-Tuning
Jaejun Lee, Raphael Tang, Jimmy Lin
arXiv
Explicit Pairwise Word Interaction Modeling Improves Pretrained Transformers for English Semantic Similarity Tasks
Yinan Zhang, Raphael Tang, Jimmy Lin
arXiv
Natural Language Generation for Effective Knowledge Distillation
Raphael Tang, Yao Lu, Jimmy Lin
EMNLP-IJCNLP 2019 DeepLo Workshop
Incorporating Contextual and Syntactic Structures Improves Semantic Similarity Modeling
Linqing Liu, Wei Yang, Jinfeng Rao, Raphael Tang, Jimmy Lin
EMNLP-IJCNLP 2019 (Oral)
Honkling: In-Browser Personalization for Ubiquitous Keyword Spotting
Jaejun Lee, Raphael Tang, Jimmy Lin
EMNLP-IJCNLP 2019 Demo
Challenges and Opportunities in Understanding Spoken Queries Directed at Modern Entertainment Platforms
Ferhan Ture, Jinfeng Rao, Raphael Tang, Jimmy Lin
SIRIP 2019
DocBERT: BERT for Document Classification
Ashutosh Adhikari, Achyudh Ram, Raphael Tang, Jimmy Lin
arXiv
Distilling Task-Specific Knowledge from BERT into Simple Neural Networks
Raphael Tang*, Yao Lu*, Linqing Liu*, Lili Mou, Olga Vechtomova, Jimmy Lin
arXiv
*Equal contribution.
Yelling at Your TV: An Analysis of Speech Recognition Errors and Subsequent User Behavior on Entertainment Systems
Raphael Tang, Ferhan Ture, Jimmy Lin
SIGIR 2019
Universal Voice-Enabled User Interfaces using JavaScript
Jaejun Lee, Raphael Tang, Jimmy Lin
IUI 2019 Demo
Rethinking Complex Neural Network Architectures for Document Classification
Ashutosh Adhikari*, Achyudh Ram*, Raphael Tang, Jimmy Lin
NAACL 2019 (Oral)
*Equal contribution.
2018
Streaming Voice Query Recognition Using Causal Convolutional Recurrent Neural Networks
Raphael Tang, Gefei Yang, Hong Wei, Yajie Mao, Ferhan Ture, Jimmy Lin
arXiv
FLOPs as a Direct Optimization Objective for Learning Sparse Neural Networks
Raphael Tang, Ashutosh Adhikari, Jimmy Lin
NIPS 2018 Workshop on Compact Deep Neural Networks with Industrial Applications
Progress and Tradeoffs in Neural Language Models
Raphael Tang, Jimmy Lin
arXiv
JavaScript Convolutional Neural Networks for Keyword Spotting in the Browser: An Experimental Analysis
Jaejun Lee, Raphael Tang, Jimmy Lin
arXiv
Adaptive Pruning of Neural Language Models for Mobile Devices
Raphael Tang, Jimmy Lin
arXiv
An Experimental Analysis of the Power Consumption of Convolutional Neural Networks for Keyword Spotting
Raphael Tang, Weijie Wang, Zhucheng Tu, Jimmy Lin
ICASSP 2018
Deep Residual Learning for Small-Footprint Keyword Spotting
Raphael Tang, Jimmy Lin
ICASSP 2018
2017
Honk: A PyTorch Reimplementation of Convolutional Neural Networks for Keyword Spotting
Raphael Tang, Jimmy Lin
arXiv


Projects