Le Minh Khoi

Hi there, I’m Khôi. I’m currenly a AI Resident at VinAI Research Residency Program. I am also currently remote intern in NTU AI Language group of Assistant Professor Luu Anh Tuan.

My research aims to develop effective and efficient methods for advancing and democratizing artificial intelligence in the era of large language models (LLMs). Specific areas of focus include:

Data sampling: effectively pre-determining the needed data resources for adapting the LLMs to new skills.
Low-resource adaptation: injecting data-scarced knowledge into LLMs, including knowledge from low-resource languages.
Parameter-efficient transfer learning: efficiently transferring knowledge across tasks, languages, and modalities.
Model distillation: scaling down model without minimal trade-offs.

CONTACT: minhkhoile2001nsg [at] gmail.com

news

May 16, 2024	One paper accepted at ACL 2024.
Dec 09, 2023	One paper accepted at AAAI 2024.

selected publications

ACL

UniBridge: A Unified Approach to Cross-Lingual Transfer Learning for Low-Resource Languages

Khoi Le, Trinh Pham, and Luu Anh Tuan

Proceedings of ACL, 2024

Abs arXiv Code

In this paper, we introduce UniBridge (Cross-Lingual Transfer Learning with Optimized Embeddings and Vocabulary), a comprehensive approach developed to improve the effectiveness of Cross-Lingual Transfer Learning, particularly in languages with limited resources. Our approach tackles two essential elements of a language model: the initialization of embeddings and the optimal vocabulary size. Specifically, we propose a novel embedding initialization method that leverages both lexical and semantic alignment for a language. In addition, we present a method for systematically searching for the optimal vocabulary size, ensuring a balance between model complexity and linguistic coverage. Our experiments across multilingual datasets show that our approach greatly improves the F1-Score in several languages. UniBridge is a robust and adaptable solution for cross-lingual systems in various languages, highlighting the significance of initializing embeddings and choosing the right vocabulary size in cross-lingual environments.
AAAI

LAMPAT: Low-Rank Adaption for Multilingual Paraphrasing Using Adversarial Training

Khoi Le, Trinh Pham, Tho Quan, and Luu Anh Tuan

Proceedings of AAAI, 2024

Abs arXiv Code

Paraphrases are texts that convey the same meaning while using different words or sentence structures. It can be used as an automatic data augmentation tool for many Natural Language Processing tasks, especially when dealing with low-resource languages, where data shortage is a significant problem. To generate a paraphrase in multilingual settings, previous studies have leveraged the knowledge from the machine translation field, i.e., forming a paraphrase through zero-shot machine translation in the same language. Despite good performance on human evaluation, those methods still require parallel translation datasets, thus making them inapplicable to languages that do not have parallel corpora. To mitigate that problem, we proposed the first unsupervised multilingual paraphrasing model, LAMPAT (Low-rank Adaptation for Multilingual Paraphrasing using Adversarial Training), by which monolingual dataset is sufficient enough to generate a human-like and diverse sentence. Throughout the experiments, we found out that our method not only works well for English but can generalize on unseen languages as well.