Ikuya Yamada, Ph.D.

Chief Scientist, Studio Ousia
Visiting Professor, Nagoya University
Visiting Scientist, RIKEN AIP
ikuya (atmark) ikuya.net
X (Twitter) GitHub Google Scholar

What's New

• September, 2024 I have been appointed as a visiting professor at Nagoya University
• September, 2024 A new Japanese book, "Introduction to Large Language Models II" has been published
• May, 2024 LEIA, a method that enhances language models using cross-lingual transfer, has been accepted to ACL Findings
• July, 2023 A new Japanese book "Introduction to Large Language Models" has been published
• May, 2023 A new Japanese book "Natural Language Processing with Deep Learning" has been published
• October, 2022 STEEL, our new entity disambiguation model, was accepted to EMNLP Findings
• September, 2022 M-BoE, our new multilingual text classification model, was accepted to CoNLL
• April, 2022Two papers (link, link) were accepted to NAACL
• February, 2022mLUKE, the multilingual extension of LUKE, was accepted to ACL
• January, 2022I co-organize the Multilingual Information Access Workshop at NAACL
• June, 2021 BPR, our efficient passage retrieval method based on learning to hash technique for open-domain question answering, was accepted to ACL 2021
• May, 2021 LUKE, our deep contextualized representations of words and entities, was added to HuggingFace Transformers
• May, 2021 I gave a keynote speech at AWS Summit Online Japan 2021, one of the largest tech conference in Japan

Publications

A full and up-to-date publication list is available at Google Scholar.

Books

• Introduction to Large Language Models II (in Japanese)
  Ikuya Yamada, Masatoshi Suzuki, Sosuke Nishikawa, Kazuki Fujii, Kosuke Yamada, Ryokan Ri
  Gijutsu-Hyohron Co., Ltd., 2024
• Introduction to Large Language Models (in Japanese)
  Ikuya Yamada, Masatoshi Suzuki, Kosuke Yamada, Ryokan Ri
  Gijutsu-Hyohron Co., Ltd., 2023
• Natural Language Processing with Deep Learning (in Japanese)
  Ikuya Yamada, Tomohide Shibata, Hiroyuki Shindo, Ryuji Tamaki
  Kyoritsu Shuppan Co., Ltd., 2023
• Natural Language Processing (in Japanese)
  Jun Suzuki, Seiji Tsuchiya, Kazuki Motohashi, Kanji Takahashi, Akihiro Tamura, Ikuya Yamada, Kenji Araki, Shinsuke Mori, Shinichi Watanabe, Shin Hara, Tomoya Mizumoto, Takeshi Shimizu
  Johokiko Co., Ltd., 2020
• Introduction to Machine Learning Programming in Python (in Japanese)
  Tatsuro Shimada, Naoto Koshimizu, Atsushi Hayakawa, Ikuya Yamada
  Gijutsu-Hyohron Co., Ltd., 2019

Journal Papers

• Trick Me If You Can: Human-in-the-loop Generation of Adversarial Question Answering Examples
  Eric Wallace, Pedro Rodriguez, Shi Feng, Ikuya Yamada, Jordan Boyd-Graber
  TACL 2019
• Studio Ousia's Quiz Bowl Question Answering System
  Ikuya Yamada, Ryuji Tamaki, Hiroyuki Shindo, Yoshiyasu Takefuji
  The NIPS '17 Competition: Building Intelligent Systems, The Springer Series on Challenges in Machine Learning, 2018
• Linkify: Enhancing Text Reading Experience by Detecting and Linking Helpful Entities to Users
  Ikuya Yamada, Tomotaka Ito, Hideaki Takeda, Yoshiyasu Takefuji
  IEEE Intelligent Systems 2018
  [Dataset]
• Learning Distributed Representations of Texts and Entities from Knowledge Base
  Ikuya Yamada, Hiroyuki Shindo, Hideaki Takeda, Yoshiyasu Takefuji
  TACL 2017
  [Code]

Selected Conference Papers

• LEIA: Facilitating Cross-lingual Knowledge Transfer in Language Models with Entity-based Data Augmentation
  Ikuya Yamada, Ryokan Ri
  ACL Findings 2024
  [Code]
• Entity Embedding Completion for Wide-Coverage Entity Disambiguation
  Daisuke Oba, Ikuya Yamada, Naoki Yoshinaga, Masashi Toyoda
  EMNLP Findings 2022
  
• A Multilingual Bag-of-Entities Model for Zero-Shot Cross-Lingual Text Classification
  Sosuke Nishikawa, Ikuya Yamada, Yoshimasa Tsuruoka, Isao Echizen
  CoNLL 2022
  
• Global Entity Disambiguation with BERT
  Ikuya Yamada, Koki Washio, Hiroyuki Shindo, Yuji Matsumoto
  NAACL 2022
  [Code]
• EASE: Entity-Aware Contrastive Learning of Sentence Embedding
  Sosuke Nishikawa, Ryokan Ri, Ikuya Yamada, Yoshimasa Tsuruoka, Isao Echizen
  NAACL 2022
• mLUKE: The Power of Entity Representations in Multilingual Pretrained Language Models
  Ryokan Ri, Ikuya Yamada, Yoshimasa Tsuruoka
  ACL 2022
• Efficient Passage Retrieval with Hashing for Open-domain Question Answering
  Ikuya Yamada, Akari Asai, Hannaneh Hajishirzi
  ACL 2021
  [Code]
• NeurIPS 2020 EfficientQA Competition: Systems, Analyses and Lessons Learned
  Sewon Min, Jordan Boyd-Graber, Chris Alberti, Danqi Chen, Eunsol Choi, Michael Collins, Kelvin Guu, Hannaneh Hajishirzi, Kenton Lee, Jennimaria Palomaki, Colin Raffel, Adam Roberts, Tom Kwiatkowski, Ikuya Yamada, other competition participants
  Proceedings of Machine Learning Research 2021
• LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention
  Ikuya Yamada, Akari Asai, Hiroyuki Shindo, Hideaki Takeda, Yuji Matsumoto
  EMNLP 2020
  [Code]
• Wikipedia2Vec: An Efficient Toolkit for Learning and Visualizing the Embeddings of Words and Entities from Wikipedia
  Ikuya Yamada, Akari Asai, Jin Sakuma, Hiroyuki Shindo, Hideaki Takeda, Yoshiyasu Takefuji, Yuji Matsumoto
  EMNLP 2020 (demonstration)
  [Code]
• Neural Attentive Bag-of-Entities Model for Text Classification
  Ikuya Yamada, Hiroyuki Shindo
  CoNLL 2019
• Representation Learning of Entities and Documents from Knowledge Base Descriptions
  Ikuya Yamada, Hiroyuki Shindo, Yoshiyasu Takefuji
  COLING 2018
  [Code]
• Named Entity Disambiguation for Noisy Text
  Yotam Eshel, Noam Cohen, Kira Radinsky, Shaul Markovitch, Ikuya Yamada, Omer Levy
  CoNLL 2017
• Segment-Level Neural Conditional Random Fields for Named Entity Recognition
  Motoki Sato, Hiroyuki Shindo, Ikuya Yamada, Yuji Matsumoto
  IJCNLP 2017
• Joint Learning of the Embedding of Words and Entities for Named Entity Disambiguation
  Ikuya Yamada, Hiroyuki Shindo, Hideaki Takeda, Yoshiyasu Takefuji
  CoNLL 2016
  [Code]
• Evaluating the Helpfulness of Linked Entities to Readers
  Ikuya Yamada, Tomotaka Ito, Shinsuke Takagi, Shinnosuke Usami, Hideaki Takeda, Yoshiyasu Takefuji
  Hypertext 2014
  [Dataset]

Achievements

• 2nd place, NeurIPS 2020 Efficient Open-Domain Question Answering Competition (restricted 6GB track), 2020.
• 3rd place, NeurIPS 2020 Efficient Open-Domain Question Answering Competition (unrestricted track), 2020.
• 1st place, ISWC 2020 Semantic Web Challenge on Tabular Data to Knowledge Graph Mining, 2020.
• 1st place, NIPS 2017 Human-Computer Question Answering Competition, 2017.
• 2nd place, WSDM Cup 2017 Triple Scoring Task, 2017.
• Competition master, Kaggle, 2016.
• 1st place, Shared Task in NAACL 2016 Workshop on Human-Computer QA, 2016.
• 1st place, Shared Task #1 in ACL 2015 Workshop on Noisy User-generated Text (W-NUT 2015), 2015.
• 1st place, NEEL Challenge in WWW 2015 Workshop on Making Sense of Microposts, 2015.

Open-source Softwares

• Wikipedia2Vec: unified embedding of words and entities from Wikipedia
• LUKE: pre-trained contextualized representation of words and entities based on transformer
• mojimoji: fast Cython-based converter for Japanese characters

Professional Service

• Organizer: MIA Workshop @ NAACL 2022
• Senior Area Chair: ACL Rolling Review (2024-)
• Area Chair: ACL Rolling Review (2023-2024), EACL (2022)
• Program Commitee: ACL Rolling Review (2021, 2022), ACL (2021), NAACL (2018), EMNLP (2020, 2021), AACL (2020, 2022), ISWC (2020, 2021, 2022)