1 code implementation • 2 Apr 2024 • Keon-Hee Park, Kyungwoo Song, Gyeong-Moon Park
In this paper, we argue that large models such as vision and language transformers pre-trained on large datasets can be excellent few-shot incremental learners.
no code implementations • 22 Feb 2024 • Haeji Jung, Changdae Oh, Jooeon Kang, Jimin Sohn, Kyungwoo Song, Jinkyu Kim, David R. Mortensen
Approaches to improving multilingual language understanding often require multiple languages during the training phase, rely on complicated training techniques, and -- importantly -- struggle with significant performance gaps between high-resource and low-resource languages.
no code implementations • 3 Nov 2023 • Changdae Oh, Hyesu Lim, Mijoo Kim, Jaegul Choo, Alexander Hauptmann, Zhi-Qi Cheng, Kyungwoo Song
Robust fine-tuning aims to ensure performance on out-of-distribution (OOD) samples, which is sometimes compromised by pursuing adaptation on in-distribution (ID) samples.
no code implementations • 12 Jun 2023 • Hyeondey Kim, Jinwoo Nam, Minjae Lee, Yun Jegal, Kyungwoo Song
To do so, knowledge tracing systems should trace the knowledge state of the students by utilizing their problem-solving history and knowledge about the problems.
no code implementations • 21 Apr 2023 • Yewon Kim, Yongtaek Lim, Dokyung Yoon, Kyungwoo Song
To improve the generalization performance on few-shot learning, there have been diverse efforts, such as prompt learning and adapter.
1 code implementation • CVPR 2023 • Changdae Oh, Hyeji Hwang, Hee-young Lee, Yongtaek Lim, Geunyoung Jung, Jiyoung Jung, Hosik Choi, Kyungwoo Song
In this work, we propose black-box visual prompting (BlackVIP), which efficiently adapts the PTMs without knowledge about model architectures and parameters.
1 code implementation • 23 Feb 2023 • SeungHwan An, Kyungwoo Song, Jong-June Jeon
We present a new supervised learning technique for the Variational AutoEncoder (VAE) that allows it to learn a causally disentangled representation and generate causally disentangled outcomes simultaneously.
1 code implementation • Proceedings of the 40th International Conference on Machine Learning 2023 • Yoon-Yeong Kim, Youngjae Cho, JoonHo Jang, Byeonghu Na, Yeongmin Kim, Kyungwoo Song, Wanmo Kang, Il-Chul Moon
Specifically, our proposed method, Sharpness-Aware Active Learning (SAAL), constructs its acquisition function by selecting unlabeled instances whose perturbed loss becomes maximum.
no code implementations • 24 Oct 2022 • Taero Kim, Sungjun Lim, Kyungwoo Song
Moreover, we propose a new algorithm, Adaptive Sharpness-aware Group Distributionally Robust Optimization (ASGDRO), to learn sufficient invariant features across domains or groups.
no code implementations • 14 Sep 2022 • Seyun Bae, Hoyoon Byun, Changdae Oh, Yoon-Sik Cho, Kyungwoo Song
A graph has an adjacency matrix different from other dataset domains such as text and image, and it is not trivial to handle the topological information, relational information, and canonical positional information.
1 code implementation • ACM SIGKDD Conference on Knowledge Discovery and Data Mining 2022 • Changdae Oh, Heeji Won, Junhyuk So, Taero Kim, Yewon Kim, Hosik Choi, Kyungwoo Song
We provide a new type of contrastive loss motivated by Gaussian and Student-t kernels for distributional contrastive learning with theoretical analysis.
1 code implementation • 15 Jun 2022 • JoonHo Jang, Byeonghu Na, DongHyeok Shin, Mingi Ji, Kyungwoo Song, Il-Chul Moon
Therefore, we propose Unknown-Aware Domain Adversarial Learning (UADAL), which $\textit{aligns}$ the source and the target-$\textit{known}$ distribution while simultaneously $\textit{segregating}$ the target-$\textit{unknown}$ distribution in the feature alignment procedure.
2 code implementations • 2 May 2022 • HeeSun Bae, Seungjae Shin, Byeonghu Na, JoonHo Jang, Kyungwoo Song, Il-Chul Moon
We suggest a new branch of method, Noisy Prediction Calibration (NPC) in learning with noisy labels.
1 code implementation • NeurIPS 2023 • Changdae Oh, Junhyuk So, Hoyoon Byun, Yongtaek Lim, Minchul Shin, Jong-June Jeon, Kyungwoo Song
Such a lack of alignment and uniformity might restrict the transferability and robustness of embeddings.
no code implementations • 29 Sep 2021 • Dongjun Kim, Seungjae Shin, Kyungwoo Song, Wanmo Kang, Il-Chul Moon
From the theory side, the difficulty arises in estimating the high precision diffusion because the data score goes to $\infty$ as $t \rightarrow 0$ of the diffusion time.
1 code implementation • 10 Jun 2021 • Dongjun Kim, Seungjae Shin, Kyungwoo Song, Wanmo Kang, Il-Chul Moon
This paper investigates with sufficient empirical evidence that such inverse correlation happens because density estimation is significantly contributed by small diffusion time, whereas sample generation mainly depends on large diffusion time.
Ranked #2 on Image Generation on CIFAR-10 (Inception score metric)
1 code implementation • NeurIPS 2021 • Yooon-Yeong Kim, Kyungwoo Song, JoonHo Jang, Il-Chul Moon
Active learning effectively collects data instances for training deep learning models when the labeled dataset is limited and the annotation cost is high.
1 code implementation • 15 Feb 2021 • Dongjun Kim, Kyungwoo Song, Seungjae Shin, Wanmo Kang, Il-Chul Moon, Weonyoung Joo
A simulation is useful when the phenomenon of interest is either expensive to regenerate or irreproducible with the same context.
no code implementations • 24 Nov 2020 • Hyemi Kim, Seungjae Shin, JoonHo Jang, Kyungwoo Song, Weonyoung Joo, Wanmo Kang, Il-Chul Moon
Therefore, this paper proposes Disentangled Causal Effect Variational Autoencoder (DCEVAE) to resolve this limitation by disentangling the exogenous uncertainty into two latent variables: either 1) independent to interventions or 2) correlated to interventions without causality.
no code implementations • NeurIPS 2021 • Yoon-Yeong Kim, Kyungwoo Song, JoonHo Jang, Il-Chul Moon
Active learning effectively collects data instances for training deep learning models when the labeled dataset is limited and the annotation cost is high.
no code implementations • Findings of the Association for Computational Linguistics 2020 • Seungjae Shin, Kyungwoo Song, JoonHo Jang, Hyemi Kim, Weonyoung Joo, Il-Chul Moon
Recent research demonstrates that word embeddings, trained on the human-generated corpus, have strong gender biases in embedding spaces, and these biases can result in the discriminative results from the various downstream tasks.
1 code implementation • 15 Oct 2020 • Dongjun Kim, Kyungwoo Song, YoonYeong Kim, Yongjin Shin, Wanmo Kang, Il-Chul Moon, Weonyoung Joo
This paper introduces a new sampling approach, called Neural Proposal (NP), of the simulation input that resolves the biased data collection as it guarantees the i. i. d.
no code implementations • 12 Jun 2020 • Yohan Jung, Kyungwoo Song, Jinkyoo Park
To improve the training, we propose an approximate Bayesian inference for the SM kernel.
no code implementations • 11 Jun 2020 • Kyungwoo Song, Yohan Jung, Dongjun Kim, Il-Chul Moon
For the attention in Transformer and GAT, we derive that the attention is a product of two parts: 1) the RBF kernel to measure the similarity of two instances and 2) the exponential of $L^{2}$ norm to compute the importance of individual instances.
no code implementations • 13 Apr 2020 • Dongjun Kim, Weonyoung Joo, Seungjae Shin, Kyungwoo Song, Il-Chul Moon
Generative Adversarial Network (GAN) can be viewed as an implicit estimator of a data distribution, and this perspective motivates using the adversarial concept in the true input parameter estimation of black-box generators.
no code implementations • 7 Apr 2020 • Seungjae Shin, Kyungwoo Song, JoonHo Jang, Hyemi Kim, Weonyoung Joo, Il-Chul Moon
Recent research demonstrates that word embeddings, trained on the human-generated corpus, have strong gender biases in embedding spaces, and these biases can result in the discriminative results from the various downstream tasks.
no code implementations • 15 Nov 2019 • Mingi Ji, Weonyoung Joo, Kyungwoo Song, Yoon-Yeong Kim, Il-Chul Moon
This work merges the self-attention of the Transformer and the sequential recommendation by adding a probabilistic model of the recommendation task specifics.
1 code implementation • 25 May 2019 • Kyungwoo Song, JoonHo Jang, Seung jae Shin, Il-Chul Moon
Long Short-Term Memory (LSTM) infers the long term dependency through a cell state maintained by the input and the forget gate structures, which models a gate output as a value in [0, 1] through a sigmoid function.
1 code implementation • 26 Apr 2019 • Kyungwoo Song, Wonsung Lee, Il-Chul Moon
Understanding politics is challenging because the politics take the influence from everything.
1 code implementation • 26 Apr 2019 • Kyungwoo Song, Mingi Ji, Sungrae Park, Il-Chul Moon
The analyses on the user history require the robust sequential model to anticipate the transitions and the decays of user interests.
2 code implementations • 22 Apr 2019 • Sungrae Park, Kyungwoo Song, Mingi Ji, Wonsung Lee, Il-Chul Moon
Successful application processing sequential data, such as text and speech, requires an improved generalization performance of recurrent neural networks (RNNs).
no code implementations • ICLR 2019 • Su-Jin Shin, Kyungwoo Song, Il-Chul Moon
The joint optimization of representation learning and clustering in the embedding space has experienced a breakthrough in recent years.