Search Results for author:
Found 8 papers, 3 papers with code
MPIrigen: MPI Code Generation through Domain-Specific Language Models
This study first investigates the performance of state-of-the-art language models in generating MPI-based parallel programs.
The Landscape and Challenges of HPC Research and LLMs
Recently, language models (LMs), especially large language models (LLMs), have revolutionized the field of deep learning.
Domain-Specific Code Language Models: Unraveling the Potential for HPC Codes and Tasks
Specifically, we start off with HPC as a domain and build an HPC-specific LM, named MonoCoder, that is orders of magnitude smaller than existing LMs but delivers similar, if not better performance, on non-HPC and HPC tasks.
Scope is all you need: Transforming LLMs for HPC Code
With easier access to powerful compute resources, there is a growing trend in the field of AI for software development to develop larger and larger language models (LLMs) to address a variety of programming tasks.
Memory in humans and deep language models: Linking hypotheses for model augmentation
The computational complexity of the self-attention mechanism in Transformer models significantly limits their ability to generalize over long temporal durations.
Memory-Augmented Graph Neural Networks: A Brain-Inspired Review
We provide a comprehensive review of the existing literature on memory-augmented GNNs.
Slower is Better: Revisiting the Forgetting Mechanism in LSTM for Slower Information Decay
Altogether, we found that LSTM with the proposed forget gate can learn long-term dependencies, outperforming other recurrent networks in multiple domains; such gating mechanism can be integrated into other architectures for improving the learning of long timescale information in recurrent neural networks.
Multi-timescale Representation Learning in LSTM Language Models
Earlier work has demonstrated that dependencies in natural language tend to decay with distance between words according to a power law.
