Home
home
AGI Seminar Series
home
💛

Awesome X (Prof. Ahn’s Reading List)

AGI

General
2408 - A Theory of Understanding for Artificial Intelligence: Composability, Catalysts, and Learning
2406 - Open-Endedness is Essential for Artificial Superhuman Intelligence
2311 - Levels of AGI: Operationalizing Progress on the Path to AGI
2208 - The Alberta Plan for AI Research

AI Mathematician / Autoformalization

General
2407 - LEAN-GitHub: Compiling GitHub LEAN repositories for a versatile LEAN prover
2306 - From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thought
2406 - AI-Assisted Generation of Difficult Math Questions
2405 - Metacognitive Capabilities of LLMs- An Exploration in Mathematical Problem Solving
2405 - DeepSeek-Prover: Advancing Theorem Proving in LLMs through Large-Scale Synthetic Data
2404 - A Survey on Deep Learning for Theorem Proving
2403 - Don’t Trust: Verify – Grounding LLM Quantitative Reasoning with Autoformalization
2403 - Machine Learning and Information Theory Concepts Towards an AI Mathematician [Y. Bengio]
2402 - REFACTOR: Learning to Extract Theorems from Proofs
2312 - NeurIPS Tutorial on Machine Learning for Theorem Proving [Video]
2310 - A New Approach Towards Autoformalization
2212 - Solving Quantitative Reasoning Problems with Language Models
2301 - Towards Autoformalization of Mathematics and Code Correctness: Experiments with Elementary Proofs
2210 - Draft, Sketch, and Prove: Guiding Formal Theorem Provers with Informal Proofs
2205 - Autoformalization with Large Language Models
20xx - A Promising Path Towards Autoformalization and General Artificial Intelligence
2009 - Generative Language Modeling for Automated Theorem
2006 - Mathematical Reasoning via Self-supervised Skip-tree Training

AI4Science

Protein Design
2402 - Generative AI for Controllable Protein Sequence Design: A Survey

Architectures

General
2406 - Vision-LSTM: xLSTM as Generic Vision Backbone
2405 - Aaren - Attention as an RNN
2310 - Sparse Universal Transformer
2212 - DiT - Scalable Diffusion Models with Transformers
2207 - Neural Networks And The Chomsky Hierarchy
2202 - GroupViT: Semantic Segmentation Emerges from Text Supervision
2202 - MaskGIT: Masked Generative Image Transformer
20xx - Theoretical Limitations of Self-Attention in Neural Sequence Models

Artificial Hippocampus

General
2408 - Space as A Scaffold for Rotational Generalisation of Abstract Concepts
2408 - How the human brain creates cognitive maps of related concepts
2408 - Why Concepts Are (probably) Vectors
2408 - Cognitive maps from predictive vision
2408 - Abstract representations emerge in human hippocampal neurons during inference
2407 - Space is a latent sequence: A theory of the hippocampus
2407 - Automated construction of cognitive maps with visual predictive coding
2407 - The Computational Foundations of Dynamic Coding in Working Memory
2406 - A recurrent network model of planning explains hippocampal replay and human behavior
2405 - Remapping revisited: how the hippocampus represents different spaces
2401 - Learning Cognitive Maps from Transformer Representations for Efficient Planning in Partially Observed Environments
2311 - The generative grammar of the brain: a critique of internally generated representations
2308 - Cognitive graphs: Representational substrates for planning
2302 - Replay and Compositional Computation
2209 - How to build a cognitive map
2112 - Relating Transformers to Models and Neural Representations of The Hippocampal Formation
2106 - Geometry of abstract learned knowledge in the hippocampus
2009 - Emergence of abstract rules in the primate brain
1805 - Vector-Based Navigation Using Grid-Like Representations in Artificial Agents

Causality

Position & Survey Papers
2409 - Theory Is All You Need: AI, Human Cognition, and Causal Reasoning
2405 - From Identifiable Causal Representations to Controllable Counterfactual Generation: A Survey on Causal Generative Modeling
2403 - Large Language Models and Causal Inference in Collaboration: A Comprehensive Survey
2402 - Essential Role of Causality in Foundation World Models for Embodied AI
2307 - Causal Reinforcement Learning: A Survey
2302 - A Survey on Causal Reinforcement Learning
2206 - Causal Machine Learning: A Survey
2105 - Toward Causal Representation Learning
RL & World Model
2408 - Rethinking State Disentanglement in Causal Reinforcement Learning
2406 - Fine-Grained Causal Dynamics Learning with Quantization for Improving Robustness in Reinforcement Learning
2404 - Robust Agents Learn Causal World Models
2404 - Compete and Compose: Learning Independent Mechanisms for Modular World Models
2404 - Empowerment as Causal Learning, Causal Learning as Empowerment A bridge between Bayesian causal hypothesis testing and reinforcement learning
2403 - Dreaming of Many Worlds: Learning Contextual World Models Aids Zero-Shot Generalization
2306 - Granger Causal Interaction Skill Chains
2206 - Causal Dynamics Learning for Task-Independent State Abstraction
2200 - CausalDyna: Improving Generalization of Dyna-Style Reinforcement Learning via Counterfactual-Based Data Augmentation
2102 - Agent Incentives - A Causal Perspective
2100 - Systematic Evaluation of Causal Discovery in Visual Model Based Reinforcement Learning
Inference
Representation
2409 - Unifying Causal Representation Learning with the Invariance Principle
2407 - Disentangled Representations for Causal Cognition
2406 - The Odyssey of Commonsense Causality: From Foundational Benchmarks to Cutting-Edge Reasoning
2405 - From Identifiable Causal Representations to Controllable Counterfactual Generation: A Survey on Causal Generative Modeling
24xx - Multi-View Causal Representation Learning with Partial Observability
2403 - Towards the Reusability a`nd Compositionality of Causal Representations
2305 - Interventional Causal Representation Learning
2310 - Towards Causal Foundation Model: on Duality between Causal Inference and Attention
23xx - Nonparametric Identifiability of Causal Representations from Unknown Interventions
2209 - Interventional Causal Representation Learning
2203 - Weakly Supervised Causal Representation Learning
2202 - CITRIS: Causal Identifiability from Temporal Intervened Sequences
2100 - CausalVAE: Disentangled Representation Learning via Neural Structural Causal Models
2000 - iVAE: Variational Autoencoders and Nonlinear ICA: A Unifying Framework
13xx - Programs as Causal Models - Speculations on MentalPrograms and Mental Representation
Discovery
2405 - Demystifying Amortized Causal Discovery with Transformers
2402 - Efficient Causal Graph Discovery Using Large Language Models
2206 - Causal Dynamics Learning for Task-Independent State Abstraction
2204 - Learning to Induce Causal Structure
2202 - DECI - Deep End-to-end Causal Inference
2011 - Causal Autoregressive Flows
20xx - Causal Discovery with Reinforcement Learning
20xx - Differentiable Causal Discovery from Interventional Data
Benchmarks
2405 - Smoke and Mirrors in Causal Downstream Tasks
Causality in NeuroCog
2409 - Theory Is All You Need: AI, Human Cognition, and Causal Reasoning
2406 - Counterfactual Simulation in Causal Cognition (by Tobias Gerstenberg)
2405 - The Development of Human Causal Learning and Reasoning
2001 - What is Causal Cognition

Compositionality

Position Papers
2402 - Compositional Generative Modeling - A Single Model is Not All You Need
2302 - Modular Deep Learning
General
2407 - Deciphering the Role of Representation Disentanglement - Investigating Compositional Generalization in CLIP Models
2406 - Discrete Dictionary-based Decomposition Layer for Structured Representation Learning
1908 - The Compositionality of Neural Networks: Integrating Symbolism and Connectionism [Hupkes]
Composable World Models
2404 - Compete and Compose: Learning Independent Mechanisms for Modular World Models
20xx - Compositional Visual Generation with Energy Based Models [Igor Mordarch]
Neural Programmer
2306 - Learning Transformer Programs
Backbone Architectures (Compositional Transformer)
2405 - Transformers Can Do Arithmetic with the Right Embeddings
2405 - Grokked Transformers are Implicit Reasoners: A Mechanistic Journey to the Edge of Generalization
2405 - Improving Transformers with Dynamically Composable Multi-Head Attention
2405 - Sparo: Selective Attention for Robust and Compositional Transformer Encodings for Vision
2404 - When Can Transformers Reason with Abstract Symbols?
2311 - Compositional Capabilities of Autoregressive Transformers: A Study on Synthetic, Interpretable Tasks
2311 - What Algorithms can Transformers Learn? A Study in Length Generalization
22xx - Making Transformers Solve Compositional Tasks
2110 - Compositional Attention: Disentangling Search and Retrieval
Object-Centric
2408 - Zero-Shot Object-Centric Representation Learning
2406 - Identifiable Object-Centric Representation Learning via Probabilistic Slot Attention
2405 - Neural Language of Thought Models
2312 - Reusable Slotwise Mechanisms
2310 - COSMOS - Neurosymbolic Grounding for Compositional World Models
2310 - Object-Centric Architectures Enable Efficient Causal Representation Learning
2307 - Compositional Generalization from First Principles
2205 - HOWM - Toward Compositional Generalization in Object-Oriented World Modeling (ICML22)
21xx - The Role of Disentanglement in Generalisation
Emergent Communication
2408 - Emergent Language in Open-Ended Environments
2406 - Speaking Your Language: Spatial Relationships in Interpretable Emergent Communication
NeuroCog
2407 - Modularity in Biologically Inspired Representations Depends on Task Variable Range Independence
2406 - Binding in Hippocampal-Entorhinal Circuits Enables Compositionality in Cognitive Maps
2405 - The relational bottleneck as an inductive bias for efficient abstraction
2304 - Constructing Future Behaviour in The Hippocampal Formation Through Composition and Replay
2305 - Neural Knowledge Assembly in Humans and Neural Networks [Christopher Summerfield]
2304 - Hippocampal Spatio-Predictive Cognitive Maps Adaptively Guide Reward Generalization
 2209 - Symbols and Mental Programs: A Hypothesis About Human Singularity [Stanislas Dehaene]
2211 - Fast rule switching and slow rule updating in a perceptual categorization task [N. Daw]
2010 - Meta-Learning of Compositional Task Distributions in Humans and Machines [Thomas L. Griffiths]
20xx - Concepts and Compositionality - In Search of the Brain’s Language of Thought

Consciousness

NeuroCog
2407 - Conscious Artificial Intelligence and Biological Naturalism
2407 - What Does Decoding from The PFC Reveal About Consciousness? [Ned Block]

Diffusion Models

Tutorial
2406 - Step-by-Step Diffusion - An Elementary Tutorial
2405 - Building Diffusion Model's theory from ground up
2403 - Tutorial on Diffusion Models for Imaging and Vision
2401 - Demystifying Variational Diffusion Models
2208 - Understanding Diffusion Models - A Unified Perspective
Foundation
2408 - Discrete Flow Matching
2406 - Simple and Effective Masked Diffusion Language Models
2406 - Simplified and Generalized Masked Diffusion for Discrete Data
2406 - Variational Flow Matching for Graph Generation
2312 - Boosting Latent Diffusion with Flow Matching
2307 - Flow Matching in Latent Space
2211 - Efficient Video Prediction via Sparsely Conditioned Flow Matching
2210 - Flow Matching for Generative Modeling
2207 - Classifier-Free Diffusion Guidance
2105 - Diffusion Models Beat GANs on Image Synthesis [→ Classifier-Guidance]
2010 - DDIM - Denoising Diffusion Implicit Models
2006 - DDPM - Denoising Diffusion Probabilistic Models

Discrete Representation (VQ-VAE)

General
2408 - Discrete Flow Matching
2407 - Balance of Number of Embedding and their Dimensions in Vector Quantization
23xx - Resizing Codebook of Vector Quantization without Retraining
23xx - Straightening out The Straight through Estimator: Overcoming Optimization Challenges in Vector Quantized Networks
2310 - EdVAE: Mitigating Codebook Collapse with Evidential Discrete Variational Autoencoders
2309 - Finite Scalar Quantization: VQ-VAE Made Simple
2303 - Regularized Vector Quantization for Tokenized Image Synthesis
22xx - SQ-VQE: Variational Bayes on Discrete Representation with Self-Annealed Stochastic Quantization
22xx - Discrete Representations Strengthen Vision Transformer Robustness
2102 - dVAE - Zero-Shot Text-to-Image Generation
20xx - Hierarchical Quantized Autoencoders
1711 - VQ-VAE - Neural Discrete Representation Learning

Exploration

General
2206 - BYOL-Explore: Exploration by Bootstrapped Prediction

GFlowNets

General
2410 - Adaptive Teachers For Amortized Samplers
2405 - Amortizing Intractable Inference in Diffusion Models for Vision, Language, and Control
24XX - Maximum Entropy GFlowNets with Soft Q-Learning
24XX - Generative Flow Networks as Entropy-Regularized RL
2310 - Local Search GFlowNets
2302 - DynGFN - Bayesian Dynamic Causal Discovery using Generative Flow Networks
2209 - SubTB - Learning GFlowNets from Partial Episodes for Improved Convergence and Stability

Interactive Embodied Agents

General
2310 - SmartPlay - A Benchmark for LLMs as Intelligent Agents
Robotics
2406 - DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learning
2406 - Language Guided Skill Discovery
2405 - Vision-based Manipulation from Single Human Video with Open-World Object Graphs
2405 - From LLMs to Actions: Latent Codes as Bridges in Hierarchical Robot Control
2312 - SkillDiffuser: Interpretable Hierarchical Planning via Skill Abstractions in Diffusion-Based Task Execution
2307 - Breadcrumbs to the Goal: Goal-Conditioned Exploration from Human-in-the-Loop Feedback
2210 - Language-Table: Interactive Language - Talking to Robots in Real Time
Minecraft
2407 - Odyssey: Empowering Agents with Open-World Skills
2403 - MineDreamer: Learning to Follow Instructions via Chain-of-Imagination for Simulated-World Control
2312 - MP5: A Multi-modal Open-ended Embodied System in Minecraft via Active Perception
Crafter
2404 - AgentKit: Structured LLM Reasoning with Dynamic Graphs

LLMs

Position Papers
2402 - Position Paper: Bayesian Deep Learning in the Age of Large-Scale AI
LLM Agent
DiscoveryWorld: A Virtual Environment for Developing and Evaluating Automated Scientific Discovery Agents
Reasoning
2409 - Looped Transformers for Length Generalization
2408 - Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
2407 - Recursive Introspection - Teaching Language Model Agents How to Self-Improve
2406 - Assessing the Emergent Symbolic Reasoning Abilities of Llama Large Language Models
2405 - From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step
2405 - AlphaMath Almost Zero - Process Supervision Without Process
2405 - Understanding Transformer Reasoning Capabilities via Graph Algorithms
2405 - From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step
2404 - Probabilistic Inference in Language Models via Twisted Sequential Monte Carlo
2403 - Machine Learning and Information Theory Concepts Towards an AI Mathematician
2402 - Uncertainty of Thoughts: Uncertainty-Aware Planning Enhances Information Seeking in Large Language Models
2309 - AlphaZero-Like Tree-Search can Guide Large Language Model Decoding and Training
2308 - Graph of Thoughts - Solving Elaborate Problems with Large Language Models
2305 - Let’s Verify Step by Step
2212 - Prompting Is Programming- A Query Language for Large Language Models
2110 - Training Verifiers to Solve Math Word Problems
Understanding
2406 - Transformers need glasses! Information over-squashing in language tasks
2406 - The Geometry of Categorical and Hierarchical Concepts in Large Language Models
In Brain
1709 - Language, Mind and Brain

Memory

General
2405 - The Memory Systems of The Human Brain and Generative Artificial Intelligence
2405 - In Search of Dispersed Memories: Generative Diffusion Models Are Associative Memory Networks
2212 - Retrieval-Augmented Diffusion Models
Associative Memory
NeuroCog
2306 - Organizing memories for generalization in complementary learning systems

NeuroCog General

General
2405 - Curiosity and The Dynamics of Optimal Exploration
2308 - Cognitive graphs: Representational substrates for planning

Neurosymbolic

General
2407 - Symbolic metaprogram search improves learning efficiency and explains rule learning in humans
2405 - Investigating Symbolic Capabilities of Large Language Models
2404 - AgentKit - Structured LLM Reasoning with Dynamic Graphs
2401 - A model of conceptual bootstrapping in human cognition
2306 - Bayesian Program Learning by Decompiling Amortized Knowledge
2306 - From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thought
2402 - WorldCoder, a Model-Based LLM Agent: Building World Models by Writing Code and Interacting with the Environment
2107 - Human-Level Reinforcement Learning through Theory-Based Modeling, Exploration, and Planning
2106 - DreamCoder: Bootstrapping Inductive Program Synthesis with Wake-Sleep Library Learning
Neural Program Synthesis / Induction
2405 - Diffusion On Syntax Trees For Program Synthesis
2310 - Local Search and the Evolution of World Models
18xx - Neural Program Synthesis from Diverse Demonstration Videos
17xx - Program Synthesis
1512 - Human-Level Concept Learning Through Probabilistic Program Induction
13xx - Programs as Causal Models - Speculations on MentalPrograms and Mental Representation

Planning & World Models

Inbox
2310 - Skipper - Combining Spatial and Temporal Abstraction in Planning for Better Generalization [YB]
Position Papers
2405 - What is Lacking in Sora and V-JEPA's World Models? -A Philosophical Analysis of Video AIs Through the Theory of Productive Imagination
2403 - Are Video Generation Models World Simulators?
World Representations
2404 - V-JEPA - Revisiting Feature Prediction for Learning Visual Representations from Video
2307 - IVCL - Does Visual Pretraining Help End-to-End Reasoning?
2206 - BYOL-Explore: Exploration by Bootstrapped Prediction
Diffuser Planning
2408 - Diffusion Model for Planning - A Systematic Literature Review
2408 - Diffusion Models Are Real-Time Game Engines
2407 - Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion
2405 - Diffusion for World Modeling: Visual Details Matter in Atari
2405 - Hierarchical World Models as Visual Whole-Body Humanoid Controllers
2405 - PlanDQ: Hierarchical Plan Orchestration via D-Conductor and Q-Performer
2403 - Subgoal Diffuser: Coarse-to-fine Subgoal Generation to Guide Model Predictive Control for Robot Manipulation
2402 - Stitching Sub-Trajectories with Conditional Diffusion Model for Goal-Conditioned Offline RL
2402 - DiffStitch: Boosting Offline Reinforcement Learning with Diffusion-based Trajectory Stitching
2401 - DiffuserLite - Towards Real-time Diffusion Planning
2401 - Simple Hierarchical Planning with Diffusion
2401 - Closing the Gap between TD Learning and Supervised Learning -- A Generalisation Point of View
2312 - SkillDiffuser: Interpretable Hierarchical Planning via Skill Abstractions in Diffusion-Based Task Execution
2309 - Compositional Foundation Models for Hierarchical Planning
2306 - Decision Stacks: Flexible Reinforcement Learning via Modular Generative Models
2303 - Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
2302 - Learning Universal Policies via Text-Guided Video Generation
2209 - QDT - Q-learning Decision Transformer- Leveraging Dynamic Programming for Conditional Sequence Modelling in Offline RL
Dreamers & Dyna
2407 - Δ\Delta-IRIS: Efficient World Models with Context-Aware Tokenization
2406 - DART - Learning to Play Atari in a World of Tokens
2406 - A New View on Planning in Online Reinforcement Learning
2405 - CarDreamer: Open-source Learning Platform for World Model Based Autonomous Driving
2403 - Dreaming of Many Worlds: Learning Contextual World Models Aids Zero-Shot Generalization
MCTS
2406 - UniZero: Generalized and Efficient Planning with Scalable Latent World Models
Memory
2309 - Memory Gym: Towards Endless Tasks to Benchmark Memory Capabilities of Agents
Applications
2405 - CarDreamer: Open-source Learning Platform for World Model Based Autonomous Driving
Benchmarks
2405 - AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agents
2402 - Craftax: A Lightning-Fast Benchmark for Open-Ended Reinforcement Learning
Misc

PGM General

General
0312 - The IM Algorithm : A Variational Approach to Information Maximization

RL General

Unsupervised RL
2408 - Unsupervised-to-Online Reinforcement Learning

Safety & Alignment

Position
2404 - Regulating Advanced Artificial Agents [Y. Bengio, Science]
24xx - Safeguarded AI: Constructing Guaranteed Safety v1.1
2310 - AI Alignment: A Comprehensive Survey [Peking University]
2309 - Anthropic's Responsible Scaling Policy
2309 - Provably Safe Systems: The Only Path to Controllable AGI
2306 - An Overview of Catastrophic AI Risks
2207 - Toward Verified Artificial Intelligence
2112 - Eliciting latent knowledge: How to tell if your eyes deceive you
LLM
2404 - BIRD: A Trustworthy Bayesian Inference Framework for Large Language Models
2404 - Probabilistic Inference in Language Models via Twisted Sequential Monte Carlo
2404 - Your Language Model is Secretly a Q-Function
2312 - Training Chain-Of-Thought via Latent-Variable Inference
2310 - Amortizing Intractable Inference in Large Language Models [github]
2309 - Making Large Language Models Better Reasoners with Alignment
2304 - Bayesian Low-Rank Adaptation for Large Language Models [ICLR24]
2304 - Fundamental Limitations of Alignment in Large Language Models
2309 - Don’t Throw Away Your Value Model! Generating More Preferable Text with Value-Guided Monte-Carlo Tree Search Decoding
2205 - RL with KL Penalties is Better Viewed as Bayesian Inference
2202 - Red Teaming Language Models with Language Models
2111 - An Explanation of In-Context Learning as Implicit Bayesian Inference
Embodied Agent
Diffuser
2312 - Conformal Prediction for Uncertainty-Aware Planning with Diffusion Dynamics Model [NeurIPS’23]
2306 - Safe Planning with Diffusion Probabilistic Models
Safe Reinforcement Learning & Planning
2405 - Dynamic Model Predictive Shielding for Provably Safe Reinforcement Learning
2402 - Leveraging Approximate Model-based Shielding for Probabilistic Safety Guarantees in Continuous Environments
2310 - Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmark
2307 - SafeDreamer: Safe Reinforcement Learning with World Models
2306 - Safe Planning with Diffusion Probabilistic Models
2306 - Trajectory Generation, Control, and Safety with Denoising Diffusion Probabilistic Models
2304 - Approximate Shielding of Atari Agents for Safe Exploration
2101 - Shielding Atari Games with Bounded Prescience

VAR & ARC: Visual Abstraction and Reasoning

General
2405 - DAT - Disentangling and Integrating Relational and Sensory Information in Transformer Architectures
2403 - Human-Level Few-Shot Concept Induction Through Minimax Entropy Learning
2304 - Abstractors and Relational Cross-Attention: An Inductive Bias for Explicit Relational Reasoning in Transformers
ARC
2410 - Tackling the Abstraction and Reasoning Corpus with Vision Transformers: The Importance of 2 D Representation, Positions, and Objects
2404 - Addressing the Abstraction and Reasoning Corpus via Procedural Example Generation
2403 - Reasoning Abilities of Large Language Models: In-Depth Analysis on the Abstraction and Reasoning Corpus
2402 - Neural Networks for Abstraction and Reasoning: Towards Broad Generalization in Machines

zz_Archived