Working Memory Constraints Scaffold Learning in Transformers under Data Scarcity
We investigate the integration of human-like working memory constraints into the Transformer architecture and implement several cognitively inspired attention variants, including fixed-width windows based and temporal decay based attention mechanisms. Our modified GPT-2 models are trained from scratch on developmentally plausible datasets (10M and 100M words). Performance is evaluated on grammatical judgment tasks (BLiMP) and alignment with human reading time data. Our results indicate that thes
Record details
Published: 22 April 2026
Source: arXiv
Category: Research
Topics: Safety & alignment · Finance, VC & PE
Retrieved: 14 July 2026
Related evidence
These records share source-supplied organisations, an exact publisher byline, automatic topics or regions. The reason is shown on every link; related does not mean supporting, agreeing with or verifying this record.
Can Persona-Prompted LLMs Emulate Subgroup Values? An Empirical Analysis of Generalisability and Fairness in Cultural Alignment
arXiv · 14 April 2026
Subtitle-Aligned Fine-Tuning of Whisper for Swiss German ASR: Benchmark Contamination, Convention Mismatch, and an Honest Baseline at 25.6% WER (13.8% cWER)
arXiv · 29 May 2026
GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Models
OpenAlex · 17 March 2023
Fields Medalist who published a paper on AI-driven human extinction now works for OpenAI
The Decoder · 8 August 2026
When AI Tells You What You Want to Hear: Sycophantic Behavior of Large Language Models in Dementia Care Settings
arXiv · 13 April 2026
From GPT-3 to GPT-5: Mapping their capabilities, scope, limitations, and consequences
arXiv · 11 April 2026
How to cite this record
ethics.ai (22 April 2026), “Working Memory Constraints Scaffold Learning in Transformers under Data Scarcity,” evidence record 5491, https://ethics.ai/record/5491 (originally published by arXiv).
Use and limitations
This page is a stable index and citation surface for a source record. ethics.ai did not author the underlying report and has not independently verified every claim. Automatic topics may be imperfect. For consequential use, quote and cite the original publisher.