Transformer
12 articles tagged with this topic
500M Parameters, 11 Patterns — GitHub Project Pries Open AI's Black Box
A GitHub project flips large models inside-out: 500M-parameter models may have just 11 independent patterns — a breakthrough for interpretability.
AI Quant Trading Goes Open Source: Tech Barrier Falls, Profit Barrier Doesn't
A Juejin AI quant tutorial goes viral: full code from data to RL live trading. Does the retail-institution gap shrink when tech barriers collapse?
Pathway's new architecture matches GPT-2 — a crack in the "you must stack GPUs" rule
Pathway's BDH post-Transformer architecture matches GPT-2 performance from 10M to 1B parameters on regular GPUs, potentially loosening the industry's
Self-Attention Powers AI Context — But Few Firms Truly Understand It
Self-attention is the core of mainstream AI, enabling simultaneous word relationship analysis. Understanding it is key to evaluating AI costs and ROI.
Transformer Book Read 3 Times: LLM Race Shifts from API Calls to Foundational Logic
A deep learning book read 3 times. While most only call LLM APIs, understanding principles like attention mechanisms now dictates AI app success and c
Million-Param GPT on Journey to the West: Demystifying LLMs Is the New Imperative
Training a million-param mini Chinese GPT on Journey to the West locally reflects the industry's urgent need to demystify the LLM black box and master
7 Years of Transformer Dominance: LLM Architecture Awaits the Next Reshuffle
Transformer underpins LLMs via self-attention, fixing old algorithms' parallel and long-context flaws. Grasping it reveals LLM capability limits and b
Transformer Attention Explained: The 2017 Engine Behind LLMs' Long Memory
Attention is a core LLM principle, solving AI amnesia by weighting key info. Understanding it isn't for coding—it reveals long-text limits and compute
C++ Transformer From Scratch Demystifies LLMs, But Won't Shift Compute Paradigm
A zero-dependency C++17 GPT (0.83M params) demystifies LLMs, but its 75x efficiency lag vs. industrial frameworks proves foundational innovation still
Transformer: 7 Years, 120K Citations—Key to the LLM Race
Google's 2017 Transformer is the LLM bedrock, replacing RNNs with parallel attention. Grasping it reveals who takes shortcuts in the LLM race.
Decade of Seq2Seq: The True Technical Starting Point of LLMs
Google's 2014 Seq2Seq architecture is the shared technical foundation of LLMs like GPT and BERT. Understanding its encoder-decoder division and info b
Compiling a Calculator Into AI Weights: A New Path to Decode Transformers
A dev compiled an RPN interpreter into Transformer weights. The 1.1GB basic-math model's value: offering a new way to bypass training and decode AI in