content aware chunking
khazzz1c 0 0 0 18 6.3K 11
Zartbot
@zartbotF
"Chase after the truth like all hell and you'll free yourself, although you never touch its coat tails."
7KFollowers
128Following
110Posts
15Media
71Likes
@zartbotF reposted
最近 RL 圈两个热度很高的工作——BPO 和 Score Centering——看起来是两条完全不同的路： 一个从 PMD（策略镜像下降）+ Bellman 重构出发，推 critic-free 的轨迹目标； 一个从 off-policy drift 分解出发，解释 TIM 为什么制造漂移并把它消掉。
这篇解读把两边的公式推到底，得到一个很干净的结论：
在完整形式下，它们共享同一个 full-vocabulary token 梯度。 条件是四条：同一个 rollout policy μ、相同的 Â、精确计算 full-vocab reverse-KL / expected score、且都不加工程近似与 clipping。
而背后的核心洞见是这个分解： E_q[Rs] = E_q[R]·s̄（drift） + Cov_q(R, s)（signal）
几个值得记住的判断：
① drift 的本质是"向 sampler 蒸馏"——s̄ = ∇Σ q_v log p_v = −∇L_CE，也就是训练在把 trainer 拉向量化、陈旧的 sampler 副本，偏差被反复注入，可能形成正反馈直至奖励崩溃
② group centering 只能缩小 drift，不能消除：advantage 总和为零，但不保证每个前缀下的条件均值为零
③ Score Centering 是 bias correction，不是经典的方差缩减：减去非零的 E_q[s] 是刻意改变梯度均值，与 reward baseline 那种 control variate 不同
④ 为什么有了 IS 还需要它：精确 IS 权重方差爆炸，裁剪后又重新引入 drift；Score Centering 不依赖 importance ratio、不需要裁剪
工程上也很实用：用 stop-gradient 构造成标量 loss 就能靠自动微分得到 (s_{y_t} − s̄)；完整词表太贵时用 top-k 近似，实验里 k=32/128 已接近完整版（文中有可直接用的 JAX 实现）。
一句话总结：梯度等价，但原始理论目标不同——BPO 从 PMD 最优性条件与 Bellman 出发，Score Centering 从 drift 出发。
#RL #RLVR #后训练 #LLM
这篇解读把两边的公式推到底，得到一个很干净的结论：
在完整形式下，它们共享同一个 full-vocabulary token 梯度。 条件是四条：同一个 rollout policy μ、相同的 Â、精确计算 full-vocab reverse-KL / expected score、且都不加工程近似与 clipping。
而背后的核心洞见是这个分解： E_q[Rs] = E_q[R]·s̄（drift） + Cov_q(R, s)（signal）
几个值得记住的判断：
① drift 的本质是"向 sampler 蒸馏"——s̄ = ∇Σ q_v log p_v = −∇L_CE，也就是训练在把 trainer 拉向量化、陈旧的 sampler 副本，偏差被反复注入，可能形成正反馈直至奖励崩溃
② group centering 只能缩小 drift，不能消除：advantage 总和为零，但不保证每个前缀下的条件均值为零
③ Score Centering 是 bias correction，不是经典的方差缩减：减去非零的 E_q[s] 是刻意改变梯度均值，与 reward baseline 那种 control variate 不同
④ 为什么有了 IS 还需要它：精确 IS 权重方差爆炸，裁剪后又重新引入 drift；Score Centering 不依赖 importance ratio、不需要裁剪
工程上也很实用：用 stop-gradient 构造成标量 loss 就能靠自动微分得到 (s_{y_t} − s̄)；完整词表太贵时用 top-k 近似，实验里 k=32/128 已接近完整版（文中有可直接用的 JAX 实现）。
一句话总结：梯度等价，但原始理论目标不同——BPO 从 PMD 最优性条件与 Bellman 出发，Score Centering 从 drift 出发。
#RL #RLVR #后训练 #LLM
2 29 1 174 10.8K 171
@zartbotF reposted
The rate of improvement from original GPT to GPT-3 is impressive. If this rate of improvement continues, GPT-5 or 6 could be indistinguishable from the smartest humans. Just my opinion, not an endorsement. I left OpenAI 2 to 3 years ago. Am a neutral outsider at this point.
245 1K 581 9.6K 0 1.9K
网络可靠性和安全性他们没有，缺我
pepejam@dingcheng_55397 · Oct 5感谢分享，注意力算力法+网络可靠性+安全性，这些胜负手梁文峰都有，他们走在正确的道路上，鲸鱼的10T可以期待
Open quoted post →
2 0 0 12 2.1K 1
what's the infra requirement for next-gen frontier models ? github.com/zartbot/blog/i…
0 12 1 83 6.8K 76
GIF
tarunyaa 1 0 0 4 1.4K 1
we build next-generation CIPU which is twice faster than nVidia BlueField-4 / CX9, with much more reliable on transport and more secure for Cloud Service Provider.
zartbot.github.io/blog/arch/cipu…
zartbot.github.io/blog/arch/cipu…
0 6 0 30 3.7K 13
Chinese version on WeChat official： mp.weixin.qq.com/s/_COtKEdtYzL_…
Zartbot@zartbotF · Sep 22What the DPU Is in the Agentic Era ? we give the answer.zartbot.github.io/blog/arch/cipu…
Open quoted post →
0 0 0 7 2.5K 4
What the DPU Is in the Agentic Era ? we give the answer.zartbot.github.io/blog/arch/cipu…
0 1 1 21 4.1K 13
chinese version on wechat official :mp.weixin.qq.com/s/h5aqT-wbPZMd…
0 1 0 14 4.1K 9
0 3 1 12 5.6K 7
✅arch is simple, verification is hard
Max Lv@m0d8ye · Sep 18西门子EDA近日发布《2026功能验证研究》报告。其中最受关注的一项数据是：在IC/ASIC受访项目中，仅有5% 的团队实现了首版芯片（First Silicon）一次成功，而2024年这一比例为14.4%。
文里讲的和我观察最贴切的一条原因：
软件已经进入“芯片正确性”的定义。 这是我认为最重要的变化。很多 bug 只有在真实 firmware + workload + power management + OS/driver 联合运行时才出现。
我们这几年花了非常多的时间在 presilicon 阶段跑 full system SW verification 也是这个原因。
blogs.sw.siemens.com/verificationho…
Open quoted post →文里讲的和我观察最贴切的一条原因：
软件已经进入“芯片正确性”的定义。 这是我认为最重要的变化。很多 bug 只有在真实 firmware + workload + power management + OS/driver 联合运行时才出现。
我们这几年花了非常多的时间在 presilicon 阶段跑 full system SW verification 也是这个原因。
blogs.sw.siemens.com/verificationho…
0 1 0 18 4.3K 8
YLC really know the big picture of math?
Yann LeCun@ylecun · Sep 16Being better than humans at searching and writing down the formal proof of a theorem does not equate being "beyond human level at math".
This is not all of math, any more than arithmetics, or computing integrals (symbolically or numerically) is all of math. It's one "mechanical" task in the whole activity that happens to be automatable.
Mathematicians invent new concepts, new frameworks, new abstractions, new definitions, and formulate conjectures.
This requires intuition and creativity that current AI systems do not have (yet).
Open quoted post →This is not all of math, any more than arithmetics, or computing integrals (symbolically or numerically) is all of math. It's one "mechanical" task in the whole activity that happens to be automatable.
Mathematicians invent new concepts, new frameworks, new abstractions, new definitions, and formulate conjectures.
This requires intuition and creativity that current AI systems do not have (yet).
0 0 0 2 2K 0
A deepdive analysis on DeepSeek-V4.1-Flash, zartbot.github.io/blog/model_arc…
5 75 3 438 47.9K 375
lol re-define low-hanging fruit😄
Ligeng Zhu 0 0 0 7 2.1K 0
great!
leo 🐾@synthwavedd · Sep 10I am told the Hodge Conjecture is very close to being verified by OpenAI, and that one of OpenAI or Anthropic are also close to solving Birch-Swinnerton-Dyer. The race to be 'next' behind the scenes is unlike anything I've had described to me before
If true - and it may not be, given the scale of the rumour mill right now - it could mean 3 Millennium Problems fall in the space of a month. Crazy times
Open quoted post →If true - and it may not be, given the scale of the rumour mill right now - it could mean 3 Millennium Problems fall in the space of a month. Crazy times
0 0 0 4 1.9K 0
but there is a hard limit on training / inference GPU cluster's VRAM...
jietang@jietang · Sep 10Are you sure? Finding the optimal model size is tricky: data volume, active parameter count, the number of environments, and the target inference cost. Model performance also depends on many other factors, each introducing its own variability.
Open quoted post → 0 2 0 12 2.9K 1
The difference is on how we distinguish "loop" and "recursive", generally "loop" term means re-inject residuals to previously layers, but it may impact both Q K V on attention blocks. So, we need a "recursive" mode to control Q/K/V
mldwit@mldwit_ · Sep 10looped transformers help increase compute and simulate depth.
not sure how you can claim that looping is wrong and in the same breath say that depth is important lol
Open quoted post →not sure how you can claim that looping is wrong and in the same breath say that depth is important lol
1 0 0 5 1.8K 3
reinvent new math!
momoai沫沫🫧@momoai_daily · Sep 9陶哲轩（Terence Tao）刚发的一条帖子，提出了一新思路：与其让各家AI公司争相"第一个宣布解决某个未解决的数学难题"，不如换个竞赛标准——比哪个AI能"第一个提出一个全新的数学洞见（insight）
A\ + O家：
「我将不眠不休地研究用户们的对话」
A\ + O家：
「我将不眠不休地研究用户们的对话」
Open quoted post →
0 0 0 9 3.3K 2
good question...
Alex Rampell@arampell · Sep 8Suppose you get a 2 billion line of code (LoC) Lean proof of the Riemann Hypothesis or the Collatz Conjecture. What have you actually learned?
The proof of Fermat’s Last Theorem is ~13M LoC in Lean, but at least there’s a human-readable mathematical spine underpinning it: Wiles/Taylor-Wiles, and you can follow the argument end to end.
A kernel-verified proof with no human-compressible argument feels like it could be a genuinely different object: proof without understanding.
What happens when the shortest known proof certificate is machine-scale, but the shortest known explanation is nonexistent?
Open quoted post →The proof of Fermat’s Last Theorem is ~13M LoC in Lean, but at least there’s a human-readable mathematical spine underpinning it: Wiles/Taylor-Wiles, and you can follow the argument end to end.
A kernel-verified proof with no human-compressible argument feels like it could be a genuinely different object: proof without understanding.
What happens when the shortest known proof certificate is machine-scale, but the shortest known explanation is nonexistent?
0 0 1 4 2.3K 0