Zartbot
@zartbotF
The difference is on how we distinguish "loop" and "recursive", generally "loop" term means re-inject residuals to previously layers, but it may impact both Q K V on attention blocks. So, we need a "recursive" mode to control Q/K/V
mldwit@mldwit_ · Sep 10looped transformers help increase compute and simulate depth.
not sure how you can claim that looping is wrong and in the same breath say that depth is important lol
Open quoted post →not sure how you can claim that looping is wrong and in the same breath say that depth is important lol
1 5