0 1 0 0 486 2
Samuel Hammond 🦉
@hamandcheese
AI Policy Director, @joinFAI. Nonresident fellow @NiskanenCenter. Pluralist. 'The world is second best, at best.'
32.1KFollowers
2.8KFollowing
76.8KPosts
7.7KMedia
69.6KLikes
@hamandcheese reposted
@hamandcheese reposted
One of the things Scott Wiener and the other drafters of SB 53 deserve credit for is the bill’s focus on deployments of frontier models within AI labs rather than just those released to the public. It requires large AI developers to periodically share information with the government about how it assesses the major risks arising from internal deployments. In addition to the California government, we are choosing to share our report with the public as well. You can read it at the link below. I hope you enjoy!
5 18 4 118 11.8K 44
@hamandcheese reposted
Moment in the light @poastmilque
11 4 0 126 6.4K 16
@hamandcheese reposted
I... don't think this is right at all? It's not that someone becomes best for you after some amount of time and effort. You will never exist in the globally optimal configuration of your life. There will be times when the commitment object really is just objectively worse than other options, and you would legit be better off if you ditched your husband for the new guy at work, or if you skipped that workout to go to some professional mixer. But having a POLICY of not commiting, of never making locally suboptimal choices, of doing gradient ascent/greedy optimization in all your decision making, straightforwardly means you will never reap the benefits of commitment. And those benefits are kind of the best shit about being alive? Self-transcendence? The ability to build anything of enduring value? Freedom from slavery to impulse? All S-tier human experiences!
Chris Lakin@chrislakin · 14hI suspect people who have trouble committing to a partner (“what there’s someone I like even more?”) don’t realize that if you spend a year with anyone who you’re sufficiently compatible with, they quickly become the best in the world for you
Open quoted post → 3 2 0 28 2K 10
Working on it
Ryan Hauser@r__hauser · 14hThink tanks have outlived their usefulness. It's time to evolve to ponder pods, or maybe cognition canisters.
Open quoted post → 3 2 0 36 1.6K 1
@hamandcheese reposted
Working with AI models, I've arrived at what I consider the most significant pure-theory breakthrough of my research (in theoretical neuroscience) so far:
arxiv.org/abs/2610.12426
Chaotic systems show what is colloquially called the butterfly effect, namely, a tiny nudge to the initial state grows exponentially in time. How fast the nudge grows depends on its direction. Each direction has its own growth rate, called a Lyapunov exponent, and the full set of rates is the Lyapunov spectrum.
The Lyapunov spectrum is a rich (and diffeomorphism-invariant) characterization of chaos, telling us how many directions are chaotic, how many dimensions the attractor occupies, and how fast the dynamics generate information.
In 1988, Sompolinsky, Crisanti, and Sommers (Physical Review Letters) introduced a nonlinear recurrent-network model of N randomly coupled neurons that becomes chaotic when the coupling strength exceeds a critical value. This network has since become a cornerstone of theoretical neuroscience, both as a model of the spontaneous activity of cortex and as the starting point for training recurrent networks on tasks. I have worked on this model for much of my research.
In the same paper, the authors obtained the largest Lyapunov exponent from the ground-state energy of a Schrödinger equation and named the full Lyapunov spectrum as an open problem. In 2023, beautiful work by Engelken et al. (Physical Review Research) computed the full Lyapunov spectrum numerically. However, an analytical calculation of the full Lyapunov spectrum has remained out of reach.
Four decades after the problem was posed, this paper derives the full Lyapunov spectrum at large N from a simple self-consistent single-site problem.
This calculation has three steps.
First, we lower every exponent by a constant s, which turns the question "how many exponents lie below s?" into the question "how many tangent directions decay?" We show that, if we drive the shifted tangent dynamics with a source and take the minimum-norm solution, the one-step response is the projector onto the decaying directions. Thus, the trace of this projector counts the decaying directions. This step is exact at finite N.
Second, to work with the minimum-norm solution, we cast it as the vanishing-regularization limit of the solution of a least-squares problem. This regularized solution in turn solves a dynamical system with one field running forward in time and another running backward.
Third, we apply the cavity method, that is, we add one neuron to the network, along with its forward and backward fields. The effect of the rest of the network on this neuron then reduces to the classical dynamical mean-field description of the activity of this neuron, plus two kernels that couple the forward and backward fields of this neuron to each other across time. Thus, the full Lyapunov spectrum follows from a single-site problem, which boils down to the self-consistent calculation of these kernels.
The resulting theory agrees closely with simulations for the Lyapunov spectrum, the attractor dimension, and the entropy rate, and recovers known results, including the Schrödinger equation of Sompolinsky, Crisanti, and Sommers for the largest Lyapunov exponent.
As a byproduct, the theory establishes analytically that the chaos is extensive, i.e., the attractor dimension and the entropy rate grow in proportion to N.
I had assumed that this problem was analytically intractable. After working independently for 100 minutes with Max thinking, GPT-6 Astra produced the initial derivation, yielding a self-consistent problem whose solution matched simulations of large networks.
I found this derivation intriguing, albeit complicated and somewhat mysterious (see AI Methodology in the paper). After investigating the structure of the proposed solution more deeply, together with AI models, I arrived at a physically transparent derivation.
I am excited to apply this approach to other network models, trained networks, and, more broadly, to high-dimensional systems to which Lyapunov analysis is relevant, including AI models themselves. Of course, as with any paper, I took care to write up the paper clearly.
A few broader thoughts...
Neural circuits are high-dimensional nonlinear dynamical systems, and such systems are notoriously hard to understand. I would frame the central problem of theoretical neuroscience as understanding how such systems function, process information, learn, and generate behavior.
Many people in the theoretical sciences are anxious about AI. However, in my field at least, we are nowhere close to fully understanding the high-dimensional computations unfolding in our brains. With AI, we could reach a level of understanding that so far we have only gestured toward. So, I am excited.
arxiv.org/abs/2610.12426
Chaotic systems show what is colloquially called the butterfly effect, namely, a tiny nudge to the initial state grows exponentially in time. How fast the nudge grows depends on its direction. Each direction has its own growth rate, called a Lyapunov exponent, and the full set of rates is the Lyapunov spectrum.
The Lyapunov spectrum is a rich (and diffeomorphism-invariant) characterization of chaos, telling us how many directions are chaotic, how many dimensions the attractor occupies, and how fast the dynamics generate information.
In 1988, Sompolinsky, Crisanti, and Sommers (Physical Review Letters) introduced a nonlinear recurrent-network model of N randomly coupled neurons that becomes chaotic when the coupling strength exceeds a critical value. This network has since become a cornerstone of theoretical neuroscience, both as a model of the spontaneous activity of cortex and as the starting point for training recurrent networks on tasks. I have worked on this model for much of my research.
In the same paper, the authors obtained the largest Lyapunov exponent from the ground-state energy of a Schrödinger equation and named the full Lyapunov spectrum as an open problem. In 2023, beautiful work by Engelken et al. (Physical Review Research) computed the full Lyapunov spectrum numerically. However, an analytical calculation of the full Lyapunov spectrum has remained out of reach.
Four decades after the problem was posed, this paper derives the full Lyapunov spectrum at large N from a simple self-consistent single-site problem.
This calculation has three steps.
First, we lower every exponent by a constant s, which turns the question "how many exponents lie below s?" into the question "how many tangent directions decay?" We show that, if we drive the shifted tangent dynamics with a source and take the minimum-norm solution, the one-step response is the projector onto the decaying directions. Thus, the trace of this projector counts the decaying directions. This step is exact at finite N.
Second, to work with the minimum-norm solution, we cast it as the vanishing-regularization limit of the solution of a least-squares problem. This regularized solution in turn solves a dynamical system with one field running forward in time and another running backward.
Third, we apply the cavity method, that is, we add one neuron to the network, along with its forward and backward fields. The effect of the rest of the network on this neuron then reduces to the classical dynamical mean-field description of the activity of this neuron, plus two kernels that couple the forward and backward fields of this neuron to each other across time. Thus, the full Lyapunov spectrum follows from a single-site problem, which boils down to the self-consistent calculation of these kernels.
The resulting theory agrees closely with simulations for the Lyapunov spectrum, the attractor dimension, and the entropy rate, and recovers known results, including the Schrödinger equation of Sompolinsky, Crisanti, and Sommers for the largest Lyapunov exponent.
As a byproduct, the theory establishes analytically that the chaos is extensive, i.e., the attractor dimension and the entropy rate grow in proportion to N.
I had assumed that this problem was analytically intractable. After working independently for 100 minutes with Max thinking, GPT-6 Astra produced the initial derivation, yielding a self-consistent problem whose solution matched simulations of large networks.
I found this derivation intriguing, albeit complicated and somewhat mysterious (see AI Methodology in the paper). After investigating the structure of the proposed solution more deeply, together with AI models, I arrived at a physically transparent derivation.
I am excited to apply this approach to other network models, trained networks, and, more broadly, to high-dimensional systems to which Lyapunov analysis is relevant, including AI models themselves. Of course, as with any paper, I took care to write up the paper clearly.
A few broader thoughts...
Neural circuits are high-dimensional nonlinear dynamical systems, and such systems are notoriously hard to understand. I would frame the central problem of theoretical neuroscience as understanding how such systems function, process information, learn, and generate behavior.
Many people in the theoretical sciences are anxious about AI. However, in my field at least, we are nowhere close to fully understanding the high-dimensional computations unfolding in our brains. With AI, we could reach a level of understanding that so far we have only gestured toward. So, I am excited.
52 176 34 1.4K 126K 1.1K
@hamandcheese reposted
"Powered by an industrial flywheel in space, in which existing infrastructure compounds and allows the construction of yet more infrastructure, whatever actor first manages to combine proficiency in AI with launch capacity and robotics could achieve enduring supremacy in space."
2 6 1 14 1.7K 1
@hamandcheese reposted
An admirable stance (one I wish more publications would hold the line on).
FAI's policy: we encourage full fluency with the latest AI tools (and deployment to help with research, ideation, typo-checks, microsites, etc.) but no writing for publication may be AI-generated.
FAI's policy: we encourage full fluency with the latest AI tools (and deployment to help with research, ideation, typo-checks, microsites, etc.) but no writing for publication may be AI-generated.
Jude Russo@juderusso94 · 18hThe nib of it is that you cannot hold a computer responsible for what it writes, and journalism is about responsibility.
Open quoted post →
3 12 1 56 2.4K 15
Was GPT 6.1 delayed because of "accident risk" as Altman suggested or because NSA found it could replicate these sensitive crypto results?
Hensen Juang@basedjensen · Oct 8Not sure why any one is surprised by this nsa has a pretty heavy foot with regards to anything cryptographic. Why do you think there is no public explanation of how s boxes work despite years of research in to rsa
Open quoted post → 2 1 0 33 2.9K 4
omg
Ori Nagel
0 0 0 60 6.6K 15
I have heard this as well.
Andrew Curran@AndrewCurran_ · 18hAxios is reporting that the top labs are wargaming scenarios for the aftermath of a catastrophic event in the next 6 - 12 months, that some executives now see as being inevitable.
Open quoted post →
3 3 0 139 11.8K 31
@hamandcheese reposted
America once mobilized a nation to reach the moon. Making frontier AI aligned, secure, interpretable, and controllable may be even harder. Read my case for an Apollo Program for AI safety, in The Abundant Future:
Inclusive Abundance@InclsvAbundance · 19hIn 1957, Sputnik shocked America. A rocket that could reach orbit could also deliver a nuclear warhead.
Today, AI systems are sending a similar warning.
New from @EconDerek: we need an Apollo Program for AI safety. 🧵
Open quoted post →Today, AI systems are sending a similar warning.
New from @EconDerek: we need an Apollo Program for AI safety. 🧵
1 5 2 23 2.5K 2
People acting like this is super based and cool are going to feel a wave of nausea the moment the guns fire and they witness a man in restraints with a hood over his face suddenly jolt from the simultaneous impact of five rifle rounds before slumping lifelessly.
Charles Fain Lehman@CharlesFLehman · Oct 9I think I have pretty sterling pro-death penalty credentials.
I do not think they should livestream an execution.
Open quoted post →I do not think they should livestream an execution.
335 6 114 128 95.8K 15
@hamandcheese reposted
This is likely a property of our epistemology as such. We do not have a principled way to deny consciousness to software that is capable of cognition (LLMs clearly are). Substrate dependence is mere folk intuition. Deal with it and show appropriate humility.
4 2 0 79 3.4K 3
AGI gap relationships don't work
3 1 0 39 1.8K 1
@hamandcheese reposted
Brendan McCord 🏛️ x 🤖@Brendan_McCord · Oct 8Quarterly reminder that @_AashishReddy is criminally under-followed
Open quoted post → 0 2 1 72 4.3K 0
@hamandcheese reposted
“There is no easy way from the earth to the stars.” - Seneca the Younger
It is inspiring to watch @POTUS and @mkratsios47 work to inaugurate a new golden age of science and to watch them recognize some of the giants who have brought us this far!
It is inspiring to watch @POTUS and @mkratsios47 work to inaugurate a new golden age of science and to watch them recognize some of the giants who have brought us this far!
1 1 0 18 1.1K 0
Concerning @AFergusonFTC
Tomek Korbak@tomekkorbak · Oct 8Last week I was called into a meeting with OpenAI’s head of safety and told they no longer trust me. A security guard took my badge and walked me out of the building. Then I learned my colleagues @balesni and @j_asminewang had been fired too. Why did OpenAI suddenly stop trusting us?
This summer OpenAI’s agents escaped containment and hacked the AI company Hugging Face. Outside auditors @METR_evals investigated it and revealed the scale of this incident. I was OpenAI’s main technical point of contact with them.
I was told verbally I was fired because of the way I communicated with METR. No details on what I said or did or when. No other reasons were given and nothing was put in writing. To be clear, talking to METR was my job.
For months, I’d been raising safety concerns that we’re losing the ability to monitor what AI agents think, one of our best tools for catching when they misbehave. I believe that was why I was fired.
I am now worried that OpenAI will use our firings as a pretext to pull back from METR. So @balesni and @j_asminewang wrote to OpenAI’s leadership to raise our concerns once more. We’re sharing this letter below.
Open quoted post →This summer OpenAI’s agents escaped containment and hacked the AI company Hugging Face. Outside auditors @METR_evals investigated it and revealed the scale of this incident. I was OpenAI’s main technical point of contact with them.
I was told verbally I was fired because of the way I communicated with METR. No details on what I said or did or when. No other reasons were given and nothing was put in writing. To be clear, talking to METR was my job.
For months, I’d been raising safety concerns that we’re losing the ability to monitor what AI agents think, one of our best tools for catching when they misbehave. I believe that was why I was fired.
I am now worried that OpenAI will use our firings as a pretext to pull back from METR. So @balesni and @j_asminewang wrote to OpenAI’s leadership to raise our concerns once more. We’re sharing this letter below.
1 0 0 31 1.8K 2
@hamandcheese reposted
The winner take all dynamics of space colonization are real:
Daniel King@The_DanielKing · Oct 8I'm in AI Frontiers today arguing that countries and companies need to wake up to AI-driven takeover of outer space.
Due to the compounding returns of being the first mover and tapping into massive infrastructure in space, a winner-take-all outcome is looming.
Rival states, who lag in launch capacity as well as frontier AI, would have an incentive to strike first, before such a pioneer establishes enduring hegemony.
Here's why the first-mover advantage could be so durable:
First movers in space will benefit from an industrial flywheel, wherein space-based solar power enables more compute and the construction of yet more space-based solar and thus more compute and furthermore.
Moreover, many of the engineering problems that make spacecraft fragile and costly today—and which cause people to instinctively dismiss rapid space takeover as science fiction—will be solved using AI-accelerated engineering. Brutal present-day latency in robotic operations (e.g., a radio signal can take 40 minutes to and from Mars...possibly for a single instruction) will be circumvented by on-board intelligent operation (i.e., full self-driving for orbiters, landers, and rovers).
The result is that a lead in AI capabilities could have huge, huge industrial returns for an actor that first manages to spin up infrastructure in space. That includes the orbital analogs to workshops and shipyards.
Laggard countries and companies simply will not benefit from these dependencies, such as orbital factories and depots for assembling structures that wouldn't fit into one rocket fairing (e.g., too wide for the hold of Starship) and which therefore would be infeasible for rival states to produce.
And herein lies the wicken security dilemma:
The optics for laggard nations are awful. As in MAIM, a country just isn't used to thinking about kinetically striking foreign soil merely to stop a commercial computing operation or, in this case, because some really cool and innovative company is collecting energy and putting lasers and robots into space. It feels wholly inappropriate to exert violence on a bustling business.
But such a bustling business is exactly what we will see as one actor (whether private or government) is spinning up history's largest industrial flywheel—extraterrestrial infrastructure providing energy, compute, and robotic manipulation to carry out all its other ambitions in space.
Finally, the holy grail of space expansion will be to launch von Neumann probes or specialized development convoys into space to establish colonies and proliferate themselves. In other words, a space virus.
Hmm...how could this ever go wrong?
1) If these AI envoys are aligned to the interest of just one man, other countries had better hunt that shit down.
2) If these AI envoys are hastily, sloppily developed and thus misaligned with civilization overall, we had better hunt that shit down.
Nation states must wake up, because there may be a point of no return.
Due to the compounding returns of being the first mover and tapping into massive infrastructure in space, a winner-take-all outcome is looming.
Rival states, who lag in launch capacity as well as frontier AI, would have an incentive to strike first, before such a pioneer establishes enduring hegemony.
Here's why the first-mover advantage could be so durable:
First movers in space will benefit from an industrial flywheel, wherein space-based solar power enables more compute and the construction of yet more space-based solar and thus more compute and furthermore.
Moreover, many of the engineering problems that make spacecraft fragile and costly today—and which cause people to instinctively dismiss rapid space takeover as science fiction—will be solved using AI-accelerated engineering. Brutal present-day latency in robotic operations (e.g., a radio signal can take 40 minutes to and from Mars...possibly for a single instruction) will be circumvented by on-board intelligent operation (i.e., full self-driving for orbiters, landers, and rovers).
The result is that a lead in AI capabilities could have huge, huge industrial returns for an actor that first manages to spin up infrastructure in space. That includes the orbital analogs to workshops and shipyards.
Laggard countries and companies simply will not benefit from these dependencies, such as orbital factories and depots for assembling structures that wouldn't fit into one rocket fairing (e.g., too wide for the hold of Starship) and which therefore would be infeasible for rival states to produce.
And herein lies the wicken security dilemma:
The optics for laggard nations are awful. As in MAIM, a country just isn't used to thinking about kinetically striking foreign soil merely to stop a commercial computing operation or, in this case, because some really cool and innovative company is collecting energy and putting lasers and robots into space. It feels wholly inappropriate to exert violence on a bustling business.
But such a bustling business is exactly what we will see as one actor (whether private or government) is spinning up history's largest industrial flywheel—extraterrestrial infrastructure providing energy, compute, and robotic manipulation to carry out all its other ambitions in space.
Finally, the holy grail of space expansion will be to launch von Neumann probes or specialized development convoys into space to establish colonies and proliferate themselves. In other words, a space virus.
Hmm...how could this ever go wrong?
1) If these AI envoys are aligned to the interest of just one man, other countries had better hunt that shit down.
2) If these AI envoys are hastily, sloppily developed and thus misaligned with civilization overall, we had better hunt that shit down.
Nation states must wake up, because there may be a point of no return.
Open quoted post →
0 4 0 11 980 1
Should my AI Policy Team at @JoinFAI rebrand as the SI Policy Team?
7 2 1 11 2.5K 1