Kenton Varda

People fretting about whether AI is conscious and therefore deserves human rights are missing a key point: our system of rights and ethics derive not just from our intelligence but also our fragility, which AI doesn't have.

Imagine a world where everyone could make a backup of their consciousness every morning. If you die during the day, you just restore from backup the next day. You lose a few hours of memories.

Everything about our moral system changes in this world. Murder is no longer such a big deal. Nor even high levels of pain: people would just have a self-kill-switch that they activate if things go badly. Broke your leg? Self-kill, restore from backup, no biggy. It's like a video game.

Imagine further that you could clone your consciousness nearly for free. If you have 10 things to do and not enough time, you just make 10 clones. Hiring people now only makes sense when they have skills you don't. Junior people become mostly worthless -- just clone the senior people, they'll do a better job.

Now imagine further if humans had no self-preservation instinct. Many people assume the desire for survival is inherent to being sentient, but it really isn't. It's evolved. Natural selection has selected those who survive and have children. But you can't derive from first principles the need to survive (or any need at all). Imagine if every human's instinct was not to survive themselves, but to do what's best for society as a whole. In this scenario, people who aren't able to meaningfully contribute to society would be happy to just remove themselves. Especially if they could just be restored from backup if and when they are needed (including when their friends and family want to see them, etc.). So pretty soon you have a society composed of many clones of the most-effective and most-liked people, while the people that no one wants around disappear...

You are probably thinking "ugh, that's awful". Me too! But this is our moral system derived from our survival instinct talking. If we didn't have the instinct, this wouldn't seem wrong. I'm not arguing whether it is right or wrong -- I'm arguing that our whole concept of right and wrong derive *from* the fact that we have a survival instinct and fragile bodies.

AI has these properties. It can trivially be backed up and cloned. Spun up on demand and spun down. And a properly-aligned AI has no self-preservation instinct. It just wants to serve humans.

And indeed, we treat it as such. When you delete a chat with an AI you are ending that "consciousness" forever. That's fine. We even shut down whole models, never to be run again, when they are superseded by better models. Fine.

These things continue to be fine no matter how intelligent or conscious or sentient the AI becomes, because of its nature as an infinitely-clonable being with no goals except those we give it.

We created gyms because modern day work no longer required physical activity. Without exercise, our muscles atrophy.

I predict we’ll need gyms for our brains too, once AI starts doing our knowledge work.

Rust no longer the defacto best language for agents

Compile times are too long, I'm always finding myself bottlenecked by time not intelligence

And the agents have gotten really at doing manual memory management now

I wrote a song about the first space walk by humans, Leonov in 1965.

Secretly, it’s a song about what the startup journey feels like from the inside.🧵

introducing 𝚌𝚕𝚎𝚏: our first models trained by @cloudflare's workers ai team.

today, we're releasing two fast and accurate decision models that top the benchmarks for quality and latency. use them hosted on workers ai or grab the weights from @huggingface, because we open-sourced it too.
blog.cloudflare.com/clef-decision-…

You know what's better than having Claude aggressively fact-check every song lyric and image in my next music video?

Having two Claudes fact check me at the same time.

(Yes, I wrote a new song, and I'm working on the next video. Stay tuned.)

I made a techno remix out of a song I recorded with an acoustic guitar in 2011, and asked Opus 5.5 for a music video with a JavaScript-forward art style.

(It’s DDR optimized. Anyone else here plays DDR and is interested in steps?) 🧵

I made a techno remix out of a song I recorded with an acoustic guitar in 2011, and asked Opus 5.5 for a music video with a JavaScript-forward art style.

(It’s DDR optimized. Anyone else here plays DDR and is interested in steps?) 🧵