Elena

pat gelsinger (@PGelsinger) designed chips at intel in a period when doing so could involve writing your own hardware description language, which he did for the 486. his team also built a compiler and tools for automating placement and routing. “I wrote my own language,” he says in this pod with @RaghuRaghuram and @appenz. it's a good reminder of how many things had to be invented alongside the processors themselves, and why the prospect of AI taking over more of that work is so exciting.

pat then asks us to imagine designing an excellent AI accelerator in three months. getting it fabricated, packaged, and into a working rack takes roughly nine more in his example (the relevant unit is now the rack: “nothing's a chip anymore, it's a rack”). so there's a period in which your very clever design has to sit and wait for its physical existence, while the models whose workloads you've designed it for can keep evolving. his description of graphcore is that it “wasn't a bad design, but the world moved on.” i think about how often people announce a new model or a new way of using one, and nine months seems like a very long commitment to your current understanding of what the hardware ought to do.

guido's argument is that agents should make it easier to support a greater variety of architectures, because writing all the software to use an unfamiliar chip has historically been one of the barriers. pat's answer is that somebody still has to pay for manufacturing and for a place to put all these chips. he expects consolidation, even with better software tools. if you're a customer committing to a data center and its electricity supply, you're making a rather different bet than the engineer who's just got a new design to work.

which leaves a lot for hardware people to do. pat is especially dissatisfied with hbm (high-bandwidth memory), pointing to problems with density, bandwidth, and heat; he calls it “a hideous memory,” and says he's just funded a new memory company that's still in stealth. he wants better memory closer to compute, optical connections between systems, and less electricity lost in conversion on its way to the chip. he also seems very happy to talk about cooling. “engineers are becoming plumbers,” is how he puts it.

if we're excited about being able to design more things with AI, we should be excited about the work that allows us to manufacture and operate them, too. pat has already lived through a period of having to invent quite a lot of the surrounding machinery. it makes sense that he'd see another one as an opportunity.

an important part of the lana del rey fan journey is realizing this isn’t a popstar we’re talking about, but a timeless musical genius on par with frank sinatra or beethoven

of all the takes that age terribly, “dont put ads in a consumer product” is probably the one that’s most guaranteed to make you look sanctimonious and self-righteous in retrospect. every single great consumer internet product and platform with >1bn users delivers some form of ad unit, because that’s how you scale a product to the largest group of people possible. ads are, quite literally, why we can have nice things.

and so it shouldn’t have been a surprise that back in jan / feb the following things happened 1) openai announced they were rolling out ads to subsidize their enormous base of free users 2) a bunch of people tried to paint this as a bad thing. now, with the benefit of ~240 days of hindsight, we can audit just how ridiculous this kneejerk reaction was.

as of august, oai has ~$1bn in arr for their ad product. in this podcast recorded for the 7th edition of the top 100 consumer ai apps leaderboard, i sat down with @omooretweets, author of the bi-yearly report, and @joshelman, to discuss why ads remain such an important part of the consumer experience, and why subscriptions can only take us so far. some takeaways:

- the subscription market is a peculiar slice of the consumer market. olivia's yipit panel suggests around 4.5% of US consumers pay for an ai subscription, and spending is concentrated in people using it to build, create, and get work done.
-a founder who needs every user to pay has a strong reason to keep building for that audience (a rather circumscribed group of people who value saving time over spending it). advertising creates room to serve people with different interests and willingness to pay. developers / creators who view ai as a way to get leverage over their lives is an important group of paying subscribers, but a smaller cohort compared with people who view ai solely as a source of information or mild entertainment for now.
- josh pointed out that ai companies can become afraid of growing too fast because every new user brings meaningful expense (this explains why, for example, meta seems to be pushing muse a little less aggressively than a product like threads, which cost far less marginally to serve to users). but as inference gets cheaper, ads and transaction fees can support more of the experiences people might want to try.
- josh found its ads useful while planning a trip. olivia thinks the context people supply could make the advertising more effective. that creates a product-design opportunity: help someone find what they want, with the sponsorship made overt and a clear value add.
- a valuable audience doesn't have to be everyone. olivia points to openevidence's early success with advertising among doctors. specialized ai products can gather audiences whose shared needs are useful to particular businesses. there is room for companies with much more focused purposes than a general assistant.

i think this is why the ads question matters beyond oai’s revenue. the business model helps determine who gets products built for them. ads are what enable ai to get built for everyone.

when we think about ai in the physical world, we usually default to thinking about humanoids or self-driving cars or or models helping scientists discover new drugs and materials. but there’s another version of physical-world ai that i think deserves more attention: using intelligence to orchestrate all the benign-seeming things that have to happen for something to get built.

which is especially important because the physical world is this wildly chaotic unpredictable place. what happens when the physical world refuses to cooperate. your shipment falls off a ship. or a strait closes. where, exactly, can an agent intervene?

this is the problem we get into in my conversation with @askvladi, cofounder of lio, and @seema_amble. lio builds ai agents for procurement, in other words the work of figuring out what a company needs to buy, finding suppliers, negotiating terms, and making sure the right things arrive. when you’re building an aircraft or a data center, those decisions connect engineering, logistics, and finance in very consequential ways.

vlad’s answer to the question of how to wrangle the physical world was about probability. suppliers have different records of reliability and routes have different risks. understanding those things in concert changes what you should buy / what you should pay: a cheap component that holds up an entire project can become a very expensive component. this all requires a much richer understanding of the world than the price and delivery date sitting in an enterprise database.

i think this is a very compelling application of ai. a lot of our ability to build more ambitious things depends on making thousands of these decisions well. robots justifiably get a lot of attention; getting everything required to build them in the same place at the right time is just as important!

my proposal for “pacing the frontier” is that model releases should be limited to 2x/year and coordinated as staggered seasonal collections, like fashion week.

so excited to see this launch. @eriktorenberg and @david__booth have been laser-focused on networks and founder ecosystems for their entire careers, and the startup world is better because of it. @cosign is the culmination of this work, but also still just the beginning...

all the world’s a sim / and all the men and women merely agents

ncaa but for h16n / uatx / thiel fellows / yc / speedrun / 1517 etc

who's building this?

in 1882, thomas edison's pearl street power station (the first commercial power plant in the us) began serving 59 customers in lower manhattan. that same fall, mit introduced the country's first electrical engineering curriculum. this co-evolution of enterprise on one side, and pedagogy on the other is one of the many forces that brought america firmly into the industrial era in the late 1800s/early 1900s.

i think this is useful context for the conversation between @eriktorenberg @bhorowitz @gaganbiyani about what education for the ai era should look like. the institutions we now treat as permanent fixtures had to make decisions about emerging technologies and whether it was worth teaching a field whose applications were still being discovered.

in this pod announcing horowitz andreessen academy (@theacademysf), ben makes the case that a technological change on the scale of ai should prompt a similarly ambitious rethinking of education. one of my favorite details is ben recounting a conversation with stanford's dan boneh about making problems hard enough that students *need* ai to solve them. that seems like a much more interesting educational ambition than spending the next decade trying to establish whether someone used a chatbot on their homework.

there's something very regressive about knowing young people have access to tools that can help them do unprecedented things, and then evaluating them entirely on what they can do without them. we should all be excited to see what happens when a school like haa takes the opposite approach, and prepares students for the machine age.

one of the more reliable rules of the internet is that if a sufficiently committed group of people in silicon valley decides something is good/bad/going to change the world/kill us all, you will inevitably learn all about it (and then form an opinion about it yourself). this has been a law that governs everything from ai discourse to health/biohacking to what you eat to how to track your body’s functions. i think people are only just beginning to appreciate just how culturally influential the bay area is.

a funny byproduct of this phenomenon is that sf people will sometimes take credit/co-opt cultural trends they aren’t even directly responsible for. so i talked with @being_on_line about how to disentangle a true bay area export from a hijacked narrative, and more generally how to figure out what is actually happening in the world. an unsurprisingly tall task.

sf sometimes encounters a subculture years after it’s been incubated, and immediately claims and mainstreams it. chinese peptides are a particularly good example: earlier this year, a lot of writng/blogging about peptides attempted to paint it as a sf-exported phenomenon, but it was really a trend that started in the south and midwest, among the bodybuilding community. a 2013 DOJ case even documents an illinois distributor importing them in 2010–11 for the bodybuilding market.

sf is ofc capable of homegrown culture (and productive neuroses). last year, nat friedman's plasticlist project spent about $500k over six months testing 296 food products for 18 plastic-associated chemicals. (specifically chemicals like phthalates and bisphenols). i personally believe this kind of citizen science / gonzo testing is only going to become more in-demand as people increasingly care about food provenance/toxins.

i think this is why these trends are worth paying attention to even when parts of them look ridiculous. a small group can change what gets tested, funded, packaged, and eventually made easy for everyone else to do.

i also don't think the endpoint of the current body-optimization boom is that everyone gets conventionally hotter and starts to look like clavicular. i think people are going to do increasingly specific, strange things to themselves, depending on what they personally think is worth optimizing (an example of this is that people are now working on wearables to achieve the equivalent of a full night’s sleep in four hours).

full convo below!

greg brockman says “we’re now in the AGI era.” one of the most interesting things about this conversation with @bhorowitz and @eriktorenberg is how long he’s been thinking about when that would happen.

at the beginning of the pod, @gdb recalls doing some math on compute with ilya around 2016/2017. following the trajectory of moore’s law, they thought agi might be about 15 years away. if you were willing to build enormous supercomputers and spend hundreds of billions of dollars, maybe you could get there in 10. the accelerated scenario they were envisioning a decade ago is roughly the one we’re living through now.

the faster timeline depended on people deciding to build an enormous amount of infrastructure. greg describes the present moment as a lot of long-running forces coming together: “it kind of makes sense it’s happening now.”

im the kind of person who resists the idea that it’s possible to predict the future, but if you were reading the right research papers 10 years ago, there were a lot of things you could extrapolate out and be roughly right about a decade later. brockman remembers openai discussing agents that could use a screen, keyboard, and mouse at an offsite in napa in 2015. he also points back to 2017/2018-era ideas for how humans might supervise systems that are smarter than they are. put differently, people were thinking about app-layer innovations like computer-use, and ai safety long before capabilities existed.

greg now thinks astra is reasonably described as agi, while acknowledging that its abilities remain uneven. in a recent essay of his titled “the defenders window” he describes safety requirements during training and evaluation, and getting powerful defensive tools into the hands of organizations before comparable capabilities spread to attackers (there is now a $ 1bn fund for this, seeded by oai).

a personal illustration of this is his own website. after the hugging face incident, greg asked an agent to check his simple blog and got back 13 security vuln findings. then it went into his cloudflare dashboard, changed settings, migrated the site, checked its work, and scheduled a follow-up to finish the email protections. there was still a bunch of ordinary preparation to do on the personal website of someone who had spent a decade thinking about agi.

which is kind of funny. you can have a pretty good sense of when the technology/agi will arrive and still have an enormous amount of work to do to get the world ready for it. greg’s account brings those two timescales together: the decade spent building toward this moment, and the relatively shorter window to act on what is now possible.

one of my favorite weekly habits is reading lunch with the ft.

so i made a world of lunches

every single "lunch with the ft" mapped, built in 3 hours with astra and minimal supervision
lunch-atlas.pages.dev

it’s the machine age now. plan accordingly.

the datacenter-themed chanel spring summer ‘17 RTW collection.

you couldn’t do this kind of thing today, they’d cancel you.

pleased to announce jalapeno chip early access at the a16z office

in true borderless fashion, i just ran into @GEVS94 on his way to the airport

extremely scott kupor voice: the secrets of sfo

in my mid-20's, a time when about half of my friends hated their jobs (a very normal phenomenon if you're in your mid-20s), i noticed a group of people my age who looked extremely happy and actualized at work. and all of these people somehow seemed to work at stripe. i'll call out @tamarawinter and @orbuch specifically as some of the first people who made me realize that it was possible to be 25 years old and do *singular* work every day, and not just whatever garden-variety boilerplate the company demands.

this podcast with @gaybrick and @DavidGeorge83 helps explain why this was, and still is, the case. will says stripe increasingly is a platform for founders internally, just as it's always been a platform for founders externally. he wants each employee to feel like an “auteur”: an engineer, product thinker, and creative person with enough context and authority to make something real.

we're now seeing a new generation of this pattern, with stripe's support of stuff like tempo (a team that might have one of the highest concentrations of founder-mindsetted people im aware of? cc @gakonst @dwr @varunsrin @liamihorne etc) the embrace of teams like privy and bridge, and other incubated initiatives like minions.

stripe is trying to make employees more founder-like while preserving a shared standard of quality (one of my favorite details on this point: stripe creates simulated customer accounts so teams can experience realistic UIs for disputes and refunds before exposing products to real users). this is something that's actually very tough to do well, so it's very cool to hear will talk about it on this pod.

back in 2017 i was interning as an equities analyst at a fund in nyc, which had a 9am morning call every weekday. the purpose of this call was to discuss quarterly earnings or current events or whether we should be buying nvidia (at the time, on a tear because of the '17 crypto boom), but one morning in june the call opened with a bunch of people freaking out because some guy named "travis" had resigned. after a couple of rounds of "travis is gone" "did they fire him" "bloomberg says he resigned," i was especially confused because the firm had a portfolio manager named travis, and i looked over at his desk and he was still sitting there, not being escorted out by security or anything. who was travis?

it was only after things had calmed down that i realized the people on this call were talking about *travis kalanick* of uber, a figure who loomed so large in the minds of wall street investors that even *two years before* uber ipo-ed, the news of his departure was treated like a six sigma event. everyone wanted to know: if travis was gone, what would happen to his company?

even in his uber days, travis was the bits and atoms guy (watch the video clipped in the first ~2 minutes of the below podcast for proof of that). and in order to be a bits and atoms guy - as in, the kind of guy who's so discontent with the slow, staid, schlerotic state of things that he has to painstakingly reorchestrate the physical world with computers, perhaps pissing off some powerful interests along the way - some people might call you difficult. some people will not understand your vision. some people will try to get you fired.

but if you're a bits and atoms guy, you never stop being one. you might retreat into near-obscurity, or spend years wandering in the wilderness, biding your time in exile. but you will not stop building. and one day you will return triumphant like a modern-day charles de gaulle, even better than ever, and ready to prove wrong everyone who didn't understand the first time around.

being in a room with travis leaves you pretty pumped up. you can see why he's such a rallying leader, and why (by my count) at least six top uber execs have chosen to follow him to his new company, atoms. it isn't simply that the job isn't finished (although that's a big part of it, as travis attests). it's also because it's clear that there are very few leaders like travis out there: people who have such a holistic worldview that they can get people who would otherwise be founders themselves to join the ride and help build an extremely specific vision for the future.

here are some things that stood out to me from this event:
-travis is a branding/comms mastermind. he deploys memorable phrases like “food computer,” “wheelbase for robots,” “atoms as bits” and describes his vision for cheap meal delivery as "autonomous burritos." you can see why people get excited for his vision for the future. there's extraordinary complexity in what atoms is trying to do, but travis makes each step along the way feel manageable because he can distill these ideas into extremely concrete images.
-atoms is building full-stack autonomation for three different divisions: food, mining, and transportation. food supplies the real estate, software, robotics, and logistics laboratory; mining supplies a commercially deployed autonomy business; and transport is intended to become the reusable movement platform connecting multiple physical industries. travis argues humanoids make sense for diverse, low-volume work in spaces designed for people. but at industrial scale (which atoms is doing), purpose-built machines should win on throughput, space, cost, and reliability.
-the best way to get over your last company may be the same as the best way to get over your ex: fall in love again. travis says anger and fear of failure can carry a founder through years of pain, but they also produce long nights where very little gets done and sharp elbows felt by everyone around you. atoms is what happened when he learned to build toward a future rather than against an enemy.
-doing hard things is a reminder you can do more hard things. in the q&a ben remembers travis describing attacks by chinese ride-sharing companies as proof that, if he could survive china, the rest of the world would be manageable. travis still deliberately creates problems when things become easy. but the rate of problem creation cannot exceed the organization’s rate of problem-solving.

such a fun talk and q+a with @travisk @bhorowitz @eriktorenberg ! check it out.

i've always enjoyed reading about the dot com era and its direct aftermath. it's a great reminder that tech cycles are long, unpredictable, and full of troughs of disillusionment before things that feel "inevitable" ever really come to pass. it's particularly funny to think about the crop of stanford kids graduating in '01-'05 (which includes a list of guys like balaji, stephen cohen, joe lonsdale, temporarily sam altman, and ofc garry tan) who started out when most people thought tech was deeply uncool and perhaps even completely over.

so i really love the way this pod begins - with @garrytan recounting what it was like graduating in '03, taking a job at microsoft because it was the practical thing for a young guy with student loans to do, turning down an early offer at palantir (which he eventually did accept as employee #10) then heading deeper into the world of startups and yc.

this podcast hosted by anish (@illscience) is actually a great study in how to get ever-closer to yourself and trusting your gut. there's also (this being anish and garry) a lot about building very useful ai products and living 3-5 (or in garry's case 20) years in the future.

doing what you instinctively know is right, being unhedged, being earnest, or in garry's words, not LARPing, isn't the fastest but it's undoubtedly the best path toward actually having an impact. this conversation is basically an instruction manual in how to do that in all areas of life - from career decisions, to building, to local politics (we probably have garry and a handful of other people to thank for the fact that sf is now a nicer city to live in than nyc).

good pod! check it out! enjoy!