Podcast: How an anonymous blog during the neural network winter led to Japan’s national AI champion
I love a good story, and David Ha of Sakana has a great one. A few weeks ago on the “Securities” podcast, I talked about Lux’s $30 million founding seed round into the world-class generative AI research lab in Tokyo, Japan. One aspect of our thesis I emphasized was the unique macro trends that are converging to make Tokyo a tech darling right now among entrepreneurs (and thus VCs). Running parallel to that macro story though is the micro case study of co-founder and CEO David’s life while at Goldman Sachs as a derivatives trader, well before he became the renowned machine learning expert and popularizer.
David moved to Tokyo after college to do derivatives trading, seeing parallels with the Bayesian probability that he experimented with as an undergraduate. He worked in Roppongi Hills, a cavernous modern commercial campus in central Tokyo, and Goldman’s culture at the time still offered lengthy lunches, a remnant of a trading culture where tips and trades were exchanged over sushi and bowls of ramen. David used his Roppongi Hills loyalty points card to study up on machine learning at the corporate campus library.
He then — quietly — started putting that learning into practice. He built experimental machine learning demos and shared them on a then-anonymous blog. The interactivity of those demos would help them spread virally at a time when neural networks and AI were just on the cusp of emerging from a frozen winter of disinterest from research scientists and certainly the commercial world. That virality would lead to a fellowship with Google Brain and ultimately to his founding of Sakana alongside Llion Jones and Ren Ito, once again returning to Tokyo as an expat to build machine learning models.
We talk about a lot in this episode, but obviously as a writer, I love stories where people are thoughtful, experimental and put their work out on the internet where it engages others they never thought to meet and now wish they had met earlier. It harkens back to the original joy of the internet before what Cory Doctorow has dubbed “enshittification”: passionate people browsing curiously, all connecting across the virtual world no matter who they are or where they come from.
🔊 Listen to How an anonymous blog during the neural network winter led to Japan’s national AI champion
The Lost Subways of North America
One reason for Tokyo’s influx of talent the past few years has been its indomitable public transit system, which is among the most extensive and comprehensive in the world. While videos of subway platform pushers compressing humans into metal boxes seems to be what the system is most popularly known for in America, the reality is that all aspects of the system are designed to offer joy to passengers in every detail, from the sound effects used to the wayfinding inside stations and train cars.
Transitioning back to New York City or the rest of North America from an Asian megacity can be a downright stupefying experience. America was a pioneer in trains and subways in much the way it pioneered the airplane, inventing whole new modes of transit to help a rapidly growing and sprawling country connect itself together. Los Angeles once had one of the most extensive networks of streetcars in the world, a fact buried under the puny subway system that is finally being expanded this decade.
Why was America so far ahead, and how did it fall so far behind? Jake Berman set out to investigate with a new book titled “The Lost Subways of North America,” in which he surveys 23 cities alphabetically from Atlanta to Washington DC to explore what was once present, what was lost, and why. Unlike books like Robert Caro’s The Power Broker that take their narrative from a singular person (Robert Moses) or city (New York), Berman is seeking patterns across the continent, synthesizing interesting connections as he meanders (although he’d prefer the train) from city to city.
The “lost” of the title refers to the hundreds of train lines that have been ripped out over the past century. Through dozens of gorgeously reproduced maps, Berman showcases just how extensive the rail systems were in cities like Los Angeles and Boston before their removal with the rise of the automobile. Far from being failures, many of these systems were actually extraordinarily profitable monopolies, often owned by the electric company that supplied the trains their power.
Those monopolies though were part of the problem. The swiftly growing urban cities of the Americas of the late 1800s and early 1900s desperately needed these trains — and there was typically only one company or a small cartel offering service. The public’s ire was intensely focused on these companies, who would use their excess profits to buy out politicians and prevent any form of competition or regulation (in one choice example, the attorney general of California Tirey Ford resigned to become head of the legal department for San Francisco’s Market Street Railway, so good was the private-sector compensation. He would go to prison for bribery several years later). U.S history textbooks generally cover railroads and antitrust in the context of farmers, but I thought it was interesting to connect the antecedents all the way into urban cores.
The automobile — that abomination and nightmare of every utopian urban planner — was the light of freedom from these rapacious monopolists. Within just a few decades, the extraordinary power of the railway companies would be reduced to nothing, with company after company declaring bankruptcy and most of their assets bought at fire-sale prices by municipal authorities. Public ownership ushered in a cycle of decay: these old assets desperately needed rehabilitation, but in the postwar economy of the 1940s and 1950s, the priority was on connecting newly-built suburbs into cities via highway. Add in America’s toxic brew of racial politics, and railways have struggled to secure investment all the way to the present day. Today, not a single city in America or Canada has a world-class subway system, although New York’s is certainly among the most extensive.
What went wrong? Berman recites the evidence across cities given his chosen format, but there are a couple of points I would synthesize. First, the stochastic chaos of politics. American planning around infrastructure and particularly transportation is driven by a complex amalgam of local, county, state and federal governments, in addition to a panoply of other agencies and authorities that can all have overlapping jurisdictions. This morass means that plans ebb and flow much like the tides, and only occasionally do all the politics line up and down our federal system to approve one.
Second, and related, is the challenge of prioritization. After decades of failed starts around transportation, there is palpable cynicism from American voters that any plan will fully come to fruition. Therefore, voters desperately want their demands to be met first, and they will vociferously argue against any plan that starts with others’ interests. This trend can be hard to see in the daily headlines, but it’s obvious across nearly two dozen case studies. City leaders will endlessly debate the exact prioritization of different lines or services, while years and often decades pass by. In many cases, all of the plans could have been implemented in the time it took to debate what should be implemented first.
Finally, there is a sense that one needs to implement grandiose and comprehensive plans rather than evolutionary improvements that just make these transportation systems better over time. Part of this is the bias of Berman’s narrative, which synchronizes around the release of watershed plans that defined (or didn’t define) these different systems. These big visions are designed to generate momentum to overcome the chaos of politics while trying to make everyone simultaneously happy to avoid the prioritization problem. America’s decentralized governance guarantees — as part of its design — that such comprehensive plans will never see fruition. The rare cases like Washington DC’s Metro come from direct federal authorization in a city which lacked home rule at the time, or from high-pressure timelines created by major events, such as Montreal’s ’67 Expo and Vancouver’s ’86 Expo.
Berman’s book is fascinating, but it does have its flaws. The high-level overviews of so many cities across so few pages unfortunately means that the treatment of some historical anecdotes is superficial at best. Transportation planning requires remarkably robust local knowledge to be effective, and the curious out-of-towner is — in all likelihood — wrong about the details and possibly wrong about the overarching narrative as well. That’s what you get for covering a century of history in three pages.
But the upshot is that in one comprehensive and compact coffee table book, we can peruse through city after city and imagine, “What if?” What if America had chosen a different course? What would it be like if our cities were connected as they are in Europe, Asia and growing regions of Latin America and now Africa? Silicon Valley likes to say that ideas are worthless since all the value is in the execution. I’ve always hated that concept as an ideas merchant myself, but one wonders if all of the energy behind these grand plans had been placed on execution, whether we might well have a Tokyo-quality system in some of our most important cities.
Is Silicon Valley out of ideas?
Finally, talking about ideas versus execution, it used to be a widely-held belief among Silicon Valley CEOs that recourse to the share buyback was the last stand for a dying tech business. Given the exponential growth of technology over the past few decades, it was anathema to sound governance to willingly hand over cash to shareholders rather than reinvest it into R&D.
The exception was always the largest tech giants, who by dint of their monopolization and extraordinary profits, truly don’t know how to spend hundreds of billions of dollars. Apple alone bought back more than $600 billion worth of stock through 2023, with Alphabet and Microsoft similarly buying back tens of billions of dollars in shares each.
Such buybacks are increasingly heading down market though:
- Meta announced earlier this month a $50 billion authorization for a share buyback after a $40 billion authorization last year, hoping to align its financial positioning with the other tech giants.
- Uber, which after billions of venture capital and a decade of operations is now throwing off full-year profits for the first time in 2023, authorized its inaugural share buyback of $7 billion this week.
- Airbnb announced a $6 billion share buyback as well (along with a focus on “reinventing” itself).
- Nvidia announced a $25 billion buyback in August last year, despite its aggressive stock performance which would tend to argue against a buyback program.
Amid layoffs, restructurings, expense reductions and now more share buybacks, it is notable just how quickly Silicon Valley companies have gone from desperately trying to build the future to just holding on to their profit models for as long as possible. I smell incumbency — and disruptive opportunity.
Lux Recommends
- Our scientist-in-residence Sam Arbesman enjoyed Stephen Hsu’s essay on the need for “A Holistic View of the Cell,” which I also found quite compelling. “By equating living cells and computers, one has mistaken the map for the territory. A cell cannot be fully understood by studying all of its components in isolation, as a steam turbine or other machines can be. Cells are stuffed with billions of interacting molecules, the behavior of which changes from one environment to the next. And yet, biologists have long devised methods to study cellular components individually, rather than as continuously changing parts of a whole. The mental model of ‘cells as machines’ has negatively impacted the tools and methods used in biology.”
- Nadia Asparouhova has a great reflection in The New Atlantis on what effective accelerationists (those “e/acc” folks on X/Twitter) are trying to do with their political program. “Effective accelerationists don’t want us all to stop caring. Just the opposite: they wish that everyone cared more. They don’t want to be known for bleating platitudes about “a better future,” then airily waving away the details. Rather, their moral vision is one where more people — including and especially those who consider themselves hands-off today — actively engage with emerging technology and identify concrete plans for its development and stewardship, rather than reflexively backing away from what they don’t understand.”
- Tess Van Stekelenburg brings attention to Jennifer Listgarten’s comment in Nature Biotechnology on "The perpetual motion machine of AI-generated data and the distraction of ChatGPT as a ‘scientist’.” “A major distinguishing factor of the sciences (specifically, biology, chemistry and physics), as compared to AI fields such as natural language processing and computer vision, is the relative lack of publicly available data suitable for these domains… As one example … ‘The replacement cost of the entire PDB [protein data bank] archive is conservatively estimated at ∼US$20 billion’…”
- Jack Watling has an interesting argument for War on the Rocks that "Automation Does Not Lead to Leaner Land Forces.” “Although autonomous systems can often magnify the capabilities of a unit, there are irreducible minimums in warfare as to how many people are needed to perform basic tasks.”
- Finally, two pieces on China. One from Kevin Xu at Interconnected profiling Wang Chuanfu, the founder of BYD which recently surpassed Tesla as the best-selling EV manufacturer in the world. Finally, Chris Buckley has a great profile of China’s nuclear ambitions in “Fear and Ambition Propel Xi’s Nuclear Acceleration.”
That’s it, folks. Have questions, comments, or ideas? This newsletter is sent from my email, so you can just click reply.
