Chip demand and the future of the climate with Mythic AI’s Mike Henry

Shahin Farshchi:
Testing one, two, three. It lands, it peaks out pretty much in the middle.

Mike Henry:
Test, test, test. Yeah, that's perfect.

Speaker 3:
We can always boost and subtract, but if we're cutting out, we can't fix that. All right. Who's ready to do a podcast?

Shahin Farshchi:
Let's do it.

Speaker 3:
All right. We've got some sound effects here. All right, we're ready to go. Three, two, one.

Danny Crichton:
Hello and welcome to Securities by Lux Capital, a podcast and newsletter that focuses on science, technology, finance, and the human condition. I'm your host, Danny Crichton. Today's subject is chips, not the good Kind, the edible kind. That's for my other podcast Ruffle Some Feathers, but actually the silicon kind, semiconductors, chips, and specifically chips and climate data centers processing capabilities have expanded tremendously over the last several decades thanks to Moore's Law and the demand from smartphones and all of our devices. But that's having a huge deleterious scale problem with the climate, and I wanted to bring on two special guests today, Shahin Farshchi of Lux Capital, and Mike Henry of Mythic AI to talk about more of that. Thank you so much for joining us.

Shahin Farshchi:
Thanks for having me, Danny.

Mike Henry:
Happy to be here. Thank you.

Danny Crichton:
I want to start out with you, Mike. Over the last couple of years, we've seen huge new fab facilities being constructed, particularly in the United States. I believe TSMC has announced a 20 billion initiative, and recently Intel launched a 20 billion fab outside of Columbus, Ohio. How should we think about these huge investments of capital and is it something valuable?

Mike Henry:
I mean, we need them. It's that simple. I've seen cases where we're talking about systems that are just absolutely essential to the security of the U.S. for example, things that let us sleep well at night, and then you see on the ground the types of hardware they're thinking about putting into them. And these are hardware systems made in Taiwan. At the top, you hear a lot of people saying, "oh, we need to secure our semiconductor supply chain and we need to make sure that we can source things globally". But then when you get onto the ground, you look at the program managers, the engineers, they want to hit deadlines. They're just going to use Nvidia because it's easy. The software's easy, it's available, it works, right? And there really is no good U.S. sourced alternative at the moment in order for systems that are really secure for the country to be able to source parts that aren't going to be at a geopolitical flashpoint. So I think it's absolutely critical.

Shahin Farshchi:
It's a simple supply demand equation. I mean, you look at the increase in semiconductor demand in 2021, I think it was 20% plus increase over the prior year and is forecasted to increase by eight to 10% every year for the next 10 years. That supply absolutely needs to be there. To Mike's point, that supply absolutely needs to be distributed. It can't be concentrated in flashpoints. And so I think this is something that is well overdue.

Danny Crichton:
I want to pivot to the main theme of this episode, which is the impact of chips and climate change. I heard two intersecting lines. One is on the embedded side, you have chips that are actually empowering the electric vehicle revolution, being able to charge faster, more efficiently, being able to do then more flexible locations or being able to accept a variety of different ways to charge, particularly as superchargers from Tesla and elsewhere are not fully deployed all across the country. And then on the data center side, obviously chips are using an incredible amount of energy, but some studies showing as high as one to 3% of energy users coming from that sector. And some countries, obviously those with crypto mining seem to shoot up even higher as you're crunching on those numbers. I'm curious, when you look at the next 10 years of this huge explosion in the number of chips, are chips helping or hurting the climate?

Shahin Farshchi:
They're absolutely an enabler to moving the climate revolution forward. Although they do consume power themselves, it's my expectation that the amount of power savings that comes from semiconductors will dwarf the amount of energy they actually consume. The amount of productivity increases that come from a lot of the software tools that we use today, the fact that you don't need local servers, the fact that you don't have to get into a car to go to meet somebody and effect a transaction and all that is being hosted on the cloud is saving significantly more energy than the amount of energy that's actually being consumed in the cloud to enable these services. So when you look at the numbers, the numbers may seem large, but if you look at the effect, it is a fraction of the effect that it's enabling.

Mike Henry:
Hardware manufacturing does impact the environment, the power and cooling. If you can amortize that over a lot of people, that's net very positive. So I think data centers are actually very good for the cloud. If you actually said, "forget data centers, let's put a box in everybody's house", that would be very negative. If you want to make a big difference with chips and climate, if I could snap my fingers and change something, I would just make cryptocurrency disappear. Right now, it's burning the same amount of power as an industrial nation and within the semi industry. When you think about what is exciting, all of the people in the semi industry, it's AI, it's 5G, high performance computing, it's super computer solving, protein folding and things like that. I actually get very little excitement in the semi industry about cryptocurrency, even though cryptocurrency in semi is so tightly linked.

I think there's two reasons for that. The first is that in the semi industry, people are in the business of making very big bets and making those bets very carefully. So you get a lot of people who are really good at deep analysis and thinking through things and very deep thinkers, and I think they look at crypto and they just say, I don't see it. And then I think the second thing is in the semi industry, it's a lot of the old guard. They've seen it all before. They've seen speculative bubbles, they've seen pump and dumps, sometimes fueled by things like venture capital now fueled by Twitter and Discord. I think the general feeling in the semi industry around crypto is that this is just Dutch tulip bulbs and railroad bonds packaged up in a different way. And so you do have this feeling in the industry of, Hey, everybody's talking about climate and chips. I'm just like make cryptocurrency go away. And you'll have you make a huge dent in the problem here. And then we can focus on the really positive things like supercomputing for solving really tough problems.

Danny Crichton:
You're obviously very ensconced at the semiconductor industry, but it's fascinating because even publicly like Nvidia, it knows its chips are being used for cryptocurrency widely, and a huge part of its market evaluation today is built around the centrality of its chips for blockchain and cryptocurrency purposes. Nonetheless, it's gaming customers. Other high performance computing customers want to prioritize it. So it actually has to create chips that are beneficial to some of these communities that have been harmed by Bitcoin hashing up to it, including the point that the hacking group two, three weeks ago, I want to say was LAPSUS with a dollar sign. I don't actually know how to say that out loud. The one with the 16-year-old hacking all these computers, put Nvidia up for ransom to try to say, "Hey, you've got to make these chips work better for blockchain purposes". And so I think it's interesting that the private conversations are sort of matching the public demeanor of some of these companies.

Mike Henry:
If we're going to be burning power and chips, we should be solving real society problems with things like AI solving things in medicine and autonomous driving and all of these real world problems where you got how many people die a year in a car crash? Let's solve those problems. How many people die from diseases? Let's solve those problems. So that's kind of the feeling internally. And not many people talk about it publicly, but it's there.

Danny Crichton:
Well, talking about efficiency, again, I had a theory that when Amazon Lambda, which is a functions of a service product from Amazon, came out in 2014, I had the theory that as developers started figuring out how much each of their functions cost it was going to charge basically per cycle on the processor, we'd actually have some sort of efficiency revolution. There's a classic quote in computer science that the greatest evil is premature optimization, but that maybe when you actually put a cost to each line of code, maybe it wouldn't be so evil. I even had this idea for startup seven, eight years ago when Lambda was launched, you'd actually have an IDE, your integrated development environment with the actual Lambda cost per line of code built right into the code editor. That way you could get a sense of how much each line of code cost, and you'd be able to improve the ones that were least efficient.

So when I look at the AI world, I'm looking at the model building costs where I've seen numbers as high as 50 to a hundred million dollars per model, and I don't see the cost going down. I don't see this efficiency revolution. Or maybe the models are just getting so complicated that the complication is outrank the efficiency over time. But Mike and Shahin, do you see the chip industry getting more efficient?

Mike Henry:
This kind of almost speaks back to what I was talking about with where we say we have these critical national security systems and developers saying, I'm just going to use Nvidiai. It's easy. You're like, wait a minute. Hey, there's a big problem here and no, I know, but it's easy. Yeah. Developers have this attitude of give me convenience or give me death.

Danny Crichton:
Good slogan for a nation state, yes.

Mike Henry:
I think it's a problem that needs to be solved, but you have to still have this really easy to use environment for developers because at the end of the day, developers are on a schedule, they're on a deadline. That's their number one priority, and there's typically a disconnect between what are the ESG goals of the company at the board level, and what are the developers doing on the ground. So everything you said is very worthwhile, but it's up to the semi industry and it's up to the software industry to build that all on the backend and just hide that as much as possible and give a very easy to use system to a developer so they can work. But just keep in mind, they're all on very tight deadlines, and so things that we think should be first order tend to be second order, and typically they're just focused on getting their job done.

Shahin Farshchi:
Going back to the topic of raw consumption, it may seem wasteful, it may seem inefficient, but if you look historically, it was consumption that may have seen unnecessary. That has led to the creation of many new industries. If you look at the mathematicians, for example, during the Renaissance and shortly after, if you look at the alchemists that were trying to turn silver or copper into gold, everything that they discovered, even though it was not useful at the time, and it may have seemed like a waste of time, actually turned out to be extremely useful for generations that came after. And if you look at, for example, semiconductors in recent past, back in the nineties, a lot of GPU usage was used for gaming. A lot of the development of GPUs was for gaming, which many would say unless you're practicing on a flight, simulator to become a pilot, would've been a complete waste of societal time.

And it turned out to be an enabler for ML and AI to solve all the problems that Mike was talking about earlier. If you look at AI and ML, they did not have access to GPUs so they just cost too much back in the eighties and nineties. But once they became commoditized for gaming, they became useful. There was another renaissance, and here we are today where you can train a gigantic model. Yes, it costs a lot of money, but there's continual advancements on the semiconductor side. There's even faster advancements going on the algorithm side to reduce the compute requirements for training a large neural network. We're getting to the point where all of those efficiencies are coming together. Again, commoditizing what used to be extremely expensive.

Danny Crichton:
What's fascinating to me is the utility people, the people making high quality things, mobility, autonomous vehicles, they're getting more efficient because there's obviously a huge economic and financial incentive to do that. But on the crypto side, they're actually increasingly burning energy to actually produce the tokens in the first place because they don't have those same incentives. But I just think it's interesting how those two are misaligned. But I want to pivot our conversation a little bit though. We're in 2022. There's been huge development in AI chips over the last five, six years. A lot of canonical companies have come out, everything from Graphcore, to Sci-Fi and the risk five world, et cetera. I'm curious, when you look forward to today, what are you seeing in 2022 and where's the world going in the next couple of years?

Mike Henry:
It seems pretty clear that NVIDIA is still dominant. The people who are responsible for the bulk of data center purchases either build internally or source from Nvidia AMD and Intel, and there's a number of reasons for that. The smarter data center, AI hardware companies are going fully vertical. That's kind of the playbook, is if you're not getting broad platform adoption, go vertical, find a market there, just get your niche, get established. Others who aren't doing that are likely going to find that they're just going to run out of money.

Danny Crichton:
My last question here, obviously semiconductor policy has become national policy in many countries everywhere from Europe to the United States, Taiwan to China, and I'm curious, the U.S. Congress is currently debating and has been debating quite long now, a fairly substantial, potentially 30 to 50 billion commitment to the chips industry, which has come under a couple of different names, CHIPS Act and some others. Is that relevant from a startup perspective, specifically from a entrepreneurial kind of Silicon Valley startup perspective? Is this money that's going to go to smaller companies, or do you think that this is going to be absconded and absorbed by the largest companies in the space, Intel, AMD and Nvidia et cetera?

Mike Henry:
The CHIPS Act is great in the sense that it now shows that this is a serious issue and that it was a very bipartisan piece of legislation in a very polarized world, really just underlines that everybody gets it. And seeing pictures of Joe Biden holding up chip wafers, it's really great to see that. And then the other positives, it's going to fund a lot of really important R&D in the university research institution world. All of that is really critical to moving us beyond this Moore's Law arms race that's happening. And I think the U.S. Will continue to excel in semiconductor in that new world because we're really good at that kind of thing of building an ecosystem around this whole set of disparate technologies onto the issues with the CHIPS Act. It's exactly like you said. It's fundamentally, either it's going to DARPA research which is eight to 10 year research, or it's going to large companies that are right now really fighting with each other to divvy up the pie.

When it comes to entrepreneuralship, it does nothing to solve what I consider as the donut hole problem or the valley of death, where you have a lot of funding for DARPA hard problems, seven to 10 years out, and then if you're at production or close to production, then you're just kind of existing in the normal world of getting your product out the door and getting sales and things like that. But it's a very long journey from that DARPA hard research to production. We've gone through it ourselves at Mythic, and thankfully we went through it in a very boom time for semiconductor funding. But if it's a bus time, that is a very serious valley of death and it's not really being addressed by this legislature, it's not. It's never been really successfully addressed. It will result in good technology dying because it just couldn't make it through that valley.

There's a lot of ways that people propose to solving it. I honestly think the government might need to make equity investments. They might need to pick winners and say, we're going to invest in this company and keep it going through this valley because this technology is so critical. Don't wait the whole contracting thing or work through this prime contractor or go through this government grant program. No, they just got to pick winners and freaking invest in them, and that's going to piss off large companies. They're going to be like, "Hey, government, you're investing in a company that's going to disrupt my large business. What the hell?" that happens and they have to say, I don't care. It's important to the country. But right now, the interests of the large companies have a lot of influence, and that kind of thing just doesn't happen.

Shahin Farshchi:
It's the interstate highway system. It is extremely important to us. It is going to lay down the foundation for many, many industries that you and I could not even fathom today. If you look at the amount of government dollars that are going into all the major semiconductor producing countries, they are all very large numbers, and we have to be at the top and we are currently at the bottom. That is unacceptable. It is critical that we get this money into this industry and build a foundation that's going to make us more competitive, more secure in the future.

Danny Crichton:
Well, with that note, Shahin, Mike, thank you so much for joining us.

Shahin Farshchi:
Lovely.

Mike Henry:
It was lot of fun. Thank you.

Danny Crichton:
The effect of semiconductors on the climate is only going to deepen in the years ahead. Demand is skyrocketing and is only going to get more heavy over time. The metaverse is coming, which means that all of us are going to demand real time, high resolution 3D graphics surrounding us, and that could be not just a couple of hundred million people in the West, but billions of people in Asia, Africa, Latin America, and all across the world. On top of that, we're going to see the rise of autonomous vehicles. If we're really going to create safe AVs for the entire planet, we're going to need real time edge compute across dozens of sensor packages in every car that's possibly what, two, three billion cars on the planet, maybe even more. Add in space exploration, add in underwater cables, whatever the case may be. We're going to have huge demands on compute going forward, but at the same time, we're also hitting limits on Moore's Law.

Moore's Law has been slowing down in the last couple of years as scientists and chip engineers struggle to break the physical barriers of silicon. Right now, we're at five nanometers approaching three nanometers with TSMC's node. But the reality is that we can't go too many nanometers smaller before we hit sort of fundamental physical limits, which means that as we move forward into 20, 20 and 2030s, we're going to see skyrocketing demand just as our chips can't get more efficient due to physical limits, which means we really need to start directing our attention both to innovation chip design, as well as policies to ensure the chips are used for the best possible purposes and are not gas guzzling. And what I'm getting at, of course, is crypto. We can't afford to have all of this energy expended in data sensors and all of these AI chips to process Bitcoin.

We can't afford as a planet that's heating up, that's getting ever warmer and across all climates from the equator to the Arctic to increasingly put carbon into the atmosphere for a phantom token. Thankfully, we're seeing Ethereum move towards a proof of stake model, and that's going to be helpful, and it could serve as a paradigm for all other cryptocurrencies going forward. But nonetheless, as venture capitalists, as industrially policy analysts, as policymakers, we all need to be taking a look and as smart take on exactly what data centers are doing and ensuring that they're as efficient as possible and being used for the best possible purposes that create progress, that create prosperity, and ensure that all of us have a quality of life as well as for our children.