See also:
Georg: Interesting to hear the adoption rate on AI tools. That's not surprising. I've been in gaming for well over 20 years at this point, or related with gaming one way or another. Even right now, my company's consulting for gaming companies in part and I would consider gaming the edge of technology. We're always at the edge. We're always under this pressure. Actually, what is happening with AI right now might be a bit faster than usual but it's the same that we've always done.
Georg: We work at the edge of technology. And that's what makes game developers so useful. That's why I've mostly hired game developers at Facebook and I still do that. Because it's people who are comfortable operating in a very quickly changing technological environment and an environment that is very brittle, which is shared with AI.
Georg: AI at this point is simultaneously revolutionary, and simultaneously really bad. I have a contrarian feel. I do a lot of consulting right now, and I tell people don't adopt it". And if you don't adopt it, it will kill you. And I think that summarizes the challenge that a lot of companies and people are facing when it comes to AI. I use a metaphore here. You're back in the early nineties. You're standing in front of a store. You see this beautiful CRT Monitor, and you want to buy it, and a voice whispers in your ear, Next month there will be an LCD at one tenth of the price", and that is AI. Every month.
Georg: We started in January last year, and people did not have access to on-premise LLMs, at least not any competitive ones. There were a bunch of available options, but Llama really changed that. And then Llama 2. When we started, and to illustrate the challenge, especially for game developers if you're starting a new project, when we started to jump on it, in November, ChatGPT comes out. You understand immediately how to apply it. Everyone understands how to apply it, right? You jump into it.
Georg: And the context window was some 2048 tokens. So a lot of your R&D went into figuring out how to compress as much useful information as possible in the context window, to be able to retrieve in the end, leave some space for retrieval, and so on. And you stumble onto all those issues like: in 9 out of 10 cases, you'll get a useful JSON file back when you ask for it. But in 10%, not. And the compression of the information is obviously lossy. You try to solve all of that. You spend a lot of R&D budget on it. You're really fast. You come to the market, and some 8 months later, when you come into the market, people look at you and go like you did rack what?
Georg: Because in their world an LLM has 200,000 tokens context window. Most of the energy you devoted to overcoming the limitations of the technology were solved in the meantime. And that is an early adopter's dilemma. And in AI at this point in time, it's extreme.
Georg: At any given month, the right answer to adopting the technology is probably Not yet!". At the same time everyone understands that the technology is paradigm shifting. Eventually it will replace probably a lot of things. But the timing is an interesting question. And the open source aspect of it is another question. Whatever you think you do, if you're trying to create a technological mode which worked for the last couple of decades of technology for a while, you find that whatever you do with the technology, you step off the rocket ship, you start building with the technology. By the time you come to the market every one has something better already, because open source already fixed it.
Georg: So I advocate actually to companies to take a breather and not do what they want to do, to take a stop and not go and implement.
Georg: Because in most cases, when you have a deeper conversation, you understand that the reflex to adopt is comforting. We take something that is new and potentially disruptive, and we bring it into our pace of operation, into the comfy environment where we are experts, game developers, whatever we are. And that gives us a feeling of control over the technology. And we can say that we're adopting AI, and we're becoming ready for it. And it seems to check that box. But it doesn't.
Georg: Because when you zoom out, when you truly look at what's happening right now, we are in a period of completely unprecedented technological acceleration.
Georg: Primarily, because we solved all the requirements for that acceleration, years and years before. We built all the protocols. We built app stores. We pulled fiber lines. We built protocols, we deployed 5G. We did all of these things. And so I'm a huge advocate of looking at primitives. And the primitive for most of generative AI technology is a function call.
Georg: And a function call that could be, thanks to open source contributions, from stable diffusion stability AI, from Meta and others, that could be on premise. But it could also be somewhere in the cloud. It doesn't really matter.
Georg: The primitive is: there's a binary artifact, the weights, and it takes a number of parameters, the prompt, temperature, a bunch of other things depending on what kind of AI you're using, and it gives a single result, an inference result. And it's high quality, or not, depending on a whole bunch of factors. And you're throwing that you can invoke that from everywhere. What does that mean. First, it means that any programmer is an AI programmer.
Georg: We see a lot of limitations on the market right now, where people are like I don't know yet who to hire". But it's a typical problem. When I started in game development there were no universities. there was no degrees. There was no nothing. You had to show up. I made multiple [missed]. And that's how I ended up at Bioware, and everyone else, hired at the same time, was hired like that.
Georg: You experiment. The explorers get those jobs. Eventually, schools come and they create curriculums. And it has always been challenging game developments specifically because it moves so fast. If you project it right now, it doesn't work. right. Whatever you teach right now in AI, at a doubling rate of capabilities that is basically double month over month on almost every dimension, energy, efficiency, quality, whatever it is.
Georg: It's really hard to figure out how you would even structure a multi-year course. Because most of the things you learn will be as relevant as the DOS command line is to your your, let's say, modern Photoshop experience, or something like that, down the road. Because tools get better and all of these things. So, when you look at the primitive, the primitive is a function call, and that's great. Web development, we understand that, we know remote function call APIs, we know everything about where we put it, probably not into the main loop, maybe, and so on. So that makes it easy to understand. It also means that you don't have to hire AI programmers.
Georg: You have to hire someone who can understand the function call. And the other one thing that I want to say about this topic is that most people have it fundamentally wrong when they talk about LLMs and generative AI, from a point of perspective.
Georg: The perspective is, we have built all the technology that we've been used as software engineers for the last... And you know, I'm being inclusive here, I don't mean to limit this to programmers or engineers, game developers are a very broad bunch, including artists. But we are all the same in front of this technology. I'll get to that in a bit. Bu we have built everything by hand. We built the first compiler, and we bootstrapped the IDE, and we we built this incredible depth tooling that we have, but the primary things of everything we've done in computer science was an engineering effort, a building effort. There's a bug, and there's a feature, nothing in between.
Georg: But this technology is not like that. This technology is a science. It was discovered. There are 2,000 papers a week, more by now, in AI being published, and that is true because these papers are discovery papers, and some of them are ground breaking and mundane at the same time. Papers, saying, if you use the words, let's think about this step by step", in a prompt, the quality of the generated code as judged by humans reading it in human eval, increases 50%.
Georg: And we pack that into 15 pages of scientific jargon. And we publish it. And it's a good thing to publish right now, because everything is moving so fast, and you know no one is peer reviewing the papers, anyway. But at the specific point in time on these models, this is true! The same as there's a paper that basically says something about giving ChatGPT a $500 tip, but not $50, or like teasing it so you get better results. Discovery!
Georg: This technology is discovered and not built. But the conversations we are having, the conversations we are even having on a public policy level are about as if we built it. And that is because companies take the technology, they put it out and say, this is a search engine. I don't want to make anyone feel bad about their employer and so on. But we understand that companies have to meet their business model, and I'm in the fortunate situation of being free now. I run my own company, and I don't sell AI, because I don't actually think it's ready, even. But I can explain AI. And when you say it's a search engine, you get the answer But it lies!". It's a terrible search engine. It makes stuff up!
Georg: And it has all these other properties. It has biases. We don't want that in a search engine, and then a lot of air is consumed in the conversation about the application of this case, and it mutates into the definition. People say all AI is flawed because it is hallucinating or because it has biases, but that is the perspective. That is because a company took the technology and told you, I found it on the ground but believe me, it's a search engine".
Georg: Like the technology was discovered in the labs of Meta and Google and so on, years and years ago. But it was left on the floor because the people that were working there were allowed to publish it. If you prevent scientists from publishing their leave, so you couldn't do that. But to make the technology go anywhere where it would be commercialized requires you to go to PMs and explain those PMs how the technology will be incremental to their current KPIs.
Georg: And if it's not, then they are not going to spend any significant amount of money on it. Keep that in mind when you see conversations about technology and about this because, for game development specifically, the discovery is a much more interesting part. We discovered something that can do things we could never do before, like efficient creation of story or imagery, and so on.
Georg: The second problem we have is the framing. And the effect of the technology. I'm going to be brutally blunt right now. From some perspective, the business model of AI from, let's say, some companies, some perspective, some people who invest a lot of money into AI, comes down to human knowledge labor moved to elastic cloud.
Georg: The ability to take a specific job and move it into a elastic environment where you can get spin it up. And we'll take game development. I worked at Bioware where we built phenomenally complicated games that were often larger than a Lord of the Rings novel in terms of writing.
Georg: Translation was a big deal. Translation is a big deal for anyone, but translation for Bioware games. Oh, boy! And that meant that it was really expensive to make major changes like getting rid of a character, adding a character, even changing potentially a character's gender or something like that, because the moment you cross past localization deadlines, you would have to book a slot, maybe at the Madrid lab to get that done again, and it would be costly.
Georg: The entirety of the system was built in a way that you do it as late as possible. And then afterwards, you have a whole bureaucracy whenever you want to ship a patch or something like that, because it's expensive. The impact of on demand translation in the cloud, should the technology be good enough, is massive.
Georg: Because at that point you have real time translation at every single step through your localization pipeline. Your game, your vertical slices, at any point in time could be translated and at current cost. It would cost you a hundredth to a thousandth of what it costs to do traditional translation. The benefits are not just logistical. They are also, of course, monetary and flexible. AI doesn't get sick. AI doesn't take... and so on.
Georg: Then people are usually on 2 sides. Either it's all over, the technology is coming for all of us. Or the technology will never be smart enough.
Georg: Right? I think when we look at art and the deep conversations that are happening around the impact of the technology on art, anyone who's not in art, especially software engineers, need to have a lot of humility and look at the pattern and understand that the same thing will happen to the software development profession. The technology has already done it to translation this year. Measurably. There's scientific studies. There is empirical evidence that translation as a profession has crossed an inflection point.
Georg: AI is good enough, and that means you can spin it up in the cloud on demand, and the profession goes into decline. For game developers, that means a few things. For Indies, it means you're empowered. You now and progressively over time will have increasingly a virtual army of AI at your disposal. That will make you more and more capable of running larger and larger studios yourself. On the other hand, there's market competition. Your competitors no longer pay for localization. They have the flexibility to react much faster and change more often.
Georg: It's somewhat inevitable in capitalism. We have to be honest about that. What happens next. I'm steeply uncomfortable because the conversations are... People's livelihood in our chosen economic system is tied to their value economically. And when an AI can do that cheaper, that is a problem. We saw that when the Vision Transformer was put out, within days of the Vision Transformer hitting the market, websites where you could get a passable accessibility audit for your website. Which took basically vision and code, and passed it through Open AI, probably, ChatGPT in the transcode, and for 60 cents to $1.29, which is certainly under the cost of inference, you can get something that previously you could have gotten from a web shop which might be usually probably outsourced somewhere in Southeast Asia, but still in the realm of $120.
Georg: No one can work for $120, but even more like no one can respond in 60s. You can change accessibility issues, and just send it again instantly, and very progressively work through it. We are in a time where the where it goes from scientific paper to implementation, to direct accessibility as a product, in a number of months or weeks.
Georg: Which is deeply challenging when you're trying to forecast. So I leave you in that regard with a few thoughts. Number one, not understanding what the technology is capable of right now. And I rarely meet people who have really a good appreciation of what the technology is capable of, because what most people do is a cursory touch with the technology. And ChatGPT was kind of OK but not that useful. It is a completely different world than when you are at the cutting edge, especially of game development. When we are looking at the last 3 months around Gaussian splats, around 4D, 3D + animations, which is the frontier now on a lot of these models, and when we are looking at the acceleration curve, we can get a pretty good idea that things are going to get really wild in terms of game development. Translation is the easiest thing to pick, because every one understands it, and it's involved in every major game.
Georg: But every single function at the same time right now is experiencing this exploration. And you know Nvidia has forecasts. I don't know what's the number? 100,000 times more more inference power in GPUs within 9 years.
Georg: When you're looking at Moore's law, which we are very comfortable with on the desktop. There's this line, and it's been looking like the same forever. But take look at the version of Moore's law, Huang's law that governs GPU. It is widely different. And this technology is tied to Huang's law, not packed to what we know from x86 desktop.
Georg: When you look at that and you see 9x in training performance between generation of graphics, cards. Compare that to what you get normally. It's a very different story.
Georg: The last primitive money. Always look at the money. Who makes money in AI right now? Nvidia makes money in AI. 70% profit margins. Even the layer right above Nvidia, the layer of people who buy ground layers hardware. The layer above is renting compute right?
Georg: You take an Nvidia card and you put it in the data center and you try to saturate it and sell it by the hour. Even in that layer it is challenging for people to make money at the current prices, which are artificially deflated to create demand for the higher levels of the stock.
Georg: You cannot make money if the card gets obsoleted within a year. And when your generational difference between an A100 and an H100 is 9 x. That means, you ever leaving any A100 in your data centre if you could get an H100, you are wasting 8 additional data centers.
Georg: So your time window for renting out the compute even becomes limited. The higher you go on the stock the more staggering the losses are.
Georg: But people are also investing. You have to ask ask the question: why are they investing? What is the ultimate end game that is being pursued there? What is the investment case?
Georg: For game developers, if you adopt... We all know, don't put ChatGPT call into the main loop, don't simulate every one of your NPCs. Every problem of the technology at once would hail on you, from prompt injection to misinference. And it would bankrupt you.
Georg: When you think about more mundane cases, when you look at where is the technology more and more stable, like translation, it becomes more obvious of where, in the short term, the frontier for game development is. Now, web developers are interesting because they're generally agile. They generally are relatively low cost operations. And the guys at Phaser, Babylon or Godot, which is kind of mixed at this point, have done a great job building a really decent tooling here. But it' generally lower complexity and different economics. The challenge is, how do you make money with AI. The 40% that make money according to the survey that we just heard about. When your inference cost is massively, artificially subsidized currently.
Georg: I leave you with a thought here which is: what is the ceiling for the inference cost that a company could ask from you once you have laid off your worker, and it's in your pipeline? That is something, as a game developer, you might want to have to consider. And on edge inference where we are getting better and better, of course, thanks to the probably entirely self serving donation of Meta Llama to the world, there is a world where AI will get to a point when we can integrate some of these great capabilities and basically make the client pay. You use your own GPU.
Georg: But cloud-based GPU, when you're thinking about the costs, which are completely unpredictable at this point, because everything is subsidized. And we're in this period of companies increasing value extraction out of their system like fee increases, and so on. We should be fairly well aware that that is something that will happen when a company gets into a lot of trouble. We've seen that with Unity. Keep that in mind when you're talking about AI adoption, sustainable AI adoption.
Georg: My advice right now would be, look at the tooling, look at the things where it can be, make you more efficient in the things you are doing, and if you want, it's beautiful to experiment. The guys at AI Dungeaon did a great job even around GPT 2. In exploring the space and showing a lot of the limitations. But the more commercial you get, the more your economics matter. I would read this survey of 60% people who are not being paid, and of a lot of people being still very happy, if you work on your your passion project as a game developer, then that is entirely true. The majority of the people that you are serving right now are not commercial game developers. And for those people who are exploring, who are pursuing passion projects, the limitations are very, very different. Just don't bankrupt yourself by putting an open ChatGPT API into your application.
Georg: That's basically the majority of what I had to say. We can have a conversation. We can talk about about a bunch of these things if you want.
Noël: Thank you, Georg.
Georg: One thing I wanted to mention. I have to be a little bit self serving. I'm basically doing 2 companies right now. One is in the consulting and education space around AI. The other one is, we build tooling. The whole idea of selling shovels in the gold rush. So we built an AI tool. It's called Omnitool. It's open source. It's free to use. Omnitool.ai. It's meant to allow you to take experiment with AI extremely quickly, all in one interface. We think it's pretty good. We also think it's quite premature, because most people don't really think yet about why you would want to combine multiple AI models. But I would encourage you to have a look and see. It might be quite interesting. We've done some interesting gaming experiments with it, throw Babylon in it, or see if ChatGPT can play King's quest using a Visual Transformer. And you know, spoiler, it cannot. It misidentifies the crocodile as a sword, and tries to pick it up, and it ends all in tears. But these are the fun parts of of AI. It's probably pretty good, for, like things like hackathons, not for production setups. More of a lab that can explode terribly in your face. But someone has to have a lab exploding in their face for the technology to become understandable. So someone has to do that.