5/25/2026 - Why Running Large Language Models Locally Is the Future of Data Privacy

Transcript

1. 0:01[music] [music] [music]
2. 0:22[music] [music] Wait, wait for me. Wait.
3. 0:37Hey, what's happening, good people? Welcome to the AI Learning Lab. It is Monday. It's Memorial Day. Hope you had
4. 0:45a great weekend. had some uh Did you Did you do the the festivities, the the obligatory hot dogs, corn on the cob,
5. 0:53and burgers and baked beans and getting in raging fights with the relatives? I mean, um uh [laughter] celebrating and remembering
6. 1:08[clears throat] uh the uh the the people that keep us safe. I hope you've done that. I hope you spend some good time with your family.
7. 1:17Welcome, welcome, welcome. Um, Mondays are nights where here at the AI learning lab. If you're new to the AI salon,
8. 1:24welcome. Uh, in the chat here, there's a chat. It's buried underneath the AI learning lab logo, which is right there.
9. 1:31You can also pop out this screen as a pop out and you can open other tabs and do other things with having this thing
10. 1:37as a pop out. Down here is the pop out button. Up there is the chat button. do me a favor and just if if you've got
11. 1:45chat open, say hello so I can see it can so I can see that the chat is working. And then I'm going to move this chat actually
12. 1:55on my side over to a different screen so when I share I can see y'all talking. Hello. Hello Rick the AI guy. What's
13. 2:06happening Rick? What's what's going down? Valerie Cox in the house. Yeah, I think it's going to be a light night
14. 2:14tonight. So, we can we can do anything. I'm gonna I'm gonna be talking about a couple of things. I'm going to show you
15. 2:23um how I use this computer to tunnel into my other computer, my AI supercomput. Well, supercomput for me.
16. 2:33It's [laughter] like a at this point it's like a medium quality laptop. You know, two years ago this
17. 2:40would have been a just a baller machine. [laughter] Now it's like, yeah, whatever. If you don't have 512 gigabytes of unified RAM,
18. 2:49you're not doing it right. [laughter] Um, but but yeah, if anyone has any questions or anything you want to you
19. 2:59want to talk about, throw that in the comments there. Um, if you're new to the AI Salon, welcome. Check things out. Um,
20. 3:06there's a little welcome section at the top. Um, the uh this this channel's right at the top there. Um, there's an
21. 3:13event section up top. There's a a little section in the welcome section or a little space in the welcome section
22. 3:19called introduce yourself. If you haven't introduced yourself to the community, go do that. If you are in the
23. 3:25community and haven't checked out introduce yourself for a while, just go say hi to someone. No quantum computer
24. 3:31in the in the freezer. Not yet. Not yet. I I'm gonna give us I'm gonna give it We'll go five years. We'll go five years
25. 3:44because quantum computers right now still require, you know, cooling cooling the uh the cubit to near absolute zero.
26. 3:55So we figure within three years they'll figure out how to how to do quantum cubit superp position without super cold
27. 4:02temperatures and then five years we'll have our first quantum phone or laptop which [laughter] you know which will be
28. 4:12able to you know we will be able to each personally calculate all of the equations in all of the history of humanity in a nancond.
29. 4:21[laughter] So so that's coming. So, we've got that. And we'll still be waiting for the custom GPT store. Brandon is still
30. 4:29bitter about the GPT store not launching. Um, yeah, let's see. I would seem like we could have Cold Fusion before then. Well, we're going to
31. 4:41need cold fusion because cuz so sitting sitting next to your your 2 cubit chip on your laptop is going to be um a a
32. 4:52chip-sized cooling unit and then sitting beside that is going to be a chipsized cold fusion unit which will power your
33. 5:00laptop indefinitely. So, so in 5 years we will have laptops that can solve all of the things that will last forever and
34. 5:09we'll still have to upgrade them once a year. [laughter] My hard drives full. But you have infinite capacity. I know. I still
35. 5:18filled it up. [laughter] Oh my god. Okay, so here's a fun one. Here's a fun one. Head over to um head over to the Irregulars channel and uh
36. 5:32this is a good one, an image challenge. What does the future look like? Just go make some future [ __ ] over in the
37. 5:37irregulars. Go make some images of of what the future looks like. Let's let's see what you got. Um and if you have
38. 5:43access to uh to the new Gemini Omni model, you can be feel free to make videos, too. What the future looks like.
39. 5:51Um that's a good one. Um what else we got going on? It's it's a holiday. Uh last week was Google IO, so there were a
40. 6:02lot of announcements from Google. Um it's pretty clear that probably we're coming up to June. Probably by mid June
41. 6:15um we will probably have Mythos from Claude from Enthropic. We will probably have chat GPT's next model which is GPT 5.6 I think. Um
42. 6:30so we'll have it from those two. Will we have it? They'll probably be something similar from Google [laughter]
43. 6:40like Gemini 3.5 Pro might end up being sort of a super high level thing and then Elon Musk is continuing to push his
44. 6:48things forward. So um so probably within the next within the next 3 weeks um we may have things of a different
45. 7:00nature. We have we may have models of a different nature. That said, I don't think it's going to change our
46. 7:06day-to-day life [laughter] at all. I don't I don't um I think the thing that I'm going to talk about tonight
47. 7:15potentially um has has the potential to do that. Okay. So So let's begin. Let's jump in here. For the past two years
48. 7:30and kind of hot and heavy for the past year, people have been talking about agents and [sighs]
49. 7:45Twitter talked about agents and LinkedIn talked about agents and Frontier model companies talked about agents and
50. 7:52agentic this and agentic that. And most people when they were talking about agents were not talking about agents. They were
51. 8:04talking about automations. Joy parody. Hey Joy. Um they were talking about automations and agentic frameworks were kind of new,
52. 8:16right? And so in the past six nine months, how long's claude code been out? No, it's been out longer than that. It's
53. 8:23been out for year and a half. Claude Code was one that came out, it was aimed primarily at coders. Codeex from OpenAI came out. It was aimed
54. 8:35primarily at coders. Um, and then all of a sudden what the coders realized is that because you accessed those agentic
55. 8:49coding tools via your command line and your command line has access to your file system locally on your computer.
56. 8:57They realized that these agentic systems that were writing code and pulling things down from GitHub and pushing
57. 9:04things back to GitHub, all the stuff that you do when you're coding, but you're pulling it down to your local
58. 9:10computer. They realize that these things were good for day-to-day work that wasn't coding, right? File management and, you know,
59. 9:18doing social media marketing campaigns and whatever it might be, creative workflows. So Claude, Claude was the
60. 9:25first one that kind of explicitly put this out there that that we're going to call Claude code. We're going to give it
61. 9:31a new tab. And that new tab is going to say co-work. And co-work is essentially the same thing, but instead of coding, you're
62. 9:40going to use code to do other things on your computer. And now everyone has copied everyone. and everyone is copying all of their um
63. 9:51all of their features and all of their interface. In fact, the new anti-gravity 2.0 just came out and it looks like all the rest.
64. 10:01So, so and and the interface for the most part just looks like chat GPT. Just looks like Claude. It just looks
65. 10:10like chat GBT, but it's different. So, how's it different? Why is it different? What's different? All right.
66. 10:16So that's that's what we're gonna that's what we're going to look at tonight. So let me show you a couple of things.
67. 10:26Kelly Camp, I've been digging into Claude quite a bit lately and their user interface is not the best. Yeah. And and
68. 10:33Claude also doesn't do images and it doesn't do video. So the new Gemini Gemini 3.5 Flash does any input, any output. So you can
69. 10:43output audio, you can output video, you can output pictures, you can output code, you can write and you can input
70. 10:49anything like that. Claude is is much more focused than that. And I saw an interview with Dario Amade, I don't
71. 10:56know, last week or the week before where he was like, I don't see any real value in pictures. I was like, that's telling. [laughter]
72. 11:05Guess they're not going to be hiring that li liberal arts major anytime soon. Although they did just hire a they just
73. 11:10did just hire a philosopher. So, um, I'm trying to teach new users, users new to AI how to navigate Claude.
74. 11:20It's a mess. So, okay. So, so Kelly, that's I'm glad you used that that technical term. It's a mess. Here's my biggest takeaway from the
75. 11:34little bit that I've learned about these agentic harnesses. They're really complicated and they're really nuanced.
76. 11:46They keep changing the rules. How they do what they do, what what they do keeps changing. How they do what they do keeps changing.
77. 11:55I don't know if there's a universal standard yet. People do seem to be coalescing around this idea of skills,
78. 12:03which Anthropic is the one that invented skills. the skill.md file. [clears throat] Um,
79. 12:22all that said, it seems to me that understanding what these things are and starting to use them is actually really important. I do
80. 12:33think that this is the way that everything is going. Hang on. Oh, yeah. I saw that. Yeah, Microsoft was told to stop using Claude because
81. 12:48their bills were getting too high. [laughter] So, one of the things you'll notice if you use Claude is that the um
82. 13:00that that it rate limits a lot that it's expensive and and if you don't pay a lot of money, you don't get a lot of use of
83. 13:06it. Um I'm noticing that with with OpenAI all all of them. The Okay, so so a couple of things. So, conceptually, where are we going
84. 13:17from? Where are we going to? We're going from an application. So, chat GPT is an application that runs in a browser and
85. 13:32you are the driving activity generator, right? You log into Chat GPT. You give ChatGpt a prompt. That prompt goes up to the cloud with
86. 13:44your context, whatever you've been talking about before. Somewhere in a bunch of GPUs up in the cloud, it comes up with an answer and it
87. 13:52streams it back to you and types out your answer. Right? That's how all these things have worked. what these agentic systems are. The
88. 14:05harness is the application. So it's a it's similar but instead of a web application, it's a downloadable application
89. 14:14that you can run as a desktop application or you can run in your command line. So on the Mac it's called the terminal. On
90. 14:25PC it's called what? PowerShell. Is that what it's called? The terminal. But it's the terminal window. So, if you're if
91. 14:31you're a Yeah. Okay. If you're a if you're a geek and a nerd, you know what the terminal is and you're
92. 14:39you're you're totally content typing cd minus-r space dot dot colon semicolon curly bracket mdir
93. 14:54mkdir poop. Right. >> [laughter] >> like a local LLM. Not quite. I'm going to get there. I'm going to get there.
94. 15:09I'm going to get to the local LLM, but not quite. Sort of. Some are some are, but not like a local LLM. So, the local
95. 15:20LLM is actually just like Chat GPT. This this I tonight tonight tonight tonight is going to be a a meander fest largely because
96. 15:33my level of knowledge at this point around these agentic systems is just this side of understanding it
97. 15:42conceptually. I get it conceptually. I don't quite get where all the piece parts are. I don't quite get how to use
98. 15:50it. Nostalgia on doss. Exactly. Meanwhile, Gemini Deep Mind are creating multimodal world models. Picture is worth a billion words. Okay.
99. 16:04So, Chat GPT and Midjourney and Sunno, all of those are web apps that are calling some larger model sitting up in the cloud.
100. 16:14They're storing all of your data in the cloud. I heard a horrific um story this morning on Tik Tok on one of the one of
101. 16:24the better podcasts that um an AI startup that was using Claude for everything, someone in their organization or some
102. 16:38agent in their organization did something that violated Anthropic's terms of of service, terms of use and
103. 16:48they shut down their entire account. So they didn't have access to their applications that they had written to
104. 16:56their context to their prompt libraries all of the [ __ ] that this company was was running in clawed and probably claude code.
105. 17:08Anthropic just turned it off like that. Boom. No warning. No human review. I don't know squat. [laughter]
106. 17:20I know. Welcome. Welcome to the show, [laughter] [laughter] giant concepts.
107. 17:36So, we're now living in we're now living in this risky time where the government has taken notice of the frontier model companies, right? Um
108. 17:48Trump just at the last minute, I think David Sachs got in Trump's ear right before he signed some executive order
109. 17:57about the government having to review Frontier Model Companies before they're live. I'm not quite sure what the
110. 18:02executive order was, but anyway, he didn't sign it at the last minute. I think it was David Saxs, one of the guys
111. 18:08from the All-In podcast who who heads up AI for the president, AI adviser for the president, told him not to sign it. Um,
112. 18:19these big frontier model companies can just change the rules. like Chat GPT 40. A lot of people are calling that Chat
113. 18:27GPT's masterpiece and everything, all the 5 series sucks and everyone's lamenting 40. Um, chat GP OpenAI just
114. 18:36said screw it, we're going to retire that. You don't get to play with that anymore. So, so, so we're absolutely in in in a new era where
115. 18:48these scrappy these scrappy little AI companies are now starting to act not scrappy. They're starting to act corporate,
116. 18:59right? And they're starting to respond to their corporate overlords. They've got to exist in a power structure
117. 19:07and they're starting to act like that. And what what that's doing for us is is it's slowing our access to the tools. It
118. 19:16is doing things like if you're running a startup solely on one of these platforms and they go away
119. 19:26or they just shut your [ __ ] down because they have some algorithm that says you did bad, delete it until further notice, right? >> [sighs]
120. 19:41>> So a thing we need to start exploring is local large language models because they're going to get better and better
121. 19:47and better. The hardware is going to get better and better and better. So the large language models are going to get
122. 19:52smaller and more efficient. The hardware is going to get better. The combination of that is that pretty much starting
123. 19:58this year and just improving from here forward, you can run models on a local computer, not up in the cloud,
124. 20:08that can do powerful stuff, can do powerful AI stuff. He announced a partnership with Anthropic. I know that. So, Pope Leo put
125. 20:21out a whole kind of manifesto thing that was pretty much, you know, don't forget the humans. I didn't realize he cut a he
126. 20:28cut a deal with anthropic. So, was that that wasn't a joke, right? That was an actual thing, Brandon. Okay, cool. [laughter] So, so
127. 20:41Anthropic is now Pope approved. [laughter] So, so if you're if you're uh if you're uh if you're swinging the rosary um next next Sunday,
128. 20:57Anthropics, your your frontier model company. [laughter] Okay. All right. [clears throat] I should actually make a slide on this,
129. 21:13shouldn't I? Anthropic was his consultant pope approved.
130. 21:27Okay. We had web- based apps that use cloud-based models and cloud-based storage. Then you have open- source models that you can run locally
131. 21:44and you can do local storage, right? Or you can do cloud-based, you know, private instances where it's in the
132. 21:52cloud but it's all your stuff. Um, and now and now we've got these things called harnesses.
133. 22:08[sighs] Trying to think what to do next. Go look at something. I'm going to go look at my local machine here.
134. 22:44That's cool. Here, I'll show I'll show you something fun.
135. 22:58All
136. 23:14right. Okay. So, what I'm going to show you here, this is what you're looking at here is
137. 23:29codeex. So, if you see my upper leftand corner, um, it says codec. So, this is this is a Mac application. This isn't a browser.
138. 23:39This is a Mac application. It's running locally on this computer. and this project. Um, I just saw an X thing about this. Do a full readonly
139. 23:55analysis on my MacBook to help me optimize storage. So, this is my main daily driver MacBook. This is my M1. Um,
140. 24:04and I had it do that and it found um lots of storage. So, I've got 891 gigabytes with 8.2 gigabytes free. So,
141. 24:14so I'm I'm very full. I I am beyond 90% full, which is really bad for a hard drive. Um, so I've got 352 gigabytes of
142. 24:25photos and pictures from 2016 and 17 alone. I've got 89 gigabytes of applications. Adobe is the biggest. So
143. 24:33Adobe always does this. They give you a new version of the app, but they don't delete the old one. So I got a bunch of
144. 24:39that. I got downloads. 31 gigabytes of downloads. 10 gigabytes on the desktop. 61 gigabytes of user app data. 9.6
145. 24:50gigabytes from Claude. Huh, look at there. iRacing. How do I have iRacing? Because iRacing is not a Mac app, is it? But anyway, I
146. 25:00can get rid of that. So anyway, so so this can do things like like chatbt can like it can run things except it's it's
147. 25:09also got access to my um to my desktop. All right. And then it's got plugins and so you can connect it to all sorts of other applications.
148. 25:24And then you can tell it like once you have it connected to Gmail, you can just say, "Hey, go look at my Gmail.
149. 25:31um find all the instances where I emailed someone from a pharma company and make me a spreadsheet of you know
150. 25:40those people their emails and summarized the last things we talked about and and when we talked about it and so it can
151. 25:47just go off and do that. Now, this isn't dissimilar if you've used GenSpark or you you've used Manis. Those are cloud-based
152. 25:57agentic systems that have tool use. So, the thing that makes a system agentic is that it's got the the large language model component
153. 26:07and then it's got tools and then it's got files and memories and storage, right? So, it's got it's got these these
154. 26:14three things that work together where these systems seem to be migrating and and the OpenAI one is is the most
155. 26:24here is that they're storing your memory files locally on your computer and so the files that you create get stored
156. 26:33here locally. Claude code and Claude co-work stores them up in their cloud. So if if something happens like happened
157. 26:40to that company where Claude just cuts off access, they've lost access to everything. Um, okay. So, so that's kind of what this is. You can
158. 26:52do automations with this. Um, you can set it up so that you can access codecs from your mobile phone. So this is very
159. 27:00similar to um to claw to uh openclaw where you could I was talking to Adam in telegram and then it would he would run
160. 27:10on my local computer. We've lost access to your screen. Thank you.
161. 27:25Um so you can set this up so so you can check it on your phone. So, Codex is is is really impressive. The other thing
162. 27:32about Codeex is Codeex, this application is is open- sourced. So, even though it's defaulted to use the OpenAI 5.5 GPT model up in the
163. 27:44cloud, this is running locally and all of its file structure stuff is happening locally on this computer. Okay, why is that important? Well,
164. 27:57do you know how do you know how sometimes you go over to Claude and you're like, "Oh, but I don't have any of my chat GPT history over
165. 28:06here." And then if you were diligent and and and and had a good work ethic, unlike myself, you could export all of your
166. 28:16data out of chat GPT and go over to Claude and import all your data over in in Claude and then it would it could
167. 28:22sort of pick up where chat GBT left off, right? You could do that. You may or may not have, but you could do that. The
168. 28:30thing about having all of this memory stuff locally on your computer is that you own it. So all of the things that it's creating,
169. 28:39all of the applications that it create that it creates, all of the the skills files that it generates or that you
170. 28:46install are all all installed locally. And assuming that skills become um a universal standard across these
171. 28:56different kinds of platforms, what it means is you'll be able to move from platform to platform and just have your
172. 29:02your single source of truth for sort of who you are and how you talk and how you write and the things you care about. All
173. 29:10that stuff that we've essentially given to Open AI, we can start to reclaim that, right? And so like one of the things that I've been
174. 29:20doing is I go into chat GPT and I say hey based on what you know about me come up with a skill for how to talk like I
175. 29:28talk and and you know a skill for for codeex or claude code to understand the kinds of projects that I have and what
176. 29:36I'm working on things like that. All right so let me see that. So codeex will remember your chats throughout other agent harnesses. Kyle,
177. 29:48what are we doing tonight? We're talking tonight. I connected Claude to Canva today. It worked so great. Yeah, these things are
178. 29:56power. The fact that these things can use tools is quite powerful. So, we're talking about agentic platforms, what
179. 30:02they're now calling harnesses, which are these locally running applications, um, that call other models, but they can
180. 30:12also call local models and they can also do they do all sorts of things, but depending on how you have them set up, Mary, yes,
181. 30:23not not all of them are transferable to everyone else without some amount of work. Right now, there's no universal
182. 30:32standard that says here's here's how you should maintain and manage your context locally. Everyone's sort of having to
183. 30:41figure it out on their own. This is this is one of the reasons I wanted to talk about this tonight. Even though
184. 30:48this probably isn't going to be super clear tonight cuz I don't quite have this understood yet. But what I do know
185. 30:55is that it is clear to me that owning your own context, owning your own data is really important and going to be more
186. 31:06and more and more important as these big frontier model companies get corporate and government compromised.
187. 31:13And and I'm not [ __ ] on companies for getting corporate and government compromised, right? They they're
188. 31:21corporations that have had billions of dollars invested in them. And at some point those investors are going to want
189. 31:28either power back or money back. And so they've got other people to to serve. Um but what that means is it's it's on us
190. 31:38to figure our own stuff out. Um let's see. Browser Cloud lives in the tab. Desktop Claude lives in your OS and talks
191. 31:48directly to your files and apps. Yeah, but but it doesn't always save everything. Claude doesn't always save
192. 31:55everything locally. Claude's saving some stuff to the cloud, some stuff locally. Codeex is overindexed to save stuff locally. As
193. 32:04far as I understand it, uh uh uh uh uh uh uh uh context window a million tokens on your computer. It's
194. 32:12the whole computer. Well, that's that's a bit of a that's a bit of a stretch because the context window it still has
195. 32:20to be able to process stuff, but it's got access to all of your computer storage. That's true. All right. Um Okay, so it's 8 o'clock.
196. 32:29So, let me just talk about Let me stop sharing this for a second. I might not even I'm not even sharing anymore. Okay.
197. 32:34Um so, [clears throat] it's top of the hour. So, um, this week in the AI salon, tomorrow, um, is part four of Andy Scarantino's,
198. 32:47um, decoupling your identity from your job. It's part four. Last week was about the goop. Um, if you haven't been in
199. 32:56that, you should join. Um, if you're not a part of the mastermind, all of the replays of that series are in there. Um,
200. 33:05Thursday we're doing, uh, an AI salon mastermind practice lab. Um, Wednesday, um, producer Brandon is doing inside the
201. 33:17mastermind. Um, the third of four of his lovable is it's the third one, right Brandon? Is that correct? Number three.
202. 33:25Yeah, it's the third of four. Um, lovable, how to vibe code an app. Um, so all sorts of stuff going on in the salon
203. 33:33this week. Okay. Um, so get in there. Get in there and and uh and uh if you haven't signed up for the mastermind, do
204. 33:43that. And if you're in the mastermind but haven't been coming to these things, if you've missed a few, that's okay.
205. 33:49Recordings are in there. Jump in. Learn how to do stuff in Lovable. Um, if you're thinking about what do I do now that my job is
206. 33:59transforming radically, um, the, uh, the identity workshop's a really powerful one. So, there's all sorts of cool stuff in there. Um,
207. 34:09I had I had a hard time with desktop Claude retaining my conversation I had earlier with the app Claude.
208. 34:17Frustrating. Yeah. So, here's the thing, and this is the thing, Lori, where I am not I'm not in a place, and I don't think
209. 34:24I'm going to be in a place where I can actually tell you how to do that for all these different platforms, right? That
210. 34:30used to be the thing I would do is I would learn, you know, the personality of Perplexity and Claude and Chat GPT and this and
211. 34:38that. I could because they were relatively the same. These agentic harnesses have big philosophical differences. Some are more geared toward
212. 34:49coders, some are more geared toward consumers. You've got anti-gravity, which is very much a coding tool, and
213. 34:56now they've got this new thing, um, Gemini Spark, which is a consumer version of anti-gravity. It's sort of a
214. 35:03interface layer. They all have different ways of how they're handling memory. How that memory is evolving is all over the place. So,
215. 35:19I I'll talk about where I'm coming down on these things toward the end, but I I still want to just bumble through the
216. 35:28what I know so far, and I just want to show you show you some of these things. Okay, I'll figure it out tomorrow. Just
217. 35:33noticed it about 20 minutes ago. On computer use, Claude can drive any app on your machine. Whatever you click
218. 35:40on and type, it can too. Yep. And codeexes like that. And perplexity computer is like that and anti-gravity
219. 35:47is like that. So all of again all of these things are copying all of these things. By the way, the open- source versions of these are
220. 35:58OpenClaw with Peter Steinberger's, you know, uh the thing I built atom on. He's now working for OpenAI. So claw uh codeex
221. 36:08their version of open claw is getting the they're coming into parody like what they do is philosophically the same thing. Um
222. 36:21Hermes is another it's like openclaw but you can run that locally. So so what have we got? We've got Hermes openclaw.
223. 36:29Those are the two open source ones. You've got codeex from open aai. You have Claude Code and Claude Co-work.
224. 36:35Same app, different tabs from Anthropic. You've got anti-gravity from Google. You've now got Google or Gemini Spark,
225. 36:46which is like friendly anti-gravity or co-work, right? It's the equivalent of co-work. Um, you've now got grock build.
226. 37:06So, that's at least seven of these things. Then you've got Manis, which is owned by Meta. You've got Gen Spark. And then there's probably
227. 37:17seven or eight other agentic systems that are like Manis and GenSpark that I just don't talk about. So there's
228. 37:24there's nine there's nine of these things. Um [clears throat] they're all kind of the same but kind of different.
229. 37:36But all of them require a couple of things. They require you to understand skills. So, so you have to give them context.
230. 37:44So, they've got these agents that can run 24/7 and can go do things on your behalf. So, you can give um like in codeex, they
231. 37:53just added a thing called slashgoal. And in fact, let me let me see if I can go add a slash goal and we'll see what it does. Yeah, I am.
232. 38:12All right. So, I'm in codeex. I'm going to do new chat. If I say slashgoal set a goal. All right. Slashgoal. Um, I want you to make me
233. 38:30Wait, how do I do this? Oh, I go slashgoal and I No. Oh, wait. Cool. Oh, yeah. Default permissions. Okay. Let me go slash goal.
234. 38:47And then now I've got it down here. It says it says goal. Okay. And I'm going to say um I want you to start me ailate.
235. 39:03I don't I don't know if I want to do this. No, wait. We we'll set the goal. Uh, okay. The goal is The goal is make me $2,500
236. 39:18per month with no cash outlay. >> You can make money with your [laughter] >> Okay.
237. 39:33So, so I'm going to give codeex a goal here. Failed to set goal. Why did it fail to set the goal? Oh, it didn't. Okay, I'll ground this
238. 39:45option so that we can work without startup money and then turn it into Okay, so it's now off working. Lost your
239. 39:52screen. Damn it. Okay, hang on. You know, I'm going to vibe code something that can share my screen without crashing every 15 seconds.
240. 40:10Yeah. Yeah. Exactly. Okay. I can't guarantee 2500 a month. Help local businesses turn missing leads into booked appointments. Five clients.
241. 40:23500. Oh, wait. Yeah. I should I should tell say I'm gonna say no. I don't want to do any work. I want you to do everything.
242. 40:38I want to be in economy one sitting on a bench while you make me money. Okay. [sighs]
243. 40:56So, the way these things work that's different than chatbt. You can tell chat GPT to go do all this stuff, but then
244. 41:02you have to sit there and do all the back and forth. This thing once you get it going, it will spin up agents and sub
245. 41:10agents to go do stuff. One might go do research, one might go write you code and create an application for you, one
246. 41:16might do all all this other stuff. So these systems are autonomously running things that you give access to certain
247. 41:25tools to that can make it very powerful. So it can use your computer just like you use your computer. Right now these tools are a mess and
248. 41:35they can get very very expensive um because they're running independently and the way they work they take these
249. 41:44memory files these context files these skill files and whenever they send a prompt up to chat GPT they're sending
250. 41:52like these massive amounts of context. It's not just a little prompt. It's like as it's doing research and gathering
251. 42:00things, it's building this everinccreasing blob of context that it keeps sending back to your your cloud provider, your you know,
252. 42:09whether it's OpenAI or Enthropic or or Gemini, it's burning all of your tokens. And so at some point, like I was working
253. 42:16on having Codeex help me do a uh like a speaker strategy. I want to do more paid speaking gigs. And about I don't know 2 and 1/2 or 3 hours
254. 42:29into working with it, it was like, "Oh, you're done. Sorry." You know, you can't do anything until 7 o'clock tonight. So,
255. 42:38um, so you need to be careful with them. Um, the other thing you need to do is like the goal I just gave it is kind of
256. 42:48a shitty goal. Let's see what's what's it doing here. Um, add to chat. Hi, name. I notice your business. Okay,
257. 42:54let's see. I get it. The fantasy bench mode economy one seat money arrives. Hard truth. No cash outlay and no work
258. 43:03from you. There is no legitimate reliable way to make 2500 bucks a month. [laughter] Okay. Also, no screen sharing. God damn it. Okay. Um,
259. 43:30I guess I'll leave that thing there. [snorts] So, here's what I'm going to here's what I'm going to do. Oh, here's what I'm
260. 43:36also going to do. So, so anyway, so this thing's off doing its thing. Um, if I go back, let's see. Create
261. 43:47a brainstorming app. That was cancelled. Okay. So, I'm going to show you something cool here. Um, this is codeex that's running here
262. 44:01locally on on this computer. But I also have this thing. There's a thing called Tail Scale, which allows me to connect my my other
263. 44:12devices. So, here's my MacBook Pro M5, this little second one here. And then through that, I can go over to Finder
264. 44:22and I can go connect to server and I actually connect to this is my MacBook Pro. So, I'm now basically going through this
265. 44:38computer and logging into that computer. So, I'm going to go full screen on this. So, so basically, so basically what you're witnessing
266. 44:47right now is me logging into a different computer. Um, so if you didn't know you can do this, you can do this. It's not
267. 44:54that bad to set up. It was pretty straightforward and chat GPT or Claude taught me how to do it. And so, a couple of things. One is, um,
268. 45:05I couldn't get this working on my M1, but see this little dude? This is called a PET. And so I'm in codeex and the pet
269. 45:13is it can be there. There's all sorts of different ones you can set up. So if I go to here, I'll go show you. If I go to
270. 45:20settings [clears throat] and then you go to appearance in settings and I scroll to the bottom, there's this pets section [laughter]
271. 45:33and and they've got all these all these different pets like, you know, the codeex pet and Dewey and Fireball and
272. 45:40all these sort of things. And then I created this one called Kyle. And basically all it does is while the
273. 45:46computer's thinking, it's it's taking messages and it'll it'll talk and jump up and down and if it's thinking, it'll
274. 45:53do like thinking poses. It it's really just a dumb thing. But like, how did I get this dude that looked like me and
275. 45:59has all this animation? I actually had Codeex create it for me. So, I uploaded a picture of myself and I said, "Turn
276. 46:07this into a sprite sheet for a pet thing." And I couldn't get it to work on the M1, but I did get it to work here.
277. 46:14So, this little dude's just just running around. Yeah, this is [laughter] this is sort of like Clippy. This is This is In fact, Lori,
278. 46:23[laughter] this is identical to Clippy, but now it's me. And what's funny is it floats. If I go to to other applications, it
279. 46:31actually floats on top of everything. So, it it's kind of annoying. You got you got to kind of keep moving it to a
280. 46:37place in the interface where it's not working on stuff. Okay. Um, Cosmic Rocks Revenge. Let's see. Design a local AI box.
281. 46:54Build a native asteroids clone.
282. 47:18So, here's a crappy here's a crappy game I made, but it's a native it's a native OSX game. [snorts] Um, but anyway, all of all of my um
283. 47:30all of the things that I've made are inside inside the codeex folder. So, this is what I was saying before, like
284. 47:41context. Here's Kyle and here's all these like here's an identity file. Can I make that bigger for you? No. Kyle Shannon's a storyteller
285. 47:55entrepreneur, right? So, it generated all of these like here's a style. Here's the creative style. Visual and creative
286. 48:03style. Kyle prefers original, emotionally resonant, weird. Am I not sharing again? Screen share not there. This is annoying.
287. 48:12This is annoying. >> [sighs]
288. 48:32>> So, what was I showing you? I showed you this. Um, so these little folders here are my local. So, any apps that I create, these
289. 48:42look like they're memory things. Um, here are exports. Here's a fractal pong. I guess I made a pong game in metal. Um,
290. 48:55so I'm just getting started here. So, but but again, this idea that I now have my own data. And the cool thing about
291. 49:03these tools is if this starts to get confusing, I can say, "Hey, make me an interface that makes this a lot easier
292. 49:09to find stuff." In fact, one of the things that people are talking about on on uh X is that you can use Obsidian as
293. 49:18a notetaking interface for for instead of me having to dig through all that stuff like here's um you know all of my home files and
294. 49:28things like that, like here's an AI learning lab page where it's got you know what the AI learning lab is and and
295. 49:35things like that. So these all these little nodes are put together to um help me find all my information. So that's obsidian
296. 49:44>> screen share again. Kyle, it blipped. >> Okay.
297. 50:00So that's that. So let me show you. So this is let me go to a new chat. So this is what codeex looks like. Here is perplexity,
298. 50:15right? It's got all your chats over here on the left, a chat bar on the right. When it generates something, it makes a
299. 50:20window like a a panel over here on the right. Here is Claude. Now Claude has this how much of you been using us? This sort of gamified
300. 50:33interface, but it's still the same thing. You've got the the chat bar at the bottom. You've got access to tools
301. 50:39and connectors, manage connectors, right? You can connect, it's connected to Google Drive. You can you can browse plugins,
302. 50:49all sorts of things like that. So, all of these things are kind of the same. And then I think there's one more I've
303. 50:56got in here. I've got anti-gravity in here. Here's anti-gravity. This is This is in dark mode, but creating an
304. 51:01asteroids clone. Here's the asteroids clone. Is this running? No. Let's see. Let me go run this
305. 51:17so I can play. My little dude's very happy. Allow running. Yes, this time.
306. 51:34So yes, I want to give it permission. It's not sharing again. No.
307. 51:55One task running. Okay. So, here's my This is a game that I made with Gemini with with uh anti-gravity.
308. 52:27Uh so again these things are pretty good. Um you know they can make applications. Um
309. 52:44they're not very much fun to look at. They take a long time. They're they're they're actually kind of hard to demo because
310. 52:53they're hard to demo because they're off working, right? Like like one of the new benchmarks of the frontier model companies is how many
311. 53:04sustained hours can these things work without needing you to come in and fix them? Creating a portfolio of all of our AI
312. 53:13projects will be so much easier now. Remember back in the day? Well, so this is this is where I'm going. Okay. So So now let me let me
313. 53:22if I've done anything here tonight, I hope that I've confused you. The the most important thing to know
314. 53:38is that how we use AI is about to shift. If you want to follow along, learn about these agentic locally running
315. 53:55harnesses. We'll use their language because they're going to keep calling them that. They're they're apps.
316. 54:02But start to understand things like skills, start to understand things like connectors. Um, the good news about all these
317. 54:10complicated things like skills and connectors and this and that is that everything is in a large language. Like
318. 54:16the instructions for how to do all this stuff is in the tool itself, right? [laughter] Confuse me. The bar is low.
319. 54:28Spark is a remote agent. Anti-gravity is Okay, cool. Is Is there version of codeex for anything? Yeah. Yeah. Yeah, I
320. 54:38use Perplexity 100% for looking for new info automatically daily on different projects I'm working on. It's so handy.
321. 54:46I love it. Yeah. And listen, [clears throat] there's people that are using Gen Sparks agentic stuff like this. There's people that are using
322. 54:54Manis for this. There there's lots of different ways to do this. The thing that I am increasingly obsessed about is
323. 55:02how do I run as much as I can locally so that all of this stuff that I'm creating and all of the all of the ways that I'm training these
324. 55:15tools to work for me are not owned by OpenAI or not owned by Anthropic are not owned by Google. I want them because
325. 55:27that local data, that local context, that local knowledge of your preferences, for lack of a better term, should be yours.
326. 55:42And quite frankly, when you go work for another company, you know, one thing's going to become clear. Companies are starting to now
327. 55:56require people to understand how to use AI. So, as part of the job, you're going to have to understand how to use AI. Well,
328. 56:08where I think this thing goes is we should be able to retain our own IP so that I can come in and I can say here's
329. 56:17my bucket of skills and I'm going to work for your company, but but I'm going to get a a running start because this is
330. 56:23how I work. I'm going to learn how your company works. So, I'm going to add that in there, but then when I leave, I want
331. 56:29to take my stuff with me and I want to be able to transfer that from job to job to job. Perplexity news is great 100%.
332. 56:42Are you guys using the Perplexity app or Perplexity the the website? Local data is gold. Local local data is literally gold.
333. 56:54It's literally gold, right? If you actually understand what's in your local data and it aligns with who you are and
334. 57:02what you're passionate about and the things you're really good at and the things you can talk to, Lori Blair uses both web and app.
335. 57:20So one thing is start to understand what these things are. Here's the other piece of this. I'm kind of now feeling like and and
336. 57:29this I don't like this feeling are our in the in the AI salon practice lab this this week has been play with purpose
337. 57:37that we're supposed to go play across domains and this is the way these things work is very outside of my domain. I'm
338. 57:43not I'm not a systems thinker. I I appreciate systems but I don't think in them. I think in abstractions.
339. 57:51And so these things are hard for me because I have to I have to translate my abstract understanding of them into very
340. 57:57tactical. Wait, what folder structure is it in? Where like where did you put that? Where do I want you to put that?
341. 58:04What goes in that? How do I pull the [ __ ] from here and put it into there? And how do I do that in a way that I can
342. 58:10find it again? Like that makes my my brain hurt. [clears throat] But but it's a really important skill to have.
343. 58:25Now, if you're like, "Kyle, this is I I can never." First of all, you can we we all can. You may not want to. Um
344. 58:37these things will get easier and more transparent. These things are going to become like operating systems. People
345. 58:46are going to start selling boxes that have local models on them and you'll buy the box and it will say, "Do you want to
346. 58:58import your [ __ ] from some other system or do you want to answer a bunch of questionnaires and just upload documents
347. 59:04and it'll set you all up and then you'll just be able to go like the these things are going to get way easy to use.
348. 59:10They're just not." Did you see Kelly's question? Confession. I still don't understand the difference between the web version and the app
349. 59:19version of these tools. Can someone please explain? Yes. Okay. For you to use chat GPT, you have to have two things
350. 59:38active for it to work. You have to have an internet connection to be able to get to chat GPT. And then you need to to have an active
351. 59:51Chat GPT subscription that allows you to process tokens on their servers. Okay? So you have to pay your internet
352. 1:00:02bill and you have to pay your Chat GPT bill, [laughter] right? with one of these local harnesses with codeex, I can run codeex
353. 1:00:13um without an internet connection. Now, the way it defaults is I still have to have the internet connection to be able to call the to
354. 1:00:23call the thing. Um I can I can run the app, right? And I can type in a prompt, but when I hit submit, it defaults to
355. 1:00:30using chat GPT 5.5, which is in the cloud. But I can. So, let me show you another another thing I've got running
356. 1:00:38on this on this my fancy machine. Okay. So, over here on this machine,
357. 1:00:56I've got this little um this purple icon called LM Studio. And so if I click on that, so here's a bunch of models that I
358. 1:01:04can download. So LM Studio is actually just like Wait, why did I not go to LM Studio? Oh. Oh, I see. Wait, where's my
359. 1:01:20Okay, so if I go here, No, if I go here. Yeah, and then I go new chat. Okay, and then I get rid of that. Okay,
360. 1:01:30so this is this is LM Studio and it says new unnamed chat, right? And then down here are the local models that I have
361. 1:01:40installed on this computer. So these are these are really teenytiny versions of what GPT 5.5 is, right? GPT pi 5.5 is
362. 1:01:56I don't know 500 million parameters or five 500 billion parameters or a trillion parameters or like the new the new one from
363. 1:02:07anthropic the mythos one is 10 trillion parameters um the one I'm running is is 35 billion right so so it's teenytiny
364. 1:02:15compared to those big things but it it effectively does the same thing right it's not as good But it's a large
365. 1:02:23language model and it's running locally on this machine. And so in here I can go um write me um a uh an M&M
366. 1:02:37style cipher about quantum mechanics. Yeah, kind of like MP3. And and then yo,
367. 1:02:53check the mic. One, two. Is this on? Look, they say it's really illusion, but the math don't lie. I wake up in super
368. 1:02:59position, both asleep and awake. Schroinger's cat in a box for goodness sake. Is it dead? Is it alive? Who knows
369. 1:03:04what state it is? Um, until you look in the box, the wave function hates it. So, you know, it at least understands what
370. 1:03:13quantum mechanics are and it understands what an Eminem rap is. And I'm not sharing my screen because Thank you,
371. 1:03:20Apple. No, that's not what I wanted. I wanted this one. I wanted this one. I wanted this one. I wanted this one. [clears throat]
372. 1:03:38All right. So, here's here's the the wrap thing that I wrote. So, Kelly, this this chat GPT like experience, I
373. 1:03:47don't need an internet connection for it. This is just running locally on that computer. Now, the other thing that this
374. 1:03:54has is this little page where it says up here at the top, there's a bunch of crap here you don't need to pay attention to,
375. 1:04:01but notice up here it says loaded models. What this is is this is my my server that this address right here. This this URL.
376. 1:04:14Am I still sharing? I'm not. God damn that thing. So frustrating.
377. 1:04:30Um, so up at the top here, this little HTTP thing, this is a server where these two models are running. Okay, so what I
378. 1:04:40could do, and so here's my Quinn 3.6 model. I should probably upgrade that to 7. 3.7 just came out. I can go into codeex. So here's codeex.
379. 1:04:51And if I go into my settings, rather than running the OpenAI 5.5 model, which requires me to have an account on their server and go
380. 1:05:03use their cloud-based systems, I can change my settings to use my local server in my local model. So, I have a
381. 1:05:10locally running app that's got these agentic, this ability to use tools and do all this other [ __ ] And then it's got the ability for those
382. 1:05:20agents to make calls to an LLM, which I can run my own one locally. So, not only can I have my own data locally for my
383. 1:05:29memory and all of the things that I'm building, I can actually have this thing running, I could be on the top of a
384. 1:05:34mountain somewhere with no connection to the internet and be doing all this sort of stuff. Does that make sense?
385. 1:05:44Not screen sharing again, but I'm screen sharing now, right? Please tell me I'm screen sharing now. Yes. Does that make sense?
386. 1:05:54He never remembers to wear the shock collar. [laughter] I'm a systems thinker, but I haven't really been exposed to the business
387. 1:06:03systems to hold my attention. Kelly, we can run Excel that is installed on a computer with the DVD or we can run it
388. 1:06:10on the internet in the cloud from Office 365. Yeah. Yeah. So this is [clears throat] what's wild here is that
389. 1:06:25[sighs] the way that the transformer model compresses knowledge is so efficient that that they have effectively taken
390. 1:06:41the entire corpus corpus of human knowledge that was available on the internet and you know whatever was digitized
391. 1:06:50and they have squashed it into a package small enough to sit on this. So sitting on this are some little chips
392. 1:07:05in here with a little processor, little memory, little fan, little screen, and like the entire corpus of human
393. 1:07:15knowledge, some version of it, a very compressed version of it is sitting on this. And so I don't need open AI. I don't
394. 1:07:25need anyone to be able to have this thing go do things for me. Now, if I want to use tools on the internet, it's
395. 1:07:32got to be connected to the internet, but I don't have to. Um, and so what that allows
396. 1:07:46is if I want to do things that are of a very private nature that I don't want to be on their servers,
397. 1:07:57I can just do that now. Right. So I can run it just like chat GPT or I can run one of these agentic harnesses
398. 1:08:06that can it the harness itself can choose oh for anything sensitive I've got a set of rules that I'm going to use
399. 1:08:12this local model but if I need high compute I'm going to use the model in the cloud. Question on the screen. Yeah
400. 1:08:19but isn't the info you're accessing not updated if it's run locally? I don't get that part yet. So the the core model no but in codeex
401. 1:08:33you've got different tools one of the tools that you use is web access so I can [laughter] this this gets this is
402. 1:08:47why this is hard to explain is that there's so many piece parts to it like I probably need to do a big infographic
403. 1:08:53and and may maybe I'll do that Maybe that's what we'll do Wednesday. Why don't you take a note on that? Um, [clears throat]
404. 1:09:00take a note on that, Brandon, for Wednesday that maybe what we'll do is we'll do a an infographic and maybe even an explainer video
405. 1:09:09um using the new Omni model um about [sighs] all these things. Okay. So, so wait, back to Lori Blair's question. Yeah, but
406. 1:09:19okay. The info you're accessing is not Okay. Okay. So, I need I need boxes. Where's my box? Oh [ __ ] I cleaned my
407. 1:09:26office. Okay, we have a different box. We have a different box. Okay, this is perfect. Okay, so [laughter] here's a
408. 1:09:34website. That's the the Hitachi website we built for Hitachi back in the day. [clears throat] Here's your lo here's
409. 1:09:41your your large language model. And wait, we'll get a smaller one, too. Hold, please. Okay, wait. Perfect. Okay.
410. 1:09:49Okay. Okay. This is chat GPT5 or this is GPT 5.5 the model itself. Chat GPT sits on top of it. Remember remember this old
411. 1:10:00metaphor. Here's chat GPT. This is the app that you log into. And then here's GPT 5.5. That's the model. Now I've downloaded Quen 3.6
412. 1:10:14which here's 5.5, here's 3.6. This is small enough to stick on my computer. So, now I've got this harness thing. I
413. 1:10:23need another I need another I need another uh Okay, I got this harness thing [laughter] that's here. Here's codeex. And I can I
414. 1:10:34can get this thing to run its little agents and either ask for chat GPT 5.5 or I can go to my little local model on
415. 1:10:41my machine or any combination of them. Then I can give access to this harness to the web. And so how you end up with updated
416. 1:10:51information is is I can say, "Hey, harness, when you're doing large language model processing, I only want you to do that
417. 1:11:01locally because I don't want to send all my [ __ ] to OpenAI." But if you need to get the latest news, you can go to the
418. 1:11:08web and search the web and go get that information and come back and process it on my little local model here. Make sense? Now, [laughter]
419. 1:11:23[laughter] oh my god, this is actually a really cool box. What's in there? Did that make any sense? That's what you get for cleaning up.
420. 1:11:40Wait, do you pay for the server? Does that question even make sense? No. Okay. So, there's this there's this software called LM Studio.
421. 1:11:55LM Studio is the thing that says, "Hey, you can install a local model and I'll be your chat GBT interface."
422. 1:12:06So, LM Studio is a standalone app that I run on my computer that I can just ask my little local model within LM Studio, it's got its own
423. 1:12:19server. So, there's like a little there's like a little a little sidecar that is running a server. And what the
424. 1:12:27server does is it like this interface is for me as a human. This interface is for codeex or um claude code or Gemini anti-gravity to be
425. 1:12:41able to interface with this model. So I've got a human interface and a computer interface. So that server is
426. 1:12:48free. It's part of LM Studio. LM Studio is free. These models are free. This server to have codeex access this is
427. 1:12:56free. So once I have codeex installed or cloud code um LM studio whatever my model of choice is and this server up and running that's
428. 1:13:09now a self-contained system. like self-contained on my computer where I don't need access to the internet to
429. 1:13:20do all the LLM stuff. I can choose to go out to the internet to do other things, but I don't have to go to chat GPT
430. 1:13:28anymore or to to GPT 5.5. Crazy, right? It's complicated. It's a lot. All right. Seems inefficient.
431. 1:13:45Well, yes and no. It's a lot of peace part. So, so, so what's going to happen is this.
432. 1:13:59[laughter] Um, the cost of being early is you got to figure this [ __ ] out. And as the tools are getting more sophisticated, you
433. 1:14:10know, realized that every one of these tools started as a tool for developers first and then they realized, oh, this could
434. 1:14:21be used for other [ __ ] too. So, let's instead of calling it Claude code, let's call it Claude co-work.
435. 1:14:28And then we'll just tell people to do different [ __ ] We have we'll have different prompts. [laughter] is Cloud
436. 1:14:34Co. Work and Claude Code are essentially the same thing. They've got slightly different tools, but they're the same [ __ ] [laughter]
437. 1:14:45I ask my local LLM to take my confidential data and anonymize any names, date of birth, addresses, phone
438. 1:14:51numbers. I then take the output and feed that into an online model to do the work. So Todd, that's what we're doing.
439. 1:14:57So at StoryVine, we've built a deidentification system. So, so what's happening at StoryVine is when a video comes into the system, we
440. 1:15:07take the transcript or we we we take the video, we run it through a local multimodal model that can watch the
441. 1:15:17video that can understand the words. It makes a transcript. We then take that transcript and we've got a
442. 1:15:24deidentification program that replaces all of the personal [ __ ] with variables with code. We then take that transcript with all
443. 1:15:35the code in it and no personal information and we send that out to chat GPT which generates a blog post and just
444. 1:15:43a bunch of other [ __ ] And then we bring it back into our system and we rehydrate it. So, we take all those codes where it
445. 1:15:50was like Mary Smith and then we put Mary Smith's name back in it once it's back on our local machine. So,
446. 1:15:57but to your point, um, Silver Fox, it seems unnecessarily complicated, all of the stuff that we've done,
447. 1:16:10all of the stuff that um, ChatGpt makes possible was possible a couple of years before chat GPT. Chat
448. 1:16:22GBT was just the interface that put kind of a simple layer on top of a really complicated system sitting underneath it. These harnesses
449. 1:16:32are somewhere in between a usable consumer app and and a coding tool like a coding environment. They are just by
450. 1:16:41definition more complicated than they need to be because it's it's engineers and coders in there doing all their coding [ __ ] Um,
451. 1:16:54what it will likely be is that all of the setup will get set up for you transparently in the background.
452. 1:17:06There'll be import tools where you can import your context from chat GPT or from claude or from Perplexity. It will
453. 1:17:12automatically set up your memory files, your this, your that, right? And then what it will likely be is you'll just go into chat GPT,
454. 1:17:22right? And then over here I'm going to go to codeex. And so this is now these are now chats. Like here's one. This is about my my
455. 1:17:37speaker strategy. So this is now it looks just like chat GPT. I've got a little interface down here where I can type into it. ask codeex
456. 1:17:48and then this this is going to be my interface to those agents that are going to go do all this stuff. So all of the
457. 1:17:54work is being done on my computer over there. Um and just right now these tools are very very complicated because they're
458. 1:18:03doing a lot right they're they're trying to invent memory systems which is effectively infinite context file systems
459. 1:18:15rag or the the ability to search all of your files computer use tool use and large language models and multimodal media generation
460. 1:18:28and ingestion, right? So, [laughter] so, so there's a lot of piece parts here. What it's going to look like 3 years
461. 1:18:37from now is you'll probably just have an app like this. You can sort of turn on the camera or not. You can point it at
462. 1:18:43the world and you can just say, I want to feature film about penguins that danced the samba in Buenus Aras in 1952.
463. 1:18:58Your agents will go off and start doing their research. What does he mean? Dancing the samba in Buenoseries 1952.
464. 1:19:06Ah, here's what was going on in 1952. You know, they'll come back and they'll occasionally update you. You know, do
465. 1:19:13you like style A or style B for your penguins? Oh, I want the king penguins. [laughter] We just, you know, that's three years
466. 1:19:21out. So if you want to use this now, you got to you got to be a little right right now. You got to be a little bit
467. 1:19:27resourceful. And what's what's what's interesting about this is the the reason I started the salon is that
468. 1:19:40chat GPT really represented I've I've said this a thousand times like it does for AI and machine learning what
469. 1:19:53the worldwide web did for the internet. It made it this simple, right? But as the internet got more sophisticated and
470. 1:20:00you know they invented things like stylesheets and and you know serverside applications and and browser side
471. 1:20:06applications and all that sort of stuff and you had and databases and you had to have all that [ __ ] talk to each other.
472. 1:20:12Even web development went through this incredibly technical phase before you got to things like WordPress and
473. 1:20:20Squarespace that hide all that technical stuff behind pre-built templates and tools that make it easy to do that. Like
474. 1:20:27we're we're going through a similar phase right now with AI. I'm thinking in terms of HIPPA data, keep it local, no problem. But if I need
475. 1:20:37to use an LLM for that sort of data, that's not secure. That's correct, Kelly. That's what I was talking about.
476. 1:20:42The deidentification system we built at StoryVine is for exactly that reason. So we take data that's got some people's
477. 1:20:50names in it. Personally identifying information. We we use an LLM that goes through the the document, identifies anything that
478. 1:20:59you could identify a person by by and strips out that data and replaces it with a variable and and just basically
479. 1:21:07tracks the mapping. And then you can send that that strip data over. Yes. >> Since you're ignoring my stickies, uh [laughter]
480. 1:21:17there was a company that specialized in that. I forgot the name of it, but I know you remember. >> It was a it was it was a Denver-based
481. 1:21:25company. They charged 25 grand just to just to play. And I forget the name of it. Um let me see. Let me see if I can. There's
482. 1:21:35companies that will do this securely where they'll give you a secure server to upload your data to and then they'll
483. 1:21:41scrub it all for you behind the scenes. >> Mhm. Yes. But you can So, so we ended up building our own Jon Snow Labs. I don't think that's it.
484. 1:21:58Skyflow. They might have gotten bought. Jon Snow Labs. the deidentification solution. Maybe they got bought. Spark NLP deidentification.
485. 1:22:14Looks like there's a bunch of them. Now, um the system we ended up building um there there's open-source libraries out
486. 1:22:23there that do this. So, you can you can find a service to do it, but but you've got Vibe coding now. So you can probably
487. 1:22:30go find the libraries and basically say I want to build a deidentification system and then I want to vibe code up,
488. 1:22:38you know, a front end that's going to call my So you can either you can either process things on a local smaller model
489. 1:22:46um or you can deidentify it, send it out to the big models, process it, bring it back, and then rehydrate it. My concern is having to change
490. 1:22:56anonymity. Doesn't that mean that an LLM has has it for that process, thereby rendering it unguarded? I guess you have to trust the security
491. 1:23:09level. Well, I don't quite know what you're asking here. Um, there's two different there's there's two different parts of the conversation
492. 1:23:21here. If you're running a local large language model and you're running one of these agentic systems to to write you an
493. 1:23:32application for example or I don't know write you a screenplay doesn't matter whatever it's going to write and you're
494. 1:23:38you're los using the local model everything there is happening locally now you can add tools that access the
495. 1:23:47internet but it still processes it locally right so that's that's a layer of risk but but you can just run a local large language model
496. 1:23:56completely on its own. If you have to send things out to one of the big models, right? Like if you're doing
497. 1:24:03medical stuff and you want it to be really good answers, you don't want to run that locally. You want one of the
498. 1:24:08big ones. Then then I would put a thing in the middle of it, a deidentification system that retains the anonymity but
499. 1:24:15sends the basic data off to the to the big the big guys. And then when the when your answers come back, you reconnect it
500. 1:24:23with the personal data. >> We This is actually I'm hopping up because this is my lane. We learned about this last week at the Zendus
501. 1:24:30conference. Uh nice >> where >> anything that you're sending, you don't want to send it via email. You don't
502. 1:24:36want to send it to an unpaid LLM. But if you're uploading something to say Zenesk and Zenesk is using OpenAI on the back
503. 1:24:44end to power their Aentic framework, they're under a an agreement with OpenAI at the enterprise level that says OpenAI
504. 1:24:54is not going to snoop and train on their data. And they're under a BAA or an agreement with the client that is using
505. 1:25:02the tools that says we're not going to use your data outside of your use case. So we've got silos built in silos built
506. 1:25:10in silos and so you know it it operates on the same framework of communicating with your doctor through my chart. you
507. 1:25:16know, you don't want to just send them an email, but if you're logging in and authenticating and using your thumbrint
508. 1:25:21and password, then you're doing it in a secure environment. And any local LLM that's working within that framework is
509. 1:25:31either airgapped that's not connected to something on the outside, and if it is connected to something on the outside,
510. 1:25:37they have a legal agreement in place that says it's not going to leave your grain silo. >> Yeah. Yeah. That's it. Basically, your
511. 1:25:46job if you want to professionalize your practice, it's one of the things we talk about in the in the mastermind practice
512. 1:25:52lab. If you want to professionalize your practice, it is on you to understand all of your handoffs and all of the
513. 1:26:01components in the chain of, you know, when a client logs into my app and they do something and it magically
514. 1:26:08goes off into LLM land and and does cool AI [ __ ] and brings back an answer, it's on me to understand what are all those
515. 1:26:18components and what are all those data agreements and privacy agreements with every one of those components. And so,
516. 1:26:24as Brandon said, if if Zenesk has an agreement with um with OpenAI that that this particular chain is HIPPA
517. 1:26:34compliant, then I can send personally identifiable information there because I've got legal um cover for that chunk
518. 1:26:45of the chain. But then if I take that, you know, same thing and the next thing I send it to to Claude to their 20 buck
519. 1:26:53a month subscription and I don't have that agreement in place. Well, now I' I've just sent, you know, HIPPA
520. 1:27:00information, you know, out of something that was secure and compliant into something that wasn't. That that's on me
521. 1:27:07as a developer, right, to understand all of those. We have images to look at. Okay, great. So, wait, I wanted to see this one from
522. 1:27:14Kelly. I understand the concepts, but my English major brain does not compute how to get a local large language model. Am
523. 1:27:21I missing something basic? We've got a few minutes. Let me show you. Here's how you do it, Kelly. So, so you're you're going to need
524. 1:27:34to run to run a local language model, local large language model that's decent, you're going to need some decent iron.
525. 1:27:45Right. So, a decent processor. At least 32 gigabytes of RAM. Mine's 64. I wish I'd gotten a 128. Right. So, you just need a lot of RAM
526. 1:27:57and it needs to be unified RAM that's accessible by the GPUs and the CPUs. [snorts] That's why the why the Macs are simple
527. 1:28:06is that their RAM is used by both. It's it's called unified RAM. Once you have a machine that can run it, how you get a
528. 1:28:13local large language model is this. >> The first step is you share your screen. >> I am. Okay.
529. 1:28:30So, I am going to go to LM Studio. So, you download LM Studio that I think will probably start on a a fairly blank screen like this. This is
530. 1:28:44just my little dude. Um, and then down here on the left hand side, I click on this button right here and it pops up this window
531. 1:28:56and here are these are all the local large language models that I have available to me. Right? So, Neatron is the new one from
532. 1:29:06Nvidia. Quen 3.6. The 27 billion. I think this is the mixture of experts model. Does this say anywhere? Quinn LLM.
533. 1:29:21No, maybe this one does. This one's thee one. So, this is the one I'm using. And And then um so like if I go to one of
534. 1:29:29these that I haven't downloaded, see where it says download right here? So, and it's 6.33 gigabytes. And like the
535. 1:29:35one I'm using is if I do the I'm using the the the Q6 quantized model, it's 30 gigabytes. So, just this
536. 1:29:46model is 30 gigabytes and the the the 8 quantized version is 37 GB. Now what is what is relevant about this this
537. 1:29:56number and this is why I was talking about you need 32 gigs of RAM. So this 3.6 model at the at the 6K thing
538. 1:30:08I've got 64 gigs of RAM on that machine. All of the applications that I'm running are taking up some amount of that RAM.
539. 1:30:17And then I'm also loading in those 29 gigabytes of that model all have to go into into that RAM. So, so 30 GB of my
540. 1:30:2764 are eaten by the model itself. Some other percentage of that is eaten by all the other applications that I'm
541. 1:30:35running on this machine. And then whatever's left over, the headroom they call it is what what I have left over
542. 1:30:42for context. So So when they talk about context window, it's literally like if you send too much [ __ ] into it and it it
543. 1:30:51its brain is full, it it literally can't do it. But what what happens is like I download this model and then when I'm
544. 1:30:59over here in the chat and I've got this little pull down menu here, these are the models that I've downloaded and that
545. 1:31:07I can use and and so you can just >> Okay. Um
546. 1:31:25So, this little list right here, these two little quints, I think these are the different quantized versions of
547. 1:31:31it. I'm not actually sure, but these are um these are the ones that I have downloaded local on my hard drive and I
548. 1:31:40can just load them into this chat and then start chatting with them. So, um, why are the Dallas Mavericks such a crappy team?
549. 1:31:57[laughter] Over reliance on Luca Doic. Luca [laughter] Luca doic. Um, [clears throat] okay. [laughter]
550. 1:32:13Let me come back here. So, first of all, Kelly, um, humoring my ineptness, this one of the reasons I I didn't
551. 1:32:23really want to talk about this tonight is that I just barely understand these concepts. Like, I am not fil with the
552. 1:32:32tech at all. I don't really know how to set them up. I just barely get it. But but what I'm getting is that
553. 1:32:42this idea of multiple agents running 247 on our behalf is where AI is going. You're still going to be able to if you
554. 1:32:52want to go into midjourney and make pictures and go back and forth and be a human interacting with AI, that's always
555. 1:33:00going to be available to us. But the bulk of where tokens will be generated and and will provide us value is these systems that
556. 1:33:10just do [ __ ] We give it a goal and it goes off and it does [ __ ] So, so that's where the opportunity is
557. 1:33:20and and the sooner we can learn that first conceptually and and then next technically and then basically do what
558. 1:33:28we've always done here which is figure out where the boundaries are, where's it good, where's it not, where are the
559. 1:33:33compromises and then basically just wait for the industry to catch up for these things to be good. And then the minute they're good,
560. 1:33:42we pounce on them and go do fun, interesting, cool [ __ ] And then three years after that will be the consumer version. That'll make it
561. 1:33:51really easy for everyone else. So that's where we are right now. Okay, let me hop over to um AI Salon because there were
562. 1:34:02some pictures put up there in the irregulars. So let me share my screen.
563. 1:34:16>> [clears throat] >> Where? Oh, there we are. [sighs] Is it in the chat? Chat. Hello, chat.
564. 1:34:36The future. We have everything. [laughter] There's a really funny video that was kind of like this. It was heads
565. 1:34:42up display where the world kind of looked like this. AI mayor zero work day, zero work weeks. That's awesome.
566. 1:34:56[laughter] Oh, that's good. The penguins dancing the samba. That's good. In uh Buenosar's 1952. See, see, we can just
567. 1:35:06go there now. [laughter] That's awesome. All right, kids. Uh, it is 9:06. We're a little bit after. Um, happy Memorial Day, everyone. Thank
568. 1:35:20you to all of our um military for doing what they do so we can dick around here and talk about AI. Um,
569. 1:35:31we'll dig deeper into this. I I think I think creating an infographic on Wednesday probably makes sense. We'll
570. 1:35:39just keep talking about this because um I don't feel comfortable right now not understanding this stuff more deeply
571. 1:35:50than I do. Oh, this is the final thing that I'll I'll leave you with is this. I think at this point these things are
572. 1:36:00so complicated that it actually makes sense to just make a bet, right? If if you if you don't really need images or like even if you need
573. 1:36:11images or video, you can still do cloud code and claude co-work because it can use tools because of MCP and and because
574. 1:36:19of skills, you can make video with external tools. So, I don't think it matters which one you pick. I mean,
575. 1:36:28I would pick one of the big three personally. I'd pick Open AI, um, Claude, or anti-gravity, you know, the the Spark anti-gravity combo. Um,
576. 1:36:42I'm going with Open AI just because they were there first and the fact that they're defaulting to everything being
577. 1:36:50local files. I think I want to stick there. But daily, I'm thinking like maybe I should go with Claude. Like I
578. 1:36:58haven't really decided yet, but I but I just these things are so complicated that I think it's it just doesn't make sense to have
579. 1:37:07a whole set of files over for this one and a different set of files here. Maybe you can point them to the same file
580. 1:37:13directory. I don't know. There's people here that probably know this better than I do. Anyway, um I I think we're getting
581. 1:37:18to a point where you're going to have to kind of make a bet of which one of these things you want to go with. Um and I
582. 1:37:25don't it probably doesn't matter. Like I don't I don't think there's a wrong answer if you go with one of the big three. Um
583. 1:37:33that's it. Peace out. Hope you had a good night tonight and I appreciate you all as always. Bye.