Local vs. Cloud LLMs: Your Data Security Secrets REVEALED!

Description

Chapters

Transcript

0:01 Well, I don't quite know what you're
0:03 asking here. Um,
0:06 there's two different there's there's
0:08 two different parts of the conversation
0:09 here.
0:12 If you're running a local large language
0:15 model and you're running one of these
0:17 agentic systems to to write you an
0:20 application for example or I don't know
0:23 write you a screenplay doesn't matter
0:24 whatever it's going to write and you're
0:26 you're los using the local model
0:29 everything there is happening locally
0:32 now you can add tools that access the
0:35 internet but it still processes it
0:37 locally right so that's that's a layer
0:39 of risk But but you can just run a local
0:42 large language model
0:44 completely on its own. If you have to
0:47 send things out to one of the big
0:50 models, right? Like if you're doing
0:51 medical stuff and you want it to be
0:53 really good answers, you don't want to
0:54 run that locally. You want one of the
0:56 big ones. Then then I would put a thing
0:59 in the middle of it, a deidentification
1:01 system that retains the anonymity but
1:03 sends the basic data off to the to the
1:06 big the big guys. And then when the when
1:09 your answers come back, you reconnect it
1:11 with the personal data.
1:13 >> We This is actually I'm hopping up
1:16 because this is in my lane. We learned
1:17 about this last week at the Zenus
1:18 conference. Uh
1:19 >> nice where
1:21 >> anything that you're sending, you don't
1:22 want to send it via email. You don't
1:24 want to send it to an unpaid LLM. But if
1:27 you're uploading something to say Zenesk
1:30 and Zenesk is using OpenAI on the back
1:33 end to power their agentic framework,
1:37 they're under a an agreement with OpenAI
1:40 at the enterprise level that says OpenAI
1:42 is not going to snoop and train on their
1:44 data. And they're under a BAA or an
1:48 agreement with the client that is using
1:50 the tools that says we're not going to
1:53 use your data outside of your use case.
1:56 So, we've got silos, built-in silos,
1:58 built-in silos. And so, you know, it it
2:00 operates on the same framework of
2:02 communicating with your doctor through
2:04 My Chart. You know, you don't want to
2:05 just send them an email, but if you're
2:07 logging in and authenticating and using
2:09 your thumbrint and password, then you're
2:11 doing it in a secure environment. And
2:14 any local LLM that's working within that
2:17 framework is either airgapped that's not
2:21 connected to something on the outside.
2:23 And if it is connected to something on
2:25 the outside, they have a legal agreement
2:26 in place that says it's not going to
2:29 leave your grain silo.
2:31 >> Yeah. Yeah. That's it. Basically, your
2:34 job if you want to professionalize your
2:37 practice, it's one of the things we talk
2:38 about in the in the mastermind practice
2:40 lab. If you want to professionalize your
2:42 practice,
2:44 it is on you to understand all of your
2:48 handoffs and all of the components in
2:50 the chain of,
2:52 you know, when a client logs into my app
2:54 and they do something and it magically
2:56 goes off into LLM land and and does cool
3:00 AI and brings back an answer, it's
3:02 on me
3:04 to understand what are all those
3:06 components and what are all those data
3:08 agreements and privacy agreements with
3:10 every one of those components. And so,
3:12 as Brandon said, if if Zenesk has an
3:16 agreement with um with OpenAI that that
3:20 this particular chain is HIPPA
3:22 compliant, then I can send personally
3:25 identifiable information there because
3:27 I've got legal um cover for that chunk
3:33 of the chain. But then if I take that,
3:36 you know, same thing and the next thing
3:38 I send it to to Claude to their 20 buck
3:41 a month subscription and I don't have
3:44 that agreement in place. Well, now I'
3:46 I've just sent, you know, HIPPA
3:48 information, you know, out of something
3:51 that was secure and compliant into
3:53 something that wasn't. That that's on me
3:55 as a developer, right, to understand all
3:58 of those.
3:59 We have images to look at. Okay, great.
4:01 So, wait, I wanted to see this one from
4:03 Kelly. I understand the concepts, but my
4:04 English major brain does not compute how
4:07 to get a local large language model. Am
4:09 I missing something basic? We've got a
4:11 few minutes. Let me show you.
4:13 >> Watch the full replay at
4:14 community.thesalon.ai.