Local vs. Cloud LLMs: Your Data Security Secrets REVEALED!

Transcript

1. 0:00Well, I don't quite know what you're asking here. Um, there's two different there's there's two different parts of the conversation here.
2. 0:11If you're running a local large language model and you're running one of these agentic systems to to write you an
3. 0:19application for example or I don't know write you a screenplay doesn't matter whatever it's going to write and you're
4. 0:25you're los using the local model everything there is happening locally now you can add tools that access the
5. 0:35internet but it still processes it locally right so that's that's a layer of risk But but you can just run a local large language model
6. 0:44completely on its own. If you have to send things out to one of the big models, right? Like if you're doing
7. 0:51medical stuff and you want it to be really good answers, you don't want to run that locally. You want one of the
8. 0:56big ones. Then then I would put a thing in the middle of it, a deidentification system that retains the anonymity but
9. 1:03sends the basic data off to the to the big the big guys. And then when the when your answers come back, you reconnect it
10. 1:11with the personal data. >> We This is actually I'm hopping up because this is in my lane. We learned about this last week at the Zenus
11. 1:18conference. Uh >> nice where >> anything that you're sending, you don't want to send it via email. You don't
12. 1:23want to send it to an unpaid LLM. But if you're uploading something to say Zenesk and Zenesk is using OpenAI on the back
13. 1:32end to power their agentic framework, they're under a an agreement with OpenAI at the enterprise level that says OpenAI
14. 1:41is not going to snoop and train on their data. And they're under a BAA or an agreement with the client that is using
15. 1:50the tools that says we're not going to use your data outside of your use case. So, we've got silos, built-in silos,
16. 1:58built-in silos. And so, you know, it it operates on the same framework of communicating with your doctor through
17. 2:03My Chart. You know, you don't want to just send them an email, but if you're logging in and authenticating and using
18. 2:08your thumbrint and password, then you're doing it in a secure environment. And any local LLM that's working within that
19. 2:17framework is either airgapped that's not connected to something on the outside. And if it is connected to something on
20. 2:24the outside, they have a legal agreement in place that says it's not going to leave your grain silo.
21. 2:31>> Yeah. Yeah. That's it. Basically, your job if you want to professionalize your practice, it's one of the things we talk
22. 2:38about in the in the mastermind practice lab. If you want to professionalize your practice, it is on you to understand all of your
23. 2:47handoffs and all of the components in the chain of, you know, when a client logs into my app and they do something and it magically
24. 2:56goes off into LLM land and and does cool AI and brings back an answer, it's on me to understand what are all those
25. 3:06components and what are all those data agreements and privacy agreements with every one of those components. And so,
26. 3:12as Brandon said, if if Zenesk has an agreement with um with OpenAI that that this particular chain is HIPPA
27. 3:22compliant, then I can send personally identifiable information there because I've got legal um cover for that chunk
28. 3:33of the chain. But then if I take that, you know, same thing and the next thing I send it to to Claude to their 20 buck
29. 3:40a month subscription and I don't have that agreement in place. Well, now I' I've just sent, you know, HIPPA
30. 3:47information, you know, out of something that was secure and compliant into something that wasn't. That that's on me
31. 3:54as a developer, right, to understand all of those. We have images to look at. Okay, great. So, wait, I wanted to see this one from
32. 4:02Kelly. I understand the concepts, but my English major brain does not compute how to get a local large language model. Am
33. 4:09I missing something basic? We've got a few minutes. Let me show you. >> Watch the full replay at community.thesalon.ai.