Do you know why we need security at every layer of an AI system?

Most people say:

“To protect data.”

That’s true… but it’s only half the story. Let’s take a real example 👇

An LLM like Llama-3.1-70B, when deployed using NVIDIA NIM, is not just a model, it also includes:

• ~350 software packages
• Hundreds of dependencies
• Multiple third-party libraries (OSS)
• Deeply interconnected components

Now think about this:

If even ONE component is vulnerable…

– The entire system can be at risk.

This is why security in AI is NOT just about Data protection! It’s about Securing the entire software supply chain at EVERY layer:

• Base OS
• Containers
• Libraries
• APIs
• Model runtime
• Orchestration (Kubernetes, etc.)

Let’s say you fix one vulnerability:

• You update a package
• That breaks a dependency
• Which affects another component
• Which may impact model behavior

Security fixes can affect across the entire system. This is why AI security is different. It’s not just “secure the app”

👉 It’s “secure the ecosystem”

Security must exist at every layer!

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts