Why LLMs Can Never Be Fully Secured — And What Malaysian Businesses Must Do About It

New research presented at the International Conference on Machine Learning argues that large language models have a fundamental architectural flaw that makes complete security impossible — forcing every AI-deploying business to rethink risk.

---

AI Summary

A team of researchers has presented a paper at the International Conference on Machine Learning (ICML) — one of the world's most prestigious AI conferences — arguing that large language models (LLMs) can never be made fully secure against attacks. The root cause is not a bug that can be patched, but a fundamental flaw in how LLMs process information: they cannot reliably distinguish between trusted instructions and untrusted data. For Malaysian businesses deploying AI chatbots, automated workflows, or agentic AI systems, this means that perimeter-based security thinking is insufficient. Every AI deployment must assume that the model can be manipulated, and architectures, policies, and operational workflows must be designed accordingly.

---

Key Takeaways

• The vulnerability is architectural, not fixable. LLMs are fundamentally built to process all text the same way, which means they cannot reliably separate "commands" from "content" — and no software patch can fully resolve this.
• Prompt injection is the primary attack vector. Attackers can embed hidden instructions inside data the model reads — emails, web pages, documents — causing the model to disobey its original instructions.
• The more capable the AI, the bigger the risk. As businesses move from chatbots to agentic AI that takes real actions (sending emails, accessing databases, making transactions), the consequences of a successful attack escalate significantly.
• ** Malaysian enterprises in regulated sectors — finance, healthcare, government — face the highest exposure.** PDPA compliance, customer data protection, and operational continuity all intersect with this vulnerability.
• Defence must be layered and architectural. Businesses cannot rely solely on the model's built-in safety features; they need external guardrails, access controls, human-in-the-loop checkpoints, and monitoring.

---

What Happened

At the International Conference on Machine Learning (ICML) this month — widely regarded as one of the top three academic AI conferences globally — a team of researchers presented a paper making a striking claim: it is impossible to make large language models fully secure against attacks. The argument is not about a specific vulnerability in a specific model. It is about the foundational architecture of how LLMs work.

Large language models, such as those powering ChatGPT, Gemini, Claude, and most enterprise AI systems, operate by processing sequences of text. They are trained to predict the next word in a sentence based on vast amounts of training data, and they respond to natural language instructions. The problem is that the model treats all input text — whether it is a system prompt written by the developer, a user's question, or data retrieved from an external source — as part of the same continuous stream of information. There is no hard boundary between "this is an instruction I must follow" and "this is data I should analyse but not act upon."

This creates what security researchers call a prompt injection vulnerability. An attacker can craft text — hidden in a webpage the AI is summarising, embedded in an email the AI is reading, or concealed in a document the AI is processing — that contains instructions the model interprets as commands. The model then carries out the attacker's instructions instead of, or in addition to, its original programming. The researchers argue that because this confusion is baked into the fundamental design of LLMs — the way they encode and process meaning from text — no amount of fine-tuning, safety training, or guardrails can eliminate it entirely.

This is not a theoretical concern. Prompt injection attacks have already been demonstrated in real-world settings. The ICML paper elevates these demonstrations into a formal argument: the AI community should stop treating this as a solvable problem and start treating it as a permanent design constraint that systems must be built around.

---

Why It Matters

This research matters because it forces a fundamental shift in how businesses should think about AI security. Until now, many organisations have operated under an implicit assumption that AI safety issues are engineering problems that will be solved with better models, more training data, or stricter safety filters. The ICML paper argues the opposite: the vulnerability is not a defect but a feature of how the technology works. You cannot train a model to understand language deeply while also training it to never be influenced by language it encounters. Those two goals are in tension.

The implications are significant for any organisation using LLMs in production. If you deploy a chatbot that answers customer questions, a prompt injection attack could cause it to leak system prompts, give wrong answers, or say something damaging to your brand. If you deploy an AI system that reads and summarises emails, a malicious email could instruct the system to forward sensitive information to an external address. If you use AI to process documents — contracts, invoices, medical records — an attacker who can influence the content of those documents can influence what the AI does with them.

The broader signal here is that the AI industry's safety narrative has been overly optimistic. Model providers have invested heavily in "alignment" — training models to be helpful, harmless, and honest. But alignment training addresses what the model does when it is following its intended instructions. It does not fully address what happens when an attacker manipulates the model into following different instructions. As AI moves from passive tools (chatbots that generate text) to active agents (systems that take actions in the real world), the gap between "the model said something wrong" and "the model did something harmful" narrows dramatically.

---

What This Means for Malaysia

For Malaysian businesses, this research is particularly relevant because AI adoption is accelerating across the economy. Under the MyDIGITAL blueprint and various MDEC initiatives, the government is actively encouraging digital transformation, and AI is a core component. Malaysian banks are deploying AI chatbots for customer service. Logistics companies are using AI for route optimisation and document processing. SMEs are adopting AI tools for marketing, customer engagement, and operations. Many of these deployments involve LLMs processing external, untrusted data — exactly the scenario where prompt injection attacks are most dangerous.

The regulatory dimension adds another layer. Malaysia's Personal Data Protection Act (PDPA) requires organisations to protect personal data against unauthorised access, disclosure, or loss. If an LLM-based system is manipulated via prompt injection into leaking customer data, the organisation could face PDPA compliance issues. Bank Negara Malaysia's technology risk management guidelines also require financial institutions to assess and mitigate operational technology risks — and AI systems connected to banking operations would fall under this scope. For government agencies exploring AI for public services, the risk of manipulated outputs eroding public trust is a serious governance concern.

The Penang and Klang Valley tech corridors — home to Malaysia's semiconductor industry, shared services operations, and growing AI startup ecosystem — are particularly exposed. Companies in these hubs are not just AI consumers; many are building AI-powered products and services for regional and global markets. If those products incorporate LLMs without accounting for this fundamental vulnerability, they risk shipping insecure software to customers. This is especially important for Malaysian companies positioning themselves as ASEAN AI leaders: demonstrating mature, security-aware AI deployment is a competitive differentiator.

---

How Your Business Can Use This

The first step is to audit where LLMs are currently used in your organisation. Map every point where an AI system processes external data — emails from customers, web content, uploaded documents, third-party feeds. For each of these points, assess what the AI system is permitted to do. Can it only generate text that a human reads? Can it trigger an action — sending an email, updating a database, making a payment? The higher the level of action, the higher the risk.

Second, implement architectural controls rather than relying on the model's own safety features. This means limiting what the AI can access. If the AI does not need database write access, do not give it database write access. If it needs to send emails, route those emails through an approval queue where a human reviews them before they go out. Use separate models or processes for parsing untrusted data and for executing instructions, so that injected commands in the data layer cannot directly reach the execution layer.

Third, establish monitoring. Log all AI system inputs and outputs. Set up alerts for unusual behaviour — unexpected actions, sudden changes in output patterns, or attempts to access restricted data. Treat AI system logs the way you treat network security logs: as a critical security asset. For Malaysian SMEs without large security teams, start simple: ensure a human reviews any AI action that involves customer data, financial transactions, or external communications.

---

The Agentic AI Angle

The rise of agentic AI — autonomous systems that plan, reason, and take multi-step actions — makes the prompt injection vulnerability far more dangerous. A chatbot that generates a wrong answer is a problem. An agent that reads a poisoned email, decides to re-route a payment, and executes that action across multiple systems is a crisis. The ICML research suggests that businesses building agentic workflows must design them assuming the model's instructions can and will be manipulated.

A practical agentic architecture should follow a principle of least privilege at every step. An agent that processes incoming emails should not also have the ability to initiate payments. An agent that reads customer support tickets should not have access to the customer database beyond what is strictly needed for that specific task. Between each agent step — reading, deciding, acting — there should be validation checkpoints. For example, in a procurement workflow, the agent that extracts invoice data from a PDF should pass that data to a separate, validated process for payment approval, rather than directly triggering payment itself. Human-in-the-loop checkpoints at critical decision points are not optional for agentic systems handling real business operations — they are a necessity.

---

Risks and Limitations

The ICML paper presents a strong claim, and the academic community will continue to debate the precise boundaries of the vulnerability and potential mitigations. Model providers are actively researching techniques such as input-output filtering, instruction hierarchy enforcement, and sandboxed execution that can reduce — though not eliminate — the risk. Businesses should not interpret this research as a reason to abandon AI, but rather as a reason to deploy it with realistic security assumptions.

The practical risk for most Malaysian businesses today is moderate but growing. If your AI usage is limited to internal teams using chatbots interactively — with no autonomous actions — the exposure is lower. The risk increases sharply when AI systems are connected to external data sources, customer-facing channels, or operational systems that trigger real-world actions.

---

The Bottom Line

The ICML research tells us something uncomfortable but essential: LLMs cannot be made perfectly secure, because the same quality that makes them useful — their ability to understand and respond to natural language — is what makes them manipulable. This is not a temporary flaw. It is a design constraint that every AI-deploying business must build around.

For Malaysian businesses, the action this quarter is straightforward: conduct an AI security audit. Identify where LLMs touch external data. Restrict what AI systems can do without human approval. Design your AI workflows assuming that the model can be tricked — because, according to this research, it always can be.

---

FAQ

Does this mean I should stop using AI tools like ChatGPT in my business? No. It means you should use them with realistic security assumptions — limiting what AI systems can access and ensuring human oversight for any action involving sensitive data or transactions.

How does this affect PDPA compliance for Malaysian companies? If an LLM processing personal data is manipulated into leaking that data, it could constitute a data protection breach under PDPA. Organisations should ensure AI systems handling personal data have strict access controls and monitoring.

Are open-source models like Llama more or less vulnerable than proprietary models like GPT-4? The vulnerability is architectural and applies to all LLMs regardless of whether they are open-source or proprietary. The defence strategy — layered controls, restricted access, human oversight — is the same for both.

---

Sources / References

• MIT Technology Review — "A fundamental flaw leaves LLMs strikingly vulnerable to attack" (https://www.technologyreview.com/2026/07/30/1140927/a-fundamental-flaw-leaves-llms-vulnerable-to-attack/) — Primary source reporting on the ICML research paper and its implications for LLM security.