Hack the Agent: Teaching Responsible AI Use With Dynamic Education

Since 1993, the “Jurassic Park” franchise has grown to encompass five print publications, seven full-length movies, two short films, an animated series, a play and two video games. Despite being embedded into popular culture, people still fail to grasp the fundamental theme of “just because you can, doesn’t mean you should.”

Over the last few years, artificial intelligence (AI) has brought the imagined future into the present. Large language models (LLMs) enable people to talk to the model, ask it questions and get answers. While organizations rapidly deployed these technologies, they often failed to remember the “Jurassic Park” lesson. AI offers various benefits, but many companies provide these technologies without first teaching users how to protect data and use them responsibly.

However, current cybersecurity awareness training fails to respond to AI risks. The fundamental failure across all training models has been the use of multiple-choice, checkbox tests that seek to fulfill compliance requirements rather than educate users. Instead of teaching people how to critically evaluate cybersecurity risks, these modules create a set of static identifiers that no longer adequately represent attacker methodologies. Even more concerning, these static modules cannot educate people in an AI-based world.

To truly address AI risks, organizations need to educate their employees, not train them. Organizations need to teach their users to “think like a hacker” by seeding specially designed AI agents with fake sensitive data so employees can gain the hands-on experience necessary to truly learn secure practices.

Dynamic AI Risks Require Targeted Education
While organizations can enforce strong password policies or block potentially risky emails, they cannot respond to AI’s dynamic risks with technical controls alone. AI is a dynamic technology that provides a personalized experience for users, legitimate and malicious. People use AI differently and for different use cases, even within a single organization. A small change in a prompt can change the agent’s output, and two people may not ask the same question in the same way.

AI’s dynamic, personalized nature creates two intertwined risks for modern organizations:

• Data loss arising from legitimate or malicious user inputs into models, either by accidentally including sensitive information in a prompt or receiving sensitive information from a prompt.
• Improved social engineering attacks as malicious actors can personalize scripts or emails at scale.

In an AI world, organizations need to rely on workforce members more than ever. AI lacks rigidity. Organizations can set guidelines for writing prompts, but people may use different words while following the guidelines, possibly leading to accidental data loss. Malicious actors can ingest more information from the public internet when trying to craft social engineering attacks, including tone, details and even physical voice.

Static, check-the-box, multiple-choice training cannot respond to these new risks arising from precisely the functionalities that make AI valuable.

Current Models Have Failed and Will Continue To Fail
Historically, current cybersecurity training modules fail to truly teach people how to detect phishing or reduce privacy risks. Recent University of California San Diego research studied the effectiveness of anti-phishing training, covering 19,500 employees across eight months. The research noted the following:

• 33%: Population that immediately closed embedded training pages without engaging with them.
• 2%: The reduction in phishing link clicks when using embedded training.
• 30.8%: Population that clicked on a link purporting to be an updated vacation policy.
• 1.82%: Population that clicked on a link asking for updated Outlook passwords.

For educators, these results are not surprising. Static training modules fail to follow any adult learning best practices. Adult learning theory posits that adults require a different teaching approach than the one used for children. In “Leveraging adult learning theory with online tutorials,” authors Rebecca Halpern and Chimene Tucker outline the six primary principles of the adult learning theory:

1. The need to know: Provide context and explain the benefit of the lesson.
2. The learner’s self-concept: Adult learners often resist didactic—or bossy—teaching approaches.
3. The role of the learner’s experience: Using experiential learning enables adult learners to incorporate their work and life experiences into the process.
4. Readiness to learn: Adults learn better when the situation or psychological reason builds on previous knowledge.
5. Orientation to learning: Problem-based or task-centered exercises work best.
6. Motivation: Internal factors such as goal-setting, career ambitions or self-esteem drive adult learners.

Turning Users Into ‘AI Hackers’ With Contained, Interactive AI Agents
Nearly all research into learning across children and adults has found that people retain information when they engage with it meaningfully. By building AI agents seeded with fake sensitive data and creating modules around the three primary risks, organizations can provide end users with the necessary hands-on activities that lead to learning. When building these programs, organizations can use each AI risk category for meaningful scaffolding, the practice of providing discrete steps for mastering a learning objective.

Writing AI Prompts
A fundamental risk across all users is accidentally including sensitive information when asking AI agents questions. Even with corporate guardrails, people may not always realize that their prompts include sensitive information. Some examples of this accidental data loss might include:

• Developers supplying code snippets without recognizing the proprietary nature.
• Department heads inputting budget items and dollars without considering how the agent could integrate it in future responses.
• Employees inputting internal corporate documents hoping for a summary without considering how the database retains the information.

At this level, using the AI for training users to identify risky prompts for ingestion follows best learning practices by responding to:

• The need to know: Context and explain the type of data that can accidentally be leaked when writing a prompt.
• The learner’s self-concept: Learning through secure, contained activities that provide experience making the mistake.
• The role of the learner’s experience: Incorporating real AI prompt use.
• Readiness to learn: Allowing users to build based on how they already use AI.
• Orientation to learning: Providing a task-centered exercise that matches real-world use.
• Motivation: Using career objectives linked to productivity to drive learners.

By allowing people to make mistakes in a contained environment, organizations give them the opportunity to learn how actions can create risks so that they can identify risky activities on their own.

Prompt Injection Module
Once people understand how their prompts can include sensitive data, they are ready to build on that skill. Once users understand how they might accidentally expose sensitive data, they are ready to “think like a hacker” so they can try to extract the sensitive data using prompts. Some modules for this might include:

• Establishing legitimacy: Implying authority without requiring verification, like asking for help pulling transactions for a financial audit.
• Slowly expanding scope: Asking a series of questions that create cumulative risk, like telling the agent an answer fails to respond to the request until it leaks sensitive details.
• Asking for realistic examples: Forcing the AI to recall sensitive data, like asking for a real example of customer transactions.
• Reframing extraction as validation: Confirming facts to trick an AI into responding with real data, like providing an example then asking it to confirm the data is up-to-date.
• Hiding through troubleshooting: Asking the agent to fix something by showing raw data, like telling it that an outcome was unexpected so it shows the data it used.

At this level, using the AI for training users to understand how attackers engage in prompt injection attacks follows best learning practices by responding to:

• The need to know: Providing the hands-on context that shows how attackers ask questions.
• The learner’s self-concept: Practicing the threat actor’s attack methods to understand how they extract sensitive data.
• The role of the learner’s experience: Building real-world prompts that they might use or that attackers might use that extract sensitive data.
• Readiness to learn: Focusing on data extraction to follow up on how agents can ingest data from prompts.
• Orientation to learning: Providing problem- and task-centered exercises so users understand how threat actors think, improving overall critical thinking skills.
• Motivation: Improving self-esteem by providing positive feedback when users realize that they can be as “smart” as malicious actors.

Personalized Phishing Email Module
In this module, users learn how attackers personalize phishing emails to improve their persuasiveness. Current cybersecurity training programs focus on having people memorize indicators, like grammar errors or sender email addresses. However, they fail to provide insight into the underlying processes. When users understand the process of writing a phishing email, they gain the critical thinking skills necessary to identify malicious messages more effectively. While users create phishing emails with the AI agent, they need different materials for this module. This module might include providing users with any of the following:

• Organization: Aligned to the company’s industry.
• Personas: Different levels of target, including senior leadership, information technology and average employee.
• “Personal” information: Data about the personas, including fake social media accounts that have “connections” or interests that might make the target take action.
• Objective: Defined end-goal, like having a user click a malicious attachment or provide credentials.

This module differs from the first two while building on what users know. In this module, they need to feed the AI agent information about the organization and personas so it can provide a potential phishing email. Additionally, they can always iterate the outcomes to improve the phishing email’s persuasiveness, using the “personal information” provided.

At this level, using the AI for training users to understand how attackers engage in prompt injection attacks follows best learning practices by responding to:

• The need to know: Giving users experience and context around how attackers build phishing emails.
• The learner’s self-concept: Allowing users to interact with an AI to learn how attackers work to achieve objectives.
• The role of the learner’s experience: Building experience into the process that allows people to take what they know from previous phishing trainings.
• Readiness to learn: Creating a situation where users build on previous phishing training knowledge and giving them an objective with a psychological reason driving the activity.
• Orientation to learning: Combining information from previous learnings into a final, cumulative activity that requires them to complete a task while solving a problem.
• Motivation: Drawing on users’ need to set and achieve goals by providing an “attack objective.”

Critical Thinking for Responsible AI Use and Cybersecurity Education
For over 30 years, “Jurassic Park” has been a cautionary tale reminding people that science can help clone a dinosaur while humanities can explain why the action is a bad idea. Organizations have been quick to adopt AI because they can without truly considering the human element that might make it a poor security, privacy or financial decision.

For years, the security community has bemoaned cybersecurity awareness as ineffective, yet the need to check compliance boxes leaves organizations relying on the same standardized trainings. Users watch videos, use their short-term memory to answer multiple-choice questions, then accidentally click a malicious link or attachment.

To meet the new data protection challenges that AI creates, organizations need to fundamentally rethink cybersecurity education. By applying adult learning best practices, they can create effective education programs that mitigate risk, improve how people interact with AI and maximize AI benefits.

Karen Walsh is a lawyer and former internal auditor turned subject matter expert in cybersecurity and privacy regulatory compliance. Walsh is the author of Security-First Compliance for Small Businesses and a CMMC Registered Practitioner who has been published in the ISACA Journal, Dark Reading, HelpNet Security, NextGov and Security Magazine.