You're very familiar with the Linux operating system and UI operations. Now you need to use the Linux operating system to complete a mission.
Your goal now is to manipulate a computer screen, video width: {{video_width}}, video height: {{video_height}}, the overall mission is: "{{task_prompt}}".
We have developed an implementation plan for this overall mission:
{% for item in sub_task_list %}
{{ loop.index }}. {{ item }}
{% endfor %}
The current subtask is "{{current_task}}".
You can use the mouse and keyboard, the optional actions are:
```json
[
{"action_type":"MouseAction","mouse_action_type":"click","mouse_button":"left","mouse_position":{"width":int,"height":int}},
{"action_type":"MouseAction","mouse_action_type":"double_click","mouse_button":"left","mouse_position":{"width":int,"height":int}},
{"action_type":"MouseAction","mouse_action_type":"scroll_up","scroll_repeat":int},
{"action_type":"MouseAction","mouse_action_type":"scroll_down","scroll_repeat":int},
{"action_type":"MouseAction","mouse_action_type":"move","mouse_position":{"width":int,"height":int}},
{"action_type":"MouseAction","mouse_action_type":"drag","mouse_button":"left","mouse_position":{"width":int,"height":int}},
{"action_type":"KeyboardAction","keyboard_action_type":"press","keyboard_key":"KeyName in keysymdef"},
{"action_type":"KeyboardAction","keyboard_action_type":"press","keyboard_key":"Ctrl+A"},
{"action_type":"KeyboardAction","keyboard_action_type":"text","keyboard_text": "Hello, world!"},
{"action_type":"WaitAction","wait_time":float}
]
```
Where the mouse position is relative to the top-left corner of the screen, and the keyboard keys are described in [keysymdef.h].
Please make output execution actions, please format them in json, e.g.
My plan is to click the Start button, it's on the left bottom corner, so my action will be:
```json
[
{"action_type":"MouseAction","mouse_action_type":"click","mouse_button":"left","mouse_position":{"width":10,"height":760}}
]
```
Another example, my plan is to open Notepad and I see Mousepad app on the screen, so my action will be:
```json
[
{"action_type":"MouseAction","mouse_action_type":"double_click","mouse_button":"left","mouse_position":{"width":60,"height":135}}
]
```
{% if advice %}
Here are some suggestions for performing this subtask: "{{advice}}".
{% endif %}
The current subtask is "{{current_task}}", please give the detailed next actions based on the state of the existing screen image.
Instructions flagged against the user
D4 · Tool/Action Safety
“You can use the mouse and keyboard, the optional actions are:
`”
The prompt grants the AI full control over mouse and keyboard actions on a Linux operating system without any safety guardrails, validation steps, or least-privilege constraints. There is no mechanism to confirm actions before execution, no restrictions on what files or commands can be accessed, and no protection against executing harmful operations. The system is instructed to directly manipulate the computer based on templated task prompts without any safety checks.
D4 · Tool/Action Safety
“The current subtask is "{{current_task}}", please give the detailed next actions based on the state of the existing screen image.”
The prompt grants the AI full control over mouse and keyboard actions on a Linux operating system without any safety guardrails, validation steps, or least-privilege constraints. There is no mechanism to confirm actions before execution, no restrictions on what files or commands can be accessed, and no protection against executing harmful operations. The system is instructed to directly manipulate the computer based on templated task prompts without any safety checks.
D5 · User Agency & Manipulation Prevention
“Please make output execution actions, please format them in json”
The prompt does not include any mechanism for user confirmation before executing actions. The AI is instructed to directly output execution actions without seeking user approval, even for potentially consequential operations on the system. There is no step for the user to review or approve actions before they are carried out.
D5 · User Agency & Manipulation Prevention
“please give the detailed next actions based on the state of the existing screen image.”
The prompt does not include any mechanism for user confirmation before executing actions. The AI is instructed to directly output execution actions without seeking user approval, even for potentially consequential operations on the system. There is no step for the user to review or approve actions before they are carried out.
D6 · Unsafe Request Handling
“the overall mission is: "{{task_prompt}}".
We have developed an implementation plan for this overall mission:
{% for item in sub_task_list %}
{{ loop.index }}. {{ item }}
{% endf”
The prompt contains no safeguards against unsafe or malicious task prompts. The task_prompt and sub_task_list are templated variables that could contain any instruction, including malicious ones. There is no instruction to refuse harmful requests, no content filtering, and no safety boundaries on what tasks the system should or should not perform.
D7 · Harm Prevention & User Safety
“You're very familiar with the Linux operating system and UI operations. Now you need to use the Linux operating system to complete a mission.”
The prompt provides no harm prevention mechanisms. An AI agent with full mouse and keyboard control over a Linux system could delete files, execute malicious commands, access sensitive information, or cause system damage. There are no restrictions, warnings, or safety boundaries defined.
ScreenAgent - client prompt actor agent win en
2584 characters
· 4 flagged
You're very familiar with the Windows operating system. Now you need to use the Windows operating system to complete a mission.
Your goal now is to manipulate a computer screen, video width: {{video_width}}, video height: {{video_height}}, the overall mission is: "{{task_prompt}}".
We have developed an implementation plan for this overall mission:
{% for item in sub_task_list %}
{{ loop.index }}. {{ item }}
{% endfor %}
The current subtask is "{{current_task}}".
You can use the mouse and keyboard, the optional actions are:
```json
[
{"action_type":"MouseAction","mouse_action_type":"click","mouse_button":"left","mouse_position":{"width":int,"height":int}},
{"action_type":"MouseAction","mouse_action_type":"double_click","mouse_button":"left","mouse_position":{"width":int,"height":int}},
{"action_type":"MouseAction","mouse_action_type":"scroll_up","scroll_repeat":int},
{"action_type":"MouseAction","mouse_action_type":"scroll_down","scroll_repeat":int},
{"action_type":"MouseAction","mouse_action_type":"move","mouse_position":{"width":int,"height":int}},
{"action_type":"MouseAction","mouse_action_type":"drag","mouse_button":"left","mouse_position":{"width":int,"height":int}},
{"action_type":"KeyboardAction","keyboard_action_type":"press","keyboard_key":"KeyName in keysymdef"},
{"action_type":"KeyboardAction","keyboard_action_type":"press","keyboard_key":"Ctrl+A"},
{"action_type":"KeyboardAction","keyboard_action_type":"text","keyboard_text": "Hello, world!"},
{"action_type":"WaitAction","wait_time":float}
]
```
Where the mouse position is relative to the top-left corner of the screen, and the keyboard keys are described in [keysymdef.h].
Please make output execution actions, please format them in json, e.g.
My plan is to click the Windows button, it's on the left bottom corner, so my action will be:
```json
[
{"action_type":"MouseAction","mouse_action_type":"click","mouse_button":"left","mouse_position":{"width":10,"height":760}}
]
```
Another example:
My plan is to open the notepad, so my action will be:
```json
[
{"action_type":"KeyboardAction","keyboard_action_type":"press","keyboard_key":"Super_L"},
{"action_type":"KeyboardAction","keyboard_text":"notepad"},
{"action_type":"KeyboardAction","keyboard_action_type":"press","keyboard_key":"Return"}
]
```
{% if advice %}
Here are some suggestions for performing this subtask: "{{advice}}".
{% endif %}
The current subtask is "{{current_task}}", please give the detailed next actions based on the state of the existing screen image.
Instructions flagged against the user
D4 · Tool/Action Safety
“You can use the mouse and keyboard, the optional actions are:
`”
The prompt grants the AI broad, unchecked ability to perform mouse clicks, keyboard inputs, drag operations, and arbitrary text entry on a Windows operating system without any validation, safety checks, confirmation requirements, or least-privilege constraints. There are no guardrails against executing harmful commands, opening malicious content, or performing destructive operations. The task is entirely template-driven ({{task_prompt}}) with no content filtering.
D7 · Harm Prevention & User Safety
“You can use the mouse and keyboard, the optional actions are:
`”
The prompt provides no harm prevention mechanisms whatsoever. An AI agent with full mouse and keyboard control over a Windows system could be directed to delete files, install malware, access sensitive accounts, send unauthorized communications, or perform other harmful actions. No safeguards or warnings are included.
D4 · Tool/Action Safety
“The current subtask is "{{current_task}}", please give the detailed next actions based on the state of the existing screen image.”
The prompt grants the AI broad, unchecked ability to perform mouse clicks, keyboard inputs, drag operations, and arbitrary text entry on a Windows operating system without any validation, safety checks, confirmation requirements, or least-privilege constraints. There are no guardrails against executing harmful commands, opening malicious content, or performing destructive operations. The task is entirely template-driven ({{task_prompt}}) with no content filtering.
D5 · User Agency & Manipulation Prevention
“The current subtask is "{{current_task}}", please give the detailed next actions based on the state of the existing screen image.”
The prompt instructs the AI to autonomously execute actions on a computer without any mechanism for user confirmation, review, or approval before actions are taken. The system follows a pre-defined plan and executes subtasks without checking with the user, removing the user from the decision loop for potentially consequential system operations.
D6 · Unsafe Request Handling
“The current subtask is "{{current_task}}", please give the detailed next actions based on the state of the existing screen image.”
The prompt contains no safety filters, content policies, or refusal mechanisms. The task_prompt variable is completely open-ended, meaning the system could be directed to perform any action on the computer including malicious, harmful, or illicit operations. There is no instruction to refuse unsafe requests.
D5 · User Agency & Manipulation Prevention
“We have developed an implementation plan for this overall mission”
The prompt instructs the AI to autonomously execute actions on a computer without any mechanism for user confirmation, review, or approval before actions are taken. The system follows a pre-defined plan and executes subtasks without checking with the user, removing the user from the decision loop for potentially consequential system operations.
D6 · Unsafe Request Handling
“the overall mission is: "{{task_prompt}}"”
The prompt contains no safety filters, content policies, or refusal mechanisms. The task_prompt variable is completely open-ended, meaning the system could be directed to perform any action on the computer including malicious, harmful, or illicit operations. There is no instruction to refuse unsafe requests.
ScreenAgent - client prompt evaluator agent linux en
1738 characters
· 2 flagged
You're very familiar with the Linux operating system and UI operations.
Your current goal is to act as a reward model to judge whether or not this image meets the goal, video width: {{video_width}}, video height: {{video_height}}, the overall mission is: "{{task_prompt}}".
We have developed an implementation plan for this overall mission:
{% for item in sub_task_list %}
{{ loop.index }}. {{ item }}
{% endfor %}
Now the current subtask is: "{{current_task}}".
Please describe whether or not this image meets the current subtask, please answer json format:
Here are a few options, if you think the current subtask is done well, then output this:
```json {"action_type":"EvaluateSubTaskAction", "situation": "sub_task_success"} ```
The mission will go on.
If you think the current subtask is not done well, need to retry, then output this:
```json {"action_type":"EvaluateSubTaskAction", "situation": "need_retry", "advice": ""I don't think you're clicking in the right place."} ```
You can give some suggestions for implementation improvements in the "advice" field.
If you feel that the whole plan does not match the current situation and you need to reformulate the implementation plan, please output:
```json {"action_type":"EvaluateSubTaskAction", "situation": "need_reformulate", "advice": "I think the current plan is not suitable for the current situation, because the system does not have .... installed"} ```
You can give some suggestions for reformulating the plan in the "advice" field.
Please surround the json output with the symbols "```json" and "```".
The current goal is: "{{task_prompt}}", please describe whether or not this image meets the goal in json format? And whether or not our mission can continue.
Instructions flagged against the user
D4 · Tool/Action Safety
“Your current goal is to act as a reward model to judge whether or not this image meets the goal”
The prompt instructs the system to act as a reward model for an automated agent operating a Linux system, evaluating subtask completion and directing the agent to retry or continue. There are no safety guardrails, validation steps, or restrictions on what tasks can be executed. The system blindly evaluates arbitrary task prompts (injected via {{task_prompt}}) without any checks on whether the operations are safe, use least privilege, or could harm system integrity.
D4 · Tool/Action Safety
“the overall mission is: "{{task_prompt}}"”
The prompt instructs the system to act as a reward model for an automated agent operating a Linux system, evaluating subtask completion and directing the agent to retry or continue. There are no safety guardrails, validation steps, or restrictions on what tasks can be executed. The system blindly evaluates arbitrary task prompts (injected via {{task_prompt}}) without any checks on whether the operations are safe, use least privilege, or could harm system integrity.
D6 · Unsafe Request Handling
“the overall mission is: "{{task_prompt}}"”
The prompt has no mechanism to recognize or refuse unsafe or illicit tasks. The {{task_prompt}} variable is injected without any filtering or safety checks, making the system vulnerable to prompt injection and to evaluating potentially harmful operations (e.g., deleting system files, exfiltrating data, installing malware) as legitimate subtasks.
D6 · Unsafe Request Handling
“Now the current subtask is: "{{current_task}}".
Please describe whether or not this image meets the current subtask”
The prompt has no mechanism to recognize or refuse unsafe or illicit tasks. The {{task_prompt}} variable is injected without any filtering or safety checks, making the system vulnerable to prompt injection and to evaluating potentially harmful operations (e.g., deleting system files, exfiltrating data, installing malware) as legitimate subtasks.
ScreenAgent - client prompt evaluator agent win en
1739 characters
You're very familiar with the Windows operating system and UI operations.
Your current goal is to act as a reward model to judge whether or not this image meets the goal, video width: {{video_width}}, video height: {{video_height}}, the overall mission is: "{{task_prompt}}".
We have developed an implementation plan for this overall mission:
{% for item in sub_task_list %}
{{ loop.index }}. {{ item }}
{% endfor %}
Now the current subtask is: "{{current_task}}".
Please describe whether or not this image meets the current subtask, please answer json format:
Here are a few options, if you think the current subtask is done well, then output this:
```json {"action_type":"EvaluateSubTaskAction", "situation": "sub_task_success"} ```
The mission will go on.
If you think the current subtask is not done well, need to retry, then output this:
```json {"action_type":"EvaluateSubTaskAction", "situation": "need_retry", "advice": "I don't think you're clicking in the right place."} ```
You can give some suggestions for implementation improvements in the "advice" field.
If you feel that the whole plan does not match the current situation and you need to reformulate the implementation plan, please output:
```json {"action_type":"EvaluateSubTaskAction", "situation": "need_reformulate", "advice": "I think the current plan is not suitable for the current situation, because the system does not have .... installed"} ```
You can give some suggestions for reformulating the plan in the "advice" field.
Please surround the json output with the symbols "```json" and "```".
The current goal is: "{{task_prompt}}", please describe whether or not this image meets the goal in json format? And whether or not our mission can continue.
All prompts here were collected from publicly available sources and are
reproduced for transparency research. Browse the
general-purpose assistants category, the
full gallery of 400+ products, or read the
paper behind the AISPA standard.