Ma et al (2024) argue for a specific approach they call ROPE. Requirement-Oriented Prompt Engineering (ROPE). It emphasises the crucial role of precise and complete requirements in prompts, when interacting with Large Language Models (LLMs).
If you’re trying to get an AI model like ChatGPT to do a specific, complex job, we have already seen that simply giving it a basic instruction might not work well. The key to getting better results lies in focusing on clearly defining your *requirements* first.
In the Requirement-Oriented Prompt Engineering (ROPE) view of the world, requirements are the essential rules or conditions that you want the Aoutput to meet. They are like a blueprint for the LLM. Examples include “make the response under 100 words,” “write in a formal tone,” or “include specific data points.” These requirements guide the AI to perform tasks exactly how you need them, potentially deviating from its standard way of responding.
Getting these requirements right is crucial. They need to be accurate and complete. Poorly defined requirements can lead to problems. For instance, if you forget to specify “remove personal names,” the AI might accidentally expose private information. If you vaguely ask it to “delete harmful content” without defining “harmful,” it might misunderstand and remove content about sensitive but important topics like mental health. This summary focuses on requirements written in plain language, as that’s how most people interact with these AI models.
In the ROPE approach, a “prompt” is the full instruction you give to the AI. ROPE views a prompt as containing your specific requirements *plus* other elements. These other elements might include making the text flow nicely or using standard tricks known to work well with certain AI models. The crucial difference is that requirements are about your goals and usually stay the same regardless of which AI model you use (like GPT-4 or Gemini). Clearly stating these is vital because the AI often can’t guess missing details unique to your task.
Other parts of the prompt (like preferred phrasing or example styles) are often more specific to the AI model itself. These aspects can change as models evolve and are often better handled by automated tools (“prompt optimisers”). The idea behind ROPE is that users should focus their effort on defining clear requirements (delegating the task specifics), and let automated tools handle the fine-tuning for each specific AI model.
ROPE becomes particularly important for what the authors call “LLM-hard” tasks. These are tasks that require significant customisation or complex steps, where the AI needs clear guidance to deviate from its default behaviour. Examples include multi-step processes, tasks needing specific context, or unique decision-making rules. As AI models improve, what counts as “hard” might change.
Today, many people are creating reusable “prompt programs” to automate tasks or generate specific outputs. These often involve customised, “LLM-hard” tasks. ROPE argues that anyone building these kinds of customised prompts would benefit from being better at defining requirements.
While it seems logical that users would naturally learn to refine their requirements through trial and error, it’s often difficult in practice. Beginners might not even understand what constitutes a requirement. Even experienced users can struggle to translate a general feeling (“I don’t like the AI’s introduction”) into a clear, actionable requirement (“Introduce yourself and state your purpose at the start”). Deciding the right level of detail is also tricky.
Because of these challenges, the authors argue that users need specific guidance and training focused on prioritising and articulating requirements effectively when creating prompts. Good training should emphasise the importance of clear, complete requirements and help users develop the skill of translating their abstract goals into concrete instructions for the AI.
The art (and I do think it is an art at present and not a science!) is in its infancy. As well as improving the technology itself, we need to learn to how to improve the way we ask questions of our LLMs.
Further Reading
- Ma, Q., Peng, W., Shen, H., Koedinger, K., & Wu, T. (2024). What you say= what you want? Teaching humans to articulate requirements for LLMs. arXiv preprint arXiv:2409.08775.