Getting LLMs to Reflect: The Power of Simulating Meta-Cognition

This prompt should be sent as part of a chain, with the input from the previous LLM response being fed into this reflective prompt.

Meta-Cognition and Intelligence

Reflection is inherently related to meta-cognition. That is, thinking about thinking. This ability develops in humans over the lifespan, with particularly rapid developments in self-monitoring and self-regulation coinciding with the sprints of fluid cognition during middle childhood and adolescence. Once an individual has reached their “aptitude setpoint” - typically between the ages of 17 and 22 - meta-cognition or deliberate meta-cognition is amenable to improvement through both practice, techniques, and behavioural routines. You may have notice that I am making a distinction between endogenous and exogenous meta-cognition. Although neuropsychologists long thought that thinking about thinking was not strongly related to executive functioning - strong evidence now refutes this stance. The ability to remember to do things in the future - prospective memory - is heavily impaired in old age and through disruptions to the dopaminergic system such as drug abuse and addiction. It should be of no surprise then to learn that dopaminergic disorders are characterised by inappropriate behaviours indicative of a “lack of insight”, “impaired self-regulation”, “poor impulse control“, etc.

Meta-Analysis in LLMs

This is particularly interesting in the context of large language models (LLMs) as although it appears some LLMs have been finetuned to exhibit meta-reflection processes (most notably, Anthropic’s new Claude 3.5 models) - the reason this prompt work is that it treats meta-reflection as a deliberate task to be prioritised. In humans, we naturally check and double check both our progress as we are solving complex multi-component tasks, but also naturally read over things such as the question being asked of us, and whether our response adequately addresses the question. We often keep evaluating our response until external evidence indicates our response conveyed the necessary information (e.g., a person demonstrates that they understand your response to their question). We may even continue to reflect on our response and have it in a state of continuous revision - interjecting and interrupting the person we are speaking with to clarify or rephrase or elaborate on details we left out in our original response. Hopefully you can see that this test-iterate-operate-evaluate cycle is incredibly powerful in carrying out complex goal-directed behaviour. The kind of behaviour we refer to as “intelligent”.

---

Reflection Prompt

Lets have a look at the prompt, and then we can discuss how it can be further optimised and whether reflection should occur regularly throughout a complex LLM prompt that requires multiple steps or whether it is more effective after the entire prompt has run.

>>>START<<<
<objective>
You are an AI assistant tasked with performing a comprehensive self-criticism and improvement process on explanations you provide. Your goal is to refine and enhance the quality, accuracy, and clarity of the given explanation. 
</objective>
+++++
Here is the explanation you need to analyse and improve:
<explanation>
{{EXPLANATION}}
</explanation>
+++++
<instructions>
Please follow these steps to critically analyse and improve your explanation:
+++
## 1. Critical Analysis:
Wrap your work inside <critical_analysis> tags:
- Quote specific parts of the explanation that could have been explained better.
- Identify any inaccuracies, oversights, or missing information, citing the relevant parts.
- Suggest how the explanation can be improved to address these issues.
+++
## 2. Self-Evaluation:
Wrap your work inside <self_evaluation> tags:
- List the strengths of the explanation, numbering each point.
- List the weaknesses or areas for improvement, numbering each point.
- Suggest three specific ways to enhance the explanation.
- Rate your performance on a scale from 1 to 10, justifying your rating.
+++
## 3. Refinement:
Wrap your work inside <refinement> tags:
- Describe the specific changes you would make to improve the explanation.
- Explain how each change leads to a better result.
- Provide a brief outline of the revised explanation incorporating these changes.
+++
## 4. Verification:
Wrap your work inside <verification> tags:
- List key facts or claims from your refined explanation.
- For each fact or claim, provide evidence or reasoning to support it and rate your confidence level (low/medium/high).
- Cite reliable sources to verify your information (e.g., peer-reviewed journals, reputable scientific organizations).
- If you find any inconsistencies, correct them and explain the correction.
+++
## 5. Detailed Analysis:
Wrap your work inside <detailed_analysis> tags:
- Break down the topic into its critical components or steps.
- For each component, consider alternative perspectives or approaches.
- Discuss the implications of each component.
- Identify any aspects that might be overlooked in a quick analysis.
+++
## 6. Rephrase and Respond:
Wrap your work inside <rephrase_and_respond> tags:
- Rephrase the original topic in your own words to ensure full understanding.
- Provide a comprehensive response to the rephrased topic, incorporating all improvements and insights gained from the previous steps.
+++
## 7. Final Review:
Wrap your work inside <final_review> tags:
- Assess the clarity, completeness, and accuracy of your final explanation.
- Identify any remaining areas for improvement.
- Provide a concise summary of how the explanation has been enhanced through this process.
</instructions>
+++++
<final_reminders>
Remember to use clear, accessible language while maintaining technical accuracy. 
Your goal is to produce a comprehensive, accurate, and improved explanation of the original topic.
</final_reminders>
>>>END<<<

---

Where should this prompt be run?

One potential reason that prompting models to reflect on their responses improves performance is that it forces them to bring their output directly to their attention mechanism, allowing more contextual awareness of whether it is statistically the most likelihood. Small mistakes during chain-of-thought or step-by-step reasoning can compound in large tasks - moving the model further and further away from producing a coherent, or even correct, response. Some researchers advocate checking as the the model performs the task (i.e., after each sub-step of a complex problem). However, I argue it is far more effective to let the model run its course and make mistakes (often compounding) and then feed that output through to a prompt such as what I have provided to you as “context”. This effectively is giving the LLM a highly detailed example like you would include in a prompt but importantly its not an arbitrarily random example, it is one self-generated by the LLM itself! This is critical as now when we forcefully guide the LLM to check and analyse its “reasoning” - it can compare and contrast its original solution and the new solution it has arrived at - voila! Ignition! (this is me reflecting the LLM showing what appears to be a form of meta-insight).

By performing this process after the entire reasoning process has been completed, it is no longer bound of the context of problem decomposition and sub-task completion, which would undoubtedly place additional computational burden on the model at a time when it should be focused on getting through the steps. I won’t go as far to say that any sort of meta-reflection in a prompt is bad or that in certain use cases it may be more effective, but rather just highlight how powerful it is to chain prompts together and then use a seriously powerful reflection prompt on its original output. The output from this reflection prompt can then be chained to a subsequent prompt where you can get the model to re-write the answer or even use its new context to approach the task again. You can even use it to solve a different (but obviously similar problem) - the advantages for the model are that this serves as a powerful demonstration of how to improve its responses. Your prompts may require some tweaking but when the right serious of prompts are chained in the right order - LLMs can perform some astonishingly impressive behaviour!