The ability of GPT-4 with a med prompt to beat googles dedicated med model is fascinating and comparing that with a GPT-4 simple prompt is a great illustration in the power of prompting.
I have struggled to find a reliable method of knowing which is the best prompt. From my experience with testing prompts however the best tends to be the simpler 0 shot prompts with a chain of thought style added.
It's incredible in the improved performance when simple sentences such as "think carefully about this" is added to the prompt.
Part of the issue here I think is that the labs are constantly trying to improve this by and more variations to prompts to post training and maybe changing the "best prompts" over time. The question for me though is whether this solves the problem or just papers over it
The ability of GPT-4 with a med prompt to beat googles dedicated med model is fascinating and comparing that with a GPT-4 simple prompt is a great illustration in the power of prompting.
I have struggled to find a reliable method of knowing which is the best prompt. From my experience with testing prompts however the best tends to be the simpler 0 shot prompts with a chain of thought style added.
It's incredible in the improved performance when simple sentences such as "think carefully about this" is added to the prompt.
Part of the issue here I think is that the labs are constantly trying to improve this by and more variations to prompts to post training and maybe changing the "best prompts" over time. The question for me though is whether this solves the problem or just papers over it