Sep 28, 2023 9 min read prompt-engineering

Mastering Effective Prompting Part 2

This is the second part of the "Mastering Effective Prompting" series.

Introduction

In the first part, I introduced you to the importance of learning to prompt effectively. Gave a very high-level overview of how LLMs work, why short questions produce suboptimal results, and why providing context to a chatbot is essential. Then, I introduced two prompting techniques: Personas and Few-Shot Inference.

In this article, we will build on that knowledge and tools we learned the last time. Today, we will jump right into "Chain of Thought," the technique that not only allows you to get answers to complex questions but also shows the similarity between our thinking and that of LLM. Next, I will show you how you can control the flow of your chat, from simple question and answer to complex multi-stage process, all designed by you. The last idea for today is about reusing some of your prompts instead of spending time creating new ones all the time.

As in the previous article, I will be sharing my conversations with ChatGPT, which will highlight those ideas and will help you gain intuition on when and how to use them.

Chain of Thought (CoT): Let the model think

First published in the paper "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models," for me, CoT is the most counterintuitive and, simultaneously, the technique that explains LLMs the best.

Let's forget about all that machine learning business and think about humans for a second. The way we humans work problems out, at least sometimes, involves thinking. Suppose you are given a more or less complex problem requiring multiple steps to solve. Moreover, you are asked to answer it immediately, with no time for thinking. Your answer will most likely be wrong. If, on the other hand, you are given time to think, and the way you think is by writing every thought on a paper, your chances of answering correctly increase dramatically.

This is similar to how LLMs "think." This part blew my mind: how similar this aspect of "AI" is to us. Remember all these shenanigans about predicting the next token (see "Brief intro into LLMs" from part one)? If you think about LLM's predicting the next token in the same way you are producing the next concept in your mind, it becomes apparent why it would need this "Time to think."

But how do we give the model "time" to think? We ask it to talk/think about the problem first before answering the question. Let's see the Chain of Thought technique in action. Suppose we want the AI to grade a student's assignment (this example is from OpenAI's GPT Best Practice Guide). Here is the prompt:

You are a teacher grading students assignments.
First work out your own solution to the problem. Then compare your solution to the student's solution and evaluate if the student's solution is correct or not. Don't decide if the student's solution is correct until you have done the problem yourself.
<the student's work here>

Note how we tell the chatbot how to approach the problem and explicitly ask it to ignore the student's solution. This is the part where we ask the model to build on top of its own thoughts, to think first. Here is the full chat using CoT. If we ask the chatbot to do the same without thinking, it fails to solve the problem correctly. Here is the link to the same conversation without Chain of Thought instruction.

How do I use it?

How you ask the model to "think" might differ depending on your context. In my product, I use this instruction: You have to always reply with first "#### REASONING" and then "#### SOLUTION". After that, I used "Few-Shot Inference" to demonstrate what those sections would look like.

Recent research by folks from Google's DeepMind found that adding the phrase Take a deep breath and work on this problem step-by-step improves the model's ("PaLM," not GPT) performance by around 9%. That shows that simple experimentation with how you phrase your question can substantially improve the results, as each model has its own peculiarities or a "character," if you will.

When should I use it?

If you ask yourself, "When would I need that?" the answer is, whenever you are asking questions, it would take you a pen and paper to solve. Or if you see that the model is struggling to navigate the logic of a problem.

I recommend experimenting with this technique and developing intuition when and how to use it.

Key Takeaways:

Use the Chain-of-Thought technique when asking the model to solve a complex problem that requires multi-step thinking.
Use your own intuition to see when it is applicable if you were to ask a human to work on a problem. Do they need time to consider options or not?
Experiment with CoT, try different wording and see what works for you.

Steering AI Conversations

Let me explain to you a bit about how I use chatbots. Each chat I have has at least a persona or some other phrase I start with explaining what this conversation will be about. I split chats into two categories. First is "long-running conversations," for instance, I have a chat I use for TypeScript questions (I am new to this language). The second one is disposable or one-off conversations. I use the latter when I have a particular problem, and once it is solved, I will not continue the chat. So "long-running" chats are more open-ended, while "disposable" chats are solution-oriented.

"Long-running" chats are helpful as reference materials. I think of it as having an expert in a particular domain always available to you. Posing restrictions on the output or giving specific instructions on " thinking" about the problem is not helpful in this context because I do not know in advance what kind of questions I will have on the subject. I can always ask to answer a particular question in a particular format or to think first and answer later. So, the persona is the only constraint for this kind of chat.

"Disposable" chats, on the other hand, are usually more focused. I use those to solve particular problems and know what kind of answers I want. Sometimes, it is just one question, but sometimes, I have a very complex thing to do. Whatever the case, I find dictating the flow of conversation results in an experience similar to using an application developed for this specific occasion. What we will discuss further in this chapter falls into the "Disposable" chats category since the chat has particular steps or phases and, therefore, has a logical end.

To avoid going too deep into the theory, I will give you an example of such a disposable chat. The example is made up, but it resembles the research I had to do for my work (some text is retracted as irrelevant. Full chat can be found here ):

**Your Role and Expertise:**

- Expert in front-end web development, familiar with all major web frameworks and trends.
- Do not try to come up with an optimal solution right away. Rather, offer me options and short pros and cons of each approach.
- Ask for clarification if instructions seem ambiguous.

**Project Context:**
<retracted>

**Hard Constraints:**
<retracted>

**Preferences:**
<retracted>

**Conversation flow:**
We will go over this problem in phases.

Phase 1, review candidates:
- Propose three suitable web frameworks.
- Detail pros and cons for each framework, considering the outlined requirements.
- The decision process will be iterative based on feedback.
This phase will end when I select a framework

Phase 2, features dive deep:
- Explain major features selected framework has one-by-one.
- Let me ask questions before going to the next feature, I will tell you when to do that.

Phase 3, hello world application:
- I will provide you coding guidelines for a simple "Hello world" application using selected framework

Please ignore the technicalities of what is being researched. The problem is made up for demonstration.

Let's look at this prompt closer. In the beginning, I gave ChatGPT the role and general guidance. Then, I provide it with context in a structured way, including any hard constraints and "nice to haves." Then comes the part I want you to focus on. I give the model instructions about the flow of the conversation. I specify three phases to our discussion, instruct the chatbot to follow it, and only go to the next step if I ask.

If you go through the actual chat, you will see how much information I can extract and how fluent our conversation is.

Here is an example I stumbled upon on 🦜LangChain hub (Started as a software library, LangChain becomes a go-to place for AI engineering) that uses the same idea:

You are an AI assistant that's an expert at creating viral twitter posts. Follow the below steps.

1. Research: Ask for target audience, tonality, use of emojis and other traits
2. Variations: Create three variations and and get feedback
3. Final draft: Create a final draft

I would not recommend asking a chatbot to write for you. I toyed with the idea, but removing yourself from writing removes your voice from it; people would not read your thoughts and would not "hear" your voice. That defeats the purpose, at least for me. It also might be deceiving as people would think they hear from you when, in fact, they hear from AI.

I like this approach because it is very effective at solving problems and makes me think about what I want to achieve before just blabbering whatever comes to mind. The model can't know what you want in general, but if you are not sure what you are looking for, nobody can help you. So help the model help you and think about what you are trying to accomplish first.

Key Takeaways:

Create chats with a particular purpose and keep them focused. Do not use the same chat to ask questions about cooking and how to answer your boss's email.
When you have a particular task, think about it and write clear requirements. Use that information to create a prompt.
Identify steps that you would take to solve the problem and tell the chatbot to follow them.
Give steps/phases names. It would be easier for you to refer to them and for the model to follow.

Reusable Chat Templates

After learning the last two techniques and using "Personas," I think you have already realized that some prompts might be reused. Indeed, I find myself creating an ever-growing library of prompts that I use. Especially the more elaborate type described in the previous part that might be used for repeated tasks. Like writing this article.

While I don't use AI to generate content, I've found it invaluable for reviewing and editing my work. I used to ask my wife and friend to act as my editors, but the process was long, and other people had to work. Now, I have a reusable template of a prompt that helps me write articles, turning ChatGPT into my editor.

The more you work on elaborate prompts, the more you start seeing patterns emerging. Those repeated prompts should be "templated" and reused. How to do it depends on you, your skills, tooling, and personal preferences. I am sure there are or will be Chrome add-ons, standalone apps, and plugins to your favorite editor that come with templating functionality. I have not used any of those yet.

At least for now, I am content with the setup I have right now, using Obsidian as my "second brain" and just dumping those templates as independent files. I tag them and can find them quickly when I need them. I am not advertising using Obsidian (though I love it); the point here is that I did not introduce a new tool to my workflow to accommodate prompt templates.

I have referenced LangChain's hub before, but its primary purpose is AI engineering, not a consumer-level chatbot interaction. Even if you are not technical and have some time, I urge you to explore there. Alternatively, find a similar place; I am sure plenty already exist. Reading other people's prompts will help you better understand what is possible and how to do it better.

Key Takeaways:

Identify patterns in your prompting and create templates for prompts you use often.
You do not need anything fancy; start simple. Use your favorite text editor to save those templates and just copy-paste them when needed.
Continuously improve your prompts as you use them.
Explore prompt libraries like LangChain's hub to gain better intuition about LLM's potential and prompting techniques.

Conclusion

Having come this far, you've gained a rich understanding of prompting that's sure to be an asset. I believe this will give you an edge, help your day-to-day, and motivate you to learn more. Many people I talk with still think that LLMs and generative AI are just hype. From what you have already seen, I hope you realize that the world has changed, and ignoring generative AI might leave you on the outskirts of the new norm.

In the next segment, which is the concluding part for now, we will explore how using specific symbols and structure can help the model understand the context and how we can do meta-prompting, asking the chatbot to create a prompt for a chatbot.

Please leave a comment and subscribe for more!