Prompt fix for empty intermediate steps in summarization (#660)

Adding quotation marks around {text} avoids generating empty or completely random responses from OpenAI davinci-003. Empty or completely unrelated intermediate responses in summarization messes up the final result or makes it very inaccurate. The error from OpenAI would be: "The model predicted a completion that begins with a stop sequence, resulting in no output. Consider adjusting your prompt or stop sequences." This fix corrects the prompting for summarization chain. This works on API too, the images are for demonstrative purposes. This approach can be applied to other similar prompts too. Examples: 1) Without quotation marks ![Screenshot from 2023-01-20 07-18-19](https://user-images.githubusercontent.com/22897470/213624365-9dfc18f9-5f3f-45d2-abe1-56de67397e22.png) 2) With quotation marks ![Screenshot from 2023-01-20 07-18-35](https://user-images.githubusercontent.com/22897470/213624478-c958e742-a4a7-46fe-a163-eca6326d9dae.png)
langchain-ai · Jan 20, 2023 · 65f3a34 · 65f3a34
1 parent 69998b5
commit 65f3a34
Show file tree

Hide file tree

Showing 3 changed files with 3 additions and 3 deletions.
diff --git a/langchain/chains/summarize/map_reduce_prompt.py b/langchain/chains/summarize/map_reduce_prompt.py
@@ -4,7 +4,7 @@
 prompt_template = """Write a concise summary of the following:
 
 
-{text}
+"{text}"
 
 
 CONCISE SUMMARY:"""

diff --git a/langchain/chains/summarize/refine_prompts.py b/langchain/chains/summarize/refine_prompts.py
@@ -21,7 +21,7 @@
 prompt_template = """Write a concise summary of the following:
 
 
-{text}
+"{text}"
 
 
 CONCISE SUMMARY:"""

diff --git a/langchain/chains/summarize/stuff_prompt.py b/langchain/chains/summarize/stuff_prompt.py
@@ -4,7 +4,7 @@
 prompt_template = """Write a concise summary of the following:
 
 
-{text}
+"{text}"
 
 
 CONCISE SUMMARY:"""
-Original file line number
+Diff line change
@@ Expand Up / @@ -4,7 +4,7 @@ @@
     prompt_template = """Write a concise summary of the following:
-    {text}
+    "{text}"
     CONCISE SUMMARY:"""
@@ Expand Down @@