Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Formatter: string_processing preview style #6936

Open
Tracked by #6935 ...
MichaReiser opened this issue Aug 28, 2023 · 7 comments
Open
Tracked by #6935 ...

Formatter: string_processing preview style #6936

MichaReiser opened this issue Aug 28, 2023 · 7 comments
Labels
formatter Related to the formatter preview Related to preview mode features

Comments

@MichaReiser
Copy link
Member

MichaReiser commented Aug 28, 2023

Add support for Black's improved string processing.

Black will split long string literals and merge short ones. Parentheses are used where appropriate. When split, parts of f-strings that don’t need formatting are converted to plain strings. User-made splits are respected when they do not exceed the line length limit. Line continuation backslashes are converted into parenthesized strings. Unnecessary parentheses are stripped. The stability and status of this feature is tracked in this issue.

Examples

"a somewhat long string that doesn't fit the line length. " "Testing to see what black does keep it a bit longer and longer",
"a somewhat long string that doesn't fit the line length. " \
    "Testing to see what black does keep it a bit longer and longer",

if "a somewhat long string that doesn't fit the line length. " + "Testing to see what black does keep it a bit longer and longer longer":
    pass

"a somewhat long string that doesn't fit the line length. " + "Testing to see what black does keep it a bit longer and longer longer longer longer longer longer longer longer longer longer longer"

("a somewhat long string that doesn't fit the line length. " + "Testing to see what black does keep it a bit longer and longer longer longer longer longer longer longer longer longer longer longer")

"a somewhat long string that doesn't fit the line length. " + ["aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", "Ccccccccccccccccccccccccccccccccccccccc cccccccccccccccccccccccccccccccccccccccccccccccccccccc"]

call(
    "a somewhat long string that doesn't fit the line length. "
    "Testing to see what black does keep it a bit longer and longer",
    "a second argument",
    3,
    4,
)


call(
    "a somewhat long string that doesn't fit the line length. Testing to see what black does keep "
    "it a bit longer and longer",
    "a second argument",
    3,
    4,
)

call(
    "a somewhat long string that doesn't fit the line length. "
    "shorter",
    "a second argument",
    3,
    4,
)


L1 = ["The is a short string", "This is a really long string that can't possibly be expected to fit all together on one line. Also it is inside a list literal, so it's expected to be wrapped in parens when spliting to avoid implicit str concatenation.", short_call("arg", {"key": "value"}), "This is another really really (not really) long string that also can't be expected to fit on one line and is, like the other string, inside a list literal.", ("parens should be stripped for short string in list")]
--- /home/micha/.config/JetBrains/PyCharmCE2023.2/scratches/scratch.py  2023-08-28 09:15:01.476535+00:00
+++ /home/micha/.config/JetBrains/PyCharmCE2023.2/scratches/scratch.py  2023-08-28 09:15:02.026782+00:00
@@ -1,17 +1,36 @@
-"a somewhat long string that doesn't fit the line length. " "Testing to see what black does keep it a bit longer and longer",
-"a somewhat long string that doesn't fit the line length. " \
-    "Testing to see what black does keep it a bit longer and longer",
+"a somewhat long string that doesn't fit the line length. "
+"Testing to see what black does keep it a bit longer and longer",
+"a somewhat long string that doesn't fit the line length. "
+"Testing to see what black does keep it a bit longer and longer",
 
-if "a somewhat long string that doesn't fit the line length. " + "Testing to see what black does keep it a bit longer and longer longer":
+if (
+    "a somewhat long string that doesn't fit the line length. "
+    + "Testing to see what black does keep it a bit longer and longer longer"
+):
     pass
 
-"a somewhat long string that doesn't fit the line length. " + "Testing to see what black does keep it a bit longer and longer longer longer longer longer longer longer longer longer longer longer"
+(
+    "a somewhat long string that doesn't fit the line length. "
+    + "Testing to see what black does keep it a bit longer and longer longer longer"
+    " longer longer longer longer longer longer longer longer"
+)
 
-("a somewhat long string that doesn't fit the line length. " + "Testing to see what black does keep it a bit longer and longer longer longer longer longer longer longer longer longer longer longer")
+(
+    "a somewhat long string that doesn't fit the line length. "
+    + "Testing to see what black does keep it a bit longer and longer longer longer"
+    " longer longer longer longer longer longer longer longer"
+)
 
-"a somewhat long string that doesn't fit the line length. " + ["aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb", "Ccccccccccccccccccccccccccccccccccccccc cccccccccccccccccccccccccccccccccccccccccccccccccccccc"]
+"a somewhat long string that doesn't fit the line length. " + [
+    "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
+    "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
+    (
+        "Ccccccccccccccccccccccccccccccccccccccc"
+        " cccccccccccccccccccccccccccccccccccccccccccccccccccccc"
+    ),
+]
 
 call(
     "a somewhat long string that doesn't fit the line length. "
     "Testing to see what black does keep it a bit longer and longer",
     "a second argument",
@@ -19,22 +38,35 @@
     4,
 )
 
 
 call(
-    "a somewhat long string that doesn't fit the line length. Testing to see what black does keep "
-    "it a bit longer and longer",
+    "a somewhat long string that doesn't fit the line length. Testing to see what black"
+    " does keep it a bit longer and longer",
     "a second argument",
     3,
     4,
 )
 
 call(
-    "a somewhat long string that doesn't fit the line length. "
-    "shorter",
+    "a somewhat long string that doesn't fit the line length. shorter",
     "a second argument",
     3,
     4,
 )
 
 
-L1 = ["The is a short string", "This is a really long string that can't possibly be expected to fit all together on one line. Also it is inside a list literal, so it's expected to be wrapped in parens when spliting to avoid implicit str concatenation.", short_call("arg", {"key": "value"}), "This is another really really (not really) long string that also can't be expected to fit on one line and is, like the other string, inside a list literal.", ("parens should be stripped for short string in list")]
\ No newline at end of file
+L1 = [
+    "The is a short string",
+    (
+        "This is a really long string that can't possibly be expected to fit all"
+        " together on one line. Also it is inside a list literal, so it's expected to"
+        " be wrapped in parens when spliting to avoid implicit str concatenation."
+    ),
+    short_call("arg", {"key": "value"}),
+    (
+        "This is another really really (not really) long string that also can't be"
+        " expected to fit on one line and is, like the other string, inside a list"
+        " literal."
+    ),
+    "parens should be stripped for short string in list",
+]

Observations:

  • Binary operators break before the strings
  • The formatting only applies to implicit concatenated strings, but not strings joined with the + operator

The strings must be parenthesized in some contexts, see https://github.com/psf/black/pull/3162/files

@MichaReiser MichaReiser added the formatter Related to the formatter label Aug 28, 2023
@MichaReiser MichaReiser added this to the Formatter: Stable milestone Aug 28, 2023
@MichaReiser
Copy link
Member Author

MichaReiser commented Aug 28, 2023

My first reaction is that this is similar to Prettier's/Rome's JSX formatting and that we need to use BestFitting with three variants:

  1. Join all string parts into a single string to make it fit
  2. Keep the same string parts as in the source. Test if all lines fit
  3. Use fill see prototype to fit as many words as possible on the line.

The main concern with this is performance:

  • We should avoid normalizing the same strings multiple times. This is already one of the most expensive functions today
  • We use BestFitting for the expression BestFit layout. Using BestFitting for strings would result in a nested BestFitting layout with exponential complexity. Can we remove the need to use BestFit` for Strings altogether?

@MichaReiser MichaReiser self-assigned this Sep 7, 2023
@MichaReiser MichaReiser added the preview Related to preview mode features label Sep 15, 2023
@MichaReiser MichaReiser removed their assignment Oct 27, 2023
@MichaReiser MichaReiser changed the title Improved string processing Improved string processing (string_processing) Nov 29, 2023
@MichaReiser MichaReiser changed the title Improved string processing (string_processing) Formatter: string_processing preview style Nov 29, 2023
@MichaReiser MichaReiser removed this from the Formatter: Stable milestone Nov 29, 2023
@KotlinIsland
Copy link
Contributor

Black no longer wraps implicitly concatenated strings used as func args in parens, due to this comment.
There is an open issue to re-enable it: psf/black#3955

@DetachHead
Copy link

fyi in case anyone was confused and thought this feature was removed in black version 24.1 like i did, the string processing now has to be enabled with --enable-unstable-feature=string_processing in addition to the --preview flag.

@MichaReiser
Copy link
Member Author

Black considers to not move forward with the string_processing preview style (issue)

@KotlinIsland
Copy link
Contributor

KotlinIsland commented Jun 27, 2024

A simpler aspect of this larger problem is joining strings that get put onto the same line. It constantly annoys me and would love to see it resolved:

# Before
(
    ""
    ""
)
# After
("" "")
# Desired
("") 
# OR
""

@MichaReiser
Copy link
Member Author

@KotlinIsland we agree. A solution for this is described in #9457

@raayu83
Copy link

raayu83 commented Sep 24, 2024

Would love to see this feature coming!
When auto creating new Django migrations for models with long help text, I always get files with very long strings on a single line.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
formatter Related to the formatter preview Related to preview mode features
Projects
None yet
Development

No branches or pull requests

4 participants