Document formatting causes sometimes messy and invalid code

[Delgan]

Environment

• OS and version: Ubuntu 20.04
• VS Code: 1.71.1
• C/C++ extension: 1.12.4
• OS and version of remote machine (if applicable): Ubuntu 20.04 (dev container)
• GDB / LLDB version: 10.2
• Clang-format version: 14.0

Bug Summary and Steps to Reproduce

Bug Summary:
From time to time, when saving and thus formatting a C++ file, the file ends up in an invalid "state". Code blocks are mixed together for an unknown reason. Different lines are intertwined with each others. I don't know how to reproduce this mistake systematically. It happens relatively rarely (once a week or so).

Screencast.from.05-10-2022.17_04_58.webm

Steps to reproduce:

1. Enable "Format on save" config
2. Modify the file, write lines, undo, redo, etc.
3. Try to save (ctrl+S)
4. Observe that file is messed up
5. Undo (ctrl+Z)
6. Save again
7. Observe that file is still messed up

Debugger Configurations

N/A

Debugger Logs

N/A

Other Extensions

Possibly related extensions:

• akiramiyakoda.cppincludeguard
• psioniq.psi-header
• cschlosser.doxdocgen
• twxs.cmake
• ms-vscode.cmake-tools

Additional Information

This problem happened to myself and a colleague of mine on two different computers, so it's not unique to my setup. However, we're sharing the same codebase and the same devcontainer, so it might be related to something in our dev environment.

If I remember correctly, calling the "Format document" command without saving also generates the error. The bug appeared about 2 months ago I would say.

It's not related to the content of the file. If I close VS Code and re-open it, then it works.

It is not easy to reproduce so unfortunately I do not have any additional information to provide. I'll try to update the ticket with logs when it reproduces.

[Delgan]

Ok, "great" news: the bug happened again while I disabled almost all my others extensions (only ms-vscode.cpptools and ms-vscode.cmake-tools remaining).
I'm thus pretty confident the bug comes from vscode-cpptools.

bug.mp4

DEBUG logs

textDocument/hover: /workspaces/software/WeedEngine/plugins/SegNet/util/model_runner.cpp (id: 9025)
Request canceled: 9025
textDocument/hover: /workspaces/software/WeedEngine/plugins/SegNet/util/model_runner.cpp (id: 9026)
cpptools/getCodeActions: /workspaces/software/WeedEngine/plugins/SegNet/util/model_runner.cpp (id: 9027)
cpptools/textEditorSelectionChange
cpptools/textEditorSelectionChange
cpptools/activeDocumentChange: /workspaces/software/WeedEngine/plugins/SegNet/util/model_runner.cpp
cpptools/getCodeActions: /workspaces/software/WeedEngine/plugins/SegNet/util/model_runner.cpp (id: 9028)
cpptools/formatDocument: /workspaces/software/WeedEngine/plugins/SegNet/util/model_runner.cpp (id: 9029)
Formatting input:
#include "model_runner.h"

#include <cuda_runtime.h>

#include <iostream>

#include "Inference/cudaUtility.h"
#include "Inference/segNetProcessing.h"
#include "WeedCom/profiler/nvtx.h"

namespace weedengine::segnet {

ModelRunner::ModelRunner(const std::string& identifier, logs::shared_logger logger)
    : m_identifier(identifier), m_timer(identifier), m_logger(std::move(logger))
{}

void ModelRunner::configure(int deviceId,
                            const std::filesystem::directory_entry& cacheDir,
                            const cv::Size& imageDimensions,
                            const cv::Rect& imageROI,
                            const weedcore::weedcom::CameraIntrinsic& cameraIntrinsic,
                            const weedcore::weedcom::CameraDistortion& cameraDistortion,
                            bool useDistortion,
                            bool classifyOutput)
{
    m_deviceId = deviceId;
    m_cacheDir = cacheDir;
    m_imageDimensions = imageDimensions;
    m_imageROI = imageROI;
 …
Formatting document: file:///workspaces/software/WeedEngine/plugins/SegNet/util/model_runner.cpp
Formatting Engine: clangFormat
Formatting raw output:
#include "model_runner.h"

#include <cuda_runtime.h>

#include <iostream>

#include "Inference/cudaUtility.h"
#include "Inference/segNetProcessing.h"
#include "WeedCom/profiler/nvtx.h"

namespace weedengine::segnet {

ModelRunner::ModelRunner(const std::string& identifier, logs::shared_logger logger)
    : m_identifier(identifier), m_timer(identifier), m_logger(std::move(logger))
{}

void ModelRunner::configure(int deviceId,
                            const std::filesystem::directory_entry& cacheDir,
                            const cv::Size& imageDimensions,
                            const cv::Rect& imageROI,
                            const weedcore::weedcom::CameraIntrinsic& cameraIntrinsic,
                            const weedcore::weedcom::CameraDistortion& cameraDistortion,
                            bool useDistortion,
                            bool classifyOutput)
{
    m_deviceId = deviceId;
    m_cacheDir = cacheDir;
    m_imageDimensions = imageDimensions;
    m_imageROI = imageROI;
 …
Formatting diffed output:
  .toStdString() m_logger->debug("Attempting to open cache file '{}'.", cachePath);


textDocument/didChange: /workspaces/software/WeedEngine/plugins/SegNet/util/model_runner.cpp
cpptools/textEditorSelectionChange
cpptools/getFoldingRanges: /workspaces/software/WeedEngine/plugins/SegNet/util/model_runner.cpp (id: 9030)
cpptools/getCodeActions: /workspaces/software/WeedEngine/plugins/SegNet/util/model_runner.cpp (id: 9031)
cpptools/getSemanticTokens: /workspaces/software/WeedEngine/plugins/SegNet/util/model_runner.cpp (id: 9032)
cpptools/getInlayHints: /workspaces/software/WeedEngine/plugins/SegNet/util/model_runner.cpp (id: 9033)
Checking for syntax errors: /workspaces/software/WeedEngine/plugins/SegNet/util/model_runner.cpp
Queueing IntelliSense update for files in translation unit of: /workspaces/software/WeedEngine/plugins/SegNet/util/model_runner.cpp
  tag parsing file: /workspaces/software/WeedEngine/plugins/SegNet/util/model_runner.cpp
sending 1 changes to server
cpptools/finishUpdateSquiggles
Error squiggle count: 0
Update IntelliSense time (sec): 2.333
textDocument/hover: /workspaces/software/WeedEngine/plugins/SegNet/util/model_runner.cpp (id: 9034)
cpptools/getFoldingRanges: /workspaces/software/WeedEngine/plugins/SegNet/util/model_runner.cpp (id: 9035)
textDocument/hover: /workspaces/software/WeedEngine/plugins/SegNet/util/model_runner.cpp (id: 9036)
cpptools/textEditorSelectionChange
textDocument/documentHighlight: /workspaces/software/WeedEngine/plugins/SegNet/util/model_runner.cpp (id: 9037)
cpptools/textEditorSelectionChange
cpptools/textEditorSelectionChange
cpptools/textEditorSelectionChange
cpptools/textEditorSelectionChange
cpptools/textEditorSelectionChange
cpptools/textEditorSelectionChange
cpptools/textEditorSelectionChange
cpptools/textEditorSelectionChange
cpptools/textEditorSelectionChange
cpptools/textEditorSelectionChange
cpptools/textEditorSelectionChange
cpptools/textEditorSelectionChange
cpptools/textEditorSelectionChange
cpptools/getCodeActions: /workspaces/software/WeedEngine/plugins/SegNet/util/model_runner.cpp (id: 9038)
textDocument/hover: /workspaces/software/WeedEngine/plugins/SegNet/util/model_runner.cpp (id: 9039)
textDocument/hover: /workspaces/software/WeedEngine/plugins/SegNet/util/model_runner.cpp (id: 9040)
cpptools/textEditorSelectionChange
cpptools/getCodeActions: /workspaces/software/WeedEngine/plugins/SegNet/util/model_runner.cpp (id: 9041)
cpptools/getDocumentSymbols: /workspaces/software/WeedEngine/plugins/SegNet/util/model_runner.cpp (id: 9042)
cpptools/getDocumentSymbols

Extension settings

{
    "C_Cpp.intelliSenseEngine": "Default",
    "C_Cpp.codeAnalysis.clangTidy.enabled": true,
    "C_Cpp.codeAnalysis.runAutomatically": false,
    "C_Cpp.clang_format_path": "/usr/bin/clang-format",
    "C_Cpp.autoAddFileAssociations": false,
    "C_Cpp.loggingLevel": "Debug"
}

If I manually run clang-format on the file, there is no issue.

I observed that the bug "degenerates" if lines are modified around. It starts around 0:40 of the following video, when I delete chars, then press Ctrl+Z, and then try to format the file again.

bug2.mp4

Please let me know what I can do more to help investigate this issue. 👍

[sean-mcmanus]

If you set C_Cpp.loggingLevel to "Debug" and check the C/C++ logging pane for formatting logging can you locate at which point the garbage text is occurring? My guess is that the garbage text will appear in the input of formatting (because our internal document object got corrupted somehow).

[sean-mcmanus]

Also, are you opening your workspace folder via a symlink? If so, you're hitting bug #5061 .

[Delgan]

@sean-mcmanus I attached the "Debug" logs in my second message, does it contain what you're looking for (I don't know exactly what is the "garbage text")?

The workspace is not opened with a symlink. It's a local folder which is then opened inside a devcontainer.

[sean-mcmanus]

Oh, the logging gets cut off after 1024 characters (we've increased that limit in our next release). Are you able to get a repro with < 1024 characters?

[Delgan]

Hey @sean-mcmanus, the bug happened today on another file. I removed code above and below the garbage text to limit the file size.

It seems your guess was right, the garbage text appears in the input for some unknown reason.

File:

#include "segNet.h"

bool SegNet::updateInferenceSettings(
    const weedcore::weedcom::InferenceSettings& inference)
{
    m_modelPath =
        std::visit([](auto&& arg) -> std::string { // All model types share the a "modelPath" argument.
        return arg.modelPath(); }, m_model);

    return true;
}

Logs:

Update IntelliSense time (sec): 0.263
cpptools/getSemanticTokens: /workspaces/software/WeedEngine/plugins/plugin_inference_downs/segNet.cpp (id: 15139)
cpptools/getFoldingRanges: /workspaces/software/WeedEngine/plugins/plugin_inference_downs/segNet.cpp (id: 15140)
cpptools/getCodeActions: /workspaces/software/WeedEngine/plugins/plugin_inference_downs/segNet.cpp (id: 15141)
cpptools/formatDocument: /workspaces/software/WeedEngine/plugins/plugin_inference_downs/segNet.cpp (id: 15142)
Formatting input:
#include "segNet.h"

bool SegNet::updateInferenceSettings(
    const weedcore::weedcom::InferenceSettings& inference)
{
    m_modelPath =
        std::visit([](auto&& arg) -> std::string { // All model types share the a "modelPath" argument.
        return arg.modelPath();e},;m_mod l)  return true;
}

Formatting document: file:///workspaces/software/WeedEngine/plugins/plugin_inference_downs/segNet.cpp
Formatting Engine: clangFormat
Formatting raw output:
#include "segNet.h"

bool SegNet::updateInferenceSettings(
    const weedcore::weedcom::InferenceSettings& inference)
{
    m_modelPath = std::visit(
        [](auto&& arg)
            -> std::string {  // All model types share the a "modelPath" argument.
            return arg.modelPath();
            e
        },
        ;
        m_mod l) return true;
}

Formatting diffed output:
 std::visit(
        [](auto&& arg)
            -> std::string {  // All model types share the a "modelPath" argument.
            return arg.modelPath();
            e
        },
        ;
        m_mod l) return true;

textDocument/didChange: /workspaces/software/WeedEngine/plugins/plugin_inference_downs/segNet.cpp
cpptools/textEditorSelectionChange
cpptools/getFoldingRanges: /workspaces/software/WeedEngine/plugins/plugin_inference_downs/segNet.cpp (id: 15143)
cpptools/getCodeActions: /workspaces/software/WeedEngine/plugins/plugin_inference_downs/segNet.cpp (id: 15144)
cpptools/getSemanticTokens: /workspaces/software/WeedEngine/plugins/plugin_inference_downs/segNet.cpp (id: 15145)
cpptools/getInlayHints: /workspaces/software/WeedEngine/plugins/plugin_inference_downs/segNet.cpp (id: 15146)
idle loop: reparsing the active document
Checking for syntax errors: /workspaces/software/WeedEngine/plugins/plugin_inference_downs/segNet.cpp
Queueing IntelliSense update for files in translation unit of: /workspaces/software/WeedEngine/plugins/plugin_inference_downs/segNet.cpp
sending 14 changes to server
cpptools/finishUpdateSquiggles
Error squiggle count: 3
Checking for syntax errors: /workspaces/software/WeedEngine/plugins/plugin_inference_downs/segNet.cpp
Queueing IntelliSense update for files in translation unit of: /workspaces/software/WeedEngine/plugins/plugin_inference_downs/segNet.cpp
cpptools/getFoldingRanges: /workspaces/software/WeedEngine/plugins/plugin_inference_downs/segNet.cpp (id: 15147)
cpptools/getCodeActions: /workspaces/software/WeedEngine/plugins/plugin_inference_downs/segNet.cpp (id: 15148)
cpptools/finishUpdateSquiggles
Error squiggle count: 3
Update IntelliSense time (sec): 2.279
cpptools/getSemanticTokens: /workspaces/software/WeedEngine/plugins/plugin_inference_downs/segNet.cpp (id: 15149)
cpptools/textEditorSelectionChange
cpptools/getFoldingRanges: /workspaces/software/WeedEngine/plugins/plugin_inference_downs/segNet.cpp (id: 15150)
cpptools/textEditorSelectionChange
cpptools/textEditorSelectionChange
Database safe to open
  tag parsing file: /workspaces/software/WeedEngine/plugins/plugin_inference_downs/segNet.cpp
cpptools/textEditorSelectionChange
cpptools/textEditorSelectionChange
cpptools/textEditorSelectionChange
cpptools/textEditorSelectionChange
cpptools/textEditorSelectionChange
cpptools/textEditorSelectionChange
cpptools/textEditorSelectionChange
cpptools/textEditorSelectionChange
cpptools/textEditorSelectionChange
cpptools/getCodeActions: /workspaces/software/WeedEngine/plugins/plugin_inference_downs/segNet.cpp (id: 15151)
cpptools/getDocumentSymbols: /workspaces/software/WeedEngine/plugins/plugin_inference_downs/segNet.cpp (id: 15152)
cpptools/getDocumentSymbols
Database safe to open

Proof:

bugg.mp4

[Kidsunbo]

I have encountered the same bug and it first happened weeks ago. The lint also shows some weired message, but everything works fine when I reload the window.

[chengfzy]

Same problem

[sean-mcmanus]

It appears that something is causing our internal document object (input to formatting) to be into an invalid state.

Is everyone who is reproing this on Linux?

Does downgrading to 1.11.5 "fix" the issue? It would help to narrow down the potential changes that could have regressed this.

Has anyone reproed this with 1.13.3? My guess is that it still would since we're not aware of fixing it.

[karl-lunarg]

I've been seeing the same problem on Windows with the same observations and experiences noted here already. It is hard to force reproduction, but it seems there is already a good repro case here. I'll try to pay attention if it happens again and try things in order to help with the other questions.

[Delgan]

Hey @sean-mcmanus.

I've been using v1.11.5 for the last two weeks and the issue did not occur once.

[alanbirtles]

This is happening to me again recently too, restarting vscode always fixes the issue. I'm using Linux.

[sean-mcmanus]

@Delgan It seems like some regression from a change in 1.12.0 to 1.12.3, but I looked through the changes and didn't see anything in particular that could cause it. By the time the format happens, the document is corrupted already, so it must be some previous operations that are corrupting the internal document. Does anyone know any good repro steps for corrupting the internal document? I can't seem to get a repro via edits.

[Delgan]

I spent over half an hour the other day doing edit/undo/redo in a file without observing any problem. It's difficult to know how to reproduce the error, it seems to happen randomly in the middle of a coding session.

@sean-mcmanus What makes you think the regression was introduced between 1.12.0 and 1.12.3 and not between 1.11.5 -> 1.12.0 or 1.12.3 -> 1.12.4? Do you want me to try a different version to possibly help "bisecting" the regression? Or does it exist some debugging .vsix version of the extension that I could use to trace absolutely everything happening?

[sean-mcmanus]

@Delgan My guess from reviewing the changes is that the bug doesn't repro in 1.12.0 because that only has some bug fixes that we don't believe have the potential to cause this regression -- if someone were able to get a repro with that version, it would certainly reduce the possibilities further.

1.12.1 added doxygen comment generation and code analysis fix formatting -- both of which theoretically could cause the bug, even though I don't see exactly how.

We don't currently have a debug version that could get more info -- we're thinking about that possibility.

[Delgan]

Hi @sean-mcmanus, I have some news.

I've been using v1.12.1 for more than one week without issue.
This morning I switched to v1.12.3 and I faced the problem after a few hours.
Then I switched to v1.12.2 and the issue appeared again.

Thus, I would say the regression was introduced between v1.12.1 and v1.12.2.
I'll switch back to v1.12.1 to ensure it wasn't just luck.

[sean-mcmanus]

This should be fixed with 1.13.6: https://github.com/microsoft/vscode-cpptools/releases/tag/v1.13.6 . The bug should have started reproing with 1.12.0.

[karl-lunarg]

This was the sort of problem that required a few hours of "normal" usage before the symptoms started to show. Today, I've been running with v1.13.6 all day and have seen no problems so far. Great job! And thank you!