support running from within scripts or IDEs

[protobits]

I'm running catkin from qtcreator as a custom build step and it complains about:

tput: No value for $TERM and no -T specified

I'm using the following parameters: --no-deps -v -i --no-color --no-status

[wjwwood]

It's trying to determine the terminal width, you'll need to give me more information on how it fails. Does it fail? or is it just printing that, but otherwise working?

This is the code where tput is being called:

https://github.com/catkin/catkin_tools/blob/master/catkin_tools/common.py#L251-L255

[jbohren]

@v01d

Does this help?

http://stackoverflow.com/a/14989071

[protobits]

Yes, thanks. It does not complain anymore now.
In case it is useful for you I believe a color code is still being printed
since I see this at the end (note I'm using --no-color):

]2; [build] Finished.

On Tue, Mar 10, 2015 at 5:19 PM, Jonathan Bohren notifications@github.com
wrote:

@v01s

Does this help?

http://stackoverflow.com/a/14989071

—
Reply to this email directly or view it on GitHub
#157 (comment).

[protobits]

Another issue (which may be qtcreator dependant) is that even when using -i, there's still output buffered which is periodically printed. On the console -i works as expected. Note that this did not happen when invoking catkin_make from qtcreator.

[protobits]

One final note, qtcreator fails to detect the error lines which allow you to click on them and go straight to the offending code. This appears to be due to the fact that the package name is prepended to the compiler output. It would be nice to have an option to simply pass the compiler/cmake output as-is.

[wjwwood]

]2; [build] Finished.

That's probably a place where we're not checking if color is on or off before printing an ANSI reset escape sequence.

Another issue (which may be qtcreator dependant) is that even when using -i, there's still output buffered which is periodically printed. On the console -i works as expected. Note that this did not happen when invoking catkin_make from qtcreator.

You can try to set PYTHONUNBUFFERED:

https://docs.python.org/2/using/cmdline.html#envvar-PYTHONUNBUFFERED

In this case it's not worth comparing to catkin_make's behavior, as the two tools are doing very different things.

One final note, qtcreator fails to detect the error lines which allow you to click on them and go straight to the offending code. This appears to be due to the fact that the package name is prepended to the compiler output. It would be nice to have an option to simply pass the compiler/cmake output as-is.

That could be an option, I don't have time to implement it myself, but here is the code you'd need to affect:

https://github.com/catkin/catkin_tools/blob/master/catkin_tools/verbs/catkin_build/output.py#L77-L161

Specifically the prefix_output option, which is passed in here:

https://github.com/catkin/catkin_tools/blob/master/catkin_tools/verbs/catkin_build/build.py#L589-L590

[protobits]

On Fri, Mar 13, 2015 at 4:28 PM, William Woodall notifications@github.com
wrote:

]2; [build] Finished.

That's probably a place where we're not checking if color is on or off
before printing an ANSI reset escape sequence.

I imagined so.

Another issue (which may be qtcreator dependant) is that even when using
-i, there's still output buffered which is periodically printed. On the
console -i works as expected. Note that this did not happen when invoking
catkin_make from qtcreator.

You can try to set PYTHONUNBUFFERED:

https://docs.python.org/2/using/cmdline.html#envvar-PYTHONUNBUFFERED

In this case it's not worth comparing to catkin_make's behavior, as the
two tools are doing very different things.

This made the trick, thanks. Maybe it is something worth documenting
somewhere.

One final note, qtcreator fails to detect the error lines which allow
you to click on them and go straight to the offending code. This appears to
be due to the fact that the package name is prepended to the compiler
output. It would be nice to have an option to simply pass the
compiler/cmake output as-is.

That could be an option, I don't have time to implement it myself, but
here is the code you'd need to affect:

https://github.com/catkin/catkin_tools/blob/master/catkin_tools/verbs/catkin_build/output.py#L77-L161

Specifically the prefix_output option, which is passed in here:

https://github.com/catkin/catkin_tools/blob/master/catkin_tools/verbs/catkin_build/build.py#L589-L590

Ok, I'll see wether I have time for this. I will get suficiently annoyed of
this problem eventually to motivate me to change it.

—

Reply to this email directly or view it on GitHub
#157 (comment).

[protobits]

By the way, I realized that actually there is no preffix printed before any error lines. QtCreator for some reason does not seem to be able to detect the error lines. Could this maybe due to some particular way you are printing things from catkin? I don't understand what could be causing this.

Anyone else tried it on qtcreator?

[jbohren]

Anyone else tried it on qtcreator?

@adeguet1 @zchen24 Have you guys been doing this?

[simonschmeisser]

I opened the CMakeLists.txt directly in QtCreator and get correct detection of issues. What benefits does adding the catkin step give you?

[jbohren]

I opened the CMakeLists.txt directly in QtCreator and get correct detection of issues. What benefits does adding the catkin step give you?

Running catkin build instead of just building one QtCreator CMake project will build all of the packages on which the current package relies.

[adeguet1]

As Jon mentioned, I tend to change the build step to use 'catkin build', not 'make'. This ensures that all dependencies in the workspace are built properly. On a side note, QtCreator also has a xml file that allows to define a workspace with multiple projects and inter-project dependencies. It would be awesome if catkin had an option to generate that xml file.

[jbohren]

On a side note, QtCreator also has a xml file that allows to define a workspace with multiple projects and inter-project dependencies. It would be awesome if catkin had an option to generate that xml file.

@adeguet1 Do you know if that xml schema is documented somewhere?

[protobits]

Another reason for me to use catkin as the build command is that I have too
many packages which I like to have open at the same time. Otherwise, I need
to create a package for each and open each project individually. I have a
single "ros" project and this way I can edit different projects easily and
build either everything or a single project (with or without deps) just by
changing the build command.

On Mon, Jul 6, 2015 at 10:40 AM, Jonathan Bohren notifications@github.com
wrote:

On a side note, QtCreator also has a xml file that allows to define a
workspace with multiple projects and inter-project dependencies. It would
be awesome if catkin had an option to generate that xml file.

@adeguet1 https://github.com/adeguet1 Do you know if that xml schema is
formatted somewhere?

—
Reply to this email directly or view it on GitHub
#157 (comment)
.

[simonschmeisser]

@v01d Could you explain how you do that?

@adeguet1 Yes, that meta-XML would be great to have!

[gavanderhoorn]

@adeguet1 wrote:

[..] On a side note, QtCreator also has a xml file that allows to define a workspace with multiple projects and inter-project dependencies. It would be awesome if catkin had an option to generate that xml file.

wouldn't this be more a task for CMake itself? It already has generators for a number of build systems.

[jbohren]

@adeguet1 wrote:

[..] On a side note, QtCreator also has a xml file that allows to define a workspace with multiple projects and inter-project dependencies. It would be awesome if catkin had an option to generate that xml file.

wouldn't this be more a task for CMake itself? It already has generators for a number of build systems.

Only if CMake had a mechanism to build multiple projects in a cohesive way. What @adeguet1 is describing is a file describing the topology of a catkin workspace with numerous CMake projects.

[gavanderhoorn]

Isn't the CMake Eclipse generator used to generate project files for Eclipse for entire workspaces? Or does that work only with catkin_make? I don't use it myself, just know it exists.

[jbohren]

Isn't the CMake Eclipse generator used to generate project files for Eclipse for entire workspaces? Or does that work only with catkin_make? I don't use it myself, just know it exists.

Exactly, since catkin_make combines all of your CMake projects into one big super-project, it can lean on CMake's project file generator for Eclipse. If we build them in isolation without hazardous cross-talk, then you need to open multiple Eclipse projects. In the case of QtCreator, you similarly need to open multiple CMakeLists.txt files.

[protobits]

@simonschmeisser I simply create one big project and set it as as "custom" so that I can define my own build steps, where I set catkin build as the build command (with a bunch of other flags) and set the source directory the top-level src/ of the catkin_ws

When I want to just build one project I alter the build step. I can either use --no-deps or not, or build everything.

The downside for me, at the moment, is that qtcreate does not recognize compiler errors which does not allow me to click the offending line in the console output. I'm not sure why is this.

[dseifert]

The downside for me, at the moment, is that qtcreate does not recognize compiler errors which does not allow me to click the offending line in the console output. I'm not sure why is this.

catkin_tools is redirecting all output on stderr to stdout. However QtCreator (and other IDEs) only checks stderr for compiler warnings/errors.

A work-around would be to remove ", stderr=STDOUT" from https://github.com/catkin/catkin_tools/blob/master/catkin_tools/runner/run_unix.py#L45 to allow the error messages to be printed unfiltered. It seems to work fine for me, but I don't know what catkin_tool functionality this may impact negatively.

[protobits]

Great find, thanks. It is strange that it does this, since it is not usual.

On Tue, Aug 25, 2015 at 5:10 PM, Daniel Seifert notifications@github.com
wrote:

The downside for me, at the moment, is that qtcreate does not recognize
compiler errors which does not allow me to click the offending line in the
console output. I'm not sure why is this.

catkin_tools is redirecting all output on stderr to stdout. However
QtCreator (and other IDEs) only checks stderr for compiler warnings/errors.

A work-around would be to remove ", stderr=STDOUT" from
https://github.com/catkin/catkin_tools/blob/master/catkin_tools/runner/run_unix.py#L45
to allow the error messages to be printed unfiltered. It seems to work fine
for me, but I don't know what catkin_tool functionality this may impact
negatively.

—
Reply to this email directly or view it on GitHub
#157 (comment)
.

[cbandera]

@v01d I am trying to set up catkin_tools with qtcreator as well, but can't find any information, other than your introductory comment to this issue, on how to set it up.
Could you tell me (or point me to) how to set up the build step?
EDIT: Found it http://doc.qt.io/qtcreator/creator-build-settings.html

[cbandera]

]2; [build] Finished.

That's probably a place where we're not checking if color is on or off before printing an ANSI reset escape sequence.

I have tracked back that random ]2;. It comes from this line, which is intended to rename the current terminal window, which in our case doesn't work.

I guess that doesn't really have to do anything with the --no-color option. So maybe we have to think about another flag to indicate that catkin is run from within another application?

[protobits]

There are two options: you can create a project using the project wizard,
specifying "other" (can't remember exactly), where you import existing
sources. Then, you can configure the build command to be "catkin build". As
previously mentioned, this helps dealing with dependencies. Under this
approach, I created a single "ros" project.
However, there's also the option listed in
http://wiki.ros.org/IDEs#QtCreator where you create a project for each
package, by simply "opening" the CMakeLists.txt. This also works and is
much faster to build. This also makes "issues" work. I switched to this
approach for now.

On Wed, Aug 26, 2015 at 3:40 AM, cbandera notifications@github.com wrote:

@v01d https://github.com/v01d I am trying to set up catkin_tools with
qtcreator as well, but can't find any information, other than your
introductory comment to this issue, on how to set it up.
Could you tell me (or point me to) how to set up the build step?

—
Reply to this email directly or view it on GitHub
#157 (comment)
.

[NikolausDemmel]

On a side note, QtCreator also has a xml file that allows to define a workspace with multiple projects and inter-project dependencies. It would be awesome if catkin had an option to generate that xml file.

Generating the CMakeLists.txt.user xml file was my initial idea as well, but it might actually be much smoother to create a catkin plugin for qtcreator that can natively open cmake projects in a catkin workspace, or even the entire workspace as such a 'meta-project', and generate the corresponding qtcreator project files on the fly.

Generating the xml files from outside qtcreator is a bit tricky, since for example, one needs to know something about the existing "Kits" and which to choose. This is currently a manual step when opening a CMake project in QtCreator.

There is some discussion about the xml generation and the qtcreator "catkin"-plugin over at the at qtcreator mailing list. Tobias Hunger from the Qt Company who seems to be doing a lot of work improving the QtCreator cmake integration is responding quickly and seems to be open to such ideas.

[gavanderhoorn]

@NikolausDemmel wrote:

Generating the CMakeLists.txt.user xml file was my initial idea as well, but it might actually be much smoother to create a catkin plugin for qtcreator that can natively open cmake projects in a catkin workspace, or even the entire workspace as such a 'meta-project', and generate the corresponding qtcreator project files on the fly.

Haven't yet used it myself, but Levi-Armstrong/ros_qtc_plugins is meant to do exactly that.

[NikolausDemmel]

Haven't yet used it myself, but Levi-Armstrong/ros_qtc_plugins is meant to do exactly that.

Nice! I wasn't aware of that. It seems to aim at much more integration, whereas I was focusing on just building initially (and so far only in discussion). I will raise my ideas over there. Thanks for the pointer.

[mikepurvis]

I'm seeing this issue when running catkin_tools builds in a Jenkins SSH-connected slave, where TERM=unknown. I get a lot of tput: unknown terminal "unknown" in my build output. If I SSH directly, I see a related warning:

ssh jenkins-slave@prod-vm-jenkins-slave-01 'tput cols'
tput: No value for $TERM and no -T specified

In my opinion, this check should be skipped over when the --no-status arg is given, since that's most of what it matters for. Alternatively, it should be made once, and if it fails, not tried again for the duration of the invocation.

[mikepurvis]

Note that a workaround is to prepend the catkin_tools invocation with TERM=dumb. This seems to cause tput cols to return/fail more quietly.

[wjwwood]

Hi guys, can this be closed or at least broken into more specific issues? It's hard to tell what we were originally talking about here 😄. The handling of terminal width changed in 0.4.3, see #415, and there seems to be a work around for that now, i.e. TERM=dumb.

I'd appreciate parting this out to specific issues if anyone has time. @v01d as the OP, I'll let you decide if this can be closed or not.

[protobits]

I too lost track of the status of the issue/s =b
But I agree that at least the problem I reported was solved with your suggestions. Feel free to close.