[FLINK-18069][CI] Test if Java/Scaladocs builds are passing in the compile stage

[rmetzger]

What is the purpose of the change

We are currently only checking if the Scaladocs are passing when building the flink docs on buildbot.
With this change, we are checking if generating the scaladocs is successful.

[flinkbot]

Thanks a lot for your contribution to the Apache Flink project. I'm the @flinkbot. I help the community
to review your pull request. We will use this comment to track the progress of the review.

Automated Checks

Last check on commit cc6305f (Tue Jun 02 18:46:10 UTC 2020)

Warnings:

• No documentation files were touched! Remember to keep the Flink docs up to date!

_{Mention the bot in a comment to re-run the automated checks.}

Review Progress

• ❓ 1. The [description] looks good.
• ❓ 2. There is [consensus] that the contribution should go into to Flink.
• ❓ 3. Needs [attention] from.
• ❓ 4. The change fits into the overall [architecture].
• ❓ 5. Overall code [quality] is good.

Please see the Pull Request Review Guide for a full explanation of the review process.

The Bot is tracking the review progress through labels. Labels are applied according to the order of the review items. For consensus, approval by a Flink committer of PMC member is required

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot approve description to approve one or more aspects (aspects: description, consensus, architecture and quality)
• @flinkbot approve all to approve all aspects
• @flinkbot approve-until architecture to approve everything until architecture
• @flinkbot attention @username1 [@username2 ..] to require somebody's attention
• @flinkbot disapprove architecture to remove an approval you gave earlier

[flinkbot]

CI report:

• 6890859 Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run travis re-run the last Travis build
• @flinkbot run azure re-run the last Azure build

[rmetzger]

Note: This PR depends on #12443 :)

[zentol]

Please rebase the PR on top of #12443 so we can check whether there are more failures, whether it fixes the issue and how much additional time we spend in the compile stage.

[rmetzger]

#12443 fixes the only issue we currently have: https://dev.azure.com/georgeryan1322/Flink/_build/results?buildId=334&view=results

[zentol]

it may make sense to instead build the javadocs exactly like we do on buildbot:
mvn javadoc:aggregate -Paggregate-scaladoc -DadditionalJOption="-Xdoclint:none" -Dmaven.javadoc.failOnError=false -Dcheckstyle.skip=true -Denforcer.skip=true -Dheader="<a href=\"http://flink.apache.org/\" target=\"_top\"><h1>Back to Flink Website</h1></a>"

[zentol]

Time-wise this seems fine; virtually no change for the javadocs, and scaladoc only takes 20 seconds.
It would be nice if we could write the scaladoc warnings into a file though.

[rmetzger]

Thanks a lot for your comments. I pushed a commit addressing them.

[tzulitai]

+1, LGTM

[zentol]

same deal as scala, move the output into a file.

[rmetzger]

I'm sorry that you have to point me to all cases of the same class of problems.
I have no problem with verbose logs, that's why I don't feel the urge to spend time on controlling the logging behavior. I will address this & rebase to the latest master (to see the build passing)

[rmetzger]

For the scala docs, I'm hiding all the warnings / errors and only show them if the build fails. I'm always showing the maven output.

For the Javadocs, I don't know if the errors are send to stderr.
I have the feeling that it is not worth my time producing a javadoc error to understand how the javadocs are printed (stderr vs stdout). I would rather prefer to spend my time testing the new 1.11 features.
In my opinion, it is okay to have 25000 lines of logs for a regular Flink compile.

If the compile stage passes, you usually don't check its output.
if it fails, you will see the error at the bottom of the file + a lot of helpful debug information above (which you would see anyways).

[zentol]

shouldn't matter whether it goes into stdout/stderr; you could just pipe everything into a file and dump the whole thing if an error happened.
Basically a simplified version of what you already did for scala, which should at most be a 1 minute fix.

[rmetzger]

I addressed the issue & rebased to latest master.

[zentol]

+1