[Bug][Sonarcube] Data too long for column 'component' at row 12

[Capone1983]

Search before asking

• I had searched in the issues and found no similar issues.

What happened

SonarCube scan fails the pipeline scan of a project.

attached stack trace -- stack trace: | github.com/apache/incubator-devlake/server/services.runPipeline | /app/server/services/pipeline_runner.go:79 | [...repeated from below...] Wraps: (2) Error running pipeline 14. Wraps: (3) attached stack trace -- stack trace: | github.com/apache/incubator-devlake/server/services.RunTasksStandalone | /app/server/services/task.go:217 | github.com/apache/incubator-devlake/server/services.(*pipelineRunner).runPipelineStandalone.func1 | /app/server/services/pipeline_runner.go:42 | github.com/apache/incubator-devlake/core/runner.runPipelineTasks | /app/core/runner/run_pipeline.go:90 | github.com/apache/incubator-devlake/core/runner.RunPipeline | /app/core/runner/run_pipeline.go:54 | github.com/apache/incubator-devlake/server/services.(*pipelineRunner).runPipelineStandalone | /app/server/services/pipeline_runner.go:38 | github.com/apache/incubator-devlake/server/services.runPipeline | /app/server/services/pipeline_runner.go:76 | github.com/apache/incubator-devlake/server/services.RunPipelineInQueue.func1 | /app/server/services/pipeline.go:347 | runtime.goexit | /usr/local/go/src/runtime/asm_amd64.s:1598 Wraps: (4) attached stack trace | -- stack trace: | | github.com/apache/incubator-devlake/server/services.RunTasksStandalone.func1 | | /app/server/services/task.go:189 | Wraps: (2) Error running task 870. | Wraps: (3) attached stack trace | -- stack trace: | | github.com/apache/incubator-devlake/core/runner.RunPluginSubTasks | | /app/core/runner/run_task.go:329 | | [...repeated from below...] | Wraps: (4) subtask convertIssueCodeBlocks ended unexpectedly | Wraps: (5) attached stack trace | -- stack trace: | | github.com/apache/incubator-devlake/impls/dalgorm.(*Dalgorm).convertGormError | | /app/impls/dalgorm/dalgorm.go:523 | | github.com/apache/incubator-devlake/impls/dalgorm.(*Dalgorm).CreateOrUpdate | | /app/impls/dalgorm/dalgorm.go:265 | | github.com/apache/incubator-devlake/helpers/pluginhelper/api.(*BatchSave).flushWithoutLocking | | /app/helpers/pluginhelper/api/batch_save.go:131 | | github.com/apache/incubator-devlake/helpers/pluginhelper/api.(*BatchSave).Close | | /app/helpers/pluginhelper/api/batch_save.go:147 | | github.com/apache/incubator-devlake/helpers/pluginhelper/api.(*BatchSaveDivider).Close | | /app/helpers/pluginhelper/api/batch_save_divider.go:102 | | github.com/apache/incubator-devlake/helpers/pluginhelper/api.(*DataConverter).Execute | | /app/helpers/pluginhelper/api/data_convertor.go:131 | | github.com/apache/incubator-devlake/plugins/sonarqube/tasks.ConvertIssueCodeBlocks | | /app/plugins/sonarqube/tasks/issue_code_blocks_convertor.go:77 | | github.com/apache/incubator-devlake/core/runner.runSubtask | | /app/core/runner/run_task.go:405 | | github.com/apache/incubator-devlake/core/runner.RunPluginSubTasks | | /app/core/runner/run_task.go:327 | | github.com/apache/incubator-devlake/core/runner.RunPluginTask | | /app/core/runner/run_task.go:163 | | github.com/apache/incubator-devlake/core/runner.RunTask | | /app/core/runner/run_task.go:137 | | github.com/apache/incubator-devlake/server/services.runTaskStandalone | | /app/server/services/task_runner.go:113 | | github.com/apache/incubator-devlake/server/services.RunTasksStandalone.func1 | | /app/server/services/task.go:187 | | runtime.goexit | | /usr/local/go/src/runtime/asm_amd64.s:1598 | Wraps: (6) Error 1406 (22001): Data too long for column 'component' at row 12 (500) | Wraps: (7) Error 1406 (22001): Data too long for column 'component' at row 12 | Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *withstack.withStack (6) *errutil.withPrefix (7) *mysql.MySQLError Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.leafError

What do you expect to happen

Scan the project in Sonarcube

How to reproduce

Install v1.0.1-beta4@483c93e
add sonarcube connection and have a project with large java folder structure.

Anything else

No response

Version

v1.0.1-beta4@483c93e

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[dosubot]

To resolve the "Data too long for column 'component' at row 12" error encountered during a SonarCube scan in the pipeline, you need to modify the length of the 'component' column in the relevant tables. The provided migration script 20231007_modify_component_length.go shows how to change the column type to varchar(500).

Here is the relevant code snippet from the migration script:

package migrationscripts

import (
	"github.com/apache/incubator-devlake/core/context"
	"github.com/apache/incubator-devlake/core/dal"
	"github.com/apache/incubator-devlake/core/errors"
	"github.com/apache/incubator-devlake/core/plugin"
	"github.com/apache/incubator-devlake/helpers/migrationhelper"
)

var _ plugin.MigrationScript = (*modifyComponentLength)(nil)

type modifyComponentLength struct{}

type sonarqubeHotspot20231007 struct {
	Component string `gorm:"index;type:varchar(500)"`
}

func (sonarqubeHotspot20231007) TableName() string {
	return "_tool_sonarqube_hotspots"
}

type sonarqubeIssueCodeBlock20231007 struct {
	Component string `gorm:"index;type:varchar(500)"`
}

func (sonarqubeIssueCodeBlock20231007) TableName() string {
	return "_tool_sonarqube_issue_code_blocks"
}

type sonarqubeIssue20231007 struct {
	Component string `gorm:"index;type:varchar(500)"`
}

func (sonarqubeIssue20231007) TableName() string {
	return "_tool_sonarqube_issues"
}

func (script *modifyComponentLength) Up(basicRes context.BasicRes) errors.Error {
	db := basicRes.GetDal()
	err := migrationhelper.ChangeColumnsType[sonarqubeHotspot20231007](
		basicRes,
		script,
		sonarqubeHotspot20231007{}.TableName(),
		[]string{"component"},
		func(tmpColumnParams []interface{}) errors.Error {
			return db.UpdateColumn(
				&sonarqubeHotspot20231007{},
				"component",
				dal.DalClause{Expr: " ? ", Params: tmpColumnParams},
				dal.Where("? != '' ", tmpColumnParams...),
			)
		},
	)
	if err != nil {
		return err
	}

	err = migrationhelper.ChangeColumnsType[sonarqubeIssueCodeBlock20231007](
		basicRes,
		script,
		sonarqubeIssueCodeBlock20231007{}.TableName(),
		[]string{"component"},
		func(tmpColumnParams []interface{}) errors.Error {
			return db.UpdateColumn(
				&sonarqubeIssueCodeBlock20231007{},
				"component",
				dal.DalClause{Expr: " ? ", Params: tmpColumnParams},
				dal.Where("? != '' ", tmpColumnParams...),
			)
		},
	)
	if err != nil {
		return err
	}

	err = migrationhelper.ChangeColumnsType[sonarqubeIssue20231007](
		basicRes,
		script,
		sonarqubeIssue20231007{}.TableName(),
		[]string{"component"},
		func(tmpColumnParams []interface{}) errors.Error {
			return db.UpdateColumn(
				&sonarqubeIssue20231007{},
				"component",
				dal.DalClause{Expr: " ? ", Params: tmpColumnParams},
				dal.Where("? != '' ", tmpColumnParams...),
			)
		},
	)
	if err != nil {
		return err
	}

	return nil
}

func (*modifyComponentLength) Version() uint64 {
	return 20231007145127
}

func (*modifyComponentLength) Name() string {
	return "modify component type to varchar(500)"
}

This script modifies the component column type to varchar(500) for the following tables:

• _tool_sonarqube_hotspots
• _tool_sonarqube_issue_code_blocks
• _tool_sonarqube_issues

To apply this migration, ensure that the script is executed in your database migration process. This should resolve the "Data too long for column 'component' at row 12" error ^[1][2].

To continue talking to Dosu, mention @dosu.