[Bug][framework] Pipelines page loads only first 50 pipelines

[TihomirPop]

Search before asking

• I had searched in the issues and found no similar issues.

What happened

When you land on the pipelines page, the GET api/pipelines endpoint gets called without any parameters and returns first 50 pipelines.
Going to new pages doesn't call the API to get new pipelines.
It also doesn't use the page size selector to get different page sizes.

What do you expect to happen

Going to different pipelines pages should call the API for that page with the selected page size and display the results in the pipelines table.

How to reproduce

Have more then 50 pipelines and open the pipelines page on page >= 3.

Anything else

No response

Version

v1.0.0-beta3

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[github-actions]

This issue has been automatically marked as stale because it has been inactive for 60 days. It will be closed in next 7 days if no further activity occurs.

[dosubot]

Hello @TihomirPop! I'm here to help you with any issues you encounter and answer your questions while you wait for a human maintainer. Let's work together to solve this!

The issue where the Pipelines page only loads the first 50 pipelines and doesn't call the API for new pages or use the page size selector is likely due to the useRefreshData hook not being called with the correct parameters to handle pagination.

To fix this, you need to modify the API call to include pagination parameters such as page and pageSize.

Here is the updated code:

import { useState, useMemo, useEffect } from 'react';

import API from '@/api';
import { PageHeader } from '@/components';
import { PATHS } from '@/config';
import { useRefreshData } from '@/hooks';

import { PipelineTable } from './components';

export const Pipelines = () => {
  const [page, setPage] = useState(1);
  const [pageSize, setPageSize] = useState(20);

  const { ready, data, refresh } = useRefreshData(() => API.pipeline.list({ page, pageSize }));

  const [dataSource, total] = useMemo(() => [(data?.pipelines ?? []).map((it) => it), data?.count ?? 0], [data]);

  useEffect(() => {
    refresh();
  }, [page, pageSize, refresh]);

  return (
    <PageHeader
      breadcrumbs={[
        { name: 'Advanced', path: PATHS.BLUEPRINTS() },
        { name: 'Pipelines', path: PATHS.PIPELINES() },
      ]}
    >
      <PipelineTable
        loading={!ready}
        dataSource={dataSource}
        pagination={{
          total,
          current: page,
          pageSize,
          onChange: (newPage, newPageSize) => {
            setPage(newPage);
            setPageSize(newPageSize);
          },
        }}
        noData={{
          text: 'Add new projects to see engineering metrics based on projects.',
        }}
      />
    </PageHeader>
  );
};

In this updated code:

1. The API.pipeline.list function is called with page and pageSize parameters.
2. The useEffect hook is used to refresh the data whenever page or pageSize changes.
3. The pagination object in the PipelineTable component is updated to handle page and page size changes.

Additionally, the backend implementation for collecting pipeline data from the GitLab API supports pagination through the page and per_page query parameters, as shown in the Index function:

// @Summary Get list of pipelines
// @Description GET /pipelines?status=TASK_RUNNING&pending=1&label=search_text&page=1&pagesize=10
// @Tags framework/pipelines
// @Param status query string false "status"
// @Param pending query int false "pending"
// @Param page query int false "page"
// @Param pagesize query int false "pagesize"
// @Param blueprint_id query int false "blueprint_id"
// @Param label query string false "label"
// @Success 200  {object} shared.ResponsePipelines
// @Failure 400  {string} errcode.Error "Bad Request"
// @Failure 500  {string} errcode.Error "Internal Error"
// @Router /pipelines [get]
func Index(c *gin.Context) {
	var query services.PipelineQuery
	err := c.ShouldBindQuery(&query)
	if err != nil {
		shared.ApiOutputError(c, errors.BadInput.Wrap(err, shared.BadRequestBody))
		return
	}
	pipelines, count, err := services.GetPipelines(&query, true)
	if err != nil {
		shared.ApiOutputError(c, errors.Default.Wrap(err, "error getting pipelines"))
		return
	}
	shared.ApiOutputSuccess(c, shared.ResponsePipelines{Pipelines: pipelines, Count: count}, http.StatusOK)
}

This backend function ensures that the API calls handle pagination correctly by setting the page and pagesize parameters in the query ^[1][2][3].

To continue talking to Dosu, mention @dosu.