Update the Met Museum reingestion partitions

[obulat]

Problem

The Met Museum reingestion DAG (met_museum_reingestion_workflow) experiences task timeouts in later intervals (e.g., date_shift_126). These tasks process large date ranges (180 days) and often fail due to excessive data volume or slow API responses for older data.

Description

To resolve this, we will redistribute the workload by:

1. Reducing six_month_list_length from 30 to 20, decreasing the number of 180-day tasks.
2. Increasing three_month_list_length from 18 to 38, adding more 90-day tasks.

Why

• Each 180-day task is split into two 90-day tasks, reducing per-task data volume.
• Total reingestion days remain unchanged:

  # Original: 30*180 + 18*90 = 7,020 days  
  # Updated:  20*180 + 38*90 = 7,020 days 

Alternatives

1. Increase dagrun_timeout. Drawback: Risks allowing indefinitely long runs and hides inefficiencies rather than addressing the root cause.

2. Parallelize task processing. Drawback: The Met Museum API (or other providers) may enforce rate limits. Increasing parallelism could violate these limits, leading to failed requests or temporary bans.

The proposed solution balances simplicity, reliability, and adherence to provider constraints.

Additional context

Code location

openverse/catalog/dags/providers/provider_reingestion_workflows.py

Lines 69 to 79 in 820f246

	ProviderReingestionWorkflow(
	# 64 total reingestion days
	ingester_class=MetMuseumDataIngester,
	max_active_tasks=2,
	pull_timeout=timedelta(hours=16),
	dagrun_timeout=timedelta(days=7),
	daily_list_length=6,
	one_month_list_length=9,
	three_month_list_length=18,
	six_month_list_length=30,
	),

Proposed change

ProviderReingestionWorkflow(
    # ... other parameters ...
    three_month_list_length=38,  # Previously 18
    six_month_list_length=20,     # Previously 30
),