[Matcher] Make a few adjustments related to limit validator matcher #442

Job	Run time
minitest (3.0, rails_6_1_4, vips)	32s
minitest (3.0, rails_6_1_4, mini_magick)	37s
minitest (3.0, rails_7_0, vips)	30s
minitest (3.0, rails_7_0, mini_magick)	40s
minitest (3.0, rails_7_1, vips)	36s
minitest (3.0, rails_7_1, mini_magick)	40s
minitest (3.0, rails_next, vips)	31s
minitest (3.0, rails_next, mini_magick)	35s
minitest (3.1, rails_7_0, vips)	33s
minitest (3.1, rails_7_0, mini_magick)	41s
minitest (3.1, rails_7_1, vips)	37s
minitest (3.1, rails_7_1, mini_magick)	46s
minitest (3.1, rails_next, vips)	34s
minitest (3.1, rails_next, mini_magick)	40s
minitest (3.2, rails_7_0, vips)	29s
minitest (3.2, rails_7_0, mini_magick)	37s
minitest (3.2, rails_7_1, vips)	39s
minitest (3.2, rails_7_1, mini_magick)	39s
minitest (3.2, rails_next, vips)	40s
minitest (3.2, rails_next, mini_magick)	38s
	12m 14s

Provide feedback