Add more memory tracking places to improve memory management

[wshwsh12]

Enhancement

There are a lot of known untracked memory at the moment. We plan to track them now!

1. Untracked memory in HashJoin

   • Track the memory usage of entryStore in hashMap executor: add some memory tracker in HashJoin #33918
   • Track the memory usage of golang Map in hashMap executor: add some memory tracker in HashJoin #33918
   • Iteratror and Row/RowPtrs slice reuse in HashJoin. executor: reuse iterator4Slice and Row/RowPtrs slice in HashJoin. #34878
   • Track the memory usage of probe phase in HashJoin. e.g. the memory usage of the matched row. In extreme cases, the memory usage of this part may be the size of the entire hashmap.

     tidb/executor/hash_table.go

     Line 115 in e21b435

     matched = make([]chunk.Row, 0, len(innerPtrs))

   • Optimize the memory usage of the read path in hashRowContainer. hashRowContainer.GetMatchedRowsAndPtrs --> RowContainer.GetRow --> ListInDisk.GetRow --> diskFormatRow.toMutRow util: optimize the memory usage of the read path for listInDisk #34778

2. Untracked memory in Sort

   • Track the memory usage of rowPtrs in SortedRowContainer, which record the position of each row of data in the Chunk util: track the memory usage of rowPtrs in sortedRowContainer #34128
   • Optimize the memory usage of GetChunk for listInDisk. #35636

3. Untracked memory in Spilling process, eg. ListInDisk struct

   • Track the memory usage of offsets in ListInDisk, which record the offsets of each row in the file
   • Use non-extra-memory spilling disk implementation to avoid offsets memory usages. For example, we can also spill offsets into disk. util: support spill offset into disk when spilling #34212

4. Untracked memory in IndexJoin/IndexHashJoin

   • Fix inaccurate memory tracking in IndexJoin/IndexHashJoin #34417

5. Refactor rateLimit for TableReader

6. Track aggregate memory usage more accurate executor: track the memory usage of aggregate more accurate #34732

7. Parallel HashAgg needs spill to disk