feat(ci): add trufflehog secrets detection

[McPatate]

What does this PR do?

Adding a GH action to scan for leaked secrets on each commit.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[albertvillanova]

Thanks for the security improvement.

Do we plan to use this tool in all HF open source projects?

[McPatate]

Yes!

[github-actions]

Show benchmarks

PyArrow==8.0.0

Show updated benchmarks!

Benchmark: benchmark_array_xd.json

metric	read_batch_formatted_as_numpy after write_array2d	read_batch_formatted_as_numpy after write_flattened_sequence	read_batch_formatted_as_numpy after write_nested_sequence	read_batch_unformated after write_array2d	read_batch_unformated after write_flattened_sequence	read_batch_unformated after write_nested_sequence	read_col_formatted_as_numpy after write_array2d	read_col_formatted_as_numpy after write_flattened_sequence	read_col_formatted_as_numpy after write_nested_sequence	read_col_unformated after write_array2d	read_col_unformated after write_flattened_sequence	read_col_unformated after write_nested_sequence	read_formatted_as_numpy after write_array2d	read_formatted_as_numpy after write_flattened_sequence	read_formatted_as_numpy after write_nested_sequence	read_unformated after write_array2d	read_unformated after write_flattened_sequence	read_unformated after write_nested_sequence	write_array2d	write_flattened_sequence	write_nested_sequence
new / old (diff)	0.005007 / 0.011353 (-0.006346)	0.003603 / 0.011008 (-0.007405)	0.062719 / 0.038508 (0.024211)	0.029327 / 0.023109 (0.006217)	0.250360 / 0.275898 (-0.025538)	0.265095 / 0.323480 (-0.058385)	0.004205 / 0.007986 (-0.003781)	0.002713 / 0.004328 (-0.001616)	0.049209 / 0.004250 (0.044958)	0.045162 / 0.037052 (0.008110)	0.260439 / 0.258489 (0.001950)	0.287778 / 0.293841 (-0.006063)	0.027458 / 0.128546 (-0.101088)	0.010169 / 0.075646 (-0.065477)	0.199487 / 0.419271 (-0.219784)	0.036584 / 0.043533 (-0.006949)	0.254523 / 0.255139 (-0.000616)	0.269902 / 0.283200 (-0.013298)	0.017138 / 0.141683 (-0.124545)	1.099285 / 1.452155 (-0.352869)	1.150878 / 1.492716 (-0.341839)

Benchmark: benchmark_getitem_100B.json

metric	get_batch_of_1024_random_rows	get_batch_of_1024_rows	get_first_row	get_last_row
new / old (diff)	0.092868 / 0.018006 (0.074862)	0.300421 / 0.000490 (0.299932)	0.000213 / 0.000200 (0.000013)	0.000053 / 0.000054 (-0.000001)

Benchmark: benchmark_indices_mapping.json

metric	select	shard	shuffle	sort	train_test_split
new / old (diff)	0.018810 / 0.037411 (-0.018601)	0.062341 / 0.014526 (0.047815)	0.074779 / 0.176557 (-0.101777)	0.120641 / 0.737135 (-0.616494)	0.075020 / 0.296338 (-0.221318)

Benchmark: benchmark_iterating.json

metric	read 5000	read 50000	read_batch 50000 10	read_batch 50000 100	read_batch 50000 1000	read_formatted numpy 5000	read_formatted pandas 5000	read_formatted tensorflow 5000	read_formatted torch 5000	read_formatted_batch numpy 5000 10	read_formatted_batch numpy 5000 1000	shuffled read 5000	shuffled read 50000	shuffled read_batch 50000 10	shuffled read_batch 50000 100	shuffled read_batch 50000 1000	shuffled read_formatted numpy 5000	shuffled read_formatted_batch numpy 5000 10	shuffled read_formatted_batch numpy 5000 1000
new / old (diff)	0.277782 / 0.215209 (0.062573)	2.716427 / 2.077655 (0.638772)	1.434204 / 1.504120 (-0.069916)	1.335990 / 1.541195 (-0.205205)	1.336636 / 1.468490 (-0.131854)	0.557562 / 4.584777 (-4.027215)	2.323517 / 3.745712 (-1.422196)	2.647937 / 5.269862 (-2.621925)	1.728735 / 4.565676 (-2.836941)	0.061888 / 0.424275 (-0.362387)	0.004981 / 0.007607 (-0.002627)	0.329429 / 0.226044 (0.103385)	3.324708 / 2.268929 (1.055779)	1.832641 / 55.444624 (-53.611983)	1.514386 / 6.876477 (-5.362091)	1.656912 / 2.142072 (-0.485160)	0.630706 / 4.805227 (-4.174521)	0.116250 / 6.500664 (-6.384414)	0.042598 / 0.075469 (-0.032871)

Benchmark: benchmark_map_filter.json

metric	filter	map fast-tokenizer batched	map identity	map identity batched	map no-op batched	map no-op batched numpy	map no-op batched pandas	map no-op batched pytorch	map no-op batched tensorflow
new / old (diff)	0.969217 / 1.841788 (-0.872570)	11.232580 / 8.074308 (3.158272)	9.541306 / 10.191392 (-0.650086)	0.139544 / 0.680424 (-0.540880)	0.014441 / 0.534201 (-0.519760)	0.285834 / 0.579283 (-0.293449)	0.261950 / 0.434364 (-0.172414)	0.325449 / 0.540337 (-0.214889)	0.415501 / 1.386936 (-0.971435)

PyArrow==latest

Show updated benchmarks!

Benchmark: benchmark_array_xd.json

metric	read_batch_formatted_as_numpy after write_array2d	read_batch_formatted_as_numpy after write_flattened_sequence	read_batch_formatted_as_numpy after write_nested_sequence	read_batch_unformated after write_array2d	read_batch_unformated after write_flattened_sequence	read_batch_unformated after write_nested_sequence	read_col_formatted_as_numpy after write_array2d	read_col_formatted_as_numpy after write_flattened_sequence	read_col_formatted_as_numpy after write_nested_sequence	read_col_unformated after write_array2d	read_col_unformated after write_flattened_sequence	read_col_unformated after write_nested_sequence	read_formatted_as_numpy after write_array2d	read_formatted_as_numpy after write_flattened_sequence	read_formatted_as_numpy after write_nested_sequence	read_unformated after write_array2d	read_unformated after write_flattened_sequence	read_unformated after write_nested_sequence	write_array2d	write_flattened_sequence	write_nested_sequence
new / old (diff)	0.005422 / 0.011353 (-0.005931)	0.003528 / 0.011008 (-0.007480)	0.049582 / 0.038508 (0.011074)	0.032683 / 0.023109 (0.009574)	0.277309 / 0.275898 (0.001411)	0.298598 / 0.323480 (-0.024882)	0.004325 / 0.007986 (-0.003661)	0.002741 / 0.004328 (-0.001588)	0.047933 / 0.004250 (0.043683)	0.040778 / 0.037052 (0.003726)	0.287492 / 0.258489 (0.029003)	0.311408 / 0.293841 (0.017567)	0.029482 / 0.128546 (-0.099064)	0.010630 / 0.075646 (-0.065016)	0.057745 / 0.419271 (-0.361526)	0.033501 / 0.043533 (-0.010031)	0.279880 / 0.255139 (0.024741)	0.297421 / 0.283200 (0.014221)	0.017907 / 0.141683 (-0.123776)	1.152221 / 1.452155 (-0.299934)	1.189332 / 1.492716 (-0.303385)

Benchmark: benchmark_getitem_100B.json

metric	get_batch_of_1024_random_rows	get_batch_of_1024_rows	get_first_row	get_last_row
new / old (diff)	0.094464 / 0.018006 (0.076457)	0.300769 / 0.000490 (0.300279)	0.000196 / 0.000200 (-0.000004)	0.000050 / 0.000054 (-0.000004)

Benchmark: benchmark_indices_mapping.json

metric	select	shard	shuffle	sort	train_test_split
new / old (diff)	0.022232 / 0.037411 (-0.015179)	0.076626 / 0.014526 (0.062100)	0.087807 / 0.176557 (-0.088750)	0.128847 / 0.737135 (-0.608288)	0.092135 / 0.296338 (-0.204203)

Benchmark: benchmark_iterating.json

metric	read 5000	read 50000	read_batch 50000 10	read_batch 50000 100	read_batch 50000 1000	read_formatted numpy 5000	read_formatted pandas 5000	read_formatted tensorflow 5000	read_formatted torch 5000	read_formatted_batch numpy 5000 10	read_formatted_batch numpy 5000 1000	shuffled read 5000	shuffled read 50000	shuffled read_batch 50000 10	shuffled read_batch 50000 100	shuffled read_batch 50000 1000	shuffled read_formatted numpy 5000	shuffled read_formatted_batch numpy 5000 10	shuffled read_formatted_batch numpy 5000 1000
new / old (diff)	0.299013 / 0.215209 (0.083804)	2.929788 / 2.077655 (0.852133)	1.614185 / 1.504120 (0.110065)	1.486720 / 1.541195 (-0.054475)	1.492473 / 1.468490 (0.023983)	0.563699 / 4.584777 (-4.021078)	0.928820 / 3.745712 (-2.816892)	2.597271 / 5.269862 (-2.672590)	1.716534 / 4.565676 (-2.849142)	0.062568 / 0.424275 (-0.361707)	0.005168 / 0.007607 (-0.002439)	0.353781 / 0.226044 (0.127737)	3.493732 / 2.268929 (1.224803)	2.018343 / 55.444624 (-53.426282)	1.694516 / 6.876477 (-5.181961)	1.796950 / 2.142072 (-0.345123)	0.634846 / 4.805227 (-4.170382)	0.115230 / 6.500664 (-6.385434)	0.040816 / 0.075469 (-0.034654)

Benchmark: benchmark_map_filter.json

metric	filter	map fast-tokenizer batched	map identity	map identity batched	map no-op batched	map no-op batched numpy	map no-op batched pandas	map no-op batched pytorch	map no-op batched tensorflow
new / old (diff)	0.986212 / 1.841788 (-0.855575)	11.954392 / 8.074308 (3.880084)	10.299670 / 10.191392 (0.108278)	0.128358 / 0.680424 (-0.552066)	0.016313 / 0.534201 (-0.517888)	0.289621 / 0.579283 (-0.289662)	0.124708 / 0.434364 (-0.309656)	0.325269 / 0.540337 (-0.215068)	0.415133 / 1.386936 (-0.971803)