finish aws s3 support

[rom1504]

pip install s3fs makes it work almost completely

only thing that doesn't work is the logger process, somehow the glob always return the same files, it doesn't see the new files
solutions:

• change the logging pattern, directly send the info over the network from workers to driver (but non obvious how to do it in spark)
• figure out what's up with the s3 filesystem in that case

[rom1504]

mostly done now
can probably be made faster by tuning the caching options / by making write async / by doing less write
(in the current state it's 30% slower than local write)

[rom1504]

https://filesystem-spec.readthedocs.io/en/latest/features.html#remote-write-caching looks like a good option

[rom1504]

same slowness issue on hdfs, probably also same solution

[rom1504]

I'm not observing any big slowness after the #85 (that should be improved)
may still use the cache based solution of fsspec, but it doesn't seem strictly needed (ss3fs and hdfs fs already seem to have decent in built strategies)

[rom1504]

I actually did more tests and this is working as expected, using local cache doesn't seem required
may revisit local caching in the future if needed