mbedtls: move to shared-module

[jepler]

this enables the implementation to be shared among ports, and switches away from using espressif's tls wrapper layer.

code size seems about the same.

this probably fixes #8268 but I didn't test that.

I did test that client mode code worked (on pyportal esp32s3 using the script from #8910)

[jepler]

apparently it causes at least some code growth, esp32_eye doesn't fit now.

[justmobilize]

@jepler I also tested this with this in hopes it might randomly fix #8925, but getting a lot of Error connecting socket: [Errno 116] ETIMEDOUT:

Running Adafruit

 - API hosts
   - api.coindesk.com:443 took 0.73 seconds | passed
   - api.covidtracking.com:443 took 0.59 seconds | passed
   - api.developer.lifx.com:443 took 0.93 seconds | passed
   - api.fitbit.com:443 took 0.50 seconds | passed
   - api.github.com:443 took 1.15 seconds | passed
   - api.hackaday.io:443 took 0.82 seconds | passed
   - api.hackster.io:443 took 0.37 seconds | passed
   - api.met.no:443 took 1.84 seconds | passed
   - api.nasa.gov:443 took 0.62 seconds | passed
   - api.nytimes.com:443 took 1.51 seconds | error - success:yes, fail:no, exc:Error connecting socket: [Errno 116] ETIMEDOUT
   - api.open-meteo.com:443 took 2.27 seconds | error - success:yes, fail:no, exc:Error connecting socket: [Errno 116] ETIMEDOUT
   - api.openai.com:443 took 0.94 seconds | passed
   - api.openweathermap.org:443 took 1.32 seconds | passed
   - api.purpleair.com:443 took 0.66 seconds | passed
   - api.spacexdata.com:443 took 1.63 seconds | passed
   - api.thecatapi.com:443 took 0.78 seconds | passed
   - api.thingiverse.com:443 took 0.59 seconds | passed
   - api.thingspeak.com:443 took 0.93 seconds | passed
   - api.tidesandcurrents.noaa.gov:443 took 1.53 seconds | passed
   - api.twitter.com:443 took 1.51 seconds | error - success:yes, fail:no, exc:Error connecting socket: [Errno 116] ETIMEDOUT
   - api.wordnik.com:443 took 0.67 seconds | passed

 - Common hosts
   - admiraltyapi.azure-api.net:443 took 1.82 seconds | passed
   - aeroapi.flightaware.com:443 took 1.31 seconds | error - success:yes, fail:no, exc:Error connecting socket: [Errno 116] ETIMEDOUT
   - airnowapi.org:443 took 1.23 seconds | error - success:yes, fail:no, exc:Error connecting socket: [Errno 116] ETIMEDOUT
   - certification.oshwa.org:443 took 0.46 seconds | passed
   - certificationapi.oshwa.org:443 took 1.61 seconds | passed
   - chat.openai.com:443 took 0.96 seconds | passed
   - covidtracking.com:443 took 0.48 seconds | passed
   - discord.com:443 took 0.92 seconds | passed
   - enviro.epa.gov:443 took 3.46 seconds | passed
   - flightaware.com:443 took 1.66 seconds | passed
   - hosted.weblate.org:443 took 1.92 seconds | passed
   - id.twitch.tv:443 took 4.42 seconds | error - success:yes, fail:no, exc:Error connecting socket: [Errno 116] ETIMEDOUT
   - io.adafruit.com:443 took 1.01 seconds | passed
   - jwst.nasa.gov:443 took 1.31 seconds | error - success:yes, fail:no, exc:Error connecting socket: [Errno 116] ETIMEDOUT
   - management.azure.com:443 took 1.12 seconds | passed
   - na1.api.riotgames.com:443 took 4.65 seconds | error - success:yes, fail:no, exc:Error connecting socket: [Errno 116] ETIMEDOUT
   - oauth2.googleapis.com:443 took 0.71 seconds | passed
   - opensky-network.org:443 took 1.53 seconds | passed
   - opentdb.com:443 took 1.13 seconds | passed
   - raw.githubusercontent.com:443 took 0.37 seconds | passed
   - site.api.espn.com:443 took 0.50 seconds | passed
   - spreadsheets.google.com:443 took 0.68 seconds | passed
   - twitrss.me:443 took 0.60 seconds | passed
   - www.adafruit.com:443 took 1.38 seconds | error - success:yes, fail:no, exc:Error connecting socket: [Errno 116] ETIMEDOUT
   - www.alphavantage.co:443 took 1.90 seconds | passed
   - www.googleapis.com:443 took 1.04 seconds | error - success:yes, fail:no, exc:Error connecting socket: [Errno 116] ETIMEDOUT
   - www.nhc.noaa.gov:443 took 0.77 seconds | passed
   - www.reddit.com:443 took 0.36 seconds | passed
   - youtube.googleapis.com:443 took 0.68 seconds | passed

 - Known problem hosts
   - valid-isrgrootx2.letsencrypt.org:443 took 4.33 seconds | passed
   - openaccess-api.clevelandart.org:443 took 4.59 seconds | passed

And it's not the same ones everytime

[jepler]

thanks for testing! I'm not sure why timeouts would be any different; it's using the same mbedtls library on espressif as it was before, just without the intervening esp_tls layer. Let's not hurry to merge this.

[justmobilize]

Upping the socket.connect timeout to 60 seconds, fixed it. Maybe there's a changed default somewhere?

[justmobilize]

On the other side, on the other issue we were getting a OSError: Failed SSL handshake and now it's Error connecting socket: (-12288, 'MBEDTLS_ERR_X509_FATAL_ERROR')

[jepler]

good call. The old Espressif code sets a 120s socket timeout(!) during ssl connect, overriding the regular timeout:

void common_hal_ssl_sslsocket_connect(ssl_sslsocket_obj_t *self,
    const char *host, size_t hostlen, uint32_t port) {
...
        // Connection successful, set the timeout on the underlying socket. We can't rely on 
the IDF
        // to do it because the config structure is only used for TLS connections. Generally,
 we
        // shouldn't hit this timeout because we try to only read available data. However, th
ere is
        // always a chance that we try to read something that is used internally.
        int fd;
        esp_tls_get_conn_sockfd(self->tls, &fd);
        struct timeval tv;
        tv.tv_sec = 2 * 60; // Two minutes
        tv.tv_usec = 0;
        setsockopt(fd, SOL_SOCKET, SO_RCVTIMEO, &tv, sizeof(tv));
        setsockopt(fd, SOL_SOCKET, SO_SNDTIMEO, &tv, sizeof(tv));

the code I wrote/adapted for pico w does no such thing, and now that behavior has "bled over" into esp32-land.

Same for the different SSL failure message, that's been the Pico W behavior and now it would replace the former Espressif behavior.

[justmobilize]

I'm all for making things closer to the same

And yes, grabbed my Pico W on 8.2.9 and getting the same results

[jepler]

I see that get_socket in connectionmanager defaults to a timeout of 1s. This timeout may simply be too short, since it was never being honored on espressif before this PR.

That said I'm still a bit surprised; I would have said that this timeout is about the time until the low-level socket can be read/written, not the time before an SSL-encrypted byte can be read/written. why would a 1s timeout be problematic during the SSL handshake unless it really takes 1s before the server is sending back some bytes to us...

[jepler]

.. pulling this back into "ready for review" since it seems the timeout issue may be explained.

[justmobilize]

ConnectionManager was pulled from Request.Session and get_socket has a 1 second timeout, although Request.Session.request which is what people use passes in 60 seconds

[tannewt]

Yay for sharing code across ports! Thank you!

[tannewt]

The CI ran suspiciously few boards though...

[justmobilize]

All the other boards passed the first time. It just ran on the single failure from the last commit