Errors and very slow when using a simple node.js server

[dhalbert]

https://forums.adafruit.com/viewtopic.php?f=60&t=189508, user jeremyamoore:

User is using the simple node.js server below. When I try with the simple client program, further down, fetching consecutively longer strings, requests gets stuck or runs very slowly after a few fetches.

User node.js server

const http = require('http');
const serverPort = 8080
const requestListener = function (req, res) {
   UrlPath = parseInt(req.url.split('/')[1])
   
console.log(UrlPath)
outstring = ''
if (typeof UrlPath == 'number' & UrlPath < 100000 & UrlPath > 0) {
   while (UrlPath)
   {
    outstring += 'x';
   UrlPath--;}
   
   }
   else
   {outstring = 'Bad number'}
  res.writeHead(200);
  res.end(outstring);
}

const server = http.createServer(requestListener);
console.log('Starting web server on port ' + serverPort)
server.listen(serverPort);

Simple client I tried, running on a Metro M4 AirLift Lite

import time
import board
from digitalio import DigitalInOut

import adafruit_requests as requests
import adafruit_esp32spi.adafruit_esp32spi_socket as socket
from adafruit_esp32spi import adafruit_esp32spi

from secrets import secrets
esp32_cs = DigitalInOut(board.ESP_CS)
esp32_ready = DigitalInOut(board.ESP_BUSY)
esp32_reset = DigitalInOut(board.ESP_RESET)

url_template = "http://192.168.1.222:8080/{n}"

esp = adafruit_esp32spi.ESP_SPIcontrol(board.SPI(), esp32_cs, esp32_ready, esp32_reset)
requests.set_socket(socket, esp)

print("Connecting to AP...")
while not esp.is_connected:
    try:
        esp.connect_AP(secrets["ssid"], secrets["password"])
    except RuntimeError as e:
        print("could not connect to AP, retrying: ", e)
        continue
print("Connected to", str(esp.ssid, "utf-8"), "\tRSSI:", esp.rssi)

secs = 1
while True:
    url = url_template.format(n=secs)
    print("Fetching", url)
    print(requests.get(url).text)
    time.sleep(1)
    secs += 1

I wrote a simpleFlask server , which works fine with the client above

from flask import Flask

app = Flask(__name__)

@app.route('/<length>')
def top(length):
    try:
        return 'x' * int(length)
    except ValueError:
        return "not an integer"

app.run(host="0.0.0.0", port=8080)

User gets errors like this with their client program

I see errors like this in some cases with their server, but not with the client above, unless I try https://192.168.1.222:8080/0/, and then I do get the ValueError below.

Traceback (most recent call last):
  File "code.py", line 1, in <module>
  File "code_meteobridge.py", line 69, in <module>
  File "adafruit_requests.py", line 446, in text
  File "adafruit_requests.py", line 426, in content
  File "adafruit_requests.py", line 489, in iter_content
  File "adafruit_requests.py", line 352, in _readinto
  File "adafruit_requests.py", line 241, in _recv_into
  File "adafruit_esp32spi/adafruit_esp32spi_socket.py", line 175, in recv_into
ValueError: Can only read number of bytes between 0 and length of supplied buffer

User's diagnosis

Just tested with 7.3.0-beta.2 and 20220507 libraries. Looks like in adafruit_esp32spi_socket recv_into, it gets to the last set of data in the buffer and passes in nbytes that is smaller than the buffer size (which should be OK). When it does the avail = self.available() the value of avail is more than nbytes. So the returned value is larger than nbytes. In adafruit_requests _readinto, self._remaining -= read becomes a negative number. The next call to socket recv_into gets called with a negative nbytes value and the message. If I rename the socket method recv_into to something else, it makes self._backwards_compatible in requests to be true and does the simplified read. So not sure what the root cause is but seems to be either the logic in socket recv_into or deeper into _the_interface.socket_available for the PyPortal.

Curl

I tried both servers with curl:

Flask server

$ curl -v http://192.168.1.222:8000/30
*   Trying 192.168.1.222:8000...
* Connected to 192.168.1.222 (192.168.1.222) port 8000 (#0)
> GET /30 HTTP/1.1
> Host: 192.168.1.222:8000
> User-Agent: curl/7.81.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Server: Werkzeug/2.1.2 Python/3.10.4
< Date: Fri, 20 May 2022 01:32:50 GMT
< Content-Type: text/html; charset=utf-8
< Content-Length: 30
< Connection: close
< 
* Closing connection 0
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

node.js server

$ curl -v http://192.168.1.222:8000/30
*   Trying 192.168.1.222:8000...
* Connected to 192.168.1.222 (192.168.1.222) port 8000 (#0)
> GET /30 HTTP/1.1
> Host: 192.168.1.222:8000
> User-Agent: curl/7.81.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Fri, 20 May 2022 01:33:52 GMT
< Connection: keep-alive
< Keep-Alive: timeout=5
< Transfer-Encoding: chunked
< 
* Connection #0 to host 192.168.1.222 left intact
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

Note that the node.js server is doing Transfer-Encoding: chunked, and the Python server is not.

[tannewt]

How does it do with native sockets?

[FoamyGuy]

I can try this today on ESP32-S2 based device.

[FoamyGuy]

I tested on a Feather TFT ESP32-S2 and it ran for a while with seemingly no problems. It ran for several minutes and made it up past request /400 seemed to receive the xs response pretty quickly and accurately each time.

I'll let it keep running and report if anything pops up. But so far it looks like this may be an issue for esp32spi only.