Work around sync issue when long 0xff sequence is transmitted

[igrr]

From @anisimovsergey on March 23, 2016 22:55

Basic Infos

Hardware

Hardware: olimex mod-wifi-esp8266
Core Version: 2.1.0

Description

I can successfully upload compiled code using Sketch->Upload (Ctrl+U) but whenever I try to upload a bin file for SPIFFS, the esptool.exe (0.4.8) flashing process stops at random moments (but always on flashing the actual data) with the error saying "read 0, requested 1
warning: espcomm_send_command: didn't receive command response"

I investigated the issue a bit and found out that I can actually upload compiled program (not a SPIFFS image) in the same memory area 0x00100000 without a problem (!) and it made me thinking that there must be some difference in the files content.

I checked the files and found out that the file produced by mkspiffs.exe (0.1.2) contains a lot of sequential 0xFFs (11111111). I forcefully replaced them by 0xAAs (10101010) just for sake of experiment and tried to upload the file. It worked! So I'm assuming that the there is a RS232 timer synchronization problem here. I searched on the Web and it seems like I'm not alone having this problem, so is it possible to solve it somehow? Sorry for my naive question, but does SPIFFS format requires 0xFFs or they can be replaced with 0xAA? Is it possible to make an optional parameter for mkspiffs?

Settings in IDE

Module: olimex mod-wifi-esp8266
Flash Size: 2M
CPU Frequency: 80Mhz
Flash Mode: qio
Flash Frequency: 40Mhz
Upload Using: SERIAL
Reset Method: none

Sketch

\libraries\ESP8266WebServer\examples\FSBrowser\

Debug Messages

espcomm_send_command: sending command payload
setting serial port timeouts to 15000 ms
setting serial port timeouts to 1000 ms
espcomm_send_command: receiving 2 bytes of data
writing flash

.......................... read 0, requested 1
warning: espcomm_send_command: didn't receive command response
warning: espcomm_send_command(FLASH_DOWNLOAD_DATA) failed
warning: espcomm_send_command: wrong direction/command: 0x01 0x03, expected 0x01 0x04
closing bootloader

Copied from original issue: esp8266/Arduino#1813

[igrr]

As far as I know, SPIFFS relies on the fact that "empty" (unused) space in flash is filled with 0xff-s.
My guess is that long sequences of 0xff-s make the receiver run slightly out of sync. One possible remedy would be to finish the work on compressed data transfers (https://github.com/igrr/esptool-ck/tree/feature/compression).

[anisimovsergey]

Thank you for your prompt reply. Yes, the compression looks promising.

[mamalala]

Hi,

first, thanks for keeping the tool i wrote many moons ago alive ;) Back then i didn't expect it to become rather popular. After all, it was just for fun.

Anyhow, regarding long 0xFF sequences when writing to flash. Actually, you don't really need to write those at all, since erased flash is by default 0xFF. One could look-ahead in the binary file and detect larger blocks of all 0xFF values and skip them, continuing the write at the next non-oxFF value (keeping flash block boundaries in mind, of course!).

However, since it is so long ago that i worked on the ESP stuff, not sure if the bootloader would support uploading multiple smaller blocks at different addresses, effectively skipping the erased/0xFF sections.

Greetings,

Chris

[anisimovsergey]

Chris, thank you for your update.
I think this issue can be closes since it turned out that it was a fake FTDI chip causing the problem! I found it out long time ago and described my experience here but forgot to mention it in this tread, sorry about that.