How to obtain Root certificate?

[alexdufetel]

I'm trying to run the esp8266 mqtt sample but cannot get my device to connect. I assume this is because I did not change the ca.crt file that is in the data folder before uploading it to my microcontroller. Which root crt should I be using?

[gguuss]

You use the ESP8266 Data Upload utility to upload the CERT.

[bertrandlo]

@gguuss Do you mean we just directly upload that file named 'ca.crt' ? i read the code, but could not make sure that's was Google's CA for TLS connection ?

I upload that file 'ca.crt' to SPIFFS, the MQTT connection could try to connect, and then Freeze. in the Google IoT Core Console Center, I can't see any ERROR/DEBUG message. It looks like to loop in the connect while section.

The private key HEX code already put in the sketch. but no lucky.

[gguuss]

I assume you're referring to the Esp8266-lwmqtt sample, which has the following file structure:

├── Esp8266-http
│   ├── backoff.h
│   ├── ciotc_config.h
│   ├── cli.h
│   ├── data
│   │   └── ca.crt
│   ├── Esp8266-http.ino
│   └── esp8266_wifi.h

The data folder contains files that are uploaded when you use the ESP8266 Data upload utility. Of particular importance is ca.cert, the Roots certificate file from Google, which you can retrieve as wget pki.goog/roots.pem and convert as openssl x509 -outform der -in roots.pem -out ca.crt. This file is used when verifying the connection to Google and rotates occasionally so maybe it's out of date in both the hex and non-hex cases.

[gguuss]

If you want to verify that your configuration / settings are working, I recommend starting from the HTTP sample.

[bertrandlo]

@gguuss Thanks for your reply. I never used TLS handshakre in miccontroller before. Sorry not to notice the ca.crt in git repository was updated.

Does that mean we must replace the ca.crt periodically ? and better kept in our program to check ?

[gguuss]

The right thing to do would be to use an over-the-air mechanism for securely updating the certificate if it expires or changes using a platform-specific solution for your device. This degree of functionality isn't currently planned for this library.

I haven't run the example for a little while, I'll see if it works for me with the library cert / GH master cert.

[gguuss]

I just verified the code is working for me using the older cert as well as the one I just generated for the GitHub master version.

A few troubleshooting suggestions:

• Open the serial monitor while running (Ctrl+shift+m) Are you seeing an error code?
• Double-check that you have added the public EC certificate for your device to the device registry, that you are using the correct device identifier, project id, and that your cloud region is correct. Basically any incorrect value in the ciotc_config.h file will result in the device not connecting.
• Verify the JWT is valid using a tool like jwt.io

Just out of curiosity, which 8266 board are you using? I'm verifying on an Adafruit Huzzah board.

[bertrandlo]

@gguuss thanks for the quick reply again. later i will report the same example test. Right now turn back to make sure the esp8266 early version ESP-1S(4M/1M ver) small devlep. board could stalbe-run the BearSSL or not.

I also had two small ESP-12F that more close to Adafruit Huzzah using ESP-12 module, just without additional reset/upload design.

[bertrandlo]

@gguuss after recheck, The problem look like came from the ca.crt not upload successful from Arduino IDE. I use 2.5.0-beta3 vesion. Arduino IDE SPIFFS upload tool just report upload successful. but the error code -10002 still shown.

then turn to use SPIFFS library and download the ca.crt from http and write to filesystem, That;s work. The SPIFFS could find and open ca.crt. I could not make sure the problem came from the too old IDE plugin or not. Because i also could not use the up-to-date MKSPIFFS to make BIN image and upload.

the looping look like came from the lack of ca.crt during handshake with Google IoT Core. This maybe borard-specific / Arduino-Core version issue. Need more check and test.

[alexdufetel]

Thanks for the help. I've tried with the old and new cert and still can't connect. Having issues with the http sample as well. In the serial output I'm seeing a 0 error with mqtt, and a "connection failed" message with http. I double checked project name, secret key, pubsub subscription, device name, device public key. I'm really not sure what I'm doing wrong, but at this point it looks like its not related to the cert.

[bertrandlo]

@alexdufetel Did you try the debug level with TLS/SSL/HTTP Client Level in the board manager ? Maybe the result could show more detail information ? In the begging, my test show the ca.crt load failed using the Arduino IDE upload tool plugin. if the NTP do not updated normally, the JWT would not functional normally,

in the http example, i also suffeed the authorization failed. after, i delete the device, and recreate and upload a new public key, that work.

[bertrandlo]

@gguuss both the mqtt and http example worked after re-create key-pairs and uploaded. even the old keys was checked using your suggestion in the jwt.io tool. but not accept by Google IoT Core. Not sure why

the modified code -

1. the NTP to the local one and Google NTP for more quicker response. maybe could relocate to the ciotc_config.h ? in some place, NTP server response
2. LED_BUILTIN was different related to ESP-modue/board, and that was selected in the board-config tool. if someone could not make sure. just remove that's part. Incorrect hardware configuration will make code halt there.

void getConfig() {
  // TODO(class): Move to library
  String header =
      String("GET ") + device->getLastConfigPath().c_str() + String(" HTTP/1.1");
  String authstring = "authorization: Bearer " + String(jwt.c_str());
  
  client.setInsecure();  // To check if ca.crt issue or not
  
  if (!client.connect(host, httpsPort)) {
    Serial.println("connection failed");
    return;
  } else {
    Serial.print("connection success, TCP Status: ");
    Serial.println(client.status());
  }

  // Connect via https.
  client.println(header.c_str());
  client.println(authstring.c_str());
  client.println("host: cloudiotdevice.googleapis.com");
  client.println("method: get");
  client.println("cache-control: no-cache");
  client.println();

  while (client.connected()) {
    String line = client.readStringUntil('\n');
    Serial.println(line);  // the read the https handshake status - if authorization problems easy to check 
    if (line == "\r") {
      Serial.println("headers received");
      break;
    }
  }
  while (client.available()) {
    String line = client.readStringUntil('\n');
    if (line.indexOf("binaryData") > 0) {
      String val = line.substring(line.indexOf(": ") + 3, line.indexOf("\","));
      Serial.println(val);
      if (val == "MQ==") {
        Serial.println("LED ON");
        //digitalWrite(LED_BUILTIN, HIGH);
      } else {
        Serial.println("LED OFF");
        //digitalWrite(LED_BUILTIN, LOW);
      }
      resetBackoff();
    }
  }
  client.stop();
}

client.setInsecure(); // To check if ca.crt issue or not
Serial.println(line); // printout the https handshake status - if authorization problems easy to check

this two step might make the debug more easily. if Google IoT Core configuration issue, the HTTP response code and message will provide the information to debug.

This issue maybe from the server/board configuration. so maybe closed.

[alexdufetel]

Thanks for the suggestion to run in it debug mode, I had not thought of that. I tried, but unfortunately I don't get much more useful information:
the board tries to establish the mqtt connection, but stays stuck in that while loop:
while(!mqttClient->connect(device->getClientId().c_str(), "unused", getJwt().c_str(), false))
lastError() and returnCode() both return 0.

running in debug outputs and additional "pm open,type:2 0" after initiating the mqtt connection, not sure what to make of it.

[bertrandlo]

@alexdufetel Do you get any JWT String shown in the serial output?? like
eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE1NDkzNzgyNjUsImV4cCI6MTU0OTM4MTg2NSwiYXVkIjoiYXBpLXByb2plY3QtNDUxNzA5MjIyNjAwIn0=.4hl6Gyz+aGQWuix0eWzhlMsq4vkJv6tt19bWarKVMSV32uhFLc7AeUeFCfdZ1lRBswLAsR+lyIEVKj5K5jXYSg==

Also you could try to add some code to check https response to check the handshake procedure.
in the getConfig() section, add one line in the while (client.connected()) { ... } section, like
'Serial.println(line);'

if any server-side configuration issue, this will give some http reponse code to check.

[alexdufetel]

@bertrandlo Thanks for your reply. I do get a JWT, but then I can't establish the connection. Adding a debug line in client.connected() does not help unfortunately, because it does not get called. Would that mean my JWT is invalid?
When I paste my JWT to jwt.io, I get a "Invalid Signature" message. selected algorithm is ES256, and I pasted my public key in the "verify token" section

My JWT:
eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE1NDk0Njk0MjcsImV4cCI6MTU0OTQ3MzAyNywiYXVkIjoiaW90LWRvZ2Zvb2QifQ==.w4h2EolYpqQcuPlO/MLovznI3El83TG+KlT0Lpu3C9/bEQuH1vLbHkqBMTwqpt27KO9GKkvFweYd0MUU32evdA==

My public key:
-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAERusopVBMclQdyu7SjXIBwme8hmRB
pB7X41kQNkK0Uu5te0wULHhR9eKCPYQZD5WXqt5aWuRcwh3LijeJ7XwJqQ==
-----END PUBLIC KEY-----

My Serial output:
Waiting on time sync...
Refreshing JWT
eyJhbGciOiJFUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE1NDk0Njk0MjcsImV4cCI6MTU0OTQ3MzAyNywiYXVkIjoiaW90LWRvZ2Zvb2QifQ==.w4h2EolYpqQcuPlO/MLovznI3El83TG+KlT0Lpu3C9/bEQuH1vLbHkqBMTwqpt27KO9GKkvFweYd0MUU32evdA==
Success to open ca file
loaded
Waiting: 5117
pm open,type:2 0
connection failed
connection failed

[alexdufetel]

also, for FYI : the device I'm using:
https://www.amazon.com/HiLetgo-Internet-Development-Wireless-Micropython/dp/B010N1SPRK/ref=sr_1_3?ie=UTF8&qid=1549470092&sr=8-3&keywords=esp8266+nodemcu

[gguuss]

@bertrandlo /FYI - adds definitions for generic board support

[bertrandlo]

@alexdufetel Happy Chinese New Year~ ! ^^

that Jwt.io result should verify your jwt token, you also have to past the private key to make sure the jwt work well. could you try that again? or regenerate a new pair. your last two messages "connection failed" it look like the error of JWT

[johnhmacleod]

Don't know if this is relevant to your issues, but we also just ran into an MQTT connect issue (on the IBM Watson IoT Platform) with the esp8266 version 2.5.0 board support. Reverting to 2.4.2 fixed the problem. I'm using a Wemos D1 Mini.

[gguuss]

@alexdufetel Let us know if the tip from @johnhmacleod helps or if trying alternative hardware fixes your issue. If it was related, we can update the readme and close this issue.

[alexodus]

Don't know if this is relevant to your issues, but we also just ran into an MQTT connect issue (on the IBM Watson IoT Platform) with the esp8266 version 2.5.0 board support. Reverting to 2.4.2 fixed the problem. I'm using a Wemos D1 Mini.

I'm in the same conditions: fail with 2.5.0 (with lastError=0,returnCode=0), no problem with 2.4.2 (on Wemos D1 Mini Lite)

[gguuss]

I'll add a note to the readme. Maybe open a new issue for determining working versions of the Espressif Community SDK.

[Gior80]

Enable debug or verbose mode when compiling and see datailed reason of failed handshake from esp-idf

[gguuss]

@Gior80 Is this a configuration within the Arduino IDE?

E.g. Tools > Core Debug Level > Verbose

[Gior80]

Yes exactly ,

This way u can get what’s happening at protocol level . I resolved some issues in this way looking what’s passing inside ssl handshaking

[Gior80]

Just to be clear on usage , just activate it and open serial monitor while running . All message gonna come out

[iotics]

Hi Gus, I am also trying to connect my ESP8266 with GCP. From serial monitor output, I figured out that I am able to get the JWT string and ca file also successfully loaded. Globals created for MQTT. MQTT started. And then checking wifi
Connecting.... 0.0.0.0.0.......
Series of zeros. What might be the issue?
Kindly help me with this

[gguuss]

I'm not sure, have you tried increasing the debugging settings / verbosity as described in the README - this may provide a more clear bug report.

Please include the version of the library you're using, the version of the Espressif SDK, and the type of ESP developer board.

[gguuss]

Adding this issue for tracking these types of issue

[jsmith173]

Hi,

I'm trying this project on MKR1000 but before this I have to update the firmware. It seems I have to use the Firmware updater described on https://www.arduino.cc/en/Tutorial/FirmwareUpdater to update the certificate. What address should I add exactly in the Add domains text box? google.com:443 ?

[gguuss]

@jsmith173 mqtt.googleapis.com:443 or mqtt.googleapis.com:8883

[jsmith173]

I think better when you use the FirmwareUpdater for MKR1000 (and not the hackster.io tool) to add the certificates
https://www.arduino.cc/en/Tutorial/FirmwareUpdater

When you'd have getDefaultSensorJSON (publish data as JSON), you can visualise the published data on Google Sheets (using dataflow).

I think a bit unusual that the topic must be specifed with / (for example "/my-topic"). When I not add the / , the program restarts many times (multiple wifi and mqtt reconnects)

My final question: how to receive incoming telemetry data for a specified topic? (I'd read back the data published by publishTelemetry)

[ronyeh3]

@alexdufetel
alex,
we are having the exact same issue as you describe (invalid signature), that ends with "=="
jwt.io shows an invalid signature.
also, ES256 signature actually should be 86 characters,
any ideas?

[alikilic777]

Sen kullanmak ESP8266 Veri Yükleme programı CERT yüklemek için.

Hi please share for esp32