Suppression of 413 error code (payload size exceeded)

[Ruslan05]

Hello Elastica team !
I faced a problem that i`m getting success response from requests which were failed due to exceeded size of payload (by default it`s 100 mb in Elasticsearch).

Here https://github.com/ruflin/Elastica/blob/master/lib/Elastica/Bulk.php#L335 in $response we still can find _status = 413 but in $bulkResponseSet there will be no information related to bad request and we can`t handle this issue.

Please validate this issue and give me a feedback.

Regards,
Ruslan

[Ruslan05]

Someone ? maybe @ruflin

[ruflin]

@Ruslan05 Could you share the full code you are using? What do you see in the Elasticsearch logs?

[Ruslan05]

@ruflin , i can`t share the code, but i believe this is not even necessary cause problem is in the elastica lib, as you can see from attached screenshot. Elasticsearch logs are also not available.

But i was getting this message {"Message":"Request size exceeded 104857600 bytes"} in \Elastica\Transport\Http in exec(Request $request, array $params) method.

This can be easily reproduced, just try to publish huge data (more then 100 mb) during single HTTP request and you will see the in the final response of Elastica will say that request was successful, but it`s not.

[ruflin]

I wonder if the problem might be here: https://github.com/ruflin/Elastica/blob/master/lib/Elastica/Bulk/ResponseSet.php#L67 We overwrite isOk from the response which also check for the headers, but there we don't: https://github.com/ruflin/Elastica/blob/master/lib/Elastica/Response.php#L166

Any chance you could do a var dump and share it here of the responseSet object to see if we have any useful information inside?

[Ruslan05]

Sorry for the late response.

No, i cant't provide any further details as we solved the problem with 100mb payload and it's not reproducible for us anymore.

But you can find responseSet object opened in phpstorm in issue description.

[vinzlj]

Same problem here, anyone?

[vinzlj]

I'm wondering if there's a good reason for this:

class ResponseSet
{
    ...
    public function __construct(BaseResponse $response, array $bulkResponses)
    {
        parent::__construct($response->getData()); // here we don't pass the $response->getStatus() as a second parameter

        $this->_bulkResponses = $bulkResponses;
    }
}

I'm using the Bulk API and the response data & status code get erased by the method _processResponse.

Here's the Response I get from the Client:

Elastica\Response {
  #_queryTime: 0.00088000297546387
  #_responseString: ""
  #_transferInfo: array:26 [
    "url" => "http://127.0.0.1:9200/_bulk?filter_path=-took%2C-items"
    "content_type" => null
    "http_code" => 413
    "header_size" => 60
    "request_size" => 201
    "filetime" => -1
    "ssl_verify_result" => 0
    "redirect_count" => 0
    "total_time" => 0.000824
    "namelookup_time" => 5.3E-5
    "connect_time" => 0.000235
    "pretransfer_time" => 0.000276
    "size_upload" => 0.0
    "size_download" => 0.0
    "speed_download" => 0.0
    "speed_upload" => 0.0
    "download_content_length" => 0.0
    "upload_content_length" => 3462417.0
    "starttransfer_time" => 0.000815
    "redirect_time" => 0.0
    "redirect_url" => ""
    "primary_ip" => "127.0.0.1"
    "certinfo" => []
    "primary_port" => 9200
    "local_ip" => "127.0.0.1"
    "local_port" => 55295
  ]
  #_response: []
  #_status: 413
  #_jsonBigintConversion: false
}

And the ResponseSet I get from the Bulk->send()->_processResponse():

Elastica\Bulk\ResponseSet {
  #_bulkResponses: []
  #_position: 0
  #_queryTime: null
  #_responseString: ""
  #_transferInfo: []
  #_response: []
  #_status: null
  #_jsonBigintConversion: false
}

We're losing all the _transferInfo data along with the statusCode which in my case is what I'd really need to prevent my code from running on error.

[vinzlj]

Actually my fix (#1775) won't fix this other issue related to the 413 error code.

Let's say I'm doing:

/** @var ResponseSet $response */
$response = $type->addDocuments($documents);

This won't work:

if (!$response->isOk()) {
    // do something
}

Because the ResponseSet->isOk() will only break if one of the BulkResponse is not ok, which does not work when a 413 is raised (no BulkResponse here).

You'll have to do something like:

if (!($response->getStatus() >= 200 && $response->getStatus() <= 300)) {
    // do something
}