Extremely Large HTTP Responses with Chunked Encoding Blocked

I’m trying to figure out why haproxy is blocking server responses with the “PH” flag.

The responses are VERY large, we’re talking 3-5GB responses. This particular response is 3633514102 bytes (3.4GB).

I have tried a number of things such as setting tune.bufsize to 5GB in size. I’ve tried turning on option accept-invalid-http-response however, I’m kind of just fumbling around in the dark trying to guess what the problem might be.

I’m not sure what combination of settings I need to support such large responses, or if something else is going on here?

I’m on Version 2.2.6-1~bpo10+1

I can see that it’s complaining about the error at position 130, which is the hex representation of the chunk size (3633514102). But I can’t for the life of me figure out why it’s complaining at that location.

The “show error” output is here:

Total events captured on [22/Jan/2021:18:41:25.300] : 1
 
[22/Jan/2021:18:41:20.864] backend turbogeth (#11): invalid response
  frontend http-https-in (#2), server archive04 (#2), event #0, src <redacted>:39386
  buffer starts at 0 (including 0 out), 14936 free,
  len 1448, wraps at 16336, error at position 130
  H1 connection flags 0x00000000, H1 stream flags 0x00004014
  H1 msg state MSG_CHUNK_SIZE(26), H1 msg flags 0x00001716
  H1 chunk len 0 bytes, H1 body len 0 bytes :
  
  00000  HTTP/1.1 200 OK\r\n
  00017  Content-Type: application/json\r\n
  00049  Vary: Origin\r\n
  00063  Date: Fri, 22 Jan 2021 18:41:20 GMT\r\n
  00100  Transfer-Encoding: chunked\r\n
  00128  \r\n
  00130  d8930676\r\n
  00140  {"jsonrpc":"2.0","id":1,"result":{"gas":4300872,"failed":false,"return
  00210+ Value":"","structLogs":[{"pc":0,"op":"CALLDATASIZE","gas":5810070,"gas
  00280+ Cost":2,"depth":1,"stack":[],"memory":[],"storage":{}},{"pc":1,"op":"R
  00350+ ETURNDATASIZE","gas":5810068,"gasCost":2,"depth":1,"stack":["000000000
  00420+ 0000000000000000000000000000000000000000000000000000b24"],"memory":[],
  00490+ "storage":{}},{"pc":2,"op":"RETURNDATASIZE","gas":5810066,"gasCost":2,
  00560+ "depth":1,"stack":["00000000000000000000000000000000000000000000000000
  00630+ 00000000000b24","00000000000000000000000000000000000000000000000000000
  00700+ 00000000000"],"memory":[],"storage":{}},{"pc":3,"op":"CALLDATACOPY","g
  00770+ as":5810064,"gasCost":558,"depth":1,"stack":["000000000000000000000000
  00840+ 0000000000000000000000000000000000000b24","000000000000000000000000000
  00910+ 0000000000000000000000000000000000000","000000000000000000000000000000
  00980+ 0000000000000000000000000000000000"],"memory":["0000000000000000000000
  01050+ 000000000000000000000000000000000000000000","0000000000000000000000000
  01120+ 000000000000000000000000000000000000000","0000000000000000000000000000
  01190+ 000000000000000000000000000000000000","0000000000000000000000000000000
  01260+ 000000000000000000000000000000000","0000000000000000000000000000000000
  01330+ 000000000000000000000000000000","0000000000000000000000000000000000000
  01400+ 000000000000000000000000000","000000000000000000

5 posts - 2 participants

Read full topic