skip to Main Content

I am currently working on a python application, that works with facebook api’s. As we all know, facebook loves their own technology and is working with zstd for data compression.

The problem: facebook is returning either a uncompressed response with normal json or if the response is longer, it is responding with a zstd compressed json.

My current code is something like this:

import zstd
import json


def handle_response(response)
    json = None
    try:
        json = json.loads(zstd.decompress(response.content))
    except:
        json = json.loads(response.text)

    return json

I am currently wondering, if there is a more clean way to do this and even detect zstd.

2

Answers


  1. What you’re doing is fine.

    You could, I suppose, check to see if the stream starts with the four bytes 28 b5 2f fd. If it doesn’t, it’s not a zstd stream. If it does, it may be a zstd stream. In the latter case, you would try to decompress and if it fails, you would fall back to just copying the input.

    That turns out to be exactly the same as what you’re already doing, since the first thing that zstd.decompress is doing is to look for that signature.

    Login or Signup to reply.
  2. After excute python clause ‘b = zstd.compress(str)’,the byte array always starts with ‘b'(xb5/xfd x03x19x00’ .
    So you can use ‘index()’ function to judge.

    ——————
    Does anyone know how to decompress the content which is the http response output type ‘applicaton/zstd’ when using internet browser?
    like below:

    ache-control: no-store
    cf-cache-status: DYNAMIC
    cf-ray:
    7a51c7dbad5efb2c-SJC
    content-length: 1591
    content-type: application/zstd
    date: Thu, 09 Mar 2023 07:59:49 GMT
    nel:
    {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
    report-to:
    {"endpoints":[{"url":"https://a.nel.cloudflare.com/report/v3?s=K3U4RKcx6XOP1ekNoJGTilOIZ%2FR4f43q%2BBsvbmmbEkQFHWMJQ5JvhDFDbZFHqVczdsR0rzY24pO9h4kjeehrn3fs0H76%2FO2F612s%2F7%2FjQ%2F6LRjYIb%2BOsPsFOEzIzJTU0NwSyUEmTPW0%3D"}],"group":"cf-nel","max_age":604800}
    server: cloudflare

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search