Question 397751 in Javascript Question posted in
A very good W3school tutorial can be found here.

Javascript – Error when trying to upload an image prompt to gemini api

I am using Postman and trying to get a response from Gemini API when trying to make a prompt including an image.
I am sending the requesto to: https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash:generateContent

{
  "contents":[
    {
      "parts":[
        {"text": "What is this picture?"},
        {
          "inline_data": {
            "mime_type":"image/jpeg",
            "data": "https://i.ibb.co/3mX1qcB/Document-sans-titre-page-0001.jpg"
          }
        }
      ]
    }
  ]
}

I am getting this response :

{
    "error": {
        "code": 400,
        "message": "Invalid value at 'contents[0].parts[1].inline_data.data' (TYPE_BYTES), Base64 decoding failed for "https://pastebin.com/raw/kL4WEnnn"",
        "status": "INVALID_ARGUMENT",
        "details": [
            {
                "@type": "type.googleapis.com/google.rpc.BadRequest",
                "fieldViolations": [
                    {
                        "field": "contents[0].parts[1].inline_data.data",
                        "description": "Invalid value at 'contents[0].parts[1].inline_data.data' (TYPE_BYTES), Base64 decoding failed for "https://pastebin.com/raw/kL4WEnnn""
                    }
                ]
            }
        ]
    }
}

I have tried to convert the image to a base64 raw text, upload it to pastebin and give it in the request but I have the same error
can someone help me?

Tags:

2

Answers


  1. 0

    You’re trying to use an image located at a URL, but the inline_data Part you’re trying to use is designed for you to pass a base64-encoded representation of the raw bytes of an image file.

    Instead, use the FileData Part, which accepts "URI based data":

    {
  "contents": [
    {
      "parts": [
        {
          "text": "What is this picture?"
        },
        {
          "fileData": {
            "mime_type": "image/jpeg",
            "fileUri": "https://i.ibb.co/3mX1qcB/Document-sans-titre-page-0001.jpg"
          }
        }
      ]
    }
  ]
}
    Login or Signup to reply.
  2. 0

    How about the following patterns?

    Pattern 1:

    In this pattern, inlineData is used.

    In this case, it is required to convert the image data (https://i.ibb.co/3mX1qcB/Document-sans-titre-page-0001.jpg) to the base64 data. When

    First, I created text data including the base64 data from the URL as follows. The filename is sampleRequestBody.txt.

    {"contents":[{"parts":[{"text":"What is this picture?"},{"inline_data":{"mime_type":"image/jpeg","data":"{base64 data converted from image data}"}}]}]}

    When this is used with a curl command, it becomes as follows.

    curl -s -X POST 
-H "Content-Type: application/json" 
-d @sampleRequestBody.txt 
"https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash-latest:generateContent?key={your API key}"

    When this curl command is run, the result shown in the "Testing" section is obtained.

    Pattern 2:

    In this pattern, fileData is used. In this case, the following flow is run.

    1. Upload the image data to Gemini

    curl "https://i.ibb.co/3mX1qcB/Document-sans-titre-page-0001.jpg" | curl --data-binary @- -X POST -H "Content-Type: image/jpeg" "https://generativelanguage.googleapis.com/upload/v1beta/files?uploadType=media&key={your API key}"

    By this, the following result is returned.

    {
  "file": {
    "name": "files/###s",
    "mimeType": "image/jpeg",
    "sizeBytes": "1271543",
    "createTime": "2024-07-17T00:00:00.000000Z",
    "updateTime": "2024-07-17T00:00:00.000000Z",
    "expirationTime": "2024-07-19T00:00:00.000000Z",
    "sha256Hash": "###",
    "uri": "https://generativelanguage.googleapis.com/v1beta/files/###",
    "state": "ACTIVE"
  }
}

    Please copy the value of uri from the returned value.

    2. Generate content

    Using the value of uri, content is generated as follows. Here, fileData property is used.

    curl -s -X POST 
-d '{"contents":[{"parts":[{"text":"What is this picture?"},{"fileData":{"mimeType":"image/jpeg","fileUri":"https://generativelanguage.googleapis.com/v1beta/files/###"}}]}]}' 
-H "Content-Type: application/json" 
"https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash-latest:generateContent?key={your API key}"

    Testing:

    Both patterns return the following result. The generated text will be different.

    {
  "candidates": [
    {
      "content": {
        "parts": [
          {
            "text": "This image shows the Swagger documentation for a petstore API.  Swagger is a specification and toolset for describing, documenting, and consuming RESTful web services.  This particular documentation defines the endpoints and data structures for a petstore API.  It outlines how users can interact with the API to create, read, update, and delete pets, as well as manage their inventory."
          }
        ],
        "role": "model"
      },
      "finishReason": "STOP",
      "index": 0,
      "safetyRatings": [
        {
          "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
          "probability": "NEGLIGIBLE"
        },
        {
          "category": "HARM_CATEGORY_HATE_SPEECH",
          "probability": "NEGLIGIBLE"
        },
        {
          "category": "HARM_CATEGORY_HARASSMENT",
          "probability": "NEGLIGIBLE"
        },
        {
          "category": "HARM_CATEGORY_DANGEROUS_CONTENT",
          "probability": "NEGLIGIBLE"
        }
      ]
    }
  ],
  "usageMetadata": {
    "promptTokenCount": 263,
    "candidatesTokenCount": 76,
    "totalTokenCount": 339
  }
}

    References:

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search