skip to Main Content

I have the following two files but I cannot get the fastapi server to work via docker.

I am using the command docker build -t my_project . to build it and docker run -it -p 8080:8080 my_project to run the server. Finally I use the following curl command to call the API with this error: curl: (56) Recv failure: Connection reset by peer.

curl -X POST "http://127.0.0.1:8080/predict/" -H "Content-Type: application/json" -d '{"image_sha": "example_sha"}'

Running this via python locally works perfectly fine.

server.py

import base64
import io

from fastapi import FastAPI
from typing import Any
import uvicorn
from PIL import Image
import pydantic

class ImageModelInput(pydantic.BaseModel):
    image_sha: str
    
    def get_base64_encoded_image(self) -> str:
        return "dummy_base64_image"

    def decode_image(self) -> Image.Image:
        image_data = base64.b64decode(self.image_base64)
        return Image.open(io.BytesIO(image_data))
        
class ImageModelOutput(pydantic.BaseModel):
    attributes: list[str]

class ImageModel:
    def __init__(self, model: Any):
        self.model = model
        self.app = FastAPI()
        
        @self.app.post("/predict/", response_model=ImageModelOutput)
        async def predict(input: ImageModelInput):
            image = input.get_base64_encoded_image()
            prediction = self.model_predict(image)
            return ImageModelOutput(attributes=prediction)
        
        @self.app.get("/readyz")
        async def readyz():
            return {"status": "ready"}

    def model_predict(self, image: Image.Image) -> list[str]:
        # Replace this method with actual model prediction logic
        return ["dummy_attribute_1", "dummy_attribute_2"]

    def run(self, host: str = "127.0.0.1", port: int = 8080):
        uvicorn.run(self.app, host=host, port=port)

# Example usage
if __name__ == "__main__":
    # Replace with your actual model loading logic
    dummy_model = "Your_Model_Here"
    image_model = ImageModel(model=dummy_model)
    image_model.run()

Dockerfile:

FROM python:3.12.4-slim@sha256:2fba8e70a87bcc9f6edd20dda0a1d4adb32046d2acbca7361bc61da5a106a914

WORKDIR /app

RUN --mount=type=cache,target=/root/.cache/pip 
    pip install -U pip poetry wheel

RUN addgroup --system somebody && 
    adduser --system --home /app --ingroup somebody somebody && 
    chown -R somebody:somebody /app

USER somebody

COPY requirements.txt /app/requirements.txt
RUN pip install --no-cache-dir -r requirements.txt
COPY my_project /app/my_project

ENV PYTHONUNBUFFERED=1

EXPOSE 8080
CMD python3 cx_ai_lightbox/server.py

2

Answers


  1. You’re missing the following part from your docker file

    # (change flask to what you use)
    #     |||||
    #     |||||             add this part
    #     |||||          ~~~~~~~~  ~~~~~~~~~  
    CMD ["flask", "run", "--host", "0.0.0.0"]
    

    Without it the IP mapping will not work correctly. You’ll also have to adjust the host part, as it is currently hard coded in your python file.

    Login or Signup to reply.
  2. Passing 127.0.0.1 as the uvicorn host only makes it accessible within the container.

    Simply replace it with 0.0.0.0 instead. Now you can access you endpoint at
    http://127.0.0.1:8080/predict/

    curl http://127.0.0.1:8080
    

    {"detail":"Not Found"}

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search