skip to Main Content

I have java method that gets last saved file from S3 bucket by getting list of all objects and sorting them by name (all files has timestamp in their name). The problem is that response time could be 13 seconds and up to 20, depends on how many objects in the bucket enter image description here

I guess problem lies in call rout – client – cloudflare – loadbalancer – ingress controler – my service container – ingress controler – loadbalancer – cloudflare – client, because when I call this method from localhost, response time is 1.5 ms in avarage

Is there a way to reduce response time?

Service method

public URL getLastSavedFile(Integer vehicleId) {
    try {
        S3ObjectSummary s3ObjectSummary = repository.downloadLastFile(String.valueOf(vehicleId));
        GeneratePresignedUrlRequest request = getPresignedUrl(s3ObjectSummary);
        log.info("Returned URL of last file from S3 storage, for vehicle - {}", vehicleId);
        return s3Client.generatePresignedUrl(request);
    } catch (IndexOutOfBoundsException e) {
        log.info("Failed to get last snapshot, the following - {} doesn't exist in storage", vehicleId);
        throw new RuntimeException(e);
    }
}

Repository

public S3ObjectSummary downloadLastFile(String vehicleId) {
    ObjectListing listing = s3client.listObjects( env.getProperty("INIT_S3_BUCKET"), vehicleId );
    List<S3ObjectSummary> objects = listing.getObjectSummaries();
    while (listing.isTruncated()) {
        listing = s3client.listNextBatchOfObjects (listing);
        objects.addAll (listing.getObjectSummaries());
    }
    objects.sort((o1, o2) -> o2.getKey().compareTo(o1.getKey()));
    return objects.get(0); //sorted list with most recent timestamp in the beginning
}

2

Answers


  1. Chosen as BEST ANSWER

    FIXED! It appers that all problems were due to the maven dependencies, I had old version 1.12.382 of aws-java-sdk and when I change it to version 1.12.415 of aws-java-sdk-s3 - it fixed the problem and now response time is 1-2 seconds


  2. 1.5ms vs 15 seconds is A LOT 😀

    There are at least 2 approach to improve the response time of S3:

    1. Use Transfer Acceleration. Improves both uploads and downloads, uses edge locations to route your request trough AWS internal infrastructure.
    2. Use CloudFront. Improves just downloads, uses edge locations to cache your content.
    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search