I have java method that gets last saved file from S3 bucket by getting list of all objects and sorting them by name (all files has timestamp in their name). The problem is that response time could be 13 seconds and up to 20, depends on how many objects in the bucket
I guess problem lies in call rout – client – cloudflare – loadbalancer – ingress controler – my service container – ingress controler – loadbalancer – cloudflare – client, because when I call this method from localhost, response time is 1.5 ms in avarage
Is there a way to reduce response time?
Service method
public URL getLastSavedFile(Integer vehicleId) {
try {
S3ObjectSummary s3ObjectSummary = repository.downloadLastFile(String.valueOf(vehicleId));
GeneratePresignedUrlRequest request = getPresignedUrl(s3ObjectSummary);
log.info("Returned URL of last file from S3 storage, for vehicle - {}", vehicleId);
return s3Client.generatePresignedUrl(request);
} catch (IndexOutOfBoundsException e) {
log.info("Failed to get last snapshot, the following - {} doesn't exist in storage", vehicleId);
throw new RuntimeException(e);
}
}
Repository
public S3ObjectSummary downloadLastFile(String vehicleId) {
ObjectListing listing = s3client.listObjects( env.getProperty("INIT_S3_BUCKET"), vehicleId );
List<S3ObjectSummary> objects = listing.getObjectSummaries();
while (listing.isTruncated()) {
listing = s3client.listNextBatchOfObjects (listing);
objects.addAll (listing.getObjectSummaries());
}
objects.sort((o1, o2) -> o2.getKey().compareTo(o1.getKey()));
return objects.get(0); //sorted list with most recent timestamp in the beginning
}
2
Answers
FIXED! It appers that all problems were due to the maven dependencies, I had old version 1.12.382 of aws-java-sdk and when I change it to version 1.12.415 of aws-java-sdk-s3 - it fixed the problem and now response time is 1-2 seconds
1.5ms vs 15 seconds is A LOT 😀
There are at least 2 approach to improve the response time of S3: