Question 185295 in Mongodb Question posted in
The official documentation can be found here.

Slow get operations with spring webflux + mongoDB

I have an API where I have a Ticket entity, and the endpoint to get all the tickets is taking like 25 seconds, but in the mongo db there are only 300 tickets.

The ticket entity:

@Data
@Document(collection = "tickets")
@NoArgsConstructor
@AllArgsConstructor
public class TicketData {

    @Id
    private String ticketId= UUID.randomUUID().toString().substring(0,10);
    @NotBlank(message="WalletId is required")
    private String walletId;
    @NotBlank(message="RaffleId is required")
    @Indexed(name="raffleIdIndex")
    private String raffleId;
    @NotNull(message="A price is required")
    private Double price;
    private Boolean isWinner;
    private LocalDateTime lastBoughtAt;
}

I am using the reactive mongo repository:

public interface MongoDBRepositoryTicket extends ReactiveMongoRepository<TicketData, String> {

    Flux<TicketData> findByRaffleId(String raffleId);

}

And since i’m trying to do it with a clean architecture approach, this is the adapter method to use the repository:

@RequiredArgsConstructor
public class MongoRepositoryAdapterTicket implements TicketRepositoryGateway {

    private final MongoDBRepositoryTicket ticketRepository;
    private final ObjectMapper mapper;

    @Override
    public Flux<Ticket> getAllTickets() {
        return this.ticketRepository
                .findAll()
                .map(ticket -> mapper.map(ticket, Ticket.class));
    }

I also tried to get tickets by raffleId and indexing this raffleId attribute, but it didn’t improve. When printing the db operations with:

logging.level.org.springframework.data.mongodb.core = DEBUG

It shows that the application is really slow running the onAfterLoad and onAfterConvert for each document, with these kind of logs:

2023-05-30 11:50:19,699        DEBUG [nioEventLoopGroup-3-7] ent.AbstractMongoEventListener: onAfterLoad({ "_id" : "25c2370a-4", "walletId" : "khds3", "raffleId" : "597e7098-7", "price" : 5.0, "isWinner" : false, "lastBoughtAt" : { "$date" : "2023-05-25T23:33:30.411Z"}, "_class" : "io.rottenville.ticket.infrastructure.drivenadaptrs.data.TicketData"})

2023-05-30 11:50:19,699        DEBUG [nioEventLoopGroup-3-7] ent.AbstractMongoEventListener: onAfterConvert({ "_id" : "25c2370a-4", "walletId" : "khds3", "raffleId" : "597e7098-7", "price" : 5.0, "isWinner" : false, "lastBoughtAt" : { "$date" : "2023-05-25T23:33:30.411Z"}, "_class" : "io.rottenville.ticket.infrastructure.drivenadaptrs.data.TicketData"}, { "$java" : TicketData(ticketId=25c2370a-4, walletId=khds3, raffleId=597e7098-7, price=5.0, isWinner=false, lastBoughtAt=2023-05-25T18:33:30.411) })

Any thoughts or ideas are greatly appreciated. Thank you!

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    As Gaurav said, I think the problem was the default batch size as it was set to 2. Now changing it to 100, the query takes less than a second for 300 documents.

    In this question there is a solution for that, by adding the following annotation to the repository methods:

    @Meta(cursorBatchSize = 100)


  2. 0

    I will suggest doing following changes to debug the problem.

    1. findAll() will use default batch size. With latest version, I am seeing the default batch size of two. This means the Mongo client will make roughly 150 calls to Mongo to fetch all 300 documents. You can search for settings to change default size.
    2. With change in default size, it should not result in 25 seconds. So we can debug in steps. First, remove the mapper code and simply subscribe to getAllTickets to see how long it takes for repository to get all records and do nothing over it. You can enable org.mongodb.driver.protocol.command to DEBUG, to see queries getting fired from client and when responses come back.

    If you see that responses to org.mongodb.driver.protocol class itself is slow then it has nothing to do with code but time it is taking to get the results (you should see similar behavior in any other client).

    Can you try these and confirm back?

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search