I’m trying to implement a dynamic search for a huge product collection. The object has several properties including productName
, subCategoryName
, categoryName
, brandName
, etc. The user could search for products using any of these properties. The order is fixed and the first priority for a search string is to find it in productName
and then subCategoryName
and so on.
I used aggregate
to achieve this and then unionWith
to concat records that matched with other properties. It seems to work when fired as a raw query but we also need support for pagination and I’m not being able to achieve that with Spring Data MongoDB
db.product.aggregate(
[
{ $match: { "productName" : { "$regex" : "HYPER", "$options" : "i"},
"companyNo" : { "$in" : [10000009]}, "status" : { "$in" : ["ACTIVE", "IN_ACTIVE", "OUT_OF_STOCK"]} }},
{ $unionWith: { coll: "product", pipeline: [{ $match: { "subCategoryName" : { "$regex" : "HYPER", "$options" : "i"},
"companyNo" : { "$in" : [10000009]}, "status" : { "$in" : ["ACTIVE", "IN_ACTIVE", "OUT_OF_STOCK"]}} }] } },
{ $unionWith: { coll: "product", pipeline: [{ $match: { "categoryName" : { "$regex" : "HYPER", "$options" : "i"},
"companyNo" : { "$in" : [10000009]}, "status" : { "$in" : ["ACTIVE", "IN_ACTIVE", "OUT_OF_STOCK"]}} }] } },
{ $unionWith: { coll: "product", pipeline: [{ $match: { "brandName" : { "$regex" : "HYPER", "$options" : "i"},
"companyNo" : { "$in" : [10000009]}, "status" : { "$in" : ["ACTIVE", "IN_ACTIVE", "OUT_OF_STOCK"]}} }] } },
]
)
Also, this query only works if we pass the substring of the exact name. For example, the NIVEA BODY LOTION EXPRESS HYDRATION 200 ML HYPERmart product will be returned if I search with NIVEA BODY LOTION but it won’t return anything if I search with HYDRATION LOTION
A Sample Product:
{
"_id" : ObjectId("6278c1c2f2570d6f199435b2"),
"companyNo" : 10000009,
"categoryName" : "BEAUTY and PERSONAL CARE",
"brandName" : "HYPERMART",
"productName" : "NIVEA BODY LOTION EXPRESS HYDRATION 200 ML HYPERmart",
"productImageUrl" : "https://shop-now-bucket.s3.ap-south-1.amazonaws.com/shop-now-bucket/qa/10000009/product/BEAUTY%20%26%20PERSONAL%20CARE/HYPERMART/NIVEA%20BODY%20LOTION%20EXPRESS%20HYDRATION%20200%20ML/temp1652081080302.jpeg",
"compressProductImageUrl" : "https://shop-now-bucket.s3.ap-south-1.amazonaws.com/shop-now-bucket/qa/10000009/product/BEAUTY%20%26%20PERSONAL%20CARE/HYPERMART/NIVEA%20BODY%20LOTION%20EXPRESS%20HYDRATION%20200%20ML/temp1652081080302.jpeg",
"productPrice" : 249.0,
"status" : "ACTIVE",
"subCategoryName" : "BODY LOTION & BODY CREAM",
"defaultDiscount" : 0.0,
"discount" : 7.0,
"description" : "Give your skin fast-absorbing moisturisation and make it noticeably smoother for 48-hours with Nivea Express Hydration Body Lotion. The formula with Sea Minerals and Hydra IQ supplies your skin with moisture all day. The new improved formula contains Deep Moisture Serum to lock in deep moisture leaving you with soft and supple skin.",
"afterDiscountPrice" : 231.57,
"taxPercentage" : 1.0,
"availableQuantity" : NumberLong(100),
"packingCharges" : 0.0,
"available" : true,
"featureProduct" : false,
"wholesaleProduct" : false,
"rewards" : NumberLong(0),
"createAt" : ISODate("2022-05-09T07:24:40.286Z"),
"createdBy" : "companyAdmin_@[email protected]",
"isBulkUpload" : true,
"buyPrice" : 0.0,
"privateProduct" : false,
"comboProduct" : false,
"subscribable" : false,
"discountAdded" : false,
"_class" : "com.apptmart.product.entity.Product"
}
I’m new to MongoDB. any references will be appretiated.
2
Answers
Checkout mongodb’s search indexes. You can create a search index with the needed fields to search by, and then you can add a $search stage to your aggregation. You can use the already built in dynamic tech search functionality. If you’re using mongodb atlas, I suggest looking at the atlas text search docs!
To address your auto complete needs, you can set the index to allow for autocompletions (i.e – you type "hello" and the product "hello world" shows up). In addition you can implement fuzzy searches and have other neat configurations to allow for filtering alongside the text search.
text search docs
atlas text search docs
Apologies for any typos in the code
Also take a look at the different pipeline options that can be used:
pipeline search options
Here is my working example in Spring Boot.
https://github.com/ConsciousObserver/MongoAggregationTest
You can invoke the
/product
REST service using following commandImplementation supports following
productName
(Searches by words, needs text search index)brandName
,categoryName
andsubCategoryName
pageNumber
andpageSize
All of it is implemented using Spring Data APIs. I generally avoid writing native queries in code, as they are not validated at compile time.
All classes are added to one Java file, it’s just a sample so it’s better to keep everything in one place.
Adding code below in case GitHub repository goes down.
pom.xml
MongoAggregationTestApplication.java
Here’s the query that’s being executed by
/products
, I got it fromMongoTemplate
logsHere’s log contents, after a few requests have been fired