skip to Main Content

I have an array of users and I want to grab a subset of them based on an inexpensive lookup in a redis set.

const users = [
  { _id: '1', name: 'george', age: 36 },
  { _id: '2', name: 'henry', age: 33 },
  { _id: '3', name: 'agatha', age: 28 },
  { _id: '4', name: 'janet', age: 29 },
  { _id: '5', name: 'gary', age: 21 },
  // ... 995 more users
]

const db = {/* my redis connection */}

const isInside = (db, user) => {
  return db.contains('my:set:key', user._id)
}

I have tried Array.prototype.filter but it doesn’t seem to work

users.filter(user => isInside(db, user))
// => always gives me back all the users even when I see they are not in the set

I know something is wrong here. How do I filter out users using isInside?

2

Answers


  1. The problem is that filter is always synchronous and your DB calls are asynchronous, filter function always returns true because its db.contains is a running promise, so it converts to true.

    One of the solutions could be, to create an array of promises, wait for all of them, and then filter out.

    const users = [
      { _id: '1', name: 'george', age: 36 },
      { _id: '2', name: 'henry', age: 33 },
      { _id: '3', name: 'agatha', age: 28 },
      { _id: '4', name: 'janet', age: 29 },
      { _id: '5', name: 'gary', age: 21 },
      // ... 995 more users
    ]
    
    const dbCheck = users.map(user => isInside(db, user))
    
    Promise.all(dbCheck).then((values) => {
      // here you have array of bools [true, false ...]
    
      const filteredUsers = users.filter((_, index) => values[index]))
    });
    
    
    Login or Signup to reply.
  2. You will probably be OK with Promise.all(users.map(user => isInside(db, user))) but there’s a danger of hitting the database too hard with multiple simultaneous requests, particularly with some 3rd-party cloud services.

    If so, then you can orchestrate an asynchronous filter in which db queries are performed sequentially, based on Array.prototype.reduce.

    It’s a bit of a palaver, but not too bad:

    const users = [
    	  { _id: '1', name: 'george', age: 36 },
    	  { _id: '2', name: 'henry', age: 33 },
    	  { _id: '3', name: 'agatha', age: 28 },
    	  { _id: '4', name: 'janet', age: 29 },
    	  { _id: '5', name: 'gary', age: 21 },
    	  // ... 995 more users
    ]
    
    users.reduce(function(promise, user) {
    	return promise
    	.then(arr => {
    		return isInside(null, user) // <<< the asynchronous call
    		.then(bool => { // isInside delivers Boolean
    			if(bool) arr.push(user); // act, depending on asynchronously derived Boolean
    			return arr; // deliver arr to next iteration of the reduction
    		});
    	});
    }, Promise.resolve([])) // starter promise, resolved to empty array
    .then(filtered => {
    	console.log(filtered); // Yay! a filtered array
    });
    
    // dummy isInside() function
    function isInside(db, user) {
    	return Promise.resolve(Math.random() < 0.5); // 50% probability
    }

    Of course, this will be slower than a .map() solution but if .map() doesn’t work ….

    Login or Signup to reply.
Please signup or login to give your own answer.
Back To Top
Search