Question posted in
The official documentation can be found here.

Redis – Testing a function with a @cache.memoize gives PicklingError

I have some tests for functions that use cache, for example:

Function:

@retry(stop=stop_after_attempt(3))
@cache.cached(timeout=60, key_prefix='resouce_group_list')
def get_azure_resource_groups():

    data = []
    resource_client = get_azure_resource_client()
    for item in resource_client.resource_groups.list():
        data.append(item)
    return data

Test:

@patch("dev_maintenance.machines.get_azure_resource_client")
def test_get_azure_list_rg(get_azure_resource_client):

    cache.clear()
    data = []
    with app.app_context():
        ret = get_azure_resource_groups()
        get_azure_resource_client.assert_called_once()
        expected = get_azure_resource_client.return_value.resource_groups.list.return_value
        assert len(get_azure_resource_client.return_value.method_calls) == 1
        for item in expected:
            data.append(item)
        assert ret == data
        cache.clear()

The above test works fine, it passes, no errors and the test is using cache.

But i got other tests, and the decorator here does not matter, it will give the same error if i change the decorator to @cache.cache:

Function:

@retry(stop=stop_after_attempt(3))
@cache.memoize(60)
def get_azure_machine_info(rg_name, machine_name, expand="instanceView"):

    try:
        compute_client = get_azure_compute_client()
        return compute_client.virtual_machines.get(rg_name, machine_name, expand=expand)
    except CloudError:
        return None

Test:

@patch("dev_maintenance.machines.get_azure_compute_client")
def test_get_azure_machine_info (get_azure_compute_client):

    cache.delete_memoized(get_azure_machine_info)
    with app.app_context():
        ret = get_azure_machine_info("rg1", "m1")
        print(ret)
        get_azure_compute_client.assert_called_once()
        assert len(get_azure_compute_client.return_value.method_calls) == 1
        assert (
            ret == get_azure_compute_client.return_value.virtual_machines.get.return_value
            )

        get_azure_compute_client.return_value.virtual_machines.get.assert_called_once_with(
            "rg1", "m1", expand="instanceView"
            )
        cache.delete_memoized(get_azure_machine_info)

Now here the test fails with the error on this line ret = get_azure_machine_info("rg1", "m1"):

value = None, from_value = PicklingError("Can't pickle <class 'unittest.mock.MagicMock'>: it's not the same object as unittest.mock.MagicMock")

>   ???
E   tenacity.RetryError: RetryError[<Future at 0x105c7c3d0 state=finished raised PicklingError>]

<string>:3: RetryError

I tried to mock the cache passing a patch decorator like:

@patch("dev_maintenance.machines.cache") or @patch("dev_maintenance.cache")

I tried to set the CACHE_TYPE to null in the test case, instantiating the cache object and passing the config:

cache = Cache()
cache.init_app(app, config={"CACHE_TYPE": "redis"})

but no success so far, any help?

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    I'am writing the answer here for people that need to test functions that are cached with flask-caching and have the same error then me.

    What i needed was to create an Object inside the test and make the mock_value = Object like this:

    First i create a simple class:

    class MachineInfo(object):
    pass

    Then in my test:

    @patch("dev_maintenance.machines.get_azure_compute_client")
def test_get_azure_machine_info (get_azure_compute_client):

    cache.clear()
    expected_res = MachineInfo()
    expected_res.id = "id"
    expected_res.name = "machine1"
    expected_res.location = "location"
    expected_res.hardware_profile = "hardware"
    expected_res.storage_profile = "storage"
    expected_res.network_profile = "network_profile"
    get_azure_compute_client.return_value.virtual_machines.get.return_value = expected_res
    res = get_azure_machine_info("rg1", "m1")
    assert res == expected_res
    cache.clear()

    Then i could assert function_call() == Object or function_call() == mock.return_value

    This simulates what the actual azure returns, an object, so i just make the mock return the object that i created so i can simulate the function itself.


  2. 0

    This is a reference to an old answer, but I think that generally MagicMock objects aren’t meant to be pickled: https://github.com/thadeusb/flask-cache/issues/52

    That error message is different though, and this is more similar to what you are seeing:
    Is there a way to make python pickle ignore "it's not the same object " errors

    Maybe you could replace the domain prefix to the class like the answer above, but I am not sure it will overcome the other difficulties of pickling a MagicMock class:

    `@patch("__main__.get_azure_compute_client")`
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search