Artificial Intelligence Question posted in
ChatGBT is becoming a world-wide phenomena, try it out here.

Receptive fields for feed forward network – Artificial Intelligence

I am pretty new to artificial intelligence and neural networks. I have implemented a feed-forward neural network in PyTorch for classification on the MNIST data set. Now I want to visualize the receptive fields of (a subset of) hidden neurons. But I am having some problems with understanding the concept of receptive fields and when I google it all results are about CNNs. So can anyone help me with how I could do this in PyTorch and how to interpret the results?

Tags:

2

Answers


  1. 0

    I have previously described the concept of a receptive field for CNNs in this answer, just to give you some context that might be useful in my actual answer.

    It seems that you are also struggling with the idea of receptive fields. Generally, you can best understand it by asking the question “which part of the (previous) layer representation is affecting my current input?”

    In Convolutional layers, the formula to compute the current layer only takes part of the image as an input (or at least only changes the outcome based on a this subregion). This is precisely what the receptive field is.
    Now, a fully connected layer, as the name implies, has a connection from every previous hidden state to every new hidden state, see the image below:

    fully connected network

    In that case, the receptive field is simply “every previous state” (e.g., in the image, a cell in the first turquoise layer is affected by all yellow cells), which is not very helpful. The whole idea would be to have a smaller subset instead of all available states.

    Therefore, I think your question regarding implementations in PyTorch do not really make much sense, unfortunately, but I hope that the answer still provided some clarity on the topic.

    As a follow-up, I also encourage you to think about the implications of this “connectedness”, especially when it comes to the number of tunable parameters.

    Login or Signup to reply.
  2. 0

    Generally, you can get the RF by backpropagation(which can be done with framework like pytorch) or formula.

    I’m not familiar with the last method and calculating the RF is really difficult if the net has a very complex structure.So get the RF by backpropagation may be the better way.

    Here is a blog which may be helpful to this question:
    https://learnopencv.com/cnn-receptive-field-computation-using-backprop/

    And for the visualization of RF, here is a implementation :

    https://github.com/shelfwise/receptivefield

    I suppose that you may already solve this problem since three years have past.But I must say it’s a really interesting question and worth to think.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search