Facebook API Question posted in
The official documentation for the Facebook APIs can be found here.

Facebook Not Scraping URL Properly (Blocked URL) On Fresh URL's – Facebook api

I have started to experience a really strange problem with Facebook link scraping/caching which seems random in nature.

On brand new posts when I go to check: https://developers.facebook.com/tools/debug/ with my new URL I get a message saying ‘Cannot scrape: URL blocked’

We have literally been sharing URL’s to Facebook for YEARS with no problem whatsoever. All open graph tags are present and working fine.

What is really weird is if I just manually press the ‘scrape again’ button it grabs all the bits properly and says everything is fine. It’s always just the first attempt.

I have also tried manually calling the scrap API instead of letting Facebook do it when someone shares the URL for the first time and still sometimes (not always) it returns empty (just ‘id’ and ‘url’ returned) indicating the URL was blocked.

What’s going on here it’s driving me mad? We are nowhere near our API limits if maybe that was the issue…

Why would it say blocked first go on the debug page then literally 2 seconds later you press ‘scrape again’ and it works fine.

Just to note sometimes when it says blocked it still shows maybe the image in the link preview but no title or description. It’s like it’s grabbed some tags but then given up. Very strange…

I have checked our access logs and I can see ‘Facebot’ accessing our site fine.

Hope someone can help, I am really hoping this is some weird Facebook bug as I can’t see any change in our code base that would effect this.

Facebook does not return anything helpful when this happens even with the API. I have no way in contacting Facebook direct either. If we are somehow getting blocked I would love to know why and fix that issue. But it’s impossible to debug at the moment.

Just to note I have 3 sites all running the same code base (WordPress environment). They are all experiencing this issue. All been running fine for many years.

Cheers

Tags:

2

Answers


  1. Chosen as BEST ANSWER
    0

    This was to do with rate limiting that was not visible in the Facebook APP dashboard. Helpful...

    Full details here: https://developers.facebook.com/bugs/232343637174087/


  2. 0

    Solution for your problem: You can use Facebook image button after adding link on Facebook news feed, like shown in iamge

    second thing upload image on othe site and give the image path to your website and then put it on your news feed, you can get definitely positive result using this two methods.
    also refer this image

    thanks hope this two will work for your problem

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search