Question posted in 
I have a huge file with data in the below format. (It’s the response from an API call I made to one of Twitter’s APIs). I want to extract the value of the field “followers_count” from it. Ordinarily, I would do this with jq with the following command : cat | jq -r ‘.followers_count’
But this contains special characters so jq cannot handle it. Can someone help by telling me how do I convert it in JSON (e.g. using a shell script) or alternatively how to get the followers_count field without conversion? If this format has a specific name, I would be interested to know about it.
Thanks.
SAMPLE LINE IN FILE:
b'[{“id”:2361407554,”id_str”:”2361407554″,”name”:”hakimo ait”,”screen_name”:”hakimo_ait”,”location”:””,”description”:””,”url”:null,”entities”:{“description”:{“urls”:[]}},”protected”:false,”followers_count”:0,”friends_count”:6,”listed_count”:0,”created_at”:”Sun Feb 23 19:08:04 +0000 2014″,”favourites_count”:0,”utc_offset”:null,”time_zone”:null,”geo_enabled”:false,”verified”:false,”statuses_count”:1,”lang”:”fr”,”status”:{“created_at”:”Sun Feb 23 19:09:21 +0000 2014″,”id”:437665498961293312,”id_str”:”437665498961293312″,”text”:”c.ronaldo”,”truncated”:false,”entities”:{“hashtags”:[],”symbols”:[],”user_mentions”:[],”urls”:[]},”source”:”u003ca href=”https://mobile.twitter.com” rel=”nofollow”u003eMobile Web (M2)u003c/au003e”,”in_reply_to_status_id”:null,”in_reply_to_status_id_str”:null,”in_reply_to_user_id”:null,”in_reply_to_user_id_str”:null,”in_reply_to_screen_name”:null,”geo”:null,”coordinates”:null,”place”:null,”contributors”:null,”is_quote_status”:false,”retweet_count”:0,”favorite_count”:0,”favorited”:false,”retweeted”:false,”lang”:”es”},”contributors_enabled”:false,”is_translator”:false,”is_translation_enabled”:false,”profile_background_color”:”C0DEED”,”profile_background_image_url”:”http://abs.twimg.com/images/themes/theme1/bg.png”,”profile_background_image_url_https”:”https://abs.twimg.com/images/themes/theme1/bg.png”,”profile_background_tile”:false,”profile_image_url”:”http://abs.twimg.com/sticky/default_profile_images/default_profile_normal.png”,”profile_image_url_https”:”https://abs.twimg.com/sticky/default_profile_images/default_profile_normal.png”,”profile_link_color”:”1DA1F2″,”profile_sidebar_border_color”:”C0DEED”,”profile_sidebar_fill_color”:”DDEEF6″,”profile_text_color”:”333333″,”profile_use_background_image”:true,”has_extended_profile”:false,”default_profile”:true,”default_profile_image”:true,”following”:false,”follow_request_sent”:false,”notifications”:false,”translator_type”:”none”}]’
2
Answers
This is not the valid JSON, if you want to grab some certain part from this response, you can dump this result in file and then iterate over it and get the text you want to grab.
otherwise, if response will be in JSON, it will be easily parse through jq library, you can also dump this record in file, convert it into json and then parse it !
there are multiple ways ‘grep,awk,sed’ ….. you can go for it !
remove ‘b from beginning and ‘ from bottom,it will become valid JSON !
Well i have removed the b’ from beginning and ‘ from the bottom ! and look it is a valid JSON, now we can easily use jq with it like this !
i am doing it with my file….