Memcached Question posted in
A decent usage manual can be found here.

Memcached – Why is php converting certain characters to '?'

Everything in my code is running my database(Postgresql) is using utf8 encoding, I’ve checked the php.ini file its encoding is utf8, I tried debugging to see if it was any of the functions I used that were doing this, but nothing everything is running as expected, however after my frontend sends a post request to backend server through curl for some text to be inserted in the database, some characters like ‘da’ are converted to ‘?’ in postgre and in memcached, I think php is converting them to Latin-1 again after the request reaches the other side for some reason becuase I use utf8_encode before the request and utf8_decode on the other side

 this is the code to send the request
         $pre_opp-> 
    
   Send_Request_To_BackEnd("/Settings",$school_name,$uuid,"Upload_Bio","POST",str_replace(" ","%",utf8_encode($bio)));

this is how the backend system receives this

  $data= str_replace("%"," ",utf8_decode($_POST["Data"]));
Tags:

2

Answers


  1. 0

    Don’t replace " " with "%".

    Use urlencode and urldecode instead of utf8_encode and utf8_decode – It will give you a clean alphanumeric representation of any character to easily transport your data.

    If everything in your environment defaults to UTF-8, you shouldn’t need utf_encode and utf_decode anyways, I guess. But if you still do, you could try combining both like this:

    Send_Request_To_BackEnd("/Settings",$school_name,$uuid,"Upload_Bio","POST", urlencode(utf8_encode($bio)));

    and

    $data= str_replace("%"," ",utf8_decode(urldecode($_POST["Data"])));
    Login or Signup to reply.
  2. 0

    You say this like it’s a mystery:

    I think php is converting them to Latin-1 again after the request reaches the other side for some reason

    But then you give the reason yourself:

    because I use utf8_encode before the request and utf8_decode on the other side

    That is exactly what uf8_decode does: it converts UTF-8 to Latin-1.

    As the manual explains, this is also where your ‘?’ replacements come from:

    This function converts the string string from the UTF-8 encoding to ISO-8859-1. Bytes in the string which are not valid UTF-8, and UTF-8 characters which do not exist in ISO-8859-1 (that is, characters above U+00FF) are replaced with ?.

    Since you’d picked the unfortunate replacement of % for space, sequences like "%da" were being interpreted as URL percent escapes, and generating invalid UTF-8 strings. You then asked PHP to convert them to Latin-1, and it couldn’t, so it substituted "?".

    The simple solution is: don’t do that. If your data is already in UTF-8, neither of those functions will do anything but mess it up; if it’s not already in UTF-8, then work out what encoding it’s in and use iconv or mb_convert_encoding to convert it, once. See also "UTF-8 all the way through".

    Since we can’t see your Send_Request_To_BackEnd function, it’s hard to know why you thought you needed it. If you’re constructing a URL with that string, you should use urlencode inside your request sending code; you shouldn’t need to decode it the other end, PHP will do that for you.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search