'Malformed UTF-8 characters, possibly incorrectly encoded' in Laravel

I wrote this method to handle UTF8 arrays and JSON problems. It works fine with array (simple and multidimensional).

/**
 * Encode array from latin1 to utf8 recursively
 * @param $dat
 * @return array|string
 */
   public static function convert_from_latin1_to_utf8_recursively($dat)
   {
      if (is_string($dat)) {
         return utf8_encode($dat);
      } elseif (is_array($dat)) {
         $ret = [];
         foreach ($dat as $i => $d) $ret[ $i ] = self::convert_from_latin1_to_utf8_recursively($d);

         return $ret;
      } elseif (is_object($dat)) {
         foreach ($dat as $i => $d) $dat->$i = self::convert_from_latin1_to_utf8_recursively($d);

         return $dat;
      } else {
         return $dat;
      }
   }
// Sample use
// Just pass your array or string and the UTF8 encode will be fixed
$data = convert_from_latin1_to_utf8_recursively($data);

I found the answer to this problem here

Just do

mb_convert_encoding($data['name'], 'UTF-8', 'UTF-8');

In my case I had a ucfirst on the asian letters string. This was not possible and produced a non utf8 string.

In Laravel 7.x, this helped me to get rid of this error.

$newString = mb_convert_encoding($arr, "UTF-8", "auto");
return response()->json($newString);

In my case, this causes error:

return response->json(["message" => "Model status successfully updated!", "data" => $model], 200);

but this not:

return response->json(["message" => "Model status successfully updated!", "data" => $model->toJson()], 200);

I've experienced the same problem. The thing is that I forgot to start the apache and mysql in xampp... :S

I know it's already an old question, but i had the same error today. For me setting the connection variable on model did the work.

/**
 * Table properties
 */
protected $connection = 'mysql-utf8';
protected $table = 'notification';
protected $primaryKey = 'id';

I don't know if the issue was with the database (probably), but the texts fields with special chars (like ~, ´ e etc) were all messed up.

---- Editing

That $connection var is used to select wich db connection your model will use. Sometimes it happens that in database.php (under /config folder) you have multiples connections and the default one is not using UTF-8 charset.

In any case, be sure to properly use charset and collation into your connection.

'connections' => [

    'mysql' => [
        'driver' => 'mysql',
        'host' => env('DB_HOST', '127.0.0.1'),
        'port' => env('DB_PORT', '3306'),
        'database' => env('DB_DATABASE', 'your_database'),
        'username' => env('DB_USERNAME', 'root'),
        'password' => env('DB_PASSWORD', 'database_password'),
        'unix_socket' => env('DB_SOCKET', ''),
        'prefix' => '',
        'strict' => false,
        'engine' => null
    ],

    'mysql-utf8' => [
        'driver' => 'mysql',
        'host' => env('DB_HOST', '127.0.0.1'),
        'port' => env('DB_PORT', '3306'),
        'database' => env('DB_DATABASE', 'your_database'),
        'username' => env('DB_USERNAME', 'root'),
        'password' => env('DB_PASSWORD', 'database_password'),
        'unix_socket' => env('DB_SOCKET', ''),
        'charset' => 'utf8',
        'collation' => 'utf8_unicode_ci',
        'prefix' => '',
        'strict' => false,
        'engine' => null
    ],

For more solution i have completed the solution of (the great) Tiago Gouvêa exposed before only for strings and arrays

i used md_convert_encoding() function instead of utf8_encode(), it works for me : (12 hours loosed ...)

// this object return me a big array who have multiple arrays imbricated

$get_days = program::get_days($ARR, $client);

// and i use this function for well parsing what the server return *

function convert_to_utf8_recursively($dat){
    if( is_string($dat) ){
        return mb_convert_encoding($dat, 'UTF-8', 'UTF-8');
    }
    elseif( is_array($dat) ){
        $ret = [];
        foreach($dat as $i => $d){
            $ret[$i] = convert_to_utf8_recursively($d);
        }
        return $ret;
    }
    else{
        return $dat;
    }
}
                
// use 
$data = convert_to_utf8_recursively($get_days);

what the server return * i talk to that because i test the same code in two different servers the first return me a well formatted json , without any function, as we usually do BUT the second server does not send me back anything if I do not apply this function ...

In my case, it happened twice:

I forgot to migrate again. I roll back (php artisan rollback) the latest migration operation because I want to change something on my database schema but I forgot to execute again the migrate command: php artisan migrate.

More info on migrations: https://laravel.com/docs/9.x/migrations

It also happened to me when the API is returning JSON responses with attributes of type blob (column on the database that is created with the binary() method). For my use case, I hide that attribute by adding the attribute's name to the model's array property $hidden. It works fine after that.

More info on hiding attributes on JSON: https://laravel.com/docs/9.x/eloquent-serialization#hiding-attributes-from-json

I got this error and i fixed the issue with iconv function like following:

iconv('latin5', 'utf-8', $data['index']);

Set the charset at after you made the connection to db like

// Create connection
$conn = new mysqli($servername, $username, $password, $dbname);
// Check connection
if ($conn->connect_error) {
    die("Connection failed: " . $conn->connect_error);
}

if (!$conn->set_charset("utf8")) {
    printf("Error loading character set utf8: %s\n", $conn->error);
    exit();
} else {
    printf("Current character set: %s\n", $conn->character_set_name());
}

First thing Correctly specify all settings in the .ENV file for Mail.

I LOST 6 HOURS FOR THIS

Example

MAIL_DRIVER=smtp
MAIL_MAILER=smtp
MAIL_HOST=smtp.yandex.ru
MAIL_PORT=465
MAIL_USERNAME=yourmail@yandex.ru
MAIL_PASSWORD=password //Create password for Apps in settings. NOT PASTE YOUR REAL MAIL PASSWORD
MAIL_ENCRYPTION=SSL
MAIL_FROM_ADDRESS=yourmail@yandex.ru
MAIL_FROM_NAME="${APP_NAME}"

I experienced the same issue. However, in my case I had the column in the table specified as varchar(255) and my content was larger when I was doing inserts. So it used to truncate the part which could not fit in the varchar column size.

To fix this, I updated my column type to longtext so larger text can be stored. Also, added a frontend validation to not have text larger than the column. (I was using this to store draft js escaped HTML)

Found an answer here.

The solution turned out to be in my application code, it has nothing to do with Laravel itself.

return mb_strtoupper(mb_substr($this->first_name, 0, 1) . mb_substr($this->last_name, 0, 1));

https://barryvanveen.nl/blog/67-solving-malformed-utf-8-characters-possibly-incorrectly-encoded-error-in-laravel

This was my problem and this is how I solved it, hope it works for someone else.

Make sure the encoding of the file is UTF-8, in vscode is in the right bottom part.

https://i.stack.imgur.com/LwuDn.png

I had this particular problem when I modified a file that already was in a development server, somehow the encoding when I edited that file in that server was changed to ANSI, so, make sure too that your file is encoded in UTF-8 in the server where you are doing your deployments.

In my case, the error was similar to what @dev1234 mentioned here (https://stackoverflow.com/a/71279140/7880201) as part of the suggested answers.

But the problem was as a result of laravel outputing the entire error which contained blob(s). I would advise you do not return just the error message in the case of

try {
} catch (\Throwable $th) {

    // $code = $th->getCode();
    $msg = $th->getMessage();
    // dd($msg);
    abort(404, 'THERE WAS AN ERROR: ' . $msg);

}

So to rectify this issue, just show a little friendly error even if you want to show it to yourself alone as a dev.

Check all blobs to ensure that laravel is not just outputing them from the ajax response and you will be fine.

I hope it saves you today. If not, there are still good answers to try. :)

This happened to me when Laravel tried to return an error in Json format (while developing API). After solving the bug, this error disappeared.