ChatGPT解决这个技术问题 Extra ChatGPT

UTF-8 byte[] to String

Let's suppose I have just used a BufferedInputStream to read the bytes of a UTF-8 encoded text file into a byte array. I know that I can use the following routine to convert the bytes to a string, but is there a more efficient/smarter way of doing this than just iterating through the bytes and converting each one?

public String openFileToString(byte[] _bytes)
{
    String file_string = "";

    for(int i = 0; i < _bytes.length; i++)
    {
        file_string += (char)_bytes[i];
    }

    return file_string;    
}
Why can't you just do this String fileString = new String(_bytes,"UTF-8"); ?
Alternatively, you could use BufferedReader to read into a char array.
@CoolBeans I could if I had known to do that ;) Thank you.
Your provided cide does not decode UTF-8. It does not handle any of the code points that require multiple bytes.

g
gladed

Look at the constructor for String

String str = new String(bytes, StandardCharsets.UTF_8);

And if you're feeling lazy, you can use the Apache Commons IO library to convert the InputStream to a String directly:

String str = IOUtils.toString(inputStream, StandardCharsets.UTF_8);

Or Guava's Charsets.UTF_8 if you are on JDK older than 1.7
Use Guava's Charsets.UTF_8 if you are on Android API below 19 too
And if checkstyle says: "Illegal Instantiation: Instantiation of java.lang.String should be avoided.", then what?
You can see in here the java.nio.charset.Charset.availableCharsets() map all the charsets not just the charsets in the StandardCharsets. And if you want to use some other charset and still want to prevent the String constructor from throwing UnsupportedEncodingException you may use java.nio.charset.Charset.forName()
IOUtils.toString(inputStream, StandardCharsets.UTF_8) is deprecated now.
M
Mmir

Java String class has a built-in-constructor for converting byte array to string.

byte[] byteArray = new byte[] {87, 79, 87, 46, 46, 46};

String value = new String(byteArray, "UTF-8");

T
Ted Hopp

To convert utf-8 data, you can't assume a 1-1 correspondence between bytes and characters. Try this:

String file_string = new String(bytes, "UTF-8");

(Bah. I see I'm way to slow in hitting the Post Your Answer button.)

To read an entire file as a String, do something like this:

public String openFileToString(String fileName) throws IOException
{
    InputStream is = new BufferedInputStream(new FileInputStream(fileName));

    try {
        InputStreamReader rdr = new InputStreamReader(is, "UTF-8");
        StringBuilder contents = new StringBuilder();
        char[] buff = new char[4096];
        int len = rdr.read(buff);
        while (len >= 0) {
            contents.append(buff, 0, len);
        }
        return buff.toString();
    } finally {
        try {
            is.close();
        } catch (Exception e) {
            // log error in closing the file
        }
    }
}

G
GETah

You can use the String(byte[] bytes) constructor for that. See this link for details. EDIT You also have to consider your plateform's default charset as per the java doc:

Constructs a new String by decoding the specified array of bytes using the platform's default charset. The length of the new String is a function of the charset, and hence may not be equal to the length of the byte array. The behavior of this constructor when the given bytes are not valid in the default charset is unspecified. The CharsetDecoder class should be used when more control over the decoding process is required.


And if your bytes are not in the platform's default charset, you can use the version that has the second Charset argument to make sure the conversion is correct.
@MikeDaniels Indeed, I did not want to include all the details. Just edited my answer
C
Community

You could use the methods described in this question (especially since you start off with an InputStream): Read/convert an InputStream to a String

In particular, if you don't want to rely on external libraries, you can try this answer, which reads the InputStream via an InputStreamReader into a char[] buffer and appends it into a StringBuilder.


A
Asaph

Knowing that you are dealing with a UTF-8 byte array, you'll definitely want to use the String constructor that accepts a charset name. Otherwise you may leave yourself open to some charset encoding based security vulnerabilities. Note that it throws UnsupportedEncodingException which you'll have to handle. Something like this:

public String openFileToString(String fileName) {
    String file_string;
    try {
        file_string = new String(_bytes, "UTF-8");
    } catch (UnsupportedEncodingException e) {
        // this should never happen because "UTF-8" is hard-coded.
        throw new IllegalStateException(e);
    }
    return file_string;
}

s
scottt

Here's a simplified function that will read in bytes and create a string. It assumes you probably already know what encoding the file is in (and otherwise defaults).

static final int BUFF_SIZE = 2048;
static final String DEFAULT_ENCODING = "utf-8";

public static String readFileToString(String filePath, String encoding) throws IOException {

    if (encoding == null || encoding.length() == 0)
        encoding = DEFAULT_ENCODING;

    StringBuffer content = new StringBuffer();

    FileInputStream fis = new FileInputStream(new File(filePath));
    byte[] buffer = new byte[BUFF_SIZE];

    int bytesRead = 0;
    while ((bytesRead = fis.read(buffer)) != -1)
        content.append(new String(buffer, 0, bytesRead, encoding));

    fis.close();        
    return content.toString();
}

Code edited to make the default be utf-8 to match the OP's question.
s
soulcheck

String has a constructor that takes byte[] and charsetname as parameters :)


b
bragboy

This also involves iterating, but this is much better than concatenating strings as they are very very costly.

public String openFileToString(String fileName)
{
    StringBuilder s = new StringBuilder(_bytes.length);

    for(int i = 0; i < _bytes.length; i++)
    {
        s.append((char)_bytes[i]);
    }

    return s.toString();    
}

my dear lord. String str = new String(byte[]) will do just fine.
This improves the efficiency, but it doesn't decode utf8 data properly.
d
digitaljoel

Why not get what you are looking for from the get go and read a string from the file instead of an array of bytes? Something like:

BufferedReader in = new BufferedReader(new InputStreamReader( new FileInputStream( "foo.txt"), Charset.forName( "UTF-8"));

then readLine from in until it's done.


Sometimes, it's useful to keep the original line delimiters. The OP might want that.
A
Anatoliy Pelepetz

I use this way

String strIn = new String(_bytes, 0, numBytes);


This doesn't specify a character set so you get the platform default character set which may well not be UTF-8.