Skip to content
This repository has been archived by the owner on Jun 20, 2023. It is now read-only.

io.searchbox.client.http.JestHttpClient#deserializeResponse cause oom exception #703

Open
xbcrh opened this issue Jan 21, 2022 · 0 comments

Comments

@xbcrh
Copy link

xbcrh commented Jan 21, 2022

io.searchbox.client.http.JestHttpClient#deserializeResponse cause oom exception

Hi,Recently, when using jestClient, an OOM exception occurred;The positioning problem is as follows

at org.apache.http.util.CharArrayBuffer.expand(CharArrayBuffer.java:60)
at org.apache.http.util.CharArrayBuffer.append(CharArrayBuffer.java:90)
at org.apache.http.util.EntityUtils.toString(EntityUtils.java:248)
at org.apache.http.util.EntityUtils.toString(EntityUtils.java:291)
at io.searchbox.client.http.JestHttpClient.deserializeResponse(JestHttpClient.java:198)

Because the size of the stream is not known in advance, there are many times of expand when executing toString. Each time expand will apply for a memory space that is twice as large in the heap. When there is a lot of returned information, it is easy to cause oom

org.apache.http.util.EntityUtils#toString(org.apache.http.HttpEntity, java.nio.charset.Charset)

public static String toString(
            final HttpEntity entity, final Charset defaultCharset) throws IOException, ParseException {
        Args.notNull(entity, "Entity");
        final InputStream instream = entity.getContent();
        if (instream == null) {
            return null;
        }
        try {
            Args.check(entity.getContentLength() <= Integer.MAX_VALUE,
                    "HTTP entity too large to be buffered in memory");
            int i = (int)entity.getContentLength();
            // DecompressingEntity.getContentLength() = -1;
            if (i < 0) {
                i = 4096;
            }
            Charset charset = null;
            try {
                final ContentType contentType = ContentType.get(entity);
                if (contentType != null) {
                    charset = contentType.getCharset();
                }
            } catch (final UnsupportedCharsetException ex) {
                if (defaultCharset == null) {
                    throw new UnsupportedEncodingException(ex.getMessage());
                }
            }
            if (charset == null) {
                charset = defaultCharset;
            }
            if (charset == null) {
                charset = HTTP.DEF_CONTENT_CHARSET;
            }
            final Reader reader = new InputStreamReader(instream, charset);
            // Insufficient capacity
            final CharArrayBuffer buffer = new CharArrayBuffer(i);
            final char[] tmp = new char[1024];
            int l;
            while((l = reader.read(tmp)) != -1) {
                // Internally expanded multiple times
                buffer.append(tmp, 0, l);
            }
            return buffer.toString();
        } finally {
            instream.close();
        }
    }
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant