Handling HTTP Event Collector (HEC) Content-Length too large errors without pulling your hair out

Updated 9/17/21: Some of the content below re: maximum content length may be outdated. Please reference the most current documentation here.

Once you start using HEC, you want to send it more and more data, as you do your payloads are going to increase in size, especially if you start batching. Unfortunately as soon as you exceed a request payload size of close to 1MB (for example if you use our Akamai app or send events from AWS Lambda) you’ll get an error status 413, with a not so friendly error message:

“Content-Length of XXXXX too large (maximum is 1000000) “

At this point you might feel tempted to pull your hair out, but fortunately you have options. The reason you are hitting this error is because HEC has a pre-defined limit on the maximum content length for the request. Fortunately this limit is configurable via limits.conf.

If you look in $SPLUNK_HOME$/etc/system/default/limits.conf you’ll see the following:

# The max request content length.
max_content_length = 1000000

All you need to do is up that limit in /etc/system/local/limits.conf and restart your Splunk instance and you’ll be good to go. If you are hosted in Splunk Cloud, our support folks will be more than happy to take care of it for you.

As a side note, we’ll be upping this default in our next release to 800MB, so that you are never bothered by this error again.

----------------------------------------------------
Thanks!
Glenn Block