Uncontrolled Resource Consumption in promhttp
client_golang is the instrumentation library for Go applications in Prometheus, and the promhttp package in client_golang provides tooling around HTTP servers and clients. In client_golang prior to version 1.11.1, HTTP server is susceptible to a Denial of Service through unbounded cardinality, and potential memory exhaustion, when handling requests with non-standard HTTP methods. In order to be affected, an instrumented software must use any of `promhttp.InstrumentHandler*` middleware except `RequestsInFlight`; not filter any specific methods (e.g GET) before middleware; pass metric with `method` label name to our middleware; and not have any firewall/LB/proxy that filters away requests with unknown `method`. client_golang version 1.11.1 contains a patch for this issue. Several workarounds are available, including removing the `method` label name from counter/gauge used in the InstrumentHandler; turning off affected promhttp handlers; adding custom middleware before promhttp handler that will sanitize the request method given by Go http.Request; and using a reverse proxy or web application firewall, configured to only allow a limited set of methods.
02 / AFFECTED SOFTWARE
Affected packages
31 explicit affected versions
03 / CONNECTIONS
Connected vulnerabilities
04 / EVIDENCE
Source records
client_golang is the instrumentation library for Go applications in Prometheus, and the promhttp package in client_golang provides tooling around HTTP servers and clients. In client_golang prior to version 1.11.1, HTTP server is susceptible to a Denial of Service through unbounded cardinality, and potential memory exhaustion, when handling requests with non-standard HTTP methods. In order to be affected, an instrumented software must use any of `promhttp.InstrumentHandler*` middleware except `RequestsInFlight`; not filter any specific methods (e.g GET) before middleware; pass metric with `method` label name to our middleware; and not have any firewall/LB/proxy that filters away requests with unknown `method`. client_golang version 1.11.1 contains a patch for this issue. Several workarounds are available, including removing the `method` label name from counter/gauge used in the InstrumentHandler; turning off affected promhttp handlers; adding custom middleware before promhttp handler that will sanitize the request method given by Go http.Request; and using a reverse proxy or web application firewall, configured to only allow a limited set of methods.
This is the Go client library for Prometheus. It has two separate parts, one for instrumenting application code, and one for creating clients that talk to the Prometheus HTTP API. client_golang is the instrumentation library for Go applications in Prometheus, and the promhttp package in client_golang provides tooling around HTTP servers and clients. ### Impact HTTP server susceptible to a Denial of Service through unbounded cardinality, and potential memory exhaustion, when handling requests with non-standard HTTP methods. ### Affected Configuration In order to be affected, an instrumented software must * Use any of `promhttp.InstrumentHandler*` middleware except `RequestsInFlight`. * Do not filter any specific methods (e.g GET) before middleware. * Pass metric with `method` label name to our middleware. * Not have any firewall/LB/proxy that filters away requests with unknown `method`. ### Patches * https://github.com/prometheus/client_golang/pull/962 * https://github.com/prometheus/client_golang/pull/987 ### Workarounds If you cannot upgrade to [v1.11.1 or above](https://github.com/prometheus/client_golang/releases/tag/v1.11.1), in order to stop being affected you can: * Remove `method` label name from counter/gauge you use in the InstrumentHandler. * Turn off affected promhttp handlers. * Add custom middleware before promhttp handler that will sanitize the request method given by Go http.Request. * Use a reverse proxy or web application firewall, configured to only allow a limited set of methods. ### For more information If you have any questions or comments about this advisory: * Open an issue in https://github.com/prometheus/client_golang * Email us at `[email protected]`
The Prometheus client_golang HTTP server is vulnerable to a denial of service attack when handling requests with non-standard HTTP methods. In order to be affected, an instrumented software must use any of the promhttp.InstrumentHandler* middleware except RequestsInFlight; not filter any specific methods (e.g GET) before middleware; pass a metric with a "method" label name to a middleware; and not have any firewall/LB/proxy that filters away requests with unknown "method".
05 / REFERENCES
Further evidence
- https://github.com/CVEProject/cvelistV5/tree/main/cves/2022/21xxx/CVE-2022-21698.json
- https://github.com/prometheus/client_golang/pull/962
- https://github.com/prometheus/client_golang/pull/987
- https://github.com/prometheus/client_golang/releases/tag/v1.11.1
- https://github.com/prometheus/client_golang/security/advisories/GHSA-cg3q-j54f-5p7p
- https://lists.fedoraproject.org/archives/list/package-announce%40lists.fedoraproject.org/message/2IK53GWZ475OQ6ENABKMJMTOBZG6LXUR/
- https://lists.fedoraproject.org/archives/list/package-announce%40lists.fedoraproject.org/message/2PFW6Q2LXXWTFRTMTRN4ZGADFRQPKJ3D/
- https://lists.fedoraproject.org/archives/list/package-announce%40lists.fedoraproject.org/message/36GUEPA5TPSC57DZTPYPBL6T7UPQ2FRH/
- https://lists.fedoraproject.org/archives/list/package-announce%40lists.fedoraproject.org/message/3L6GDN5S5QZSCFKWD3GKL2RDZQ6B4UWA/
- https://lists.fedoraproject.org/archives/list/package-announce%40lists.fedoraproject.org/message/4KDETHL5XCT6RZN2BBNOCEXRZ2W3SFU3/
- https://lists.fedoraproject.org/archives/list/package-announce%40lists.fedoraproject.org/message/5OGNAFVXSMTTT2UPH6CS3IH6L3KM42Q7/
- https://lists.fedoraproject.org/archives/list/package-announce%40lists.fedoraproject.org/message/7V7I72LSQ3IET3QJR6QPAVGJZ4CBDLN5/
- https://lists.fedoraproject.org/archives/list/package-announce%40lists.fedoraproject.org/message/AK7CJBCGERCRXYUR2EWDSSDVAQMTAZGX/
- https://lists.fedoraproject.org/archives/list/package-announce%40lists.fedoraproject.org/message/DLUJZV3HBP56ADXU6QH2V7RNYUPMVBXQ/
- https://lists.fedoraproject.org/archives/list/package-announce%40lists.fedoraproject.org/message/FY3N7H6VSDZM37B4SKM2PFFCUWU7QYWN/
- https://lists.fedoraproject.org/archives/list/package-announce%40lists.fedoraproject.org/message/HLAQRRGNSO5MYCPAXGPH2OCSHOGHSQMQ/
- https://lists.fedoraproject.org/archives/list/package-announce%40lists.fedoraproject.org/message/J5WPM42UR6XIBQNQPNQHM32X7S4LJTRX/
- https://lists.fedoraproject.org/archives/list/package-announce%40lists.fedoraproject.org/message/KBMVIQFKQDSSTHVVJWJ4QH6TW3JVB7XZ/
- https://lists.fedoraproject.org/archives/list/package-announce%40lists.fedoraproject.org/message/MH6ALXEQXIFQRQFNJ5Y2MJ5DFPIX76VN/
- https://lists.fedoraproject.org/archives/list/package-announce%40lists.fedoraproject.org/message/RN7JGC2LVHPEGSJYODFUV5FEKPBVG4D7/
- https://lists.fedoraproject.org/archives/list/package-announce%40lists.fedoraproject.org/message/SASRKYHT5ZFSVMJUQUG3UAEQRJYGJKAR/
- https://lists.fedoraproject.org/archives/list/package-announce%40lists.fedoraproject.org/message/ZKORFJTRRDJCWBTJPISKKCVMMMJBIRLG/
- https://lists.fedoraproject.org/archives/list/package-announce%40lists.fedoraproject.org/message/ZY2SLWOQR4ZURQ7UBRZ7JIX6H6F5JHJR/
- https://nvd.nist.gov/vuln/detail/CVE-2022-21698
- https://github.com/prometheus/client_golang
- https://lists.fedoraproject.org/archives/list/[email protected]/message/2IK53GWZ475OQ6ENABKMJMTOBZG6LXUR
- https://lists.fedoraproject.org/archives/list/[email protected]/message/2PFW6Q2LXXWTFRTMTRN4ZGADFRQPKJ3D
- https://lists.fedoraproject.org/archives/list/[email protected]/message/36GUEPA5TPSC57DZTPYPBL6T7UPQ2FRH
- https://lists.fedoraproject.org/archives/list/[email protected]/message/3L6GDN5S5QZSCFKWD3GKL2RDZQ6B4UWA
- https://lists.fedoraproject.org/archives/list/[email protected]/message/4KDETHL5XCT6RZN2BBNOCEXRZ2W3SFU3
- https://lists.fedoraproject.org/archives/list/[email protected]/message/5OGNAFVXSMTTT2UPH6CS3IH6L3KM42Q7
- https://lists.fedoraproject.org/archives/list/[email protected]/message/7V7I72LSQ3IET3QJR6QPAVGJZ4CBDLN5
- https://lists.fedoraproject.org/archives/list/[email protected]/message/AK7CJBCGERCRXYUR2EWDSSDVAQMTAZGX
- https://lists.fedoraproject.org/archives/list/[email protected]/message/DLUJZV3HBP56ADXU6QH2V7RNYUPMVBXQ
- https://lists.fedoraproject.org/archives/list/[email protected]/message/FY3N7H6VSDZM37B4SKM2PFFCUWU7QYWN
- https://lists.fedoraproject.org/archives/list/[email protected]/message/HLAQRRGNSO5MYCPAXGPH2OCSHOGHSQMQ
- https://lists.fedoraproject.org/archives/list/[email protected]/message/J5WPM42UR6XIBQNQPNQHM32X7S4LJTRX
- https://lists.fedoraproject.org/archives/list/[email protected]/message/KBMVIQFKQDSSTHVVJWJ4QH6TW3JVB7XZ
- https://lists.fedoraproject.org/archives/list/[email protected]/message/MH6ALXEQXIFQRQFNJ5Y2MJ5DFPIX76VN
- https://lists.fedoraproject.org/archives/list/[email protected]/message/RN7JGC2LVHPEGSJYODFUV5FEKPBVG4D7
- https://lists.fedoraproject.org/archives/list/[email protected]/message/SASRKYHT5ZFSVMJUQUG3UAEQRJYGJKAR
- https://lists.fedoraproject.org/archives/list/[email protected]/message/ZKORFJTRRDJCWBTJPISKKCVMMMJBIRLG
- https://lists.fedoraproject.org/archives/list/[email protected]/message/ZY2SLWOQR4ZURQ7UBRZ7JIX6H6F5JHJR
- https://pkg.go.dev/vuln/GO-2022-0322