Open Source · · 8 days

Intruder signed tampered OpenSSH RPMs with Red Hat key

During the August 2008 Fedora and Red Hat infrastructure intrusions, an attacker got tampered OpenSSH packages for RHEL 4 and RHEL 5 signed with a legitimate Red Hat package key, though Red Hat said RHN subscribers did not receive them through official channels.

The Fedora and Red Hat intrusion started in public with silence, which is why the incident still reads like a case study in distribution trust. On August 14, Fedora warned users not to download or update packages while it investigated an infrastructure issue. A week later, Fedora confirmed that one of the compromised servers was a system used for signing Fedora packages, but said the signing passphrase had not been used during the intrusion and source/package checks had found no evidence of Fedora package tampering. Fedora rotated signing keys anyway.

Red Hat's disclosure landed inside RHSA-2008:0855, an OpenSSH security update. The important supply-chain fact was separate from the ordinary OpenSSH X11-cookie fix in the same advisory: an intruder had managed to sign a small number of tampered OpenSSH packages for RHEL 4 on i386 and x86_64, and RHEL 5 on x86_64. That made the trust boundary different from a random mirrored RPM. The packages carried a legitimate Red Hat signature, which is the exact signal RPM tooling and administrators normally use to decide that an update came from the vendor.

Red Hat said it remained "highly confident" that its systems and processes had prevented the intrusion from compromising Red Hat Network or the content distributed via RHN, and accordingly believed customers updating through RHN were not at risk. The risk sat with people who might have obtained Red Hat binary packages outside the subscriber channel. To make that concrete, Red Hat published openssh-blacklist-1.0.sh, a signed detection script that checked installed packages or candidate RPM files against a list of the known tampered packages. The published page documents its PASS and ALERT output and its RPM-filename arguments; how it compares them is not stated there.

The tampered versions were compact but high-value: openssh-3.9p1-8.RHEL4.24, openssh-3.9p1-9.el4, openssh-4.3p2-26, and openssh-4.3p2-26.el5, across the OpenSSH client, server, askpass, debuginfo, and related subpackages. Public CVE records later tracked the trojaned package issue as CVE-2008-3844 with unknown impact. The lasting lesson is that package signing protects the distribution path only while the signing environment remains controlled; once that boundary is crossed, a signature can authenticate the attacker's artifact.

Notes

  • Red Hat's openssh-blacklist-1.0.sh lists the SIGMD5 of every tampered package, 43 in total across the four builds and their subpackages. SIGMD5 is the RPM header and payload signature that uniquely identifies a built package; it is not the digest md5sum returns for the .rpm file, so it is recorded as rpm_sigmd5 rather than as a file hash.
  • The tampered builds Red Hat named were openssh-3.9p1-8.RHEL4.24 (i386, x86_64), openssh-3.9p1-9.el4 (i386, x86_64), openssh-4.3p2-26 (x86_64), openssh-4.3p2-26.el5 (x86_64). The affected package and subpackage names were openssh, openssh-askpass, openssh-askpass-gnome, openssh-clients, openssh-debuginfo, openssh-server.
  • No digest of the distributed .rpm files themselves has been published. The packages were signed with the Red Hat production key, which is why identification relies on the signature rather than on a download hash.

Appendix · Affected releases

openssh rpm fixed 3.9p1-11.el4_7
3.9p1-8.RHEL4.24 no sample yet
openssh rpm fixed 3.9p1-11.el4_7
3.9p1-9.el4 no sample yet
openssh rpm fixed 4.3p2-26.el5_2.1
4.3p2-26 no sample yet
openssh rpm fixed 4.3p2-26.el5_2.1
4.3p2-26.el5 no sample yet
  • Red Hat stated that RHN and RHN-distributed content were not compromised; this record tracks abuse of the Red Hat signing trust root and the tampered signed RPMs, not confirmed delivery through RHN.
  • Fedora's related signing-system compromise is included as context because Fedora reported no package-integrity discrepancies and no evidence that the Fedora signing passphrase was used during the intrusion.

Indicators

  • packageopenssh-3.9p1-8.RHEL4.24 (i386, x86_64)
  • packageopenssh-3.9p1-9.el4 (i386, x86_64)
  • packageopenssh-4.3p2-26 (x86_64)
  • packageopenssh-4.3p2-26.el5 (x86_64)
  • rpm_sigmd500b6c24146eb6222ec58342841ee31b1
  • rpm_sigmd5021d1401b2882d864037da406e7b3bd1
  • rpm_sigmd5035253874639a1ebf3291189f027a561
  • rpm_sigmd508daefebf2a511852c88ed788717a148
  • rpm_sigmd5177b1013dc0692c16e69c5c779b74fcf
  • rpm_sigmd524c67508c480e25b2d8b02c75818efad
  • rpm_sigmd527ed27c7eac779f43e7d69378a20034f
  • rpm_sigmd52a2f907c8d6961cc8bfbc146970c37e2
  • rpm_sigmd52b0a85e1211ba739904654a7c64a4c90
  • rpm_sigmd52df270976cbbbbb05dbdf95473914241
  • rpm_sigmd52ff426e48190519b1710ed23a379bbee
  • rpm_sigmd5322cddd04ee5b7b8833615d3fbbcf553
  • rpm_sigmd535b050b131dab0853f11111b5afca8b3
  • rpm_sigmd538f67a6ce63853ad337614dbd760b0db
  • rpm_sigmd53b9e24c54dddfd1f54e33c6cdc90f45c
  • rpm_sigmd53fa1a1b446feb337fd7f4a7938a6385f
  • rpm_sigmd541741fe3c73d919c3758bf78efc437c9
  • rpm_sigmd5432b94026da05d6b11604a00856a17b2
  • rpm_sigmd554bd06ebf5125debe0932b2f1f5f1c39
  • rpm_sigmd557f7e73ee28ba0cbbaad1a0a63388e4c
  • rpm_sigmd559ad9703362991d8eff9d138351b37ac
  • rpm_sigmd571ef43e0d9bfdfada39b4cb778b69959
  • rpm_sigmd5760040ec4db1d16e878016489703ec6d
  • rpm_sigmd589892d38e3ccf667e7de545ea04fa05b
  • rpm_sigmd58a65c4e7b8cd7e11b9f05264ed4c377b
  • rpm_sigmd58bf3baa4ffec125206c3ff308027a0c4
  • rpm_sigmd5982cd133ba95f2db580c67b3ff27cfde
  • rpm_sigmd5990d27b6140d960ad1efd1edd5ec6898
  • rpm_sigmd59bef2d9c4c581996129bd9d4b82faafa
  • rpm_sigmd59c90432084937eac6da3d5266d284207
  • rpm_sigmd5a1dea643f8b0bda52e3b6cad3f7c5eb6
  • rpm_sigmd5b54197ff333a2c21d0ca3a5713300071
  • rpm_sigmd5b92ccd4cbd68b3d3cefccee3ed9b612c
  • rpm_sigmd5bb1905f7994937825cb9693ec175d4d5
  • rpm_sigmd5bc6b8b246be3f3f0a25dd8333ad3456b
  • rpm_sigmd5c0aff0b45ee7103de53348fcbedaf72e
  • rpm_sigmd5c7d520faab2673b66a13e58e0346021d
  • rpm_sigmd5ce97e8c02c146c8b1075aad1550b1554
  • rpm_sigmd5d19ae2199662e90ec897c8f753816ee0
  • rpm_sigmd5de61e6e1afd2ca32679ff78a2c3a0767
  • rpm_sigmd5dfbc24a871599af214cd7ef72e3ef867
  • rpm_sigmd5f68d010c6e54f3f8a973583339588262
  • rpm_sigmd5fc814c0e28b674da8afcfbdeecd1e18e

References

  1. Red Hat alert RHSA-2008:0855-01 (openssh)lwn.net
  2. RHSA-2008:0855 - Critical: openssh security updateaccess.redhat.com
  3. Red Hat OpenSSH blacklist script advisorysecurity.access.redhat.com
  4. Red Hat openssh-blacklist-1.0.shsecurity.access.redhat.com
  5. Something going on with Fedoralwn.net
  6. One week of infrastructure issueslwn.net
  7. What happened with Fedora - and Red Hat toolwn.net
  8. Fedora, Red Hat, and distributor securitylwn.net
  9. Re: non-disclosure of infrastructure problem a management issue?lwn.net
  10. Red Hat hack prompts critical OpenSSH updatetheregister.com
  11. Red Hat breached as hackers target Linux serversitpro.com
  12. Red Hat belatedly confirms security breachzdnet.com
  13. ASA-2008-399: openssh security update (RHSA-2008-0855)support.avaya.com
  14. CVE-2008-3844nvd.nist.gov

Source record: oss/attacks/redhat-openssh/meta.yaml