📰 TLDR AppSec Weekly 📰

Google launches Sec-Gemini v1, an AI model beating peers in threat analysis with Mandiant and OSV integrations. CaMeL by DeepMind tackles prompt injection with capability-based Python flows and data tracking—no extra AI layers. Slopsquatting rises as LLMs hallucinate package names attackers register to hijack supply chains. Socket exposes disgrasya, a PyPI package used for stealthy carding attacks via real checkout flows. Sourcegraph’s Sherlock uses LLMs to streamline security reviews and cut triage time, while MCP-Scan audits AI toolchains for prompt injection and tool poisoning.

🌶️ 🌶️ This Week in AppSec World 🌶️ 🌶️

Lessons from building Sherlock: Automating security code reviews with Sourcegraph

Sourcegraph’s internal tool Sherlock combines LLMs with Sourcegraph Cody to automate security code reviews, cutting false positives and saving engineers 30+ minutes daily. It correlates scanner alerts with contextual insights, flags edge cases, and prioritizes real risks in pull requests. Sherlock has already uncovered multiple high-severity issues across 400+ PRs—highlighting the power of AI-augmented AppSec 🛡️

Analysis of 5000+ Malicious Open Source Packages

SafeDep analyzed over 5,500 confirmed-malicious OSS packages, revealing that 96% were caught by their engine, with 72% flagged via high-confidence signals. Most came from npm and PyPI, featuring TTPs like Burp Collaborator exfiltration, preinstall script abuse, and dependency confusion via fake high-version packages. Typosquatting was rampant, with common tricks like misspelled names and version bait. 90% of packages were under 10KB, highlighting how small malicious code can be highly impactful.

How security teams fail

Veteran security leader lcamtuf reflects on why corporate security teams often fail — from ad-hoc early hires and reactive fire-fighting to ossified org structures with unclear business outcomes. He explores how cultural rifts and ignored risks culminate in breaches that validate concerns but expose strategic failure. A must-read ethnographic lens on AppSec dysfunction.

Taming the Wild West of ML: Practical Model Signing with Sigstore

Google’s Open Source Security Team released a stable model signing library leveraging Sigstore, enabling cryptographic verification of ML models to prevent tampering across the AI supply chain. This initiative addresses threats like model poisoning and backdoors by ensuring model integrity from training to deployment. The signing tool supports large models and integrates with model hubs like HuggingFace, marking a major milestone in secure AI development.

Breaking Down SharePoint Walls: Hunting for Sensitive Files

Security Friends released ShareFiltrator, a Python tool that abuses SharePoint’s _api/search/query endpoint to enumerate and bulk-download sensitive files like credentials and configs using rtFa and FedAuth cookies. It bypasses GUI restrictions, leverages FQL queries, and restores original timestamps to aid in secret hunting. The post also outlines mitigations using Defender for Cloud Apps and enforcing least privilege in Microsoft 365 environments.

Malicious PyPI Package Targets WooCommerce Stores with Automated Carding Attacks

The Socket Research Team uncovered disgrasya, a PyPI package downloaded over 34,000 times, used to automate credit card testing on WooCommerce stores integrated with CyberSource. The script emulates real shopper behavior to stealthily verify stolen cards and exfiltrates data to a malicious server. This attack bypasses traditional fraud detection by mimicking valid checkout flows—highlighting urgent need for rate-limiting, CAPTCHA, and fraud rules at the e-commerce level.

ImageRunner: A Privilege Escalation Vulnerability Impacting GCP Cloud Run

Tenable Research disclosed ImageRunner, a now-patched GCP vulnerability allowing attackers with limited run.services.update and iam.serviceAccounts.actAs permissions to pull private container images via Cloud Run. By exploiting the service agent’s elevated access, attackers could inject malicious commands and exfiltrate sensitive data. Google fixed the issue by enforcing explicit image access checks as of Jan 28, 2025. The flaw exemplifies Tenable’s “Jenga®” concept—inter-service privilege escalation risks in cloud platforms.

🤖 This Week in AI Security 🤖

CaMeL offers a promising new direction for mitigating prompt injection attacks

Simon Willison reviews DeepMind’s new paper introducing CaMeL (Capabilities for Machine Learning), a system that defends against LLM prompt injection by translating user commands into a secure, auditable Python-like script. Building on Willison’s Dual LLM pattern, CaMeL uses a custom interpreter to track data flow and apply strict capabilities-based access control. Unlike prior approaches that layer more AI, CaMeL relies on classic security engineering principles to enforce trusted execution paths and mitigate injection risks. While not a complete solution, it’s the most credible direction yet for securing AI agents.

The Rise of Slopsquatting: How AI Hallucinations Are Fueling a New Class of Supply Chain Attacks

Socket reports on slopsquatting—a rising threat where attackers register fake packages hallucinated by LLMs like GPT-4 or CodeLlama. A study analyzing 576K code samples found that 19.7% of package suggestions didn’t exist, with repeatable hallucinations making them ideal for abuse. As AI-assisted “vibe coding” grows, so does the risk of blind dependency trust. Tools like Socket’s GitHub app and browser extension help detect suspicious packages before they reach production.

MCP: Building Your SecOps AI Ecosystem

Jack Naglieri explores Model Context Protocol (MCP)—an open, standardized interface enabling LLMs to dynamically connect with security tools like SIEMs, log stores, and detection systems without custom code. MCP supports both exploratory and structured workflows, enhancing analyst productivity by reducing tool-switching and enabling context-aware automation. Unlike purpose-built AI agents, MCP acts as a universal orchestrator, empowering flexible, human-guided investigations across any SecOps stack. As vendors adopt MCP, it may redefine how security teams blend automation, AI, and human expertise.

Google announces Sec-Gemini v1, a new experimental cybersecurity model

Google unveiled Sec-Gemini v1, an experimental AI model designed to bolster cybersecurity workflows with state-of-the-art reasoning and near real-time threat intelligence. Integrating Gemini’s LLM architecture with Google Threat Intelligence and OSV data, it significantly outperforms other models on CTI benchmarks for threat analysis and root cause mapping. Sec-Gemini v1 delivers in-depth threat actor profiling and vulnerability context, and is now available for research collaboration with select organizations.

Do LLMs Consider Security? An Empirical Study on Responses to Programming Questions

A new study by Drexel University researchers tested GPT-4, Claude 3, and Llama 3 on their ability to detect security issues in code. The study found LLMs warned users about vulnerabilities in only 12.6% to 40% of cases, performing better with certain vulnerability types like sensitive information exposure while struggling with others like path traversal. When LLMs did identify issues, they provided more comprehensive information about causes, exploits, and fixes than typical Stack Overflow responses. The researchers demonstrated improvements by integrating static analysis tools like CodeQL with LLM prompts.

🏆 AppSec/AI Sec Tools of The Week 🏆

invariantlabs-ai/mcp-scan

MCP-Scan is a security auditing tool by Invariant Labs that analyzes installed Model Context Protocol (MCP) server configurations for vulnerabilities like prompt injection, tool poisoning, and cross-origin escalation (tool shadowing). It supports platforms like Claude, Cursor, and Windsurf, and includes features like tool hashing (tool pinning) to detect unauthorized changes. MCP-Scan is open source (Apache 2.0) and installable via uvx mcp-scan@latest.

And that’s a wrap for this week! If you enjoy this curated issue, feel free to forward or share it with your appsec folks, team, hackerclub. For feedback, hugs, bugs or chats drop an email to [email protected]