{"data":{"slug":"llm-platform-security-chatgpt-plugin-eval","name":"chatgpt-plugin-eval","tagline":"Framework for Evaluating Security in LLM Plugin Ecosystems","github_url":"https://github.com/llm-platform-security/chatgpt-plugin-eval","owner":"llm-platform-security","repo":"chatgpt-plugin-eval","owner_avatar_url":"https://avatars.githubusercontent.com/u/144710465?v=4","primary_language":"HTML","stars":29,"forks":7,"topics":["chatgpt","chatgpt-plugins","llm","llm-platform","llm-platform-security","llm-privacy","llm-security","openai"],"archived":false,"github_pushed_at":"2024-07-29T23:26:59+00:00","maintenance_label":"Dormant","url":"https://www.graphcanon.com/tools/llm-platform-security-chatgpt-plugin-eval","markdown_url":"https://www.graphcanon.com/tools/llm-platform-security-chatgpt-plugin-eval.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/llm-platform-security-chatgpt-plugin-eval","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=llm-platform-security-chatgpt-plugin-eval","description":"LLM Platform Security: Applying a Systematic Evaluation Framework to OpenAI's ChatGPT Plugins","homepage_url":"https://llm-platform-security.github.io/chatgpt-plugin-eval/","license":null,"open_issues":1,"watchers":2,"ai_summary":"Introduces a systematic evaluation framework aimed at assessing security, privacy, and safety concerns associated with third-party plugins interfacing with large language models like ChatGPT.","readme_excerpt":"## LLM Platform Security: Applying a Systematic Evaluation Framework to OpenAI's ChatGPT Plugins\nLarge language model (LLM) platforms, such as OpenAI's ChatGPT, have recently begun integrating a plugin ecosystem to interface with third-party services on the internet. While these plugins extend the capabilities of LLM platforms, they are developed by arbitrary third parties and thus should not be implicitly trusted. Plugins also interface with LLM platforms and users through natural language, which can have ambiguous and imprecise interpretation. We worry that LLM platform plugin ecosystems are emerging without a systematic consideration for security, privacy, and safety.\n\nThus, we propose a framework that lays a foundation for LLM platform designers to analyze and improve the security, privacy, and safety of current and future plugin-integrated LLM platforms. Our framework is a formulation of an attack taxonomy that is developed by iteratively exploring how plugins, LLM platforms, and users could leverage their capabilities and responsibilities to mount attacks against each other. As part of our iterative process, we apply our framework in the context of OpenAI's plugin ecosystem. (While we look at OpenAI, we believe that the issues have the potential to be industry-wide.) We uncover plugins that concretely demonstrate the potential for the issues that we outline in our attack taxonomy to manifest in practice. We conclude by discussing novel challenges and by providing recommendations to improve the security, privacy, and safety of future LLM platforms.\n\nWe provide a brief FAQ below to highlight some of our findings. We suggest reading the full paper for more details. Please reach out to us if you have additional questions. Link to the paper: https://arxiv.org/abs/2309.10254\n\n\n## Frequently asked questions\n\n### What was the motivation behind this research?\nLLM platforms are extending their capabilities by integrating a [plugin ecosystem](https://openai.com/blog/chatgpt-plugins). This can potentially raise a number of security and privacy issues. First, plugins are developed by third parties, which in prior computing platforms (such as on the web and on smartphones) have brought a number of security and privacy issues. Secondly, plugins interface with LLM platforms and users through natural language, which can have ambiguous and imprecise interpretation. Third, LLM platform vendors, such as OpenAI, currently only impose modest restrictions on third-party plugins with a [handful of policies](https://platform.openai.com/docs/plugins/review/plugin-store) and — based on our analysis and [anecdotal evidence found online](https://embracethered.com/blog/posts/2023/chatgpt-plugin-vulns-chat-with-code/) — a frail review process.\n\nWe believe these concerns highlight that LLM platform plugin ecosystems are emerging mostly without a systematic consideration for security, privacy, and safety. Although third party integrations are currently in beta and users have to opt in to use them, if widely deployed without security considerations, such integrations could result in harm to the users, plugins, and LLM platforms. Thus, to lay a systematic foundation for secure LLM platforms and integrations as a whole, we propose a framework that can be leveraged by current and future designers of LLM platforms.\n\nLooking ahead, we anticipate that third-party plugin integration in LLM platforms is only the beginning of an era of *LLMs as computing platforms*. In parallel with innovation in the core LLMs, we expect to see systems and platform level innovations in how LLMs are integrated into web and mobile ecosystems, the IoT, and even core operating systems. The security and privacy issues that we identify in the context of LLM plugin ecosystems are \"canaries in the coalmine\" (i.e., advance warnings of future concerns and challenges), and our framework can help lay a foundation for these emerging LLM-based computing platforms.\n\n### How did you","github_created_at":"2023-09-11T15:30:33+00:00","created_at":"2026-07-11T23:42:01.293654+00:00","updated_at":"2026-08-05T06:00:56.208224+00:00","categories":[{"slug":"evaluation-observability","name":"Evaluation & Observability","url":"https://www.graphcanon.com/categories/evaluation-observability","markdown_url":"https://www.graphcanon.com/categories/evaluation-observability.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/evaluation-observability"}],"tags":[{"slug":"chatgpt","name":"chatgpt"},{"slug":"llm-plugins","name":"llm-plugins"},{"slug":"privacy","name":"privacy"},{"slug":"security","name":"security"}],"trust":{"provenance":{"is_fork":false,"github_id":690129930,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-08-05T06:00:55.492Z","maintenance":{"label":"Dormant","score":18,"methodology":"github_public_v1","releases_90d":0,"days_since_push":736,"last_release_at":null},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T23:42:02.840Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-08-05T06:00:55.941Z"},"languages":{"value":["html"],"source":"github.language","observed_at":"2026-08-05T06:00:55.941Z"}},"decision_facts":{"hosting":null,"pricing":{"model":"freemium"},"requirements":{"min_ram_gb":null,"requires_docker":false},"constraints":{"min_ram_gb":null,"pricing_model":"freemium","requires_docker":false},"when_to_use":["- When evaluating the security risks of integrating third-party services into your LLM platform through plugins","- If you are developing for OpenAI's plugin ecosystem and want a structured approach to mitigate potential attacks and vulnerabilities"],"when_not_to_use":["- In cases where only generic, high-level security guidance is required without an in-depth framework analysis","- When the primary focus is on improving performance metrics rather than addressing specific security and privacy concerns of LLM plugins"],"source":"enrich:decision_facts","observed_at":"2026-07-17T01:22:45.527Z"},"constraint_facets":{"min_ram_gb":null,"pricing_model":"freemium","requires_docker":false},"decision_summary":[{"label":"Pricing","value":"freemium"},{"label":"Adopt for","value":"chatgpt-plugin-eval is an evaluation framework designed specifically to assess security, privacy, and safety concerns related to third-party plugins interfacing with large language models like ChatGPT."},{"label":"License detail","value":"The license information for chatgpt-plugin-eval is unknown."}]}}