{"data":{"slug":"jina-ai-langchain-serve","name":"langchain-serve","tagline":"Self-host LLM Apps with Docker Compose or Kubernetes","github_url":"https://github.com/jina-ai/langchain-serve","owner":"jina-ai","repo":"langchain-serve","owner_avatar_url":"https://avatars.githubusercontent.com/u/60539444?v=4","primary_language":"Python","stars":1639,"forks":133,"topics":["autogpt","autonomous-agents","babyagi","chatbot","fastapi","gpt","langchain","llm","production","python","slack"],"archived":true,"github_pushed_at":"2023-09-20T04:01:50+00:00","maintenance_label":"Archived","stars_delta_30d":-1,"url":"https://www.graphcanon.com/tools/jina-ai-langchain-serve","markdown_url":"https://www.graphcanon.com/tools/jina-ai-langchain-serve.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/jina-ai-langchain-serve","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=jina-ai-langchain-serve","description":"⚡ Langchain apps in production using Jina & FastAPI","homepage_url":"https://cloud.jina.ai","license":"Apache-2.0","open_issues":15,"watchers":5,"ai_summary":"Langchain apps in production using Jina & FastAPI, supports deployment via Docker Compose or Kubernetes with secrets management.","readme_excerpt":"### 🏠 Self-host LLM Apps with Docker Compose or Kubernetes\n\n  - 🚀 Export your apps as Kubernetes or Docker Compose YAMLs with single command.\n  - 👉 `lc-serve export app --kind <kubernetes/docker-compose> --path .`\n  - 📦 Deploy your app on your own internal infrastructure with your own security policies.\n  - 📞 [Talk to us if you need all the features of Jina AI Cloud on your own infrastructure.](#-reach-out-to-us)\n\n---\n\n## 🗝️ Using Secrets during Deployment\n\nYou can use secrets during app deployment by passing a secrets file to deployment with the `--secrets` flag. The secrets file should be a `.env` file containing the secrets.\n\n```bash\nlcserve deploy jcloud app --secrets .env\n```\n\n<details>\n<summary>Show details</summary>\n\nLet's take an example of a simple app that uses `OPENAI_API_KEY` stored as secrets.\n\nThis app directory contains the following files:\n\n```\n.\n├── main.py             # The app\n├── jcloud.yml          # JCloud deployment config file\n├── README.md           # This README file\n├── requirements.txt    # The requirements file for the app\n└── secrets.env         # The secrets file containing the redis credentials\n```\n\n> **Note**\n> `secret.env` in this directory is a dummy file. You should replace it with your own secrets after creating a Redis instance. (For example with [Upstash](https://upstash.com/)), such as:\n\n```text\nOPENAI_API_KEY=sk-xxx\n```\n\n`main.py` will look like:\n\n```python\n\n---\n\n## 💰 Pricing\n\nApplications hosted on JCloud are priced in two categories:\n\n**Base credits**\n\n- Base credits are charged to ensure high availability for your application by maintaining at least one instance running continuously, ready to handle incoming requests. If you wish to stop the serving application, you can either remove the app completely or put it on pause, the latter allows you to resume the app serving based on persisted configurations (refer to [`lc-serve` CLI section](#-lc-serve-cli) for more information). Both options will halt the consumption of credits.\n- Actual credits charged for base credits are calculated based on the [instance type as defined by Jina AI Cloud](https://docs.jina.ai/concepts/jcloud/configuration/#cpu-tiers).\n- By default, instance type `C3` is used with a minimum of 1 instance and [Amazon EFS](https://aws.amazon.com/efs/) disk of size 1G, which means that if your application is served on JCloud, you will be charged ~10 credits per hour.\n- You can change the instance type and the minimum number of instances by providing a YAML configuration file using the `--config` option. For example, if you want to use instance type `C4` with a minimum of 0 replicas, and 2G EFS disk, you can provide the following configuration file:\n  ```yaml\n  instance: C4\n  autoscale_min: 0\n  disk_size: 2G\n  ```\n\n**Serving credits**\n\n- Serving credits are charged when your application is actively serving incoming requests.\n- Actual credits charged for serving credits are calculated based on the credits for the instance type multiplied by the duration for which your application serves requests. \n- You are charged for each second your application is serving requests.\n\n\n**Total credits charged = Base credits + Serving credits**. ([Jina AI Cloud](https://cloud.jina.ai/pricing) defines each credit as €0.005)\n\n---\n\n### JCloud deployment failed at pushing image to Jina Hubble, what should I do?\n\n<details>\n<summary><b>Expand</b></summary>\n\nPlease use `--verbose` and retry to get more information. If you are operating on computer with `arm64` arch, please retry with `--platform linux/amd64` so the image can be built correctly.\n\n</details>","github_created_at":"2023-03-21T03:08:56+00:00","created_at":"2026-07-11T10:44:39.84304+00:00","updated_at":"2026-09-20T04:30:01.378566+00:00","categories":[{"slug":"developer-tools","name":"Developer Tools","url":"https://www.graphcanon.com/categories/developer-tools","markdown_url":"https://www.graphcanon.com/categories/developer-tools.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/developer-tools"},{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"}],"tags":[{"slug":"autogpt","name":"autogpt"},{"slug":"autonomous-agents","name":"autonomous-agents"},{"slug":"babyagi","name":"babyagi"},{"slug":"chatbot","name":"chatbot"},{"slug":"fastapi","name":"fastapi"},{"slug":"gpt","name":"gpt"},{"slug":"langchain","name":"langchain"},{"slug":"llm","name":"llm"}],"trust":{"provenance":{"is_fork":false,"github_id":616766088,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-09-18T08:20:29.274Z","maintenance":{"label":"Archived","score":8,"methodology":"github_public_v1","releases_90d":0,"days_since_push":1094,"last_release_at":"2023-08-22T10:12:03Z","stars_delta_30d":-1,"open_issues_delta_30d":0},"security_summary":{"status":"ok","scanner":"osv@v1","low_count":0,"high_count":0,"last_scan_at":"2026-09-18T08:20:31.451Z","medium_count":0,"scan_profile":"deps","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-09-18T08:20:28.135Z"},"languages":{"value":["python"],"source":"github.language+pyproject.toml","observed_at":"2026-09-18T08:20:28.135Z"},"license_spdx":{"value":"Apache-2.0","source":"github.license","observed_at":"2026-09-18T08:20:28.135Z"}},"decision_facts":{"hosting":null,"pricing":{"model":"freemium","summary":"Base credits ensure high availability by maintaining at least one instance running continuously, while serving credits are charged when the application is actively serving requests. Pricing is based, "},"requirements":{"notes":["Requires Docker Compose or Kubernetes for deployment.","Secrets management is supported via `.env` files."],"min_ram_gb":1,"requires_docker":true},"constraints":{"min_ram_gb":1,"pricing_model":"freemium","requires_docker":true},"when_to_use":["When you need to deploy Langchain applications with Jina and FastAPI on your own infrastructure.","If you require secrets management during deployment, allowing you to securely handle sensitive information like API keys.","When you want to export your applications as Kubernetes or Docker Compose YAML files with a single command.","If you are looking for a solution that integrates with Jina AI Cloud features but on your own infrastructure."],"when_not_to_use":["If you do not require or prefer not to use Jina and FastAPI for your Langchain applications.","When you do not need or want to manage secrets through a `.env` file during deployment.","If you are not interested in deploying your applications using Docker Compose or Kubernetes.","If you are looking for a fully managed cloud service without the need to manage your own infrastructure."],"source":"enrich:decision_facts","observed_at":"2026-09-18T10:45:07.239Z"},"constraint_facets":{"min_ram_gb":1,"pricing_model":"freemium","requires_docker":true},"decision_summary":[{"label":"Pricing","value":"freemium - Base credits ensure high availability by maintaining at least one instance running continuously, while serving credits are charged when the application is actively serving requests. Pricing is based, "},{"label":"Requirements","value":"Min 1 GB RAM; Requires Docker; Requires Docker Compose or Kubernetes for deployment.; Secrets management is supported via `.env` files."},{"label":"Adopt for","value":"Langchain-serve is a tool for deploying Langchain applications in production using Jina and FastAPI, with support for Docker Compose or Kubernetes deployment and secrets management."}]}}