{"data":{"slug":"albarji-mixture-of-diffusers","name":"mixture-of-diffusers","tagline":"Mixture of Diffusers for scene composition and high resolution image generation","github_url":"https://github.com/albarji/mixture-of-diffusers","owner":"albarji","repo":"mixture-of-diffusers","owner_avatar_url":"https://avatars.githubusercontent.com/u/9654655?v=4","primary_language":"Python","stars":449,"forks":41,"topics":["ai","computer-vision","diffusion-models","stable-diffusion"],"archived":false,"github_pushed_at":"2023-05-21T13:10:01+00:00","maintenance_label":"Dormant","url":"https://www.graphcanon.com/tools/albarji-mixture-of-diffusers","markdown_url":"https://www.graphcanon.com/tools/albarji-mixture-of-diffusers.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/albarji-mixture-of-diffusers","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=albarji-mixture-of-diffusers","description":"Mixture of Diffusers for scene composition and high resolution image generation","homepage_url":null,"license":"MIT","open_issues":5,"watchers":7,"ai_summary":null,"readme_excerpt":"# Mixture of Diffusers\n\n\n\n\n\n\n\nThis repository holds various scripts and tools implementing a method for integrating a mixture of different diffusion processes collaborating to generate a single image. Each diffuser focuses on a particular region on the image, taking into account boundary effects to promote a smooth blending.\n\nIf you prefer a more user friendly graphical interface to use this algorithm, I recommend trying the [Tiled Diffusion & VAE](https://github.com/pkuliyi2015/multidiffusion-upscaler-for-automatic1111) plugin developed by pkuliyi2015 for [AUTOMATIC1111's stable-diffusion-webui](https://github.com/AUTOMATIC1111/stable-diffusion-webui).\n\n## Motivation\n\nCurrent image generation methods, such as Stable Diffusion, struggle to position objects at specific locations. While the content of the generated image (somewhat) reflects the objects present in the prompt, it is difficult to frame the prompt in a way that creates an specific composition. For instance, take a prompt expressing a complex composition such as\n\n> A charming house in the countryside on the left,\n> in the center a dirt road in the countryside crossing pastures,\n> on the right an old and rusty giant robot lying on a dirt road,\n> by jakub rozalski,\n> sunset lighting on the left and center, dark sunset lighting on the right\n> elegant, highly detailed, smooth, sharp focus, artstation, stunning masterpiece\n\nOut of a sample of 20 Stable Diffusion generations with different seeds, the generated images that align best with the prompt are the following:\n\n\n \n \n \n \n \n
\n\nThe method proposed here strives to provide a better tool for image composition by using several diffusion processes in parallel, each configured with a specific prompt and settings, and focused on a particular region of the image. For example, the following are three outputs from this method, using the following prompts from left to right:\n\n* \"**A charming house in the countryside, by jakub rozalski, sunset lighting**, elegant, highly detailed, smooth, sharp focus, artstation, stunning masterpiece\"\n* \"**A dirt road in the countryside crossing pastures, by jakub rozalski, sunset lighting**, elegant, highly detailed, smooth, sharp focus, artstation, stunning masterpiece\"\n* \"**An old and rusty giant robot lying on a dirt road, by jakub rozalski, dark sunset lighting**, elegant, highly detailed, smooth, sharp focus, artstation, stunning masterpiece\"\n\n\n\n\n\nThe mixture of diffusion processes is done in a way that harmonizes the generation process, preventing \"seam\" effects in the generated image.\n\nUsing several diffusion processes in parallel has also practical advantages when generating very large images, as the GPU memory requirements are similar to that of generating an image of the size of a single tile.\n\n## Usage\n\nThis repository provides two new pipelines, `StableDiffusionTilingPipeline` and `StableDiffusionCanvasPipeline`, that extend the standard Stable Diffusion pipeline from [Diffusers](https://github.com/huggingface/diffusers). They feature new options that allow defining the mixture of diffusers, which are distributed as a number of \"diffusion regions\" over the image to be generated. `StableDiffusionTilingPipeline` is simpler to use and arranges the diffusion regions as a grid over the canvas, while `StableDiffusionCanvasPipeline` allows a more flexible placement and also features image2image capabilities.\n\n### Prerequisites\n\nSince this work is based on Stable Diffusion models, you will need to [request access and accept the usage terms of Stable Diffusion](https:/","github_created_at":"2022-08-23T16:35:21+00:00","created_at":"2026-07-11T12:29:30.363989+00:00","updated_at":"2026-07-11T12:29:35.282775+00:00","categories":[{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"},{"slug":"data-retrieval","name":"Data & Retrieval","url":"https://www.graphcanon.com/categories/data-retrieval","markdown_url":"https://www.graphcanon.com/categories/data-retrieval.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/data-retrieval"},{"slug":"computer-vision","name":"Computer Vision","url":"https://www.graphcanon.com/categories/computer-vision","markdown_url":"https://www.graphcanon.com/categories/computer-vision.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/computer-vision"}],"tags":[{"slug":"ai","name":"ai"},{"slug":"stable-diffusion","name":"stable-diffusion"},{"slug":"python","name":"python"},{"slug":"diffusion-models","name":"diffusion-models"},{"slug":"computer-vision","name":"computer-vision"}],"trust":{"provenance":{"is_fork":false,"github_id":528076983,"owner_type":"User","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-07-11T12:29:31.113Z","maintenance":{"label":"Dormant","score":18,"methodology":"github_public_v1","releases_90d":0,"days_since_push":1146,"last_release_at":"2023-05-21T13:10:01Z"},"security_summary":{"status":"findings","scanner":"osv@v1","low_count":102,"high_count":0,"last_scan_at":"2026-07-11T12:29:33.188Z","medium_count":0,"scan_profile":"deps","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-07-11T12:29:32.904Z"},"languages":{"value":["python"],"source":"github.language","observed_at":"2026-07-11T12:29:32.904Z"},"license_spdx":{"value":"MIT","source":"github.license","observed_at":"2026-07-11T12:29:32.904Z"}}}}