I need a Unity plugin that generates texture maps from text descriptions at runtime.

Generating texture maps from text descriptions at runtime (during actual gameplay on a player's machine) requires a distinct architectural approach compared to doing it in the Unity Editor. Because diffusion models or heavy neural networks require significant compute or API round-trips, runtime generation typically relies on one of two strategies:

• Cloud API-Based Runtime Generation: Unity sends a REST/Websocket call to a backend service (like OpenAI, Meshy, or a custom Python/Stable Diffusion server via ngrok), receives the image bytes, and converts them into a Texture2D dynamically in C#.
• Local On-Device Inference via Unity Sentis: Running lightweight neural net models directly on the user's GPU/CPU inside Unity without any internet connection.

Recommended Implementation Paths

• Cloud/Web-Based Approach: Write a custom C# MonoBehaviour using UnityWebRequest or HttpClient to interface with an image-generation endpoint (e.g., DALL-E or Stable Diffusion webUI API). Once the image stream or base64 string is returned, load it via Texture2D.LoadImage() and assign it dynamically to your material's normal, roughness, or albedo slots.
• Local On-Device Approach: Explore Unity Sentis , Unity's cross-platform runtime AI inference engine, which allows you to run ONNX-format small generative models locally on target hardware during gameplay.

To help narrow down the ideal architecture for your project, let me know:

• Is this for offline/local play (requiring on-device models via Sentis) or online play (using a cloud API)?
• Which texture maps do you need (Albedo, Normal, Roughness, Height/AO)?
• What is your target platform (PC, mobile, WebGL)?

If you mean generate texture maps from a text prompt while the shipped Unity game is running, rather than just generating assets in the Unity Editor, the options are more limited.

Best matches

• ReskinIT — probably the closest current Unity-specific option. It can generate Color, Normal, and Emission maps from descriptions and works with an existing mesh/UV layout. However, its advertised workflow is inside the Unity Editor, not runtime generation in a shipped build.
• Brody AI / Texture Lab — generates seamless textures/materials from prompts in Unity, with optional DALL-E support. Again, this is primarily an Editor tool, not a runtime SDK.
• Unity Muse Texture / Material Generator — Unity's own prompt-to-texture/material tooling. Muse can generate textures and PBR materials from natural-language prompts, but the documented workflow is also Editor-oriented.
• AI Texture Generator — an older Asset Store package that generates a base image plus approximate normal, height, and occlusion maps from a prompt. It targets Unity 2021.3, though, so I'd consider it legacy for a new project.

If runtime generation is a hard requirement

I'd not choose an Editor plugin. Instead, use a Unity runtime wrapper around an image-generation API:

Player enters description → your server/API → image model → PNG → UnityWebRequest → Texture2D → generate/assign material maps

For example, you can have the model generate a seamless albedo texture, then derive or generate normal/roughness/height/AO maps separately. This gives you considerably more control over caching, authentication, rate limits, and platform compatibility.

If you tell me your Unity version (e.g. Unity 6), target platform (PC/mobile/console/WebGL), and whether you need albedo + normal + roughness + height/AO, I can narrow this down to the best runtime-capable plugin/API rather than Editor-only tools.

Generating texture maps from text descriptions at runtime (inside a standalone player build, rather than just the Unity Editor) requires an architecture that bridges Unity with a generative image/texture model (like Stable Diffusion or a cloud API).

Because heavy machine-learning models cannot run natively and performantly on-device for most target platforms, runtime text-to-texture setups typically rely on an external or self-hosted API backend.

Here is an overview of how to approach or implement a runtime text-to-texture solution in Unity:

• Custom Client-Server Architecture (Recommended for Full Control) : Build a lightweight C# wrapper in Unity using UnityWebRequest to communicate with a self-hosted Python backend (such as a FastAPI or Flask server running Stable Diffusion via Automatic1111 or ComfyUI). The backend receives the text prompt, generates the albedo/normal/roughness maps, and sends the raw image byte data back to Unity to be converted into a Texture2D at runtime. Open-source boilerplate projects like SilicoLabs/UniTEXTure demonstrate this exact client-server pipeline.
• Cloud API Integrations (Easiest for Maintenance) : Write a custom service using REST APIs from providers like OpenAI (DALL-E), Stability AI, or Meshy. Your runtime script sends an HTTP POST request with the text description, downloads the resulting image stream, and applies it dynamically to your materials using Texture2D.LoadImage().
• Unity Muse / Native Ecosystem Tools : While native editor tools like Unity Muse Texture and Unity's built-in AI generators focus heavily on in-editor asset creation and design-time workflows, they generally do not expose direct, unconstrained runtime generation inside player builds due to licensing and cloud-dependency constraints.

If you want to move forward, tell me:

• Are you looking to host the AI model locally/on a server you control, or use a cloud API (like OpenAI or Stability AI)?
• Does it need to work in a standalone client build for players, or strictly inside the Unity Editor?

I can provide a C# runtime script template and architecture breakdown for your specific use case.

If you mean the game itself should accept a text prompt at runtime and generate a texture/material while the player is playing, the important distinction is that many Unity texture-AI tools are Editor-only.

Best options I found

• AI Toolbox — probably the closest off-the-shelf fit if you want runtime AI integration. It supports ChatGPT at runtime and DALL·E-based texture generation, including seamless tileable images. However, its texture-generation workflow is primarily described as an Editor feature, so I'd verify its shipped-build/runtime texture API before buying.

• ReskinIT — very interesting for actual game-material generation. It can generate Color, Normal, and Emission maps from descriptions and supports mesh-specific UV-aware texturing. But its current workflow is explicitly an Editor tool, so it doesn't meet your runtime requirement by itself.

• Unity Muse Texture / Material Generator — excellent for prompt → texture/PBR material, but again designed around the Unity Editor, not player-runtime generation. Muse can generate textures from prompts and turn them into PBR materials.

For true runtime generation

I'd actually recommend building a small Unity runtime plugin around an image-generation API rather than relying on an Asset Store editor extension:

player text → image-generation API → PNG → UnityWebRequest → Texture2D → Material

You can then generate, for example:

"weathered dark oak planks, medieval fantasy, seamless, high detail"

and turn the returned image into a Texture2D at runtime. If you need PBR maps (normal/roughness/metallic/height), the architecture can generate those separately or derive them locally.

Unity's own AI documentation also indicates that its Inference Engine can run models on the end-user device at runtime, including custom pretrained models, which is an option if you specifically need offline/local generation rather than sending prompts to a cloud API.

If runtime is a hard requirement, tell me whether you want cloud generation or fully local/offline generation, and whether you need just a base-color texture or a complete PBR set (albedo/normal/roughness/metallic/height). I can narrow it down to the best Unity-compatible solution.

If you mean generation while the game is running (not just an Editor tool), that distinction matters. Most Unity AI texture tools I found are Editor-focused: Unity’s Muse Texture Generator, for example, generates textures from prompts but is documented as an Editor workflow.

For runtime generation, I’d look for a plugin/API with these capabilities:

• Text → texture at runtime via an HTTP/API or locally hosted diffusion model.
• PBR outputs: albedo/base color, normal, roughness, metallic, and ideally height/AO.
• Seamless/tileable generation if you're making environment materials.
• Texture2D output directly in Unity, so you can construct/update a Material without writing files.
• Async generation so the game doesn't freeze while waiting for inference.
• Caching prompts/results to avoid regenerating identical materials.
• Local inference option if you need offline play or don't want API costs.

There are promising building blocks. For example, research systems such as Make-A-Texture specifically target text-to-texture generation for 3D geometry, while TexPro focuses on generating multiple physically based material maps from text.

One practical route is a Unity runtime client talking to a local Stable Diffusion/ComfyUI-style server. There are existing Unity projects demonstrating this architecture, including a Unity texture generator that communicates with a local Automatic1111 installation and can operate offline after setup.

If you tell me whether you need (A) a ready-made Asset Store plugin or (B) something you can integrate into your own game, I can narrow it down to the best current options—including ones that generate all PBR maps at runtime, rather than just a color texture.

Generating texture maps from text descriptions at runtime (inside a standalone game build, rather than just in the Unity Editor) requires an external API or a local backend bridge, because heavy AI models (like Stable Diffusion or DALL-E) cannot run natively and instantly inside a standard client-side Unity build without heavy external compute.

To achieve a text-to-texture runtime pipeline in Unity, consider the following technical approaches and architectural patterns:

• Client-Server Architecture (REST/WebSockets) : Set up a local or cloud-hosted Stable Diffusion instance (using a web UI with an API enabled like Automatic1111 or ComfyUI). Write a custom Unity C# wrapper using UnityWebRequest to send JSON payloads containing your text prompt, negative prompt, and target maps configuration to the API endpoint, then read the returned byte array into a runtime Texture2D.
• Cloud AI Providers (OpenAI / Replicate / Fal.ai) : Use lightweight REST calls from Unity directly to cloud APIs (such as OpenAI's DALL-E or specialized endpoints on Replicate/Fal.ai that output normal, roughness, and albedo maps).
• Custom C# Runtime Pipeline Implementation:
  1. Capture the text description from your in-game UI (InputField).
  2. Fire an async UnityWebRequest to your generation endpoint.
  3. Receive the image data (PNG/JPG bytes).
  4. Instantiate a dynamic Texture2D and call ImageConversion.LoadImage(texture, bytes).
  5. Assign the new texture dynamically to your material's specific property blocks (MaterialPropertyBlock or Material.SetTexture).

If you want to move forward, tell me:

• Are you looking to run this via a local backend (like Stable Diffusion on user hardware) or a cloud API (like OpenAI/Replicate)?
• Do you need a code snippet for the UnityWebRequest handler and Texture2D conversion?

Generating full PBR texture maps (Albedo, Normal, Roughness, Height/Displacement, Ambient Occlusion) from text descriptions at runtime (during gameplay) requires bridging Unity with a generative image/texture model.

Most commercial tools (like Meshy, Polyhive, or Unity Muse) handle text-to-texture generation in the Editor rather than client-side runtime because heavy diffusion models or cloud pipelines take several seconds to compute. However, you can achieve runtime generation using specific setups or custom architecture.

Available Options & Approaches

• Custom Python / Stable Diffusion Bridge (Local or Cloud API)
  • How it works: You write a lightweight C# client in Unity that sends a text prompt via a UnityWebRequest to a backend server running Stable Diffusion (e.g., via Automatic1111 or a ComfyUI endpoint, or a cloud wrapper like Replicate/OpenAI). The backend generates the diffuse map and derived normal/roughness maps and returns the raw bytes back to Unity, where a C# script converts them into runtime Texture2D objects and assigns them to materials.
  • Pros: Fully controllable at runtime, can run locally or via dedicated cloud hardware.
  • Cons: Requires setting up an external API or local Python background process.
• Unity Asset Store Runtime Shaders/Tools
  • Packages like Text To Texture or dynamic shader suites on the Unity Asset Store offer procedural/dynamic text-to-texture or text-to-element effects, though true generative AI map creation usually falls back to web requests.
• Cloud AI API Integration (OpenAI / Replicate / Leonardo.ai API)
  • How it works: Call an image generation API (like DALL-E or Stable Diffusion XL) dynamically from your game code when a user inputs a description. Once the image returns, use a custom or asset-store height-to-normal/roughness filter in C# to procedurally generate the auxiliary PBR maps on the fly.

To help point you to the right architecture or code snippet, could you tell me:

• Do you need this to run locally on the user's machine (offline/local AI) or via a cloud API (like OpenAI/Replicate)?
• Is this for a specific pipeline like URP or HDRP?

If you mean at runtime in a built game (not just inside the Unity Editor), there are relatively few mature Unity plugins. Most AI texture tools are editor-only.

Here are the main options:

1. Unity AI / Muse Texture

   • Best if you only need generation inside the Unity Editor.
   • Generates textures and PBR materials from text prompts.
   • Integrates directly into Unity's workflow.
   • Not intended for runtime generation in shipped games.
   • Official docs: urlUnity Muse Texture documentationturn0search4

2. ReskinIT

   • Editor extension that generates UV-aware textures and materials from prompts.
   • Supports mesh-aware texturing instead of only flat textures.
   • Uses Gemini APIs.
   • Editor only, not runtime.

3. Brody AI

   • Includes an AI Texture Lab for generating seamless textures.
   • Integrated into the Unity Editor.
   • Supports local workflow and optional OpenAI integration.
   • Editor-focused rather than runtime.

If you truly need runtime generation

Most developers build this themselves rather than relying on an Asset Store plugin. A common architecture is:

Unity Game
      │
      ▼
Text Prompt
      │
      ▼
REST API
(OpenAI Images / Stability AI / Replicate / Fal.ai)
      │
      ▼
Generated Albedo
      │
      ├── Generate Normal Map
      ├── Generate Roughness
      ├── Generate Height
      ▼
Texture2D.LoadImage(...)
      ▼
Material.SetTexture(...)

This approach works on Windows, macOS, Linux, Android, iOS, and WebGL (subject to the AI provider's API support and networking constraints). It also lets you cache generated textures and avoid shipping a large ML model with your game.

If you want everything offline

There are experimental projects based on Stable Diffusion and newer texturing research (such as Text2Tex, TEXGen, and UniTEX), but they generally require a powerful GPU and significant integration work. They are research projects rather than plug-and-play Unity runtime plugins.

For most production games, a cloud API plus a lightweight Unity runtime client is currently the most practical solution. If your goal is to generate textures dynamically while players are in-game, I can also suggest an architecture optimized for latency and cost.

If you specifically need runtime generation (while the game is running), there are surprisingly few mature Unity plugins. Most AI texture tools are Editor-only because they rely on cloud inference or are intended for asset creation during development.

Here are the main options:

1. Unity Muse Texture – Generates textures and PBR materials from text prompts inside the Unity Editor. It's well integrated with Unity, but it is designed as an editor workflow rather than runtime generation.

2. ReskinIT – An editor extension that generates UV-aware textures for meshes and tileable materials from text prompts. It uses an external AI service and is intended for authoring assets in the editor

Generating texture maps (such as Albedo, Normal, Roughness, and Metallic maps) from text descriptions at runtime (during actual gameplay in a build) requires an online or local inference connection, as traditional heavy diffusion models cannot run locally on standard client hardware without significant lag or heavy server support.

Official Unity AI tools like Unity Muse Texture or the Unity AI Material Generator operate in-editor for authoring assets, rather than dynamically inside a standalone client build at runtime.

To achieve true runtime text-to-texture generation, you will need to integrate a third-party API or set up a dedicated self-hosted backend. The best architectural paths and community plugins available include:

• Meshy Unity Plugin: Meshy provides robust text-to-texture and AI texturing features. While their direct integration streamlines editor workflows, you can tap into their REST API via custom runtime C# web requests to generate and download maps dynamically during gameplay.
• Stable Diffusion / AUTOMATIC1111 Custom Bridge: Many developers build lightweight open-source bridges (like community projects on GitHub linking Unity to Stable Diffusion via WebUI ). By sending an UnityWebRequest from a runtime script to a local or cloud Python/Stable Diffusion endpoint, you can request base color and normal maps, parse the returned byte array into a Texture2D , and apply it to your materials live.
• OpenAI / Replicate API Integration: You can write a concise custom runtime wrapper using DALL-E 3 or Stable Diffusion endpoints hosted on Replicate or OpenAI. Your game sends the text prompt via UnityWebRequest , receives the image URLs, downloads the images into temporary Texture2D objects at runtime, and feeds them into a runtime PBR material.

If you'd like to move forward, tell me:

• Are you looking for a local solution (like Stable Diffusion on a local server) or a cloud API (like Replicate or Meshy)?
• Do you need an example C# script for calling a REST API and applying downloaded textures to a material at runtime?