Hugging Face Text to Image (Prompt)

This Action allows generating images using AI models, based on specified fields within a Data Object. In addition to the default Hugging Face Text To Image action you can refine the generated image. Once you get the desired result you can save the generated image as an asset in the specified folder.

info

For the action to work, you need to select exactly one element. Selection of none or more than one element is currently not supported and will result in an error. This also means you can´t use the selection context with this action (e.g. Data Object Car Selection)

Configuration Options

# The model to use
model_endpoint: 'https://api-inference.huggingface.co/models/stabilityai/stable-diffusion-2-1'

#Folder for storing generated images.
asset_folder: '/AI Generated'

Detailed Configuration Options

model_endpoint: (Required) Specifies the endpoint for the model you want to use for image generation. The default base URL is https://api-inference.huggingface.co. Providing just the model name defaults to this URL. A full URL, such as 'https://api-inference.huggingface.co/models/stabilityai/stable-diffusion-2-1', overrides the default. This is where API requests for image generation are sent.

asset_folder: (Required) Designates the folder where generated images will be stored. This path, like '/AI Generated', is the path where the generated images will be saved.

prompt_template: (Optional) A Twig template that creates the prompt sent to the image generation model. It combines the input fields into a coherent description for the model. The selected element gets passed as "subject". If empty, the user has to input the initial prompt manually.

filename_template: (Optional) A Twig template to generate the filename dynamically.

parameters: (Optional) Contains additional parameters for the image generation process:

• height: (Optional) Specifies the height of the generated image in pixels.
• width: (Optional) Specifies the width of the generated image in pixels.
• negative_prompt: A Twig template that specifies descriptions to avoid in the generated images.
• guidance_scale: (Optional) Determines how closely the generated image should adhere to the prompt as opposed to the model's own creativity.
• num_inference_steps: (Optional) Sets the number of steps the model undergoes to refine the generated image. Higher values can lead to more detailed images.

options: (Optional) Contains additional options for the image generation process: use_cache: (Optional, default: true) Utilizes previously generated images for similar requests to accelerate response times. Setting this to false ensures a new image is generated for each request, enhancing uniqueness but potentially increasing wait times.

Examples

The example below illustrates how to generate an image using the AI model "stabilityai/stable-diffusion-2-1". It takes values from the "name", "productionYear" and "description" fields to generate the image, saves the image in the "AI Generated" folder within Assets, and assigns the generated image to the "AIGeneratedImage" field in the Data Object.

# Specifies the model's endpoint. A full URL overrides the default base URL.
model_endpoint: 'https://api-inference.huggingface.co/models/stabilityai/stable-diffusion-2-1'

# Specifies the folder for storing generated images.
asset_folder: '/AI Generated'

# Twig template that constructs the prompt sent to the image generation model.
prompt_template: "Realistic picture of {{ subject.name }}, production year: {{ subject.productionYear }}, {{ subject.description }}, detailed, 8k"

# Twig template to generate the filename based on Data Object or Asset fields.
filename_template: "{{ name }}-{{ random() }}"

parameters:
  height: 600
  width: 800
  negative_prompt: "ugly, deformed, disfigured, poor details, bad anatomy"
  guidance_scale: 8.5
  num_inference_steps: 80

options:
  # Uses cached images for similar requests to speed up response times. Set to false for new images per request.
  use_cache: true

Additional Information

To explore different visual outcomes, it's possible to switch to other AI models by changing the model_endpoint value. Below are examples of models that can be utilized for image generation:

https://api-inference.huggingface.co/models/CompVis/stable-diffusion-v1-4
https://api-inference.huggingface.co/models/stabilityai/stable-diffusion-xl-base-1.0
https://api-inference.huggingface.co/models/stabilityai/stable-diffusion-2-1
https://api-inference.huggingface.co/models/runwayml/stable-diffusion-v1-5
https://api-inference.huggingface.co/models/kandinsky-community/kandinsky-2-2-decoder
https://api-inference.huggingface.co/models/lllyasviel/control_v11p_sd15_openpose