Hugging Face Text Generation

This Action facilitates the generation of text using AI models based on a specified prompt template (twig) and relevant attributes or metadata of Assets and Data Objects. The generated text can be used for a variety of purposes, including but not limited to product descriptions, asset metadata, or any other textual content required. It is designed to operate in a one-shot manner, meaning it generates text based on the initial prompt without further interaction.

Importantly, this action can be executed on one to many elements of Assets or Data Objects, making it highly scalable and versatile for batch processing.

Configuration Options

#The endpoint to use
model_endpoint: 'https://api-inference.huggingface.co/models/mistralai/Mistral-7B-Instruct-v0.1'

#targets where the text generation results will be saved.
output_targets:
  save_on_subject: true
  save_on_execution_context: true
  
#Field within the Asset or Data Object where the generated asset description should be stored.
output_field: description

#Language setting for localized fields in Data Objects or Assets (metadata).
language: en

#Twig template string that outlines the structure of the prompt to be sent to the AI model.
prompt_template: "Describe a car with following parameters {{ subject.name }}, Product year: {{ subject.productionYear }}, Color: {{ subject.color|join(', ') }}."

Detailed Configuration Options

model_endpoint: (Required) Specifies the endpoint for the AI model to use. The default base URL is https://api-inference.huggingface.co. If only the model name is provided (e.g., /models/tiiuae/falcon-7b-instruct), this default base URL is used. However, if a full URL is specified, it will override the default base URL.

prompt_template: (Required) A Twig template string that outlines the structure of the prompt to be sent to the AI model. The selected element can be accessed using the subject variable.

output_targets: (Required) Specifies the targets where the text generation results will be saved.

• save_on_subject: (Required) Determines whether the generated text is saved directly on the Data Object or Asset itself. Setting this to true enables the result to be stored within the specified 'output_field' of the subject.
• save_on_execution_context: (Required) Specifies if the generated text should be saved into the job's execution context. This is useful for passing the results to subsequent steps within the same job, allowing other actions to work with the generated text.

output_field: (Required) Specifies the field within the Asset or Data Object where the generated asset description should be stored. This requires save_on_subject to be set to true.

language: (Optional) Specifies the language setting for multilingual fields within a Data Object or Asset.

output_field: (Required) specifies the destination for the generated text, which can either be an attribute of a Data Object or a metadata field of an Asset. If save_on_execution_context is set to true, this field defines the key in the execution context where the generated text will be stored.

parameters: (Optional) To customize the text generation process, several optional parameters can be configured:

• top_k: (Default: null): Integer to define the top tokens considered within the sample operation to create new text.
• top_p: (Default: null): Float to define the tokens that are within the sample operation of text generation. Add tokens in the sample for more probable to least probable until the sum of the probabilities is greater than top_p.
• temperature: (Default: 1.0). Float (0.0-100.0). The temperature of the sampling operation. 1 means regular sampling, 0 means always take the highest score, 100.0 is getting closer to uniform probability.
• repetition_penalty: (Default: null). Float (0.0-100.0). The more a token is used within generation the more it is penalized to not be picked in successive generation passes.
• max_new_tokens: (Default: null). Int (0-250). The amount of new tokens to be generated, this does not include the input length it is a estimate of the size of generated text you want. Each new tokens slows down the request, so look for balance between response times and length of text generated.
• max_time: (Default: null). Float (0-120.0). The amount of time in seconds that the query should take maximum. Network can cause some overhead so it will be a soft limit. Use that in combination with max_new_tokens for best results.
• return_full_text: (Default: false). Bool. If set to false, the return results will not contain the original query making it easier for prompting.

options: (Optional) setting includes:

• use_cache (Default: true). A boolean that determines if cached responses for similar prompts should be reused. Defaulted to true, it enhances efficiency by avoiding repetitive text generation. Setting it to false ensures fresh generation for each request, suitable for unique content needs.
• use_nl2br (Default: false). A boolean that determines if the generated text should be formatted with line breaks. Defaulted to false, it maintains the original format of the generated text.

Examples

Data Object Example

model_endpoint: 'https://api-inference.huggingface.co/models/mistralai/Mistral-7B-Instruct-v0.1'
output_targets:
  save_on_subject: true
  save_on_execution_context: true
output_field: description
language: en
prompt_template: "Describe a car with following parameters {{ subject.name }}, Product year: {{ subject.productionYear }}, Color: {{ subject.color|join(', ') }}."
parameters:
  temperature: 0.4
  max_time: 10
  max_new_tokens: 120
options:
  use_cache: false
  use_nl2br: true

Asset Example

model_endpoint: 'https://api-inference.huggingface.co/models/mistralai/Mistral-7B-Instruct-v0.1'
output_targets:
  save_on_subject: true
  save_on_execution_context: true
output_field: CarImages.text
language: en
prompt_template: "Describe a car with following parameters: {{ subject.getMetadata('title') }}."

Additional Information

To explore different text generation outcomes, it's possible to switch to other AI models by changing the model_endpoint value. Below are examples of models that can be utilized for image generation:

https://api-inference.huggingface.co/models/tiiuae/falcon-7b-instruct
https://api-inference.huggingface.co/models/mistralai/Mistral-7B-v0.1
https://api-inference.huggingface.co/models/mistralai/Mistral-7B-Instruct-v0.1

Usage of the execution context

This action can update the execution context with the generated text when save_on_execution_context is set to true. The array structure of the execution context is as follows:

$content[$id_of_the_processed_object][$output_field] = $generated_text;