Introducing New Templates in SageMaker AI: Text Ranking and Question & Answer
Unleashing the Power of Human Feedback for Model Training
Today we are excited to introduce the Text Ranking and Question and Answer UI templates to SageMaker AI customers. The Text Ranking template enables human annotators to rank multiple responses from a large language model (LLM) based on custom criteria, such as relevance, clarity, or factual accuracy. This ranked feedback provides critical insights that help refine models through Reinforcement Learning from Human Feedback (RLHF), generating responses that better align with human preferences. The Question and Answer template facilitates the creation of high-quality Q&A pairs based on provided text passages. These pairs act as demonstration data for Supervised Fine-Tuning (SFT), teaching models how to respond to similar inputs accurately.
In this blog post, we’ll walk you through how to set up these templates in SageMaker to create high-quality datasets for training your large language models. Let’s explore how you can leverage these new tools.
Enhancing Model Training with New SageMaker AI Templates: Text Ranking and Question & Answer
Today, we’re thrilled to announce the release of two powerful new templates for Amazon SageMaker AI customers: the Text Ranking template and the Question and Answer template. These tools are designed to help you create high-quality datasets that refine your large language models (LLMs) more effectively. Let’s dive into how each of these templates can enhance your model training.
Text Ranking Template
The Text Ranking template is designed to allow human annotators to rank multiple responses generated by an LLM based on specific criteria such as relevance, clarity, and factual accuracy. Annotators are presented with a prompt and a selection of model-generated responses, which they can rank according to custom guidelines tailored to your use case.
Key Features
-
Customizable Ranking Criteria: Annotators can define dimensions such as “Helpfulness” or “Inclusivity,” allowing you to align model outputs more closely with human preferences.
-
Structured Feedback: The ranked data is captured in a structured format, detailing the re-ranked indices for each criterion. This structured feedback is critical for model fine-tuning using Reinforcement Learning from Human Feedback (RLHF).
-
Quality Evaluation: The template provides a straightforward way to evaluate the quality of LLM outputs, ensuring that responses meet the desired standards.
Setting Up the Text Ranking Template in SageMaker
To set up the Text Ranking template, follow these steps:
-
Access SageMaker AI Console: Navigate to the Ground Truth section in the SageMaker AI console and select "Labeling job."
-
Create Labeling Job: Click "Create labeling job" and specify your input manifest location and output path.
-
Input Configuration: Use Manual Data Setup to input a JSON file containing your prompt and the corresponding model responses.
-
Set Dimensions: Enter your labeling instructions and define the ranking dimensions based on your project needs. You can also allow tie rankings for equal-quality responses.
-
Preview and Create Job: Review the UI template before finalizing your labeling job.
Once annotators submit their evaluations, their responses are saved to your specified S3 bucket, where you can easily access the output manifest file, including ranked responses and metadata.
Question and Answer Template
The Question and Answer template allows you to generate high-quality question-and-answer pairs from given text passages. Annotators create relevant Q&A pairs that serve as demonstration data for Supervised Fine-Tuning (SFT), guiding the model on how to handle similar tasks in the future.
Key Features
-
Flexible Input: Annotators can reference entire passages or specific text sections, adapting the Q&A to your project’s needs.
-
Visual Linking: A color-coded matching feature visually links questions to their relevant sections in the text, streamlining the annotation process.
-
Enhanced Model Training: The Q&A pairs produced enhance your model’s capability to accurately follow instructions and respond to real-world inputs.
Setting Up the Question and Answer Template in SageMaker
Here’s how to set up the Question and Answer template:
-
Access SageMaker AI Console: Similar to the Text Ranking setup, navigate to Ground Truth and select "Labeling job."
-
Create Labeling Job: Choose "Create labeling job" and specify your manifest file and output path.
-
Input Configuration: Use Manual Data Setup to upload a JSON file containing the text passage from which annotators will derive Q&A pairs.
-
Set Parameters: Customize the labeling instructions, including minimum and maximum Q&A pairs, word counts, and optional tags.
-
Preview and Create Job: Review the annotation interface before creating the labeling job.
When annotators submit their work, their detailed responses are saved in your S3 bucket, ready for your analysis.
Using the Create Labeling Job API
Beyond the SageMaker console, you can also set up labeling jobs programmatically using the Create Labeling Job API. This option provides greater flexibility for automation and integration into your existing workflows, making it easier to manage large-scale labeling tasks efficiently.
Conclusion
With the launch of the Text Ranking and Question and Answer templates, Amazon SageMaker AI offers powerful tools that streamline the process of generating high-quality datasets. These capabilities simplify model fine-tuning and increase the alignment of model outputs with human preferences, whether through supervised learning or reinforcement approaches.
If you’re ready to take advantage of these new templates and elevate your model training processes, start exploring SageMaker AI today.
About the Authors
This post was co-authored by a talented team at AWS, each contributing their expertise to enhance the SageMaker AI experience. From Generative AI Specialists to Product Managers and Software Engineers, this diverse group works passionately to provide customers with cutting-edge AI solutions.
Join us in creating efficient workflows for fine-tuning datasets and advancing the capabilities of AI models with Amazon SageMaker. Happy training!