Turn a Typeform into an AI Intake Agent: Pt 1 — Prompts and Evals

Introduction to {{item.name}}

Every day, organizations collect data through forms. Employee onboarding, customer intake, surveys - they all present users with walls of input fields and validation messages. It's tedious, impersonal, and often frustrating. But what if we could make this experience feel more natural? What if filling out forms felt like having a conversation with a helpful assistant?

In this guide, we'll show you how to build a conversational form assistant using PromptLayer. You'll learn to create an AI that naturally collects information while maintaining data quality and validation standards.

Building the Core Assistant

The Master Prompt: Your Assistant's Brain

The first step is creating your assistant's core instructions. In PromptLayer's registry, create a new prompt template. While you might be tempted to write one long instruction set, breaking it down into logical sections using snippets makes your prompt more maintainable and easier to refine.

Your prompt should cover:

• Core responsibilities (data collection and validation)
• Conversation guidelines (how to interact naturally)
• User experience rules (pacing and clarity)
• Data collection process (gathering and submitting information)
• Tool usage instructions (when and how to submit collected data)

This modular approach using snippets lets you update specific behaviors without touching the entire prompt.

Configuring Your Assistant

Your prompt needs two types of configuration: input variables for context and a tool for data submission.

Input Variables

• agent_name tells your assistant how to introduce itself
• conversation_history helps maintain context using placeholders for previous messages
• form_details defines what information to collect and validation rules

Submission Tool Call

The assistant needs a way to submit collected data. We provide a SubmitIntake tool that:

• Takes all collected fields as arguments
• Gets called only when all required data is gathered
• Follows the structure defined in form_details

When all data is collected, the assistant will (hopefully) make a tool call with the structured data, completing the form submission process naturally.

Creating the Evaluation Engine

The clever part comes in how we evaluate these behaviors. We create an LLM as a judge evaluation prompt that:

1. Analyzes the assistant's actual response
2. Returns a structured assessment:
3. Receives the test scenario details

{
  "passed": boolean, # Whether the response passed the evaluation against the expected behavior
  "reason": "Detailed explanation of why the response did/didn't meet expectations"
}

This structured output is crucial - it lets us automatically track evaluation results while maintaining detailed feedback about why responses succeed or fail. Having a structured boolean (rather than bare strings) will allow as to easily keep track of the score.

Automatic Quality Gates

After connecting your evaluation prompt and dataset to your main prompt in PromptLayer, you get automatic quality checks. Each time you update your assistant's prompt, PromptLayer offers to run your behavioral test suite. This creates immediate feedback about whether your changes improved or degraded the assistant's core behaviors.

This approach to evaluation helps ensure that while your assistant might not give identical responses every time (that would feel robotic!), it consistently exhibits the behaviors that make it effective at its job.

Comprehensive Monitoring

PromptLayer provides multiple ways to monitor your assistant's performance:

Evaluation Scores and Reports

Each prompt version in the registry includes a link to its evaluation report, showing how it performed against your test cases. These scores and detailed reports help you track improvements across versions and understand exactly how your changes affected the assistant's behavior.

Detailed Traces

The trace feature lets you inspect every interaction in detail. You can see the exact inputs provided to your prompts, the complete responses, and any tool calls made. This visibility is invaluable when debugging unexpected behaviors or optimizing your assistant's performance.

Putting It All Together

To demonstrate these concepts in action, we built a simple Streamlit application that lets organizations create and interact with their custom form assistants. The app has two main components:

Form Builder

Organizations can define their own custom forms by specifying the fields they need to collect. Each field can have:

• A name and type (text, number, date, etc.)
• A description for validation requirements
• Example values for clarity (optional)

Chat Interface

Users interact with their created assistants through a familiar chat interface. The assistant guides them through the form-filling process conversationally, collecting and validating data along the way. Behind the scenes, PromptLayer (through its SDK) manages the conversation flow and ensures data quality.

Looking Ahead

The world of conversational AI is evolving rapidly. PromptLayer's new agents builder opens up exciting possibilities. Instead of one master prompt handling everything, you could create specialized prompts for different tasks, all orchestrated by a workflow. This modular approach could make your assistant even more robust and maintainable.

Start Building

Ready to transform your forms into conversations? Begin with the basics:

Start with your form structure - what information do you need to collect? Create your master prompt, focusing on natural dialogue. Set up evaluations early - they'll guide your development. Then iterate and improve based on evaluation results.

Want to see a complete implementation? Check out our example at GitHub.

Remember, the goal isn't just to collect data - it's to make the experience feel natural and efficient for your users. With PromptLayer's tools and this guide, you're ready to start building more human-friendly forms.