How to Use This With AI
- Click the copy button (📋) in the top-right corner of the code block below
- Paste into your AI chat (ChatGPT, Claude, etc.)
- Add your test requirements after the schema
📋 Schema for AI (Click Copy Button Above)
Copy
Ask AI
# Revyl YAML Test Schema - LLM Reference
## Purpose
This document provides a structured, machine-readable reference for generating Revyl YAML tests programmatically.
## Critical Behavior for Test Generation
**IMPORTANT**: When generating tests, DO NOT include manual navigation at the start:
- **Web tests**: Automatically navigate to build URL at test start
- **Mobile tests**: Automatically open the app at test start
**Manual navigation steps are ONLY needed for**:
- `navigate`: Navigating to a DIFFERENT URL mid-test (web only)
- `open_app`: Reopening app after `close_app` (mobile only)
**Correct pattern**: Tests should start with `instructions`, `validation`, or `extraction` blocks.
## Complete Schema Structure
test:
metadata:
name: string # Required. Test name
platform: enum # Optional. Values: "web" | "ios" | "android"
build:
name: string # Required. Build variable name
pinned_version: string # Optional. Specific version
blocks:
- # Block objects (see Block Type Definitions below)
## Block Type Definitions
### instructions
type: "instructions"
step_description: string # Required. Natural language instruction
### validation
type: "validation"
step_description: string # Required. Condition to validate
### extraction
type: "extraction"
step_description: string # Required. What to extract
variable_name: string # Required. Use kebab-case
### manual
type: "manual"
step_type: enum # Required. See values below
step_description: string # Required for some step_types
**step_type values (Web)**: `wait`, `navigate`, `back`, `forward`, `refresh`, `browser_code_execution`
**step_type values (Mobile)**: `wait`, `open_app`, `close_app`
**step_description requirements by step_type**:
- `wait`: Must be a NUMBER representing seconds (e.g., "2", "5", "10")
- `navigate`: Must be a URL (e.g., "https://example.com")
- `browser_code_execution`: Must be valid JavaScript code
- `back`, `forward`, `refresh`, `open_app`, `close_app`: step_description is optional
**Examples**:
# Wait - step_description must be a number
- type: manual
step_type: wait
step_description: "3" # Wait 3 seconds
# Navigate - step_description must be a URL
- type: manual
step_type: navigate
step_description: "https://checkout.example.com"
**IMPORTANT**: Do NOT use `navigate` or `open_app` at test start.
### if
type: "if"
condition: string # Required. Natural language condition
then: array # Required. Array of blocks
else: array # Optional. Array of blocks
### while
type: "while"
condition: string # Required. Loop condition
body: array # Required. Array of blocks
## Variable System
**Definition** (in extraction block):
- type: extraction
step_description: "The product price"
variable_name: product-price
**Usage** (in any step_description):
- type: validation
step_description: "Total is $product-price$"
## High-Level Instructions (Recommended for Complex Flows)
When there is indeterminism about what actions need to be taken, **use high-level instructions** instead of specifying every step. The AI will intelligently handle all the intermediate steps.
**Example - Instead of guessing every step**:
# DON'T do this - too specific and may not match actual flow
- type: instructions
step_description: "Click sign up button"
- type: instructions
step_description: "Enter email in email field"
- type: instructions
step_description: "Enter password in password field"
# etc... you may not know all the steps
**DO this - Use high-level instruction**:
- type: instructions
step_description: "Complete the sign up flow"
The system will automatically navigate through all required screens and states without needing to know the exact flow in advance.
**When to use high-level instructions**:
- Multi-step flows where exact screens are unknown (sign up, checkout, onboarding)
- Flows that may vary based on user state
- Complex processes where you want the AI to handle the details
## Validation Best Practices
When you don't know exactly what to verify, **use broad validations** instead of guessing specific components.
**Broad validation examples**:
# General state validations - GOOD
- type: validation
step_description: "The checkout page is displayed"
- type: validation
step_description: "Login was successful"
- type: validation
step_description: "The user is on the dashboard"
**Negative validations (what shouldn't be visible)**:
# Verify errors are NOT shown - GOOD
- type: validation
step_description: "No error message is displayed"
- type: validation
step_description: "The login screen is no longer visible"
**Avoid overly specific validations when uncertain**:
# DON'T guess specific components you're not sure about
- type: validation
step_description: "The blue confirmation button in the top right is visible"
# DO use general validations
- type: validation
step_description: "The confirmation screen is displayed"
## Test Generation Rules
1. **NEVER start with manual navigation** - auto-navigate to build URL/app
2. **First block should be**: instructions, validation, or extraction
3. **Use high-level instructions for complex flows** - "Complete the checkout process" instead of specifying every step
4. **Use broad validations when uncertain** - "Login was successful" instead of guessing specific UI elements
5. **Negative validations are powerful** - "No error message is displayed" confirms success
6. **Wait step_description must be a NUMBER** - Use "3" not "Wait for page to load" (represents seconds)
7. **Use navigate/open_app ONLY when**: Changing URL/reopening closed app mid-test
8. **Always include**: test object, metadata.name, build.name, blocks array
9. **Variables**: Define with extraction, reference with `$variable$`
10. **Platform awareness**: Only use platform-appropriate step_types
## Complete Example
test:
metadata:
name: "E-commerce Purchase"
platform: web
build:
name: "Shopping Site"
blocks:
# Test auto-starts at build URL - NO navigate needed
- type: instructions
step_description: "Search for 'laptop'"
- type: instructions
step_description: "Click first result"
- type: extraction
step_description: "The product price"
variable_name: price
- type: instructions
step_description: "Click 'Add to Cart'"
- type: validation
step_description: "Cart shows price $price$"
- type: if
condition: "Discount field is visible"
then:
- type: instructions
step_description: "Enter 'SAVE10'"
- type: instructions
step_description: "Click 'Checkout'"
- type: validation
step_description: "Checkout page displayed"
## Common Validation Errors to Avoid
1. Starting test with unnecessary manual navigation (navigate/open_app)
2. Using descriptive text for wait instead of a number - "3" not "Wait briefly"
3. Missing required fields (type, step_description, etc.)
4. Invalid block type values
5. Invalid step_type for platform
6. Missing variable_name in extraction blocks
7. Empty then/body arrays in if/while blocks