XTestify Documentation

Table of Contents

Introduction

The Test Automation Platform is a comprehensive SaaS solution designed to streamline and enhance your web application testing process. Leveraging the power of Playwright, AI, and seamless Jira integration, our platform enables teams to automate test execution without writing a single line of code.

Key features include:

  • No-code test automation from Jira test cases
  • AI-powered test execution and error recovery
  • Real-time test monitoring with screenshots
  • Comprehensive HTML test reports
  • Seamless Jira and Xray integration
  • Cloud-based execution with no local setup required

Getting Started

Account Creation

To get started with the Test Automation Platform:

  1. Visit our signup page
  2. Create an account using your email or Google account
  3. Select your subscription plan (Free plan available)
  4. Verify your email address

First Login

After creating your account, you'll be directed to the dashboard. From here, you can:

  • Configure your Jira integration
  • Set up AI features
  • Run your first test
Note

The free plan includes 5 test executions per month. For teams requiring more extensive testing capabilities, check our subscription plans.

Jira Integration

Setting Up Jira Connection

To connect your Jira instance to the platform:

  1. Navigate to the "Jira Integrations" section from your profile dropdown
  2. Click "Add Integration"
  3. Enter a name for your integration (e.g., "Production Jira")
  4. Provide your Jira URL (e.g., https://yourcompany.atlassian.net)
  5. Enter your Jira username (email) and API token
  6. Add your Xray Client ID and Client Secret
  7. Click "Test Connection" to verify your credentials
  8. Save your integration

Generating a Jira API Token

To generate a Jira API token:

  1. Log in to Atlassian Account Settings
  2. Click "Create API token"
  3. Enter a label for your token (e.g., "Test Automation Platform")
  4. Copy the generated token

Xray Configuration

For Xray integration, you'll need to generate API credentials:

  1. Navigate to Xray settings in your Jira instance
  2. Go to API Keys section
  3. Generate a Client ID and Client Secret
  4. Copy these credentials to the platform
Important

Your Jira API token and Xray credentials are encrypted and securely stored. We never store your actual Jira password.

Test Execution

Running a Test

To execute a test from a Jira ticket:

  1. From the dashboard, enter the Jira ticket ID (e.g., TEST-123)
  2. Enter the target URL where the test should be executed
  3. Select the browser type (Chromium, Firefox, or WebKit)
  4. Click "Run Test" to start execution

To execute a test manually written inside the app:

  1. From the dashboard, select "Manual Test"
  2. From the Select Manual Test, pick a Manual test
  3. Click "Run Test" to start execution

Real-time Monitoring

During test execution, you can monitor progress in real-time:

  • View current step being executed
  • See real-time screenshots of the browser
  • Monitor test progress with the progress bar
  • Interact with the AI assistant via chat for help or clarification

User Interaction

Sometimes tests may require your input:

  • When a step fails, you'll be prompted to retry, modify, or confirm failure
  • For data input steps, you may be asked to provide specific information
  • You can communicate with the AI assistant to help resolve issues

AI-Powered Features

AI Configuration

The platform supports multiple AI providers:

  • Default AI Provider: Pre-configured for immediate use
  • OpenAI: Connect using your own API key
  • OpenRouter: Access multiple AI models through a single API
  • Amazon Bedrock: Use Claude, Titan, or Llama models via AWS
  • Local Models: Connect to self-hosted models via Ollama or LM Studio

To configure your AI provider:

  1. Open the AI Configuration panel
  2. Select your preferred provider
  3. Enter the required credentials
  4. Click "Test Connection" to verify

AI-Assisted Testing

Our platform leverages AI for several key functions:

  • Intelligent Test Generation: Automatically translates Jira or Manual Test steps into executable tests.
  • Smart Error Recovery: Analyzes failures and attempts recovery strategies
  • Context-Aware Element Selection: Identifies the right elements even when selectors change
  • Natural Language Assistance: Chat with the AI to get help during test execution
  • Visual Analysis: Analyzes page screenshots to provide visual context and insights.
Note

While AI features enhance the testing experience, the platform also works in "BeautifulSoup mode" without AI for users who prefer not to use AI or have privacy requirements.

Test Reports

HTML Reports

After test execution, a comprehensive HTML report is generated with:

  • Overall test summary and statistics
  • Step-by-step results with pass/fail status
  • Embedded screenshots for visual verification
  • Execution time and performance metrics
  • Error details and troubleshooting information

Accessing Reports

Reports can be accessed in several ways:

  • Directly from the test completion screen
  • From the Test History section in your profile
  • Via direct link that can be shared with team members
  • Downloaded as HTML for offline viewing or integration

Subscription Plans

Feature Free Starter Pro Team
Monthly Price €0 €15 €99 €299
Tests per month 5 35 100 500
Jira integrations 2 2 Unlimited Unlimited
Overage pricing Not available €0.25/test €0.25/test €0.20/test

For enterprise plans, on premise deployments or custom requirements, please contact our sales team.

Troubleshooting

Common Issues

Authentication Problems

If you're experiencing authentication issues with Jira or Xray:

  • Verify your Jira URL is correct and includes the protocol (https://)
  • Ensure your API token has not expired
  • Check that your account has sufficient permissions in Jira
  • For Xray, verify both Client ID and Client Secret are correct

You can test your credentials using the "Test Connection" button when setting up a Jira integration.

Test Execution Failures

If your tests are failing unexpectedly:

  • Check that your test steps in Jira are clear and actionable
  • Verify the target website is accessible and functioning properly
  • Ensure your test data is correctly formatted
  • Try running the test with a different browser
  • Check for dynamic elements or timing issues that might require wait steps

Getting Support

If you continue to experience issues, please contact our support team:

FAQ

No, our platform is designed to be completely no-code. You simply create test cases in Jira, and our platform automatically executes them without requiring any programming knowledge.

For optimal results, create clear, step-by-step test cases in Jira. Each step should describe a single action (e.g., "Click the login button" or "Enter username: testuser"). The more specific your steps are, the better our AI can interpret and execute them.

Our platform supports Chromium, Firefox, and WebKit browsers, powered by Playwright. This gives you coverage across all major browser engines for comprehensive testing.

Our AI analyzes your test steps and the application under test to determine the best way to execute each action. It can identify elements on the page, understand natural language test descriptions, and even recover from errors by trying alternative approaches when a step fails.

We use OpenAI's GPT5 series of models, to deliver the best outcomes for customers. The LLM providers we use do not retain your inputs and outputs, or use them to improve their services.

No, none of our LLM providers store the data you submit or the responses you receive. The data you submit and the responses you receive via XTestify are not used to fine-tune or improve any LLM models or services.