Bridging the Gap: Triggering GitLab E2E Tests After Vercel Deployments

Vercel offers a fantastic developer experience for deploying frontend applications, especially with its seamless Git integration and preview deployments for every branch. However, running automated end-to-end (E2E) tests against these deployments presents a common challenge: how do you ensure your tests only run *after* Vercel has finished building and deploying the unique URL?

Simply triggering your test suite immediately upon commit won't work, as the Vercel deployment takes time. Your tests would likely fail trying to hit a URL that isn't live yet.

In this post, we'll explore an approach that leverages Vercel's build process to trigger a GitLab CI pipeline, which intelligently waits for the Vercel deployment to be ready before kicking off E2E tests using Playwright.

The Core Idea

The workflow involves three main parts working in concert:

• Vercel Build: When Vercel detects changes and starts a build, we use its build command hook to execute a script.
• Trigger Script: This script calls the GitLab API to trigger a specific CI/CD pipeline, passing along crucial information like the Vercel deployment URL.
• GitLab CI Job: The triggered GitLab job uses the Vercel CLI to explicitly wait for the deployment URL to become active before proceeding to run the E2E tests.

Visualising the Flow

Here's a sequence diagram illustrating the process:

‍

Setup Requirements

GitLab Pipeline Trigger Token

• In your GitLab project, go to Settings -> CI/CD -> Pipeline triggers.
• Create a trigger token. Store this value securely.
• Add this token as an environment variable named GITLAB_PIPELINE_TRIGGER_TOKEN in your Vercel project settings.

Vercel Access Token

• In Vercel, go to Account Settings -> Tokens.
• Create an access token.
• Add this token as a *masked* CI/CD variable named VERCEL_TOKEN in your GitLab project settings (Settings -> CI/CD -> Variables).

Vercel Protection Bypass for Automation

• If you have enabled Vercel Authentication on deployments that you want to have e2e tests, you will need the Protection Bypass for Automation token, to allow Playwright to run its test against those deployments.
• In Vercel, go to the Project Settings -> Protection Bypass for Automation, and ensure the bypass secret exists.

Other Vercel Variables

Ensure GITLAB_PROJECT_ID is set as an environment variable in your Vercel project. The VERCEL_* variables used in the trigger script are typically provided automatically by Vercel's build environment.

Vercel CLI

Make sure the vercel CLI is available in the GitLab CI job's environment (either install it in the before_script, use an image that includes it, or add it as a dev dependency and call via yarn vercel).

‍

Step-by-step

Step 1: Hooking into the Vercel Build

Vercel allows you to specify custom build commands in your project settings or package.json. We'll use a dedicated script for CI builds.

In package.json, we define a build:ci script:

// package.json
{
  // ... other config
  "scripts": {
    "build": "yarn lint && yarn typecheck && nuxt build", // Regular build
    "build:ci": "nuxt build && yarn tsx scripts/trigger-pipeline.ts", // Build for CI/Vercel
    "test:e2e": "playwright test"
    // ... other scripts
  }
}

When Vercel runs its build process, using the "Build Command" configured in Vercel project settings, which should be set to yarn build:ci (or similar for npm, pnpm etc), it first builds the application (nuxt build in this Nuxt.js example) and then executes our custom script scripts/trigger-pipeline.ts.

Step 2: Triggering the GitLab Pipeline

The scripts/trigger-pipeline.ts script is the bridge between Vercel and GitLab. It uses Vercel's automatically available environment variables during the build process.

// scripts/trigger-pipeline.ts
import "dotenv/config";
import { PipelineTriggerTokens } from "@gitbeaker/rest";

function main() {
  // Essential variables provided by Vercel's build environment
  const projectId = process.env.GITLAB_PROJECT_ID; // Your GitLab Project ID
  const commitRef = process.env.VERCEL_GIT_COMMIT_REF; // The branch/commit SHA being deployed
  const pipelineTriggerToken = process.env.GITLAB_PIPELINE_TRIGGER_TOKEN; // GitLab trigger token (see setup below)
  const vercelUrl = process.env.VERCEL_URL; // The unique deployment URL (e.g., project-git-branch-user.vercel.app)
  const vercelAutomationBypassSecret = process.env.VERCEL_AUTOMATION_BYPASS_SECRET; // Vercel's bypass secret
  const pullRequestId = process.env.VERCEL_GIT_PULL_REQUEST_ID; // Optional: MR ID if applicable
  const vercelProductionUrl = process.env.VERCEL_PROJECT_PRODUCTION_URL; // Your main production URL

  if (
    !projectId ||
    !commitRef ||
    !pipelineTriggerToken ||
    !vercelUrl ||
    !vercelAutomationBypassSecret ||
    !vercelProductionUrl
  ) {
    throw Error("Pipeline trigger env vars not correctly setup");
  }

  // Variables to pass to the GitLab CI job
  let pipelineVariables: Record<string, string> = {
    VERCEL_URL: vercelUrl,
    VERCEL_AUTOMATION_BYPASS_SECRET: vercelAutomationBypassSecret,
    VERCEL_PROJECT_PRODUCTION_URL: vercelProductionUrl,
  };

  // Pass MR ID if it's a merge request build
  if (pullRequestId && pullRequestId !== "") {
    pipelineVariables = { ...pipelineVariables, CI_MERGE_REQUEST_IID: pullRequestId };
  }

  // Use @gitbeaker/rest to interact with GitLab API
  const pipelineTrigger = new PipelineTriggerTokens({});
  console.log(`Triggering GitLab pipeline for commit: ${commitRef} on project: ${projectId}`);
  pipelineTrigger.trigger(projectId, commitRef, pipelineTriggerToken, {
    variables: pipelineVariables, // Pass our collected variables
  });
  console.log("GitLab pipeline triggered successfully.");
}

main();

This script:

1. Reads necessary environment variables (provided by Vercel and configured by you).
2. Constructs a set of variables to pass to the GitLab job, including the crucial VERCEL_URL.
3. Uses the @gitbeaker/rest library (you could also use curl or another HTTP client) to hit the GitLab API's pipeline trigger endpoint.

Step 3: Waiting and Testing in GitLab CI

Now, over in your .gitlab-ci.yml, you define the job that gets triggered.

# .gitlab-ci.yml

variables:
  CI: 1 # Example variable

stages:
  - test
  - build
  - e2e # Our E2E stage
  - deploy

# ... other job definitions (lint, unit tests, etc.) ...

e2e-test:
  extends: .common_template # Assuming a common setup template
  stage: e2e
  needs: [] # Run independently as it's triggered
  rules:
    # VERY IMPORTANT: Only run this job when triggered via API
    - if: $CI_PIPELINE_SOURCE == "trigger"
  image: mcr.microsoft.com/playwright:v1.48.2-noble # Use a Playwright-compatible image
  variables:
    # Use the variables passed from the trigger script
    DEPLOYMENT_BRANCH_URL: ${VERCEL_URL}
    DEPLOYMENT_BRANCH_HTTPS_URL: "https://${VERCEL_URL}"
    PRODUCTION_HTTPS_URL: "https://${VERCEL_PROJECT_PRODUCTION_URL}"
    # VERCEL_TOKEN needs to be configured as a masked GitLab CI/CD variable
  script:
    # Install Playwright browsers
    - yarn playwright install --with-deps
    # THE MAGIC: Wait for Vercel deployment to be ready
    - echo "Waiting for Vercel deployment at https://${DEPLOYMENT_BRANCH_URL} to be ready..."
    - yarn vercel inspect ${DEPLOYMENT_BRANCH_URL} --wait --token ${VERCEL_TOKEN}
    - echo "Vercel deployment ready!"
    # Run E2E tests against the live Vercel URL
    - echo "Running E2E tests..."
    - yarn test:e2e # Playwright is configured (see playwright.config.ts section) to use the correct base URL
  artifacts:
    # ... artifact configuration ...

Key aspects of this job:

1. rules: - if: $CI_PIPELINE_SOURCE == "trigger" : Ensures this job *only* runs when triggered by our script, not on regular commits pushed to GitLab.
2. image: Uses an image with Playwright and its dependencies pre-installed.
3. variables: It receives the VERCEL_URL(and others) passed from the trigger script.
4. yarn vercel inspect ... --wait: This is the crucial command. It uses the Vercel CLI (which needs to be available, hence yarn vercel... assuming it's a dev dependency or installed globally in the image/before_script).
   1. inspect ${DEPLOYMENT_BRANCH_URL}: Targets the specific Vercel deployment URL.
   2. --wait: Tells the CLI to pause the script and continuously check the deployment status until it's "READY".
   3. --token ${VERCEL_TOKEN}: Requires a Vercel Access Token with appropriate permissions, configured as a masked CI/CD variable in GitLab.
5. yarn test:e2e : Only executes *after* vercel inspect --wait completes successfully.

Configuring Playwright for Dynamic URLs

For the E2E tests to target the correct Vercel deployment, Playwright needs to be configured to use the dynamic URL provided by the GitLab CI job. This is typically done in the playwright.config.ts file using the baseURL option within the use block.

Here's how you can set it up to read the environment variables we defined in .gitlab-ci.yml:

// playwright.config.ts
import { defineConfig, devices } from "@playwright/test";
import path from "path";
import dotenv from "dotenv";
import { fileURLToPath } from "url";

// Basic dotenv setup (optional, useful for local testing)
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
dotenv.config({ path: path.resolve(__dirname, ".env") });

// Helper function to determine the correct base URL
function getBaseUrl() {
  // Default for local development (when not in CI)
  if (!process.env.CI) return "http://localhost:3000";

  // In CI, use GitLab/Vercel provided variables
  // Note: CI_COMMIT_REF_NAME is a predefined GitLab variable
  if (process.env.CI_COMMIT_REF_NAME === "develop") {
    // Use a specific URL for the develop branch if needed
    return process.env.DEVELOPMENT_BRANCH_URL; // Set in .gitlab-ci.yml
  }

  if (process.env.CI_COMMIT_REF_NAME === "main") {
    // Use the production URL for the main branch
    return process.env.PRODUCTION_HTTPS_URL; // Set in .gitlab-ci.yml from VERCEL_PROJECT_PRODUCTION_URL
  }

  // Default for other branches/triggers (feature branches, MR previews)
  // Uses the dynamic URL passed from the Vercel trigger
  return process.env.DEPLOYMENT_BRANCH_HTTPS_URL; // Set in .gitlab-ci.yml from VERCEL_URL
}

export default defineConfig({
  testDir: "./test/e2e/",
  fullyParallel: true,
  forbidOnly: !!process.env.CI,
  retries: process.env.CI ? 2 : 0,
  workers: process.env.CI ? 2 : undefined,
  reporter: [/* ... reporters ... */],

  use: {
    /* Base URL to use in actions like `await page.goto('/')`. */
    baseURL: getBaseUrl(), // Dynamically set the base URL!

    trace: "on-first-retry",
    video: "on-first-retry",

    /* Handle Vercel Deployment Protection in CI */
    extraHTTPHeaders:
      process.env.CI && process.env.VERCEL_AUTOMATION_BYPASS_SECRET
        ? {
            // Header to bypass Vercel's password protection
            "x-vercel-protection-bypass": process.env.VERCEL_AUTOMATION_BYPASS_SECRET,
            // Optional: Header to attempt setting a bypass cookie
            "x-vercel-set-bypass-cookie": "true",
          }
        : undefined,
  },

  projects: [/* ... browser projects ... */],

  // webServer config for local development (disabled in CI)
  webServer: !process.env.CI ? [ /* ... local server config ... */] : [],
});

‍

Key points in this configuration:

1.  getBaseUrl() Function: This function checks if the tests are running in a CI environment (process.env.CI). If so, it uses standard GitLab CI variables (CI_COMMIT_REF_NAME) and the custom variables we defined in .gitlab-ci.yml (DEVELOPMENT_BRANCH_URL, PRODUCTION_HTTPS_URL, DEPLOYMENT_BRANCH_HTTPS_URL) to determine the correct URL for the specific context (develop branch, main branch, or a preview deployment).

2. baseURL: getBaseUrl(): The result of this function is assigned to the baseURL option in the use block. Playwright actions like await page.goto('/') will automatically prepend this base URL.

3.  extraHTTPHeaders: This section demonstrates how to automatically include the x-vercel-protection-bypass header when running in CI. It uses the VERCEL_AUTOMATION_BYPASS_SECRET variable (passed from the trigger script and set in .gitlab-ci.yml) to allow Playwright to access Vercel deployments that might be password-protected.

With this setup, your Playwright tests automatically target the correct live Vercel deployment URL provided during the CI run, whether it's a preview, staging, or production environment.

Potential Pitfalls and Troubleshooting

While this workflow is robust, issues can arise. Here are some common pitfalls and how to troubleshoot them:

GitLab Job Not Triggered

Check Vercel Build Logs: Ensure the yarn build:ci command completed successfully and the scripts/trigger-pipeline.ts script executed without errors. Look for the "GitLab pipeline triggered successfully" log message.

Verify Vercel Environment Variables: Double-check that GITLAB_PROJECT_ID and GITLAB_PIPELINE_TRIGGER_TOKEN are correctly set in your Vercel project's environment variables.

Check GitLab Trigger Token: Ensure the token used in Vercel matches an active trigger token in GitLab (Settings -> CI/CD -> Pipeline triggers). Regenerate if necessary.

e2e-test Job Runs Unexpectedly (e.g., on every commit)

Verify rules: in .gitlab-ci.yml: Ensure the rule if: $CI_PIPELINE_SOURCE == "trigger" is correctly applied to the e2e-test job and hasn't been accidentally overridden or removed.

vercel inspect --wait Fails or Times Out

Check VERCEL_TOKEN: Verify that the VERCEL_TOKEN CI/CD variable in GitLab is correct, not expired, and has sufficient permissions (at least read access to deployments). Ensure it's set as *masked*.

Verify Vercel Deployment Status: Manually check the Vercel dashboard for the specific deployment corresponding to VERCEL_URL. Did the build/deployment fail on Vercel's side? Address any Vercel build errors first.

Playwright Tests Fail with 404s or Target Wrong URL

Check baseURL Logic: Add logging within the getBaseUrl() function in playwright.config.ts (e.g., console.log("Using baseURL:", determinedUrl);) and check the GitLab job logs to see which URL is being selected.

Verify Variable Propagation: In the e2e-test job script in .gitlab-ci.yml, add echo "VERCEL_URL is: ${VERCEL_URL}" and echo "DEPLOYMENT_BRANCH_HTTPS_URL is: ${DEPLOYMENT_BRANCH_HTTPS_URL}" *before* the yarn test:e2e command. Check the job logs to ensure these variables have the expected values passed from the trigger.

Check Playwright goto calls: Ensure your tests use relative paths (e.g., page.goto('/login')) which rely on the baseURL, rather than hardcoding absolute URLs.

Playwright Tests Fail Due to Vercel Deployment Protection (Password)

Verify VERCEL_AUTOMATION_BYPASS_SECRET: Ensure this variable is correctly set in Vercel, passed by the trigger script, defined in the .gitlab-ci.yml variables section, and read correctly in playwright.config.ts to set the x-vercel-protection-bypass header. Check GitLab job logs for any errors related to this variable.

Check Header in Playwright: Use Playwright's debugging tools or network inspection to confirm the x-vercel-protection-bypass header is actually being sent with requests during the test run.

Conclusion

By combining Vercel's build hooks, a simple trigger script, and the power of the Vercel CLI within a GitLab CI job, you can create a reliable E2E testing workflow. This ensures your tests always run against a fully deployed Vercel environment, whether it's a preview branch or production, giving you higher confidence in your deployments. This bridge between Vercel's deployment process and GitLab's CI capabilities provides a seamless way to integrate comprehensive testing into your modern frontend development lifecycle.

‍