fix: flaky e2e patterns #16696

zomars · 2024-09-18T02:13:45Z

What does this PR do?

This PR refactors various end-to-end tests to use a new submitAndWaitForResponse utility function for form submissions and API calls. It also introduces a confirmReschedule helper function to standardize the rescheduling process across tests. These changes improve test reliability and reduce duplication.

Key changes include:

Replacing direct API calls and response waiting with submitAndWaitForResponse
Updating booking, rescheduling, and cancellation flows to use the new utility functions
Refactoring team invitation and webhook tests to use the new helpers
Updating various tests related to event types, profiles, and organizations

Mandatory Tasks (DO NOT REMOVE)

I have self-reviewed the code (A decent size PR without self-review might be rejected).
I have added a Docs issue here if this PR makes changes that would require a documentation change. If N/A, write N/A here and check the checkbox.
I confirm automated tests are in place that prove my fix is effective or that my feature works.

How should this be tested?

Run the full end-to-end test suite to ensure all tests pass with the new changes.
Pay special attention to tests involving booking, rescheduling, and API interactions.
Verify that the new utility functions are working as expected across different test scenarios.
Check that no regressions have been introduced in the affected areas (e.g., team invitations, webhooks, event types).

Checklist

I have read the contributing guide
My code follows the style guidelines of this project
I have commented my code, particularly in hard-to-understand areas
I have checked if my changes generate no new warnings

Signed-off-by: zomars <[email protected]>

vercel · 2024-09-18T02:13:56Z

The latest updates on your projects. Learn more about Vercel for Git ↗︎

2 Skipped Deployments

Name	Status	Preview	Comments	Updated (UTC)
cal	⬜️ Ignored (Inspect)	Visit Preview		Sep 19, 2024 8:50pm
calcom-web-canary	⬜️ Ignored (Inspect)	Visit Preview		Sep 19, 2024 8:50pm

zomars · 2024-09-18T02:13:59Z

fix: flaky e2e patterns #16696 👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @zomars and the rest of your teammates on Graphite

graphite-app · 2024-09-18T02:15:10Z

Graphite Automations

"Add foundation team as reviewer" took an action on this PR • (09/18/24)

1 reviewer was added to this PR based on Keith Williams's automation.

zomars · 2024-09-18T02:26:42Z

apps/web/playwright/change-username.e2e.ts

+const TESTING_USERNAMES = [
+  {
+    username: "demousernamex",
+    description: "",
+  },
+  {
+    username: "demo.username",
+    description: " to include periods(or dots)",
+  },
+];


Merged tests into parametized one since it was the exact same code with only username differences

zomars · 2024-09-18T02:27:15Z

apps/web/playwright/event-types.e2e.ts

@@ -225,7 +228,6 @@ testBothFutureAndLegacyRoutes.describe("Event Types tests", () => {
        await page.locator(`text="Cal Video (Global)"`).click();

        await saveEventType(page);
-        await page.getByTestId("toast-success").waitFor();


Already handled on saveEventType

anikdhabal · 2024-09-18T02:28:48Z

apps/web/playwright/change-password.e2e.ts

+    await submitAndWaitForResponse(page, "/api/trpc/auth/changePassword?batch=1", {
+      action: () => page.locator("text=Update").click(),
+    });


This is perfect to not depending on toast messages🙏

zomars · 2024-09-18T02:31:16Z

apps/web/playwright/lib/testUtils.ts

-/** Fastest way so far to test for saving changes and form submissions */
+/**
+ * Fastest way so far to test for saving changes and form submissions
+ * @see https://playwright.dev/docs/api/class-page#page-wait-for-response


Added reasoning for this utility.

zomars · 2024-09-18T02:32:49Z

apps/web/playwright/reschedule.e2e.ts

  }) => {
+    // eslint-disable-next-line playwright/no-skipped-test
+    test.skip(!process.env.DAILY_API_KEY, "DAILY_API_KEY is needed for this test");


IF process.env.DAILY_API_KEY is undefined videoCallUrl will be an empty string which IMO default the purpose of this test

zomars · 2024-09-18T02:33:44Z

apps/web/playwright/team/team-invitation.e2e.ts

@@ -78,7 +78,6 @@ test.describe("Team", () => {
        password: "P4ssw0rd!",
      });
      await page.locator(`button:text("${t("add")}")`).click();
-      await page.locator(`[data-testid="copy-invite-link-button"]`).click();


Already handled in getInviteLink

zomars

Ready for review

zomars

Ready for review

github-actions · 2024-09-18T02:36:18Z

E2E results are ready!

Workflow #32288.1 latest results

anikdhabal · 2024-09-18T02:49:04Z

@zomars Can you check? Some tests are failing, especially login.e2e.

zomars · 2024-09-18T02:50:20Z

@zomars Can you check? Some tests are failing, especially login.e2e.

I'm waiting on published results to do a deep dive.

zomars · 2024-09-19T19:39:44Z

apps/web/components/booking/CancelBooking.tsx

cc @anikdhabal good alternative to a hard refresh

zomars · 2024-09-19T19:40:07Z

apps/web/playwright/fixtures/bookings.ts

+      const seed = `${username}:${dayjs(startDate).utc().format()}:${new Date().getTime()}:${
+        workerInfo.workerIndex
+      }`;


Fixes rare collision case

zomars · 2024-09-19T19:41:13Z

packages/features/bookings/lib/handleCancelBooking.ts

Added waitUntil for non blocking promises and updated the unit test

…/test/webhook.test.ts

anikdhabal · 2024-09-19T20:30:58Z

Inviting an existing user and then › create a booking on new link It often gives a strange issue: net::ERR_BLOCKED_BY_RESPONSE

All checks passed🙏. @zomars can you check this weird issue. This is very flaky

zomars · 2024-09-19T20:35:41Z

Inviting an existing user and then › create a booking on new link It often gives a strange issue: net::ERR_BLOCKED_BY_RESPONSE

All checks passed🙏. @zomars can you check this weird issue. This is very flaky

It happens when we try to waitforUrl and the page does a hard refresh. Causing a response interruption.

anikdhabal · 2024-09-19T20:43:54Z

apps/web/playwright/webhook.e2e.ts

+    const scheduledTriggersAfterDisabling = await prisma.webhookScheduledTriggers.findMany({
+      where: {
+        webhook: {
+          userId: user.id,
+        },
+      },
+      select: {
+        payload: true,
+        webhook: {
+          select: {
+            userId: true,
+          },
+        },
+        startAfter: true,
+      },
+    });
+
+    expect(scheduledTriggersAfterDisabling.length).toBe(0);


Do we need this? We are already checking the response, and then we are making an extra db query just to check the length.

fix: flaky e2e patterns

c8efcd5

Signed-off-by: zomars <[email protected]>

keithwillcode added core area: core, team members only foundation labels Sep 18, 2024

zomars marked this pull request as ready for review September 18, 2024 02:14

graphite-app bot requested a review from a team September 18, 2024 02:14

dosubot bot added the automated-tests area: unit tests, e2e tests, playwright label Sep 18, 2024

zomars added the ready-for-e2e label Sep 18, 2024

vercel bot deployed to Preview – api September 18, 2024 02:21 View deployment

zomars commented Sep 18, 2024

View reviewed changes

anikdhabal reviewed Sep 18, 2024

View reviewed changes

vercel bot deployed to Preview – dev September 18, 2024 02:29 View deployment

zomars commented Sep 18, 2024

View reviewed changes

zomars added 2 commits September 17, 2024 20:02

type fixes

3f4771a

Update users.ts

6b1a838

vercel bot deployed to Preview – api September 18, 2024 03:20 View deployment

vercel bot deployed to Preview – dev September 18, 2024 03:37 View deployment

zomars added 2 commits September 17, 2024 21:24

More fixes

58299e8

Update reschedule.e2e.ts

b782a26

vercel bot deployed to Preview – api September 18, 2024 04:39 View deployment

vercel bot deployed to Preview – dev September 18, 2024 23:47 View deployment

Merge branch 'main' into fix/e2e-wait-for-response

74537c2

vercel bot deployed to Preview – api September 19, 2024 16:37 View deployment

vercel bot deployed to Preview – dev September 19, 2024 16:45 View deployment

More fixes

7da5e11

vercel bot deployed to Preview – api September 19, 2024 18:26 View deployment

vercel bot deployed to Preview – dev September 19, 2024 18:34 View deployment

zomars added 3 commits September 19, 2024 12:00

Update webhook.test.ts

ab871c0

More fixes

e0dd55e

Update reschedule.e2e.ts

6b0554c

vercel bot deployed to Preview – api September 19, 2024 19:34 View deployment

zomars commented Sep 19, 2024

View reviewed changes

apps/web/components/booking/CancelBooking.tsx Outdated

Copy link

Member Author

zomars Sep 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cc @anikdhabal good alternative to a hard refresh

zomars commented Sep 19, 2024

View reviewed changes

vercel bot deployed to Preview – dev September 19, 2024 19:42 View deployment

Update webhook.e2e.ts

0a717b2

vercel bot deployed to Preview – api September 19, 2024 19:56 View deployment

zomars added 2 commits September 19, 2024 13:00

Discard changes to packages/features/bookings/lib/handleCancelBooking.ts

ffad294

Discard changes to packages/features/bookings/lib/handleCancelBooking…

1963915

…/test/webhook.test.ts

vercel bot deployed to Preview – api September 19, 2024 20:16 View deployment

vercel bot deployed to Preview – dev September 19, 2024 20:19 View deployment

zomars marked this pull request as ready for review September 19, 2024 20:35

dosubot bot added the 💻 refactor label Sep 19, 2024

zomars requested a review from anikdhabal September 19, 2024 20:40

anikdhabal reviewed Sep 19, 2024

View reviewed changes

Merge branch 'main' into fix/e2e-wait-for-response

c7f903b

vercel bot deployed to Preview – api September 19, 2024 20:58 View deployment

vercel bot deployed to Preview – dev September 19, 2024 21:07 View deployment

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: flaky e2e patterns #16696

fix: flaky e2e patterns #16696

zomars commented Sep 18, 2024 •

edited

Loading

vercel bot commented Sep 18, 2024 •

edited

Loading

zomars commented Sep 18, 2024

graphite-app bot commented Sep 18, 2024

zomars Sep 18, 2024

zomars Sep 18, 2024

anikdhabal Sep 18, 2024 •

edited

Loading

zomars Sep 18, 2024

zomars Sep 18, 2024

zomars Sep 18, 2024

zomars left a comment

zomars left a comment

github-actions bot commented Sep 18, 2024 •

edited

Loading

anikdhabal commented Sep 18, 2024

zomars commented Sep 18, 2024

zomars Sep 19, 2024

zomars Sep 19, 2024

zomars Sep 19, 2024

anikdhabal commented Sep 19, 2024

zomars commented Sep 19, 2024

anikdhabal Sep 19, 2024

fix: flaky e2e patterns #16696

Are you sure you want to change the base?

fix: flaky e2e patterns #16696

Conversation

zomars commented Sep 18, 2024 • edited Loading

What does this PR do?

Mandatory Tasks (DO NOT REMOVE)

How should this be tested?

Checklist

vercel bot commented Sep 18, 2024 • edited Loading

zomars commented Sep 18, 2024

graphite-app bot commented Sep 18, 2024

Graphite Automations

Choose a reason for hiding this comment

Choose a reason for hiding this comment

anikdhabal Sep 18, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

zomars left a comment

Choose a reason for hiding this comment

zomars left a comment

Choose a reason for hiding this comment

github-actions bot commented Sep 18, 2024 • edited Loading

E2E results are ready!

anikdhabal commented Sep 18, 2024

zomars commented Sep 18, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

anikdhabal commented Sep 19, 2024

zomars commented Sep 19, 2024

Choose a reason for hiding this comment

zomars commented Sep 18, 2024 •

edited

Loading

vercel bot commented Sep 18, 2024 •

edited

Loading

anikdhabal Sep 18, 2024 •

edited

Loading

github-actions bot commented Sep 18, 2024 •

edited

Loading