Skip to content

test(cloudflare): Unflake integration test #20208

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
JPeer264 wants to merge 1 commit into
base: develop
Choose a base branch
from
+17 −0
Open

test(cloudflare): Unflake integration test #20208

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions dev-packages/cloudflare-integration-tests/runner.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -242,6 +242,10 @@ export function createRunner(...paths: string[]) {
`SENTRY_DSN:http://public@localhost:${mockServerPort}/1337`,
'--var',
`SERVER_URL:${serverUrl}`,
'--port',
'0',
'--inspector-port',
'0',
...extraWranglerArgs,
],
{ stdio, signal },
Expand Down
10 changes: 10 additions & 0 deletions dev-packages/cloudflare-integration-tests/suites/tracing/durableobject-spans/test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,10 +45,20 @@ it('sends child spans on repeated Durable Object calls', async ({ signal }) => {
// Expect 5 transaction envelopes — one per call.
const runner = createRunner(__dirname).expectN(5, assertDoWorkEnvelope).start(signal);

// Small delay between requests to allow waitUntil to process in wrangler dev.
// This is needed because wrangler dev may not guarantee waitUntil completion
// the same way production Cloudflare does. Without this delay, the last
// envelope's HTTP request may not complete before the test moves on.
const delay = () => new Promise(resolve => setTimeout(resolve, 50));
Copy link
Copy Markdown
Member

@nicohrubec nicohrubec Apr 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

m: can we solve this differently, specifically is there some event that we could await before moving onto the next request instead of adding a timeout? this might already help but I am worried that this will not fully resolve the flakiness

Copy link
Copy Markdown
Member Author

@JPeer264 JPeer264 Apr 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right now there is now way, as the runner doesn't provide a way of doing this. Wrangler, as of now, seems to drop waitUntil runs entirely when another request comes in. For that to work we have to change the way how the runner works, and not register all expect at once and then call the worker, but rather call the worker and wait for the expect right after. But that would be a bigger part of work AFAICT

Copy link
Copy Markdown
Member

@Lms24 Lms24 Apr 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, so to understand correctly: We delay here by 50ms so that a kicked off waitUntil task finishes before we start a new request? And we do this due to a local wrangler limitation (?)

Taking a step back: Why is this test doing 5 request repetitions? I see we always assert on the same payload, without cross-envelope checks, so what do we gain from it? (not saying we shouldn't just that it's not clear).

Given I understand correctly, I'd say the delay is fine (for the lack of better options). But can we make sure this is enough for CI? 50ms seems a bit short but then again, I'm not sure if it's necessary to wait longer. Maybe just deferring to the next tick is already enough?


await runner.makeRequest('get', '/');
await delay();
await runner.makeRequest('get', '/');
await delay();
await runner.makeRequest('get', '/');
await delay();
await runner.makeRequest('get', '/');
await delay();
Copy link
Copy Markdown

@cursor cursor bot Apr 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed-delay sleep may still cause test flakiness

Low Severity

The test introduces a hardcoded 50ms setTimeout delay between requests as a workaround for wrangler dev's waitUntil behavior. Per the review rules, timeouts or sleeps in tests are flagged as likely to introduce flakes — concrete events or signals to wait on are preferred. A 50ms delay may be insufficient under CI load, potentially causing the same flakiness this PR aims to fix. The PR discussion already acknowledges this limitation, noting that the runner doesn't currently provide an event-based mechanism.

Fix in Cursor Fix in Web

Triggered by project rule: PR Review Guidelines for Cursor Bot

Reviewed by Cursor Bugbot for commit f77c6b1. Configure here.

await runner.makeRequest('get', '/');
await runner.completed();
});
3 changes: 3 additions & 0 deletions dev-packages/cloudflare-integration-tests/vite.config.mts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,9 @@ export default defineConfig({
singleThread: true,
},
},
sequence: {
shuffle: true,
Copy link
Copy Markdown
Member

@Lms24 Lms24 Apr 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the shuffle flag it was consistently failing, this is why this is added in this PR as well

Maybe I misunderstand but shouldn't shuffle be set to false then?

},
reporters: process.env.DEBUG
? ['default', { summary: false }]
: process.env.GITHUB_ACTIONS
Expand Down
Loading