Repository navigation
[Fix-18672][UI] Allow SeaTunnel workflow tasks to select stream execution - #18673
sunlishuo25 wants to merge 36 commits into
Conversation
|
Thanks for opening this pull request! Please check out our contributing guidelines. (https://github.com/apache/dolphinscheduler/blob/dev/docs/docs/en/contribute/join/pull-request.md) |
Discard the previous task's stored execution mode only when its task type changes, so the new type uses the existing defaults. Preserve SeaTunnel's explicit mode when the type is unchanged and when a definition is reopened. Add focused source checks for task type transitions, mode preservation, custom/resource configuration round trips, and legacy defaults.
SbloodyS
left a comment
SbloodyS
left a comment
There was a problem hiding this comment.
The new E2E covers execution-type selection and save/reload, but leaves several changed paths untested.
Please add regression coverage for copying a STREAM SeaTunnel task, updating connected-edge styling when switching between BATCH and STREAM, and disabling SeaTunnel savepoint while keeping Flink savepoint enabled.
The test also never executes the workflow or checks the Stream Task instance list, so it does not verify the instance-classification outcome described in the PR.
|
Thanks for the review, @SbloodyS. Commit f1acc46 now includes regression coverage for STREAM task copying and persistence, connected-edge styling when switching BATCH/STREAM, disabled SeaTunnel savepoint with Flink savepoint preserved, and actual workflow execution followed by Stream/Batch instance-list checks. Local validation passed all 43 browser checks and the complete WorkflowSeaTunnelE2ETest class. The workflow was actually started, but the SeaTunnel tasks failed because /bin/seatunnel.sh was missing (exit 127). The STREAM classification assertions passed; this does not demonstrate successful SeaTunnel engine execution. Local validation used ARM64 with reduced JVM heap settings. The tests are in WorkflowSeaTunnelE2ETest.java and dolphinscheduler-ui/tests/task-execution-graph.cjs. The latter is included in the Frontend workflow. All nine upstream pull_request workflows for this head are still awaiting approval with zero jobs executed. |
| echo "Code format check failed! Please run \`pnpm run lint\` to format the code." | ||
| exit -1 | ||
| fi | ||
| - name: Task execution type regressions |
There was a problem hiding this comment.
Removed the extra step from frontend.yml in 1f47241; the existing lint/build checks are kept.
| import org.openqa.selenium.support.ui.ExpectedConditions; | ||
|
|
||
| @DolphinScheduler(composeFiles = "docker/basic/docker-compose.yaml") | ||
| class WorkflowSeaTunnelE2ETest { |
There was a problem hiding this comment.
This is a fake e2e test. The principle of e2e is to verify the real user scenario, which did not start the seatunnel task.
There was a problem hiding this comment.
The test now uses the real SeaTunnel 2.3.3 fixture in docker/seatunnel-task. Two complete local runs of 1f47241 passed: Console output, UI Stop, the same PID exiting, and a successful BATCH run. The local setup differences and pending upstream approval are described in the updated PR body.
| return null; | ||
| } | ||
| // Require an unclipped center so the hit test matches the native pointer location. | ||
| boolean target = Boolean.TRUE.equals(((JavascriptExecutor) driver).executeScript( |
There was a problem hiding this comment.
Removed the added hit test in 1f47241. WorkflowForm.java now uses native WebDriver interactions with the rendered task label for opening and copying. The existing drag-and-drop helper is still used for adding nodes.
Keep a task query pending while its table is visibly loading instead of repeatedly refreshing it. Read the task state before sampling the current table class, avoiding a loading-selector lookup that could wait for the next poll. Preserve prompt business assertions and the existing120-second/500ms wait.
Locate this case's exact task operation cell under the existing Batch and Stream row boundaries. Preserve the original native retry and ten-second completion wait, and check the visible Log completion after each successful click in the same iteration. Keep shared page helpers, Stop, workflow assertions and engine waits unchanged.
SbloodyS
left a comment
SbloodyS
left a comment
There was a problem hiding this comment.
WorkflowSeaTunnelE2ETest failed. Please fix it. @sunlishuo25
Give the SeaTunnel case a 90-minute budget while retaining 30 minutes for other cases. Bound both pinned fixture downloads and confirm project creation with native Enter, preserving the existing business assertions.
|
@SbloodyS I ran the full WorkflowSeaTunnelE2ETest locally, and it passed on Linux ARM64 with the default application and SeaTunnel heap settings: 1 test, 0 failures, 0 errors, 0 skipped. I've attached the test report, relevant logs, and recording. |
| needs: build | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 30 | ||
| timeout-minutes: ${{ matrix.case.name == 'WorkflowSeaTunnelE2ETest' && 90 || 30 }} |
There was a problem hiding this comment.
All e2e tests need to be completed within 30 minutes. Please optimize the implementation.
|
@SbloodyS I've restored the 30-minute timeout and fixed the E2E UI interactions. Both WorkflowSwitchE2ETest and WorkflowSeaTunnelE2ETest passed locally on Linux ARM64 with the default application and SeaTunnel heap settings. The test reports, logs, and recordings are attached in one ZIP. |
This branch has not been deployed
2 participants
Was this PR generated or assisted by AI?
YES. Code investigation, implementation, regression tests, documentation, and this description were assisted by OpenAI Codex. Local checks and their limitations are reported below.
Purpose of the pull request
Fixes #18672.
A SeaTunnel workflow node always defaults to
taskExecuteType=BATCH, even when its SeaTunnel configuration runs a streaming job, and the form has no way to select Stream Task. Add an explicit Batch/Stream choice using the existing execution-type field so an opted-in workflow task can appear in the Stream Task instance list. Batch remains the default, including for existing definitions without an execution type.The choice is independent of the SeaTunnel configuration, supporting both custom scripts and resource files without trying to parse HOCON in the browser. Users must also select the matching SeaTunnel job mode and use attached submission.
Brief change log
Verify this pull request
The current head is
19dd1014. The E2E matrix job timeout is restored to 30 minutes, removing the SeaTunnel 90-minute exception.Both complete test classes passed locally on Linux ARM64 with cached assets and the default application and SeaTunnel heap settings: WorkflowSwitchE2ETest (2 tests) and WorkflowSeaTunnelE2ETest (1 test), with no failures, errors or skips. The tested candidate differs from this head only by two whitespace-only changes. Reports, logs and recordings are attached in the validation comment.
Official AMD64 CI is awaiting maintainer approval. These local results do not establish that the full E2E matrix, including cold-start setup, completes within the 30-minute per-job limit.
Historical validation for
20b576b:--no-cachebuild exercised the candidate Dockerfile's curl RUN steps without COPY replacements or cached download layers. Download, pinned checksum verification and fixture build passed in 26 minutes 17 seconds.This run retained the default application heap (4 GiB, young generation 2 GiB) and engine heap (1 GiB); Maven used 512 MiB. The local execution used an ARM64-derived base with sudo restored, ARM64 browser support and local recording adaptations, including two FLV substitutions. These adaptations are not PR changes. The official native AMD64/default CI setup remains unverified; the local QEMU attempt with the official AMD64 Java image failed with SIGSEGV and is not a PASS. Earlier adapted runs failed with missing sudo or without a project-creation POST; those failures remain in the verification history.
The committed regression covers Cancel/Save, STREAM copying and persistence, connected-edge styling and real instance classification. Savepoint coverage uses the production renderer/service tests, retaining enabled Flink savepoint; the complete Selenium case has no savepoint-click assertion. The Node regression files remain available locally, with the added Frontend workflow steps removed as requested.
Historical validation for
1f47241: two complete local runs on October 12 passed, each with JUnit 1/0/0/0. Independent checks passed 35 source checks, 30 mounted controls and offline E2E Java compilation, and the author accepted that head after reviewing screenshots/video. Those runs used a reduced application heap, cached pinned components supplied through COPY and local FLV recording substitutions; they did not validate the network download path or the official AMD64/default-heap setup. Earlier local UI ESLint,vue-tsc --noEmit, production build and JDK 11 Spotless checks also passed.Upstream CI for this head still has nine pull_request workflows awaiting maintainer approval. The official AMD64 E2E has not run for this head. Local PASS does not mean upstream CI has passed.
This change is limited to classification of workflow-managed tasks. It does not restore standalone streaming execution, add detached-job tracking/savepoint/recovery, or fix existing engine-cancellation and cross-workflow streaming-dependency behavior.
Pull Request Notice
Pull Request Notice