feat(frontend): add the Models section to the dashboard - #7046
Draft
tanishqgandhi1908 wants to merge 26 commits into
Draft
feat(frontend): add the Models section to the dashboard#7046tanishqgandhi1908 wants to merge 26 commits into
tanishqgandhi1908 wants to merge 26 commits into
Conversation
… tests Review fixes on apache#5911: use dataset owner email in retrieveLatestDatasetVersion, replace brittle Option.get/.head with headOption, strip datasets prefix on selection-modal reopen, clarify FileResolver docs; add FileResolver/DatasetFileNode and frontend path-helper tests. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The datasets logical-path prefix strips four leading segments (datasets/owner/dataset/version); update the cover-image test's input path to include the prefix so the extracted relative path is the file name, not an empty string. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Now that models are coming as a separate resource/table, the leading resource-type segment is what selects the backing table, so an unprefixed path can no longer be routed unambiguously. Make the "datasets" prefix required instead of a tolerated fallback, and migrate existing data. - FileResolver: a dataset path must start with the "datasets" segment; unprefixed paths are no longer treated as dataset paths. - pytexera DatasetFileDocument: same rule, mirroring the backend. - FileListerSourceOpExec: parse the now-prefixed datasetVersionPath (skip the "datasets" segment); extracted into a testable helper. - Migration (sql/updates/29.sql): prepend "datasets/" to legacy paths stored in workflow.content and workflow_version.content, covering both the fileName (scan sources) and datasetVersionPath (file lister) operator properties. Only values whose first two segments match an existing (user.email, dataset.name) are rewritten, so local paths and URLs are left untouched; email format is irrelevant (owner may be a username without "@"). Uses create_missing=false and is idempotent. - Example workflows: use datasets-prefixed paths. Tests: FileResolverSpec and WorkflowExecutionsResourceSpec updated; test_dataset_file_document.py updated; new FileListerSourceOpExecSpec. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…l-path-prefix # Conflicts: # frontend/src/app/common/type/datasetVersionFileTree.spec.ts
…format python test - dataset-selection-modal.component.spec.ts: expect the datasets-prefixed selectedPath in version (non-file) mode, matching the emitted path. - test_dataset_file_document.py: apply ruff format (wrap an over-length line) so `ruff format --check` passes. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The cover-image endpoints resolve a dataset path built as
{owner}/{name}/{coverImage}. With the prefix now required by FileResolver,
these must carry the datasets/ segment; add it in the set-cover,
get-cover redirect, and cover-url handlers so cover images resolve again.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…storage # Conflicts: # sql/changelog.xml
…rage # Conflicts: # common/workflow-core/src/test/scala/org/apache/texera/amber/core/storage/DocumentFactorySpec.scala
The explicit-access branch of listModels hardcoded size 0 while the public branch computed it, so a user's own models reported no size. Compute it in both, mirroring the dataset listing, and degrade to 0 rather than dropping the row when LakeFS cannot answer.
github-actions
Bot
added
feature
engine
ddl-change
github-actions
Bot
added
frontend
Contributor
Automated Reviewer SuggestionsBased on the
|
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #7046 +/- ##
============================================
+ Coverage 77.30% 79.50% +2.20%
- Complexity 3524 3849 +325
============================================
Files 1161 1149 -12
Lines 45922 43607 -2315
Branches 5101 5191 +90
============================================
- Hits 35501 34671 -830
+ Misses 8840 7284 -1556
- Partials 1581 1652 +71
☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Contributor
|
| config | throughput | MB/s | latency | max Δ latest / 7d | |
|---|---|---|---|---|---|
| 🔴 | bs=10 sw=10 sl=64 | 363 | 0.222 | 25,238/37,117/37,117 us | 🔴 -14.4% / 🔴 +135.3% |
| 🔴 | bs=100 sw=10 sl=64 | 781 | 0.476 | 124,340/170,211/170,211 us | 🔴 +10.1% / 🔴 +58.5% |
| ⚪ | bs=1000 sw=10 sl=64 | 913 | 0.557 | 1,095,712/1,150,204/1,150,204 us | ⚪ within ±5% / 🔴 +11.9% |
Baseline details
Latest main a61702f from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---|---|---|---|---|
| bs=10 sw=10 sl=64 | throughput | 363 tuples/sec | 424 tuples/sec | 786.12 tuples/sec | -14.4% | -53.8% |
| bs=10 sw=10 sl=64 | MB/s | 0.222 MB/s | 0.259 MB/s | 0.48 MB/s | -14.3% | -53.7% |
| bs=10 sw=10 sl=64 | p50 | 25,238 us | 24,385 us | 12,305 us | +3.5% | +105.1% |
| bs=10 sw=10 sl=64 | p95 | 37,117 us | 32,669 us | 15,774 us | +13.6% | +135.3% |
| bs=10 sw=10 sl=64 | p99 | 37,117 us | 32,669 us | 18,978 us | +13.6% | +95.6% |
| bs=100 sw=10 sl=64 | throughput | 781 tuples/sec | 844 tuples/sec | 999.71 tuples/sec | -7.5% | -21.9% |
| bs=100 sw=10 sl=64 | MB/s | 0.476 MB/s | 0.515 MB/s | 0.61 MB/s | -7.6% | -22.0% |
| bs=100 sw=10 sl=64 | p50 | 124,340 us | 115,509 us | 100,616 us | +7.6% | +23.6% |
| bs=100 sw=10 sl=64 | p95 | 170,211 us | 154,643 us | 107,356 us | +10.1% | +58.5% |
| bs=100 sw=10 sl=64 | p99 | 170,211 us | 154,643 us | 113,255 us | +10.1% | +50.3% |
| bs=1000 sw=10 sl=64 | throughput | 913 tuples/sec | 919 tuples/sec | 1,031 tuples/sec | -0.7% | -11.5% |
| bs=1000 sw=10 sl=64 | MB/s | 0.557 MB/s | 0.561 MB/s | 0.63 MB/s | -0.7% | -11.5% |
| bs=1000 sw=10 sl=64 | p50 | 1,095,712 us | 1,091,668 us | 980,328 us | +0.4% | +11.8% |
| bs=1000 sw=10 sl=64 | p95 | 1,150,204 us | 1,134,182 us | 1,027,528 us | +1.4% | +11.9% |
| bs=1000 sw=10 sl=64 | p99 | 1,150,204 us | 1,134,182 us | 1,054,298 us | +1.4% | +9.1% |
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,551.04,200,128000,363,0.222,25238.28,37117.07,37117.07
1,100,10,64,20,2562.41,2000,1280000,781,0.476,124339.89,170210.79,170210.79
2,1000,10,64,20,21913.00,20000,12800000,913,0.557,1095711.50,1150204.27,1150204.27
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this PR?
Adds a Models section to the dashboard so models can be managed from the UI. Lists the models a user can access, with card and list views, keyword search, sort, inline rename and description editing, sharing, and delete.
Gated behind a new gui.tabs.models_enabled flag, defaulting to false: the list page ships before the detail page, so the section stays hidden until complete.
Any related issues, documentation, discussions?
Closes #6499
Depends on #6974 — base is main, so the diff shows that PR's commits until it merges
How was this PR tested?
cd frontend && ng test --include "src/app/dashboard/**"
sbt "ConfigService/testOnly *ConfigResourceSpec" "Config/testOnly *DefaultsConfigSpec"
229 tests across the 7 touched specs; ConfigResourceSpec 33 and DefaultsConfigSpec 5 for the new tab flag. New specs cover the model service (15) and the list page (14: filtering across name/description/framework/owner, AND-ing across chips, sort fallback for the edit-time key models lack, caching so a filter change does not refetch). Model branches added to list-item (24) and card-item (74), including that the card links to the user route when accessibleUserIds is empty, does not fetch a dataset cover, and routes rename/description/share through ModelService.
Manually verified against the local stack: sidebar appears only with the flag on, both views render, search and sort work, share opens against /access/model/*, delete removes the row.
Was this PR authored or co-authored using generative AI tooling?
Generated-by: Claude Code (Claude Opus 5)