feat(local-dev): pick a usable Python and offer to install a missing toolchain

[Yicong-Huang]

Depends on the Linux-support PR (feat(local-dev): support Linux), which this needs for the platform detection. That branch is this one's parent commit, so it shows up in this diff too — review it there, and read only the second commit here. I'll rebase once it lands.

What changes were proposed in this PR?

Two halves of the same fresh-machine problem.

The interpreter that runs Python UDFs was picked blindly. UDF_PYTHON_PATH defaulted to command -v python3 at source time — the system interpreter, which almost never has amber/requirements.txt installed. Python UDFs then failed at worker launch on a stack of import errors that pointed nowhere near the interpreter choice. The venv AGENTS.md tells contributors to create (<workspace>/venv312) was ignored, as was an already-activated $VIRTUAL_ENV.

Resolution is now lazy, and takes the first candidate that is Python 3.12 and can import what a worker needs:

$UDF_PYTHON_PATH  ->  $VIRTUAL_ENV  ->  <workspace>/venv312  ->  pyenv 3.12
  ->  python3.12 on PATH  ->  python3 on PATH

The choice is reported the way the JDK probe reports JAVA_HOME. An explicit UDF_PYTHON_PATH still wins — it is the documented override — but now says so when it can't import the deps. "No 3.12 anywhere" and "3.12 without amber's deps" are reported differently, because they need different fixes and the old default silently produced the second one. Lazy so status / logs / --help don't spawn an interpreter per candidate, and never fatal: the JVM services run fine without a Python toolchain, only UDFs don't.

A missing tool was described, never offered. _install_hint printed a suggestion and gave up, so a fresh machine meant hand-installing five or six things, re-running up after each to discover the next gap. A missing JDK 17, Node or Python 3.12 is now offered for install, after showing the exact command:

Tool	Installer
JDK 17	distro package manager (apt-get/dnf/yum/pacman/zypper), brew on macOS, SDKMAN when there is none
Node 24	nvm (bootstraps nvm first if absent)
Python 3.12	pyenv (bootstraps pyenv first if absent)

jenv is not used: it switches between JDKs, it cannot install one.

The decision helpers (_pkg_manager, _install_cmd_for) only ever print what would run; a separate step executes it. That keeps the choice unit-testable without installing anything, and it means the user sees the command — sudo included — before being asked.

Behaviour deliberately preserved for non-interactive callers: without a TTY nothing is ever prompted and the old print-a-hint-and-fail path stands, so scripted and CI runs cannot hang on a read. --install-missing answers yes without asking, --no-install only ever prints the hint, and the contradiction is refused rather than silently resolved. docker and sbt stay hint-only — docker needs a daemon, group membership and a re-login, which is not something to do behind a y/n prompt.

Any related issues, documentation, discussions?

Closes #7066 Depends on the Linux-support PR (#7065).

How was this PR tested?

New coverage in the existing infra-job suite. Everything goes through the pure decision helpers, so CI asserts on what would be installed without installing anything; the Python probes are driven by fake interpreters that echo a version or set an exit code:

$ bash bin/local-dev/tests/test_local_dev_sh.sh
...
  ✓ pkg manager: apt-get → apt-get
  ✓ pkg manager: dnf only → dnf
  ✓ pkg manager: pacman only → pacman
  ✓ pkg manager: apt-get preferred over dnf → apt-get
  ✓ pkg manager: nothing installed (refuses)
  ✓ _pkg_manager keeps the Darwin/brew branch
  ✓ install cmd: java via apt names openjdk-17-jdk
  ✓ install cmd: java via apt asks for sudo explicitly
  ✓ install cmd: java via dnf names java-17-openjdk-devel
  ✓ install cmd: java with no package manager falls back to SDKMAN
  ✓ install cmd: node uses nvm
  ✓ install cmd: python uses pyenv
  ✓ install cmd: python pins 3.12
  ✓ install cmd: refuses 'docker'
  ✓ install cmd: refuses 'sbt'
  ✓ install cmd: refuses 'definitely-not-a-tool'
  ✓ consent: refuses without a TTY (never prompts)
  ✓ consent: TEXERA_INSTALL_MISSING=1 assumes yes
  ✓ consent: TEXERA_INSTALL_MISSING=0 refuses
  ✓ python probe: 3.12 accepted
  ✓ python probe: 3.11 rejected
  ✓ python probe: missing interpreter rejected
  ✓ python probe: importable deps accepted
  ✓ python probe: missing deps rejected
  ✓ python probe: empty path rejected
  ✓ python candidates: $VIRTUAL_ENV before sibling venv312
  ✓ python candidates: venv312 before bare python3 on PATH
  ✓ _require_udf_python helper is defined
  ✓ cmd_up resolves the UDF interpreter before launching
  ✓ cmd_auto resolves the UDF interpreter before launching
  ✓ cmd_up_one resolves the UDF interpreter before launching
  ✓ _require_udf_python points at amber's requirements, not the TUI hint
  ✓ UDF_PYTHON_PATH no longer defaults to bare python3 at source time
  ✓ --help documents --install-missing / --no-install
  ✓ up rejects --install-missing together with --no-install (rc=2)

103 passed, 0 failed

$ python -m pytest bin/local-dev/tests/ -q
1 failed, 42 passed

test_is_dirty_after_seed_then_edit is pre-existing and unrelated — it reproduces identically on a pristine upstream/main checkout on this machine, and this PR touches neither tui.py nor that test. Filed as #7075 and fixed there.

Manually, on Ubuntu 24.04.4 with the amber deps installed into a sibling venv312 and nothing in UDF_PYTHON_PATH — it finds the venv rather than the system interpreter the old default would have taken:

$ bin/local-dev.sh auto
  ✓  python: /home/…/Repos/venv312/bin/python  (runs Python UDFs)

$ command -v python3            # what the old default resolved to
/usr/bin/python3
$ /usr/bin/python3 -c "import pyarrow"
ModuleNotFoundError: No module named 'pyarrow'

The consent path, forced by pinning a version that cannot exist (TEXERA_PYTHON_VERSION=3.99). With a TTY it shows the command and asks; the answer here was n, and the run continued:

  ⚠  python: no Python 3.99 found — Python UDFs will not run
  looked in: $VIRTUAL_ENV, <workspace>/venv312, pyenv, PATH
  or point at one yourself: export UDF_PYTHON_PATH=/path/to/python3.99

  python is missing. I can run:

      curl -fsSL https://pyenv.run | bash && export PYENV_ROOT="$HOME/.pyenv" && … && pyenv install -s 3.99

  Run it now? [y/N]

Without a TTY (</dev/null) the same run prints the warning, never prompts, does not hang, and exits 0. Both flags are accepted on their own (auto --no-install, auto --install-missing → rc 0) and rejected together (rc 2).

Not exercised end-to-end: actually running the JDK / Node / pyenv installers, since this machine already has all three. The commands themselves are asserted by the tests above; the execution step is a bash -lc of exactly the string shown to the user.

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Code (Claude Opus 5)

[github-actions]

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

• Committers with relevant context: @parshimers
  You can request their reviews formally with /request-review @parshimers.

• Contributors with relevant context: @aglinxinyuan
  You can notify them by mentioning @aglinxinyuan in a comment.