Databricks serverless: job env differs from notebook env, pin extras

A notebook submitted via databricks jobs submit ran on an old serverless env (mlflow 2.11, sklearn 1.3, plotly present), but the same notebook deployed with databricks bundle deploy ran on a newer env where plotly was missing (ModuleNotFoundError). Always %pip install every non-core lib (plotly, mlflow>=3) instead of trusting the preinstalled set. %pip install -q .. works from a bundle/Git-folder notebook to install the repo's pyproject.toml (hatchling, src layout), which also removes the sys.path hack for src/ imports. For sklearn models scored with mlflow.pyfunc.spark_udf, log with pyfunc_predict_fn="predict_proba" and a predict_proba signature, then use result_type="array<double>" and take element [1]; notebook-scoped %pip libs reach the serverless UDF workers.