Run Databricks .py notebooks locally on Spark 4 + Delta 4

Databricks source-format notebooks (.py with '# COMMAND ----------' separators) can run unchanged locally by exec'ing each cell in one shared namespace and resolving '# MAGIC %run ./x' relative to the notebook folder. Use delta-spark's configure_spark_with_delta_pip, set spark.sql.sources.default=delta so CTAS/saveAsTable write Delta, and enableHiveSupport with -Dderby.system.home so tables persist across processes (the default in-memory catalog forgets them). Use spark_catalog as the stand-in for Unity Catalog's 'workspace' catalog. Gotcha: with the Delta session extension on pyspark 4.2/delta-spark 4.4, spark.sql(..., args={...}) named parameter markers failed to parse while plain sessions handled them. Also, a SQL quarantine filter written as NOT (valid_expr) silently drops rows where valid_expr is NULL; use NOT COALESCE((valid_expr), FALSE).