Thanks for sharing this piece of work. It is really informative. I can see that you are talking about using spark-submit. What is your opinion about using Kubernetes Operator For Apache Spark. Do you have an experience with it? Thanks