vagrant plugin install vagrant-google Following is a sample Vagrantfile to create the VM. Vagrant.configure("2") do |config| config.vm.box = "google/gce"# Provider to set up VM in Google Cloud config.vm.provider :google do |google, override| google.google_project_id = "<Your google cloud project ID here>" google.google_json_key_location = "<Path to JSON key here>" # 2vCPU, 4GB google.machine_type='e2-medium' # Use Ubuntu 20.04 google.image_family = 'ubuntu-2004-lts' google.name = 'algorand-index-server' # A
I published these notebooks to a github repo, spark-experiments . Right now it has some basic and spark-sql based experiments. I’ll be adding more as I go.
Rather than setting up Jupyter, Spark, and everything else needed locally, I found an existing Docker image, pyspark-notebook , that contains everything I needed, including matplotlib to visualize the data as I get further along. If you have Docker installed, you just run the Docker container via a single command, and you’re off and running. See the spark-experiments installation instructions for details.