Access file/folder path in databricks

In databricks notebook, following are the way 2 ways to access the folder (example folder name: mnt):

1. If you are reading from DBFS with pyspark, then you start your filename like this ‘/mnt/….’
2. If you are reading form DBFS with regular python, you start your filename like this ‘/dbfs/mnt/…’

HTH

Setup Single Node Job Cluster from Datafactory

 To Setup Cluster from datafactory job - 

1. First we need to make the Workers to 0 - Yes set it to 0. See image below:
   
   
2. Then under Addition Cluster Settings, look for Cluster Spark conf and set following name/values.
   
   "spark.databricks.cluster.profile": "singleNode"
   "spark.master": "local[*, 4]" 
   
   
   
   
   
3. Last we need to set Tag under Cluster custom tags:
   
   
   "ResourceClass": "SingleNode" 
   
   
   

   
   
   

That's it!!!

HTH