This browser is no longer supported.
Upgrade to Microsoft Edge to take advantage of the latest features, security updates, and technical support.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(166.4, 97%, 26%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ES%3C/text%3E%3C/svg%3E)
Hi
I'm running a scheduled multiple run notebook using the below configuration, but I keep getting the below error
DAG = {
"activities": [
{
"name": "Notebook",
"path": "Notebook1",
"timeoutPerCellInSeconds": 1800,
},
{
"name": "Notebook2",
"path": "Notebook2",
"timeoutPerCellInSeconds": 1800
},
{
"name": "Notebook3",
"path": "Notebook3",
"timeoutPerCellInSeconds": 1800
},
{
"name": "Notebook4",
"path": "Notebook4",
"timeoutPerCellInSeconds": 1800
},
{
"name": "Notebook5",
"path": "Notebook5",
"timeoutPerCellInSeconds": 1800
},
{
"name": "Notebook6",
"path": "Notebook6",
"timeoutPerCellInSeconds": 1800
},
{
"name": "Notebook7",
"path": "Notebook7",
"timeoutPerCellInSeconds": 1800
},
],
"timeoutInSeconds": 43200,
"concurrency": 5
}
mssparkutils.notebook.runMultiple(DAG, {"displayDAGViaGraphviz": False})
Error :
Spark_Ambiguous_Executor_MaxExecutorFailures: Livy session has failed. Session state: Dead. Error code: Spark_Ambiguous_Executor_MaxExecutorFailures. Job failed during run time with state=[dead].
TSG:Application failed because too many executors failed. The number of acceptable
executor failures is controlled by the config spark.yarn.max.executor.failures.
To investigate this failure, look at the executor logs and error codes.
Source: Unknown.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(153.6, 16%, 24%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ERG%3C/text%3E%3C/svg%3E)
Hi Seelan,
Can you please specify the Learn Path/Module that you are going through?
Thank you.
Hi Rakesh,
I'm not going through a module, I'm creating my own notebook.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(86.4, 36%, 18%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EBM%3C/text%3E%3C/svg%3E)
Hello Seelan,
The task failed because the executor that is running was lost.
This may happen because the task crashed the JVM.
Here are the Possible causes:
Please check executor logs and error codes and see if you find any more details.
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(35.2, 11%, 13%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3ELJ%3C/text%3E%3C/svg%3E)
This is more or less the exact stderr output for the mentioned error. Would be more useful to get a hint on how to investigate further. Even if one manages to find out that the problem is caused by (for example) number 1, you still need help to investigate why the driver has memory issues.