Adpative scheduler is not recognizing by Flink 1.14.0

Question

I am trying to use adaptive schedule with flink 1.14 to run flink job based on available resources instead of waiting for required parallelism (scaling) but I don't see flink is getting recognize adaptive schedule.

Ex: flink run  -m yarn-cluster -ynm jobName -p 128 -D jobmanager.scheduler=Adaptive  -D cluster.declarative-resource-management.enabled=true -c className JarName

Reference : https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/elastic_scaling/

Caused by: java.util.concurrent.CompletionException: java.util.concurrent.CompletionException: org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException: Slot request bulk is not fulfillable! Could not allocate the required slot within slot request timeout
        at org.apache.flink.runtime.scheduler.DefaultScheduler.lambda$assignResource$8(DefaultScheduler.java:515)
        ... 37 more
Caused by: java.util.concurrent.CompletionException: org.apache.flink.runtime.jobmanager.scheduler.NoResourceAvailableException: Slot request bulk is not fulfillable! Could not allocate the required slot within slot request timeout
        at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:292)
        at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:308)
        at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:607)
        at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591)
        ... 35 more

Regards, Madan

Answer 1

See the section of the docs describing Limitations of Elastic Scaling. In particular, this part, which explains that yarn is not supported:

Deployment is only supported as a standalone application deployment. Active resource providers (such as native Kubernetes, YARN) are explicitly not supported. Standalone session clusters are not supported either. The application deployment is limited to single job applications.

The only supported deployment options are Standalone in Application Mode (described on this page), Docker in Application Mode and Standalone Kubernetes Application Cluster.