[Cluster] Distribute nodes equally to each batches when priority is the same

[jiji]

Hello dear admin,

I am working on the photoscan cluster project. I made a test in which I tried to launch 2 batches (the same priority) to the server with 2 nodes, and each batches can be cut to 10 small jobs (the waiting nodes are 10 accroding to photoscan network monitor).

So I observed that the 2 nodes were always distributed to the first batch and the second batch was always in status "pending", until that the waiting nodes of the first batch down to 0 and then the second batch began to work ...

I am wondering if it is possible to ajust this nodes distributions mechanism to distribute nodes equally to each batches (if possible) when each batches' priority are the same ?

Thanks in advance
Best regards 

[jiji]

Hello dear admin,

I am working on the photoscan cluster project. I made a test in which I tried to launch 2 batches (the same priority) to the server with 2 nodes, and each batches can be cut to 10 small jobs (the waiting nodes are 10 accroding to photoscan network monitor).

So I observed that the 2 nodes were always distributed to the first batch and the second batch was always in status "pending", until that the waiting nodes of the first batch down to 0 and then the second batch began to work ...

I am wondering if it is possible to ajust this nodes distributions mechanism to distribute nodes equally to each batches (if possible) when each batches' priority are the same ?

Thanks in advance
Best regards

[Alexey Pasumansky]

Hello jiji,

We'll try to implement that in the upcoming updates.

[jiji]

Thanks for the reply,

It is so great !

[Alexey Pasumansky]

Hello jiji,

But is there any reason, why you cannot control the priority manually or via script, for example?

[jiji]

Hello admin,

yes, for now, i upgrade all the batchs' priority to "High" in my script except for "Build dense cloud" step, and I retain it in priority "Normal" because it always takes all the nodes avaibles ...   

and that's the tempory solution that I have. But I still hope that the more reasonable nodes distribution could be implemented in the upcomming version.