分布式训练出错 #405
Description
Duplicate GPU detected : rank 0 and rank 1 both on CUDA device 5e000
[rank0]:[W922 06:11:50.704443870 ProcessGroupNCCL.cpp:1538] Warning: WARNING: destroy_process_group() was not called before program exit, which can leak resources. For more info, please see https://pytorch.org/docs/stable/distributed.html#shutdown (function operator())