前言

我通過👇復寫了模型的state_dict方法，具體就是給其增加了dynamic_mask（字典類型，里面是tensor），allocated_neuron_num（整型）。

def state_dict(self, destination=None, prefix='', keep_vars=False):state_dict = super().state_dict(destination, prefix, keep_vars)state_dict['model.dynamic_mask'] = gloVar.dynamic_maskstate_dict['model.allocated_neuron_num'] = gloVar.allocated_neuron_numreturn state_dict

結果報錯：

File "/data3/syxu/sparsenmt_exp/sparsenmt/fairseq/fairseq/models/distributed_fairseq_model.py", line 58, in DistributedFairseqModelwrapped_model = DistributedDataParallel(File "/data3/syxu/anaconda3/envs/torch18/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 580, in __init__self._sync_params_and_buffers(authoritative_rank=0)File "/data3/syxu/anaconda3/envs/torch18/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 597, in _sync_params_and_buffersself._distributed_broadcast_coalesced(File "/data3/syxu/anaconda3/envs/torch18/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 1334, in _distributed_broadcast_coalesceddist._broadcast_coalesced( TypeError: _broadcast_coalesced(): incompatible function arguments. The following argument types are supported:1. (process_group: torch._C._distributed_c10d.ProcessGroup, tensors: List[at::Tensor], buffer_size: int, src: int = 0) -> None

解決

不使用報錯中顯示的DistributedDataParallel。根據文檔，這個在fairseq中體現為–ddp-backend參數。
報錯時–ddp-backend=pytorch_ddp（默認），改為legacy_ddp、no_c10d都不會再報錯。

參考

https://fairseq.readthedocs.io/en/latest/command_line_tools.html
https://blog.csdn.net/j___t/article/details/104368597

總結

以上是生活随笔為你收集整理的[fairseq] 报错：TypeError: _broadcast_coalesced(): incompatible function arguments的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。