PyTorch训练GPU占用率低？别被任务管理器骗了，用这招看真实利用率

在Windows上使用PyTorch训练模型时，任务管理器里显示的GPU占用率比较低，这是正常的。因为PyTorch只利用了GPU中的CUDA核心，3D、视频解码等功能没有使用，平均下来，GPU的综合利用率比较低。可以使用nvidia-smi命令行工具查看真实的GPU占用率。

任务管理器里的GPU占用率

使用nvidia-smi查看真实的GPU占用率

命令（dmon表示以实时监控的形式查看）：

nvidia-smi dmon

各列含义：

sm 和 mem 高百分比表明 GPU 计算或显存带宽成为瓶颈。在深度训练时，需要关注 sm 和 mem 是否接近 100%，确保 GPU 满载。

GPU占用率确实低的解决方案

可以在PyTorch DataLoader里设置num_workers和pin_memory参数解决。注意：在windows上设置num_workers后，训练函数必须放到if name == "main"里面，否则会报错。

注意：不要把num_workers设置的太大，创建worker也需要很多时间。可以从2开始加，只要保证使用nvidia-smi查看的GPU利用率在90%以上即可。然后调整batch_size，让显存占用率也在90%左右即可。

示例代码：

import torch
import torch.nn as nn
import torch.optim as optim
import torch.utils.data as data
from torchvision import datasets, transforms

num_epochs = 20
batch_size = 100
learning_rate = 0.001

transform = transforms.Compose(
    [transforms.ToTensor(), transforms.Normalize((0.5,), (0.5,))]
)

train_dataset = datasets.MNIST(
    root="./data", train=True, transform=transform, download=True
)
test_dataset = datasets.MNIST(
    root="./data", train=False, transform=transform, download=True
)

train_loader = data.DataLoader(train_dataset, batch_size=batch_size, shuffle=True, num_workers=2, pin_memory=True)
test_loader = data.DataLoader(test_dataset, batch_size=batch_size, shuffle=False)

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")


class CNNModel(nn.Module):
    def __init__(self):
        super().__init__()
        self.layer1 = nn.Sequential(
            nn.Conv2d(1, 32, kernel_size=5, stride=1, padding=2),
            nn.ReLU(),
            nn.MaxPool2d(kernel_size=2, stride=2),
        )
        self.layer2 = nn.Sequential(
            nn.Conv2d(32, 64, kernel_size=5, stride=1, padding=2),
            nn.ReLU(),
            nn.MaxPool2d(kernel_size=2, stride=2),
        )
        self.dropout = nn.Dropout()
        self.fc1 = nn.Linear(7 * 7 * 64, 1000)
        self.fc2 = nn.Linear(1000, 10)

    def forward(self, x):
        out = self.layer1(x)
        out = self.layer2(out)
        out = out.reshape(out.size(0), -1)
        out = self.dropout(out)
        out = self.fc1(out)
        out = self.fc2(out)
        return out


model = CNNModel().to(device)

criterion = nn.CrossEntropyLoss()

optimizer = optim.Adam(model.parameters(), lr=learning_rate)

def train():
    model.train()
    for epoch in range(num_epochs):
        for i, (images, labels) in enumerate(train_loader):
            images = images.to(device)
            labels = labels.to(device)
    
            output = model(images)
            loss = criterion(output, labels)
    
            optimizer.zero_grad()
            loss.backward()
            optimizer.step()
    
            if (i + 1) % 100 == 0:
                print(
                    "Epoch [{}/{}], Step [{}/{}], Loss: {:.4f}".format(
                        epoch + 1, num_epochs, i + 1, len(train_loader), loss.item()
                    )
                )

def test():
    model.eval()
    with torch.no_grad():
        total = 0
        correct = 0
        for images, labels in test_loader:
            images = images.to(device)
            labels = labels.to(device)
    
            output = model(images)
            _, predict = torch.max(output, 1)
    
            total += labels.size(0)
            correct += (predict == labels).sum().item()
    
        print("Accuracy of 10000 test images: {} %".format(correct / total * 100))

if __name__ == "__main__":
    train()
    test()