Torch way to explore https:https://cdn.v2ex.com/navatar/2bca/b9d9/995_normal.png?m=1582782020 https:https://cdn.v2ex.com/navatar/2bca/b9d9/995_large.png?m=1582782020 2024-08-26T02:17:50Z Copyright © 2010-2018, V2EX Torch 凉了吗?这个主题怎么这么久没有新节点了 tag:www.v2ex.com,2024-08-26:/t/1067756 2024-08-26T02:20:50Z 2024-08-26T02:17:50Z RYS member/RYS 写了一个简单的二分类模型,但是训练了 N 次模型参数都没有动静 tag:www.v2ex.com,2023-10-31:/t/987091 2023-10-31T06:20:25Z 2023-10-31T09:20:37Z xing393939 member/xing393939 训练代码如下:

import numpy as np import torch # 1.prepare dataset xy = np.loadtxt("redPacket_2.csv", skiprows=1, delimiter=",", dtype=np.float32) x_data = torch.from_numpy(xy[:, :-1]) y_data = torch.from_numpy(xy[:, [-1]]) # 2.design model using class class Model(torch.nn.Module): def __init__(self): super(Model, self).__init__() self.linear1 = torch.nn.Linear(4, 2) self.linear2 = torch.nn.Linear(2, 1) self.activate = torch.nn.ReLU() self.sigmoid = torch.nn.Sigmoid() def forward(self, x): x = self.activate(self.linear1(x)) x = self.sigmoid(self.linear2(x)) return x device = torch.device("cuda" if torch.cuda.is_available() else "cpu") model = Model().to(device) x_data = x_data.to(device) y_data = y_data.to(device) # 3.construct loss and optimizer criterion = torch.nn.BCELoss(reduction="mean") optimizer = torch.optim.SGD(model.parameters(), lr=0.01) # 4.training cycle forward, backward, update for epoch in range(10000): y_pred = model(x_data) loss = criterion(y_pred, y_data) if epoch % 100 == 0: print( "epoch %9d loss %.3f" % (epoch, loss.item()), model.linear2.weight.data, model.linear2.bias.data, ) optimizer.zero_grad() loss.backward() optimizer.step() 

训练集下载在这里,我每隔 100 周期打印模型的损失值和模型参数,结果如下:

epoch 0 loss 50.000 tensor([[0.0944, 0.1484]], device='cuda:0') tensor([0.4391], device='cuda:0') epoch 100 loss 50.000 tensor([[0.0944, 0.1484]], device='cuda:0') tensor([0.4391], device='cuda:0') epoch 200 loss 50.000 tensor([[0.0944, 0.1484]], device='cuda:0') tensor([0.4391], device='cuda:0') epoch 300 loss 50.000 tensor([[0.0944, 0.1484]], device='cuda:0') tensor([0.4391], device='cuda:0') epoch 400 loss 50.000 tensor([[0.0944, 0.1484]], device='cuda:0') tensor([0.4391], device='cuda:0') epoch 500 loss 50.000 tensor([[0.0944, 0.1484]], device='cuda:0') tensor([0.4391], device='cuda:0') epoch 600 loss 50.000 tensor([[0.0944, 0.1484]], device='cuda:0') tensor([0.4391], device='cuda:0') epoch 700 loss 50.000 tensor([[0.0944, 0.1484]], device='cuda:0') tensor([0.4391], device='cuda:0') epoch 800 loss 50.000 tensor([[0.0944, 0.1484]], device='cuda:0') tensor([0.4391], device='cuda:0') epoch 900 loss 50.000 tensor([[0.0944, 0.1484]], device='cuda:0') tensor([0.4391], device='cuda:0') epoch 1000 loss 50.000 tensor([[0.0944, 0.1484]], device='cuda:0') tensor([0.4391], device='cuda:0') epoch 1100 loss 50.000 tensor([[0.0944, 0.1484]], device='cuda:0') tensor([0.4391], device='cuda:0') epoch 1200 loss 50.000 tensor([[0.0944, 0.1484]], device='cuda:0') tensor([0.4391], device='cuda:0') epoch 1300 loss 50.000 tensor([[0.0944, 0.1484]], device='cuda:0') tensor([0.4391], device='cuda:0') 

不知道为什么会不收敛,是哪里需要改进吗?

]]> [干货] 史上最全的 PyTorch 学习资源汇总 import torch as tf tag:www.v2ex.com,2019-05-07:/t/561876 2019-05-07T08:59:52Z 2020-03-01T23:08:22Z 1722332572 member/1722332572 PyTorch 学习教程、手册

PyTorch 视频教程

NLP&PyTorch 实战

CV&PyTorch 实战

PyTorch 论文推荐

PyTorch 书籍推荐

相较于目前 Tensorflow 类型的书籍已经烂大街的状况,PyTorch 类的书籍目前已出版的并没有那么多,笔者给大家推荐我认为还不错的四本 PyTorch 书籍。

欢迎 Star Fork : https://github.com/INTERMT/Awesome-PyTorch-Chinese

]]>
PyTorch 60 分钟安装入门教程 tag:www.v2ex.com,2018-12-26:/t/521234 2018-12-26T08:54:19Z 2018-12-26T08:51:19Z 1722332572 member/1722332572 http://pytorchchina.com/2018/06/25/what-is-pytorch/
PyTorch 60 分钟入门教程:自动微分
http://pytorchchina.com/2018/12/25/autograd-automatic-differentiation/
PyTorch 60 分钟入门教程:神经网络
http://pytorchchina.com/2018/12/25/neural-networks/
PyTorch 60 分钟入门教程:PyTorch 训练分类器
http://pytorchchina.com/2018/12/25/training-a-classifier/
PyTorch 60 分钟入门教程:数据并行处理
http://pytorchchina.com/2018/12/11/optional-data-parallelism/
PyTorch 60 分钟安装入门教程
http://pytorchchina.com ]]>
PyTorch 安装教程 tag:www.v2ex.com,2018-12-11:/t/516574 2018-12-11T08:55:40Z 2018-12-11T08:52:40Z 1722332572 member/1722332572 http://pytorchchina.com/2018/12/11/pytorch-windows-install-1/
PyTorch Mac 安装教程
http://pytorchchina.com/2018/12/11/pytorch-mac-install/
PyTorch Linux 安装教程
http://pytorchchina.com/2018/12/11/pytorch-linux-install/ ]]>
PyTorch 60 分钟入门教程 tag:www.v2ex.com,2018-12-11:/t/516422 2018-12-11T03:26:21Z 2018-12-11T03:23:21Z 1722332572 member/1722332572 PyTorch 是一个基于 Python 的科学计算包,主要定位两类人群:

NumPy 的替代品,可以利用 GPU 的性能进行计算。
深度学习研究平台拥有足够的灵活性和速度
开始学习
Tensors (张量)
Tensors 类似于 NumPy 的 ndarrays,同时 Tensors 可以使用 GPU 进行计算。

from __future__ import print_function
import torch
构造一个 5×3 矩阵,不初始化。

x = torch.empty(5, 3)
print(x)
输出:

tensor(1.00000e-04 *
[[-0.0000, 0.0000, 1.5135],
[ 0.0000, 0.0000, 0.0000],
[ 0.0000, 0.0000, 0.0000],
[ 0.0000, 0.0000, 0.0000],
[ 0.0000, 0.0000, 0.0000]])


构造一个随机初始化的矩阵:

x = torch.rand(5, 3)
print(x)
输出:

tensor([[ 0.6291, 0.2581, 0.6414],
[ 0.9739, 0.8243, 0.2276],
[ 0.4184, 0.1815, 0.5131],
[ 0.5533, 0.5440, 0.0718],
[ 0.2908, 0.1850, 0.5297]])


构造一个矩阵全为 0,而且数据类型是 long.

Construct a matrix filled zeros and of dtype long:

x = torch.zeros(5, 3, dtype=torch.long)
print(x)
输出:

tensor([[ 0, 0, 0],
[ 0, 0, 0],
[ 0, 0, 0],
[ 0, 0, 0],
[ 0, 0, 0]])
构造一个张量,直接使用数据:

x = torch.tensor([5.5, 3])
print(x)
输出:

tensor([ 5.5000, 3.0000])
创建一个 tensor 基于已经存在的 tensor。

x = x.new_ones(5, 3, dtype=torch.double)
# new_* methods take in sizes
print(x)

x = torch.randn_like(x, dtype=torch.float)
# override dtype!
print(x)
# result has the same size
输出:

tensor([[ 1., 1., 1.],
[ 1., 1., 1.],
[ 1., 1., 1.],
[ 1., 1., 1.],
[ 1., 1., 1.]], dtype=torch.float64)
tensor([[-0.2183, 0.4477, -0.4053],
[ 1.7353, -0.0048, 1.2177],
[-1.1111, 1.0878, 0.9722],
[-0.7771, -0.2174, 0.0412],
[-2.1750, 1.3609, -0.3322]])
获取它的维度信息:

print(x.size())
输出:

torch.Size([5, 3])
注意

torch.Size 是一个元组,所以它支持左右的元组操作。

操作
在接下来的例子中,我们将会看到加法操作。

加法: 方式 1

y = torch.rand(5, 3)
print(x + y)
Out:

tensor([[-0.1859, 1.3970, 0.5236],
[ 2.3854, 0.0707, 2.1970],
[-0.3587, 1.2359, 1.8951],
[-0.1189, -0.1376, 0.4647],
[-1.8968, 2.0164, 0.1092]])
加法: 方式 2

print(torch.add(x, y))
Out:

tensor([[-0.1859, 1.3970, 0.5236],
[ 2.3854, 0.0707, 2.1970],
[-0.3587, 1.2359, 1.8951],
[-0.1189, -0.1376, 0.4647],
[-1.8968, 2.0164, 0.1092]])
加法: 提供一个输出 tensor 作为参数

result = torch.empty(5, 3)
torch.add(x, y, out=result)
print(result)
Out:

tensor([[-0.1859, 1.3970, 0.5236],
[ 2.3854, 0.0707, 2.1970],
[-0.3587, 1.2359, 1.8951],
[-0.1189, -0.1376, 0.4647],
[-1.8968, 2.0164, 0.1092]])
加法: in-place

# adds x to y
y.add_(x)
print(y)
Out:

tensor([[-0.1859, 1.3970, 0.5236],
[ 2.3854, 0.0707, 2.1970],
[-0.3587, 1.2359, 1.8951],
[-0.1189, -0.1376, 0.4647],
[-1.8968, 2.0164, 0.1092]])
Note

注意

任何使张量会发生变化的操作都有一个前缀 ‘_’。例如:x.copy_(y), x.t_(), 将会改变 x.

你可以使用标准的 NumPy 类似的索引操作

print(x[:, 1])
Out:

tensor([ 0.4477, -0.0048, 1.0878, -0.2174, 1.3609])
改变大小:如果你想改变一个 tensor 的大小或者形状,你可以使用 torch.view:

x = torch.randn(4, 4)
y = x.view(16)
z = x.view(-1, 8) # the size -1 is inferred from other dimensions
print(x.size(), y.size(), z.size())
Out:

torch.Size([4, 4]) torch.Size([16]) torch.Size([2, 8])
如果你有一个元素 tensor,使用 .item() 来获得这个 value。

x = torch.randn(1)
print(x)
print(x.item())
Out:

tensor([ 0.9422])
0.9422121644020081

PyTorch 入门教程: http://pytorchchina.com/2018/06/25/what-is-pytorch/ ]]>
PyTorch 60 分钟入门教程:数据并行处理 tag:www.v2ex.com,2018-12-11:/t/516416 2018-12-11T03:15:52Z 2018-12-11T03:12:52Z 1722332572 member/1722332572 作者:Sung Kim 和 Jenny Kang

在这个教程中,我们将学习如何用 DataParallel 来使用多 GPU。
通过 PyTorch 使用多个 GPU 非常简单。你可以将模型放在一个 GPU:

device = torch.device("cuda:0")
model.to(device)
然后,你可以复制所有的张量到 GPU:

mytensor = my_tensor.to(device)
请注意,只是调用 my_tensor.to(device) 返回一个 my_tensor 新的复制在 GPU 上,而不是重写 my_tensor。你需要分配给他一个新的张量并且在 GPU 上使用这个张量。

在多 GPU 中执行前馈,后馈操作是非常自然的。尽管如此,PyTorch 默认只会使用一个 GPU。通过使用 DataParallel 让你的模型并行运行,你可以很容易的在多 GPU 上运行你的操作。

model = nn.DataParallel(model)
这是整个教程的核心,我们接下来将会详细讲解。
引用和参数

引入 PyTorch 模块和定义参数

import torch
import torch.nn as nn
from torch.utils.data import Dataset, DataLoader
# 参数

input_size = 5
output_size = 2

batch_size = 30
data_size = 100
设备

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
实验(玩具)数据

生成一个玩具数据。你只需要实现 getitem.

class RandomDataset(Dataset):

def __init__(self, size, length):
self.len = length
self.data = torch.randn(length, size)

def __getitem__(self, index):
return self.data[index]

def __len__(self):
return self.len

rand_loader = DataLoader(dataset=RandomDataset(input_size, data_size),batch_size=batch_size, shuffle=True)
简单模型

为了做一个小 demo,我们的模型只是获得一个输入,执行一个线性操作,然后给一个输出。尽管如此,你可以使用 DataParallel 在任何模型(CNN, RNN, Capsule Net 等等.)

我们放置了一个输出声明在模型中来检测输出和输入张量的大小。请注意在 batch rank 0 中的输出。

class Model(nn.Module):
# Our model

def __init__(self, input_size, output_size):
super(Model, self).__init__()
self.fc = nn.Linear(input_size, output_size)

def forward(self, input):
output = self.fc(input)
print("\tIn Model: input size", input.size(),
"output size", output.size())

return output


创建模型并且数据并行处理

这是整个教程的核心。首先我们需要一个模型的实例,然后验证我们是否有多个 GPU。如果我们有多个 GPU,我们可以用 nn.DataParallel 来 包裹 我们的模型。然后我们使用 model.to(device) 把模型放到多 GPU 中。



model = Model(input_size, output_size)
if torch.cuda.device_count() > 1:
print("Let's use", torch.cuda.device_count(), "GPUs!")
# dim = 0 [30, xxx] -> [10, ...], [10, ...], [10, ...] on 3 GPUs
model = nn.DataParallel(model)

model.to(device)
输出:

Let's use 2 GPUs!
运行模型:
现在我们可以看到输入和输出张量的大小了。
for data in rand_loader:
input = data.to(device)
output = model(input)
print("Outside: input size", input.size(),
"output_size", output.size())
输出:

In Model: input size torch.Size([15, 5]) output size torch.Size([15, 2])
In Model: input size torch.Size([15, 5]) output size torch.Size([15, 2])
Outside: input size torch.Size([30, 5]) output_size torch.Size([30, 2])
In Model: input size torch.Size([15, 5]) output size torch.Size([15, 2])
In Model: input size torch.Size([15, 5]) output size torch.Size([15, 2])
Outside: input size torch.Size([30, 5]) output_size torch.Size([30, 2])
In Model: input size torch.Size([15, 5]) output size torch.Size([15, 2])
In Model: input size torch.Size([15, 5]) output size torch.Size([15, 2])
Outside: input size torch.Size([30, 5]) output_size torch.Size([30, 2])
In Model: input size torch.Size([5, 5]) output size torch.Size([5, 2])
In Model: input size torch.Size([5, 5]) output size torch.Size([5, 2])
Outside: input size torch.Size([10, 5]) output_size torch.Size([10, 2])
结果:

如果你没有 GPU 或者只有一个 GPU,当我们获取 30 个输入和 30 个输出,模型将期望获得 30 个输入和 30 个输出。但是如果你有多个 GPU,你会获得这样的结果。

多 GPU

如果你有 2 个 GPU,你会看到:

# on 2 GPUs
Let's use 2 GPUs!
In Model: input size torch.Size([15, 5]) output size torch.Size([15, 2])
In Model: input size torch.Size([15, 5]) output size torch.Size([15, 2])
Outside: input size torch.Size([30, 5]) output_size torch.Size([30, 2])
In Model: input size torch.Size([15, 5]) output size torch.Size([15, 2])
In Model: input size torch.Size([15, 5]) output size torch.Size([15, 2])
Outside: input size torch.Size([30, 5]) output_size torch.Size([30, 2])
In Model: input size torch.Size([15, 5]) output size torch.Size([15, 2])
In Model: input size torch.Size([15, 5]) output size torch.Size([15, 2])
Outside: input size torch.Size([30, 5]) output_size torch.Size([30, 2])
In Model: input size torch.Size([5, 5]) output size torch.Size([5, 2])
In Model: input size torch.Size([5, 5]) output size torch.Size([5, 2])
Outside: input size torch.Size([10, 5]) output_size torch.Size([10, 2])


如果你有 3 个 GPU,你会看到:

Let's use 3 GPUs!
In Model: input size torch.Size([10, 5]) output size torch.Size([10, 2])
In Model: input size torch.Size([10, 5]) output size torch.Size([10, 2])
In Model: input size torch.Size([10, 5]) output size torch.Size([10, 2])
Outside: input size torch.Size([30, 5]) output_size torch.Size([30, 2])
In Model: input size torch.Size([10, 5]) output size torch.Size([10, 2])
In Model: input size torch.Size([10, 5]) output size torch.Size([10, 2])
In Model: input size torch.Size([10, 5]) output size torch.Size([10, 2])
Outside: input size torch.Size([30, 5]) output_size torch.Size([30, 2])
In Model: input size torch.Size([10, 5]) output size torch.Size([10, 2])
In Model: input size torch.Size([10, 5]) output size torch.Size([10, 2])
In Model: input size torch.Size([10, 5]) output size torch.Size([10, 2])
Outside: input size torch.Size([30, 5]) output_size torch.Size([30, 2])
In Model: input size torch.Size([4, 5]) output size torch.Size([4, 2])
In Model: input size torch.Size([4, 5]) output size torch.Size([4, 2])
In Model: input size torch.Size([2, 5]) output size torch.Size([2, 2])
Outside: input size torch.Size([10, 5]) output_size torch.Size([10, 2])
如果你有 8 个 GPU,你会看到:

Let's use 8 GPUs!
In Model: input size torch.Size([4, 5]) output size torch.Size([4, 2])
In Model: input size torch.Size([4, 5]) output size torch.Size([4, 2])
In Model: input size torch.Size([2, 5]) output size torch.Size([2, 2])
In Model: input size torch.Size([4, 5]) output size torch.Size([4, 2])
In Model: input size torch.Size([4, 5]) output size torch.Size([4, 2])
In Model: input size torch.Size([4, 5]) output size torch.Size([4, 2])
In Model: input size torch.Size([4, 5]) output size torch.Size([4, 2])
In Model: input size torch.Size([4, 5]) output size torch.Size([4, 2])
Outside: input size torch.Size([30, 5]) output_size torch.Size([30, 2])
In Model: input size torch.Size([4, 5]) output size torch.Size([4, 2])
In Model: input size torch.Size([4, 5]) output size torch.Size([4, 2])
In Model: input size torch.Size([4, 5]) output size torch.Size([4, 2])
In Model: input size torch.Size([4, 5]) output size torch.Size([4, 2])
In Model: input size torch.Size([4, 5]) output size torch.Size([4, 2])
In Model: input size torch.Size([4, 5]) output size torch.Size([4, 2])
In Model: input size torch.Size([2, 5]) output size torch.Size([2, 2])
In Model: input size torch.Size([4, 5]) output size torch.Size([4, 2])
Outside: input size torch.Size([30, 5]) output_size torch.Size([30, 2])
In Model: input size torch.Size([4, 5]) output size torch.Size([4, 2])
In Model: input size torch.Size([4, 5]) output size torch.Size([4, 2])
In Model: input size torch.Size([4, 5]) output size torch.Size([4, 2])
In Model: input size torch.Size([4, 5]) output size torch.Size([4, 2])
In Model: input size torch.Size([4, 5]) output size torch.Size([4, 2])
In Model: input size torch.Size([4, 5]) output size torch.Size([4, 2])
In Model: input size torch.Size([4, 5]) output size torch.Size([4, 2])
In Model: input size torch.Size([2, 5]) output size torch.Size([2, 2])
Outside: input size torch.Size([30, 5]) output_size torch.Size([30, 2])
In Model: input size torch.Size([2, 5]) output size torch.Size([2, 2])
In Model: input size torch.Size([2, 5]) output size torch.Size([2, 2])
In Model: input size torch.Size([2, 5]) output size torch.Size([2, 2])
In Model: input size torch.Size([2, 5]) output size torch.Size([2, 2])
In Model: input size torch.Size([2, 5]) output size torch.Size([2, 2])
Outside: input size torch.Size([10, 5]) output_size torch.Size([10, 2])
总结
数据并行自动拆分了你的数据并且将任务单发送到多个 GPU 上。当每一个模型都完成自己的任务之后,DataParallel 收集并且合并这些结果,然后再返回给你。

更多信息,请访问:
https://pytorch.org/tutorials/beginner/former_torchies/parallelism_tutorial.html

下载完整 Python 代码: http://pytorchchina.com/2018/12/11/optional-data-parallelism/ ]]>
ubao msn snddm index pchome yahoo rakuten mypaper meadowduck bidyahoo youbao zxmzxm asda bnvcg cvbfg dfscv mmhjk xxddc yybgb zznbn ccubao uaitu acv GXCV ET GDG YH FG BCVB FJFH CBRE CBC GDG ET54 WRWR RWER WREW WRWER RWER SDG EW SF DSFSF fbbs ubao fhd dfg ewr dg df ewwr ewwr et ruyut utut dfg fgd gdfgt etg dfgt dfgd ert4 gd fgg wr 235 wer3 we vsdf sdf gdf ert xcv sdf rwer hfd dfg cvb rwf afb dfh jgh bmn lgh rty gfds cxv xcv xcs vdas fdf fgd cv sdf tert sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf sdf shasha9178 shasha9178 shasha9178 shasha9178 shasha9178 liflif2 liflif2 liflif2 liflif2 liflif2 liblib3 liblib3 liblib3 liblib3 liblib3 zhazha444 zhazha444 zhazha444 zhazha444 zhazha444 dende5 dende denden denden2 denden21 fenfen9 fenf619 fen619 fenfe9 fe619 sdf sdf sdf sdf sdf zhazh90 zhazh0 zhaa50 zha90 zh590 zho zhoz zhozh zhozho zhozho2 lislis lls95 lili95 lils5 liss9 sdf0ty987 sdft876 sdft9876 sdf09876 sd0t9876 sdf0ty98 sdf0976 sdf0ty986 sdf0ty96 sdf0t76 sdf0876 df0ty98 sf0t876 sd0ty76 sdy76 sdf76 sdf0t76 sdf0ty9 sdf0ty98 sdf0ty987 sdf0ty98 sdf6676 sdf876 sd876 sd876 sdf6 sdf6 sdf9876 sdf0t sdf06 sdf0ty9776 sdf0ty9776 sdf0ty76 sdf8876 sdf0t sd6 sdf06 s688876 sd688 sdf86