Tensor——基本运算

broadcasting

expand、without copying data
key idea

• insert 1 dim ahead
• expand dims with size 1 to same size
• feature maps:[4,32,14,14]
• bias:[32,1,1] =>[1,32,1,1] => [4,32,14,14]

加减乘除

add(+)

>>> import torch
>>> a = torch.rand(3,4)
>>> b = torch.rand(4)
>>> a+b,torch.add(a,b)
(tensor([[1.1636, 1.1454, 1.3575, 1.1242],[1.6967, 1.2132, 0.9738, 1.2507],[0.9603, 1.6887, 1.3814, 0.7453]]), 
tensor([[1.1636, 1.1454, 1.3575, 1.1242],[1.6967, 1.2132, 0.9738, 1.2507],[0.9603, 1.6887, 1.3814, 0.7453]]))

可见使用“+”和使用add函数结果是相同的。也可以验证一下：

>>> import torch
>>> a = torch.rand(3,4)
>>> b = torch.rand(4)
>>> torch.all(torch.eq(a+b,torch.add(a,b)))
tensor(True)

sub(-)
mul(*)
div(/)

矩阵运算

torch.mm( )

只能用于二维矩阵的运算。

>>> import torch
>>> a = torch.full([2,2],3)
>>> b = torch.ones(2,2)
>>> torch.mm(a,b)
tensor([[6., 6.],[6., 6.]])

torch.matmul( )/@

可以用于二维矩阵计算，也可以是多维。

二维：

>>> import torch
>>> a = torch.full([2,2],3)
>>> b = torch.ones(2,2)
>>> torch.matmul(a,b)
tensor([[6., 6.],[6., 6.]])

>>> import torch
>>> a = torch.full([2,2],3)
>>> b = torch.ones(2,2)
>>> a@b
tensor([[6., 6.],[6., 6.]])

降维度

>>> import torch
>>> a = torch.rand(4,784)
>>> x = torch.rand(4,784)
>>> w = torch.rand(512,784)
>>> (x@w.t()).shape
torch.Size([4, 512])

>二维

比如四维，计算的时候就是前两维不变，后两维进行计算。

 >>> import torch
>>> a = torch.rand(4,3,28,64)
>>> b = torch.rand(4,3,64,32)
>>> torch.matmul(a,b).shape
torch.Size([4, 3, 28, 32])

broadcast
通道数目不同时，符合broadcast情况：

>>> import torch
>>> a = torch.rand(4,3,28,64)
>>> b = torch.rand(4,1,64,32)
>>> torch.matmul(a,b).shape
torch.Size([4, 3, 28, 32])

维度不同、通道数目也不等时。不符合broadcast时会报错：

>>> import torch
>>> a = torch.rand(4,3,28,64)
>>> b = torch.rand(4,64,32)
>>> torch.matmul(a,b).shape
Traceback (most recent call last):File "<stdin>", line 1, in <module>
RuntimeError: The size of tensor a (3) must match the size of tensor b (4) at non-singleton dimension 1

pow

两种方法：
①任意次幂

>>> import torch
>>> a = torch.full([2,2],6)
>>> a.pow(3)
tensor([[216., 216.],[216., 216.]])

②仅限平方

>>> import torch
>>> a = torch.full([2,2],6)
>>> a**2
tensor([[36., 36.],[36., 36.]])

sqrt/sqrt

平方根：

>>> import torch
>>> a = torch.full([2,2],1024)
>>> a.sqrt(),a.rsqrt, a**0.5
(tensor([[32., 32.],[32., 32.]]),tensor([[32., 32.],[32., 32.]]))
>>> 

平方根的倒数：

>>> import torch
>>> a = torch.full([2,2],1024)
>>> a.rsqrt()
( tensor([[0.0312, 0.0312],[0.0312, 0.0312]]))

exp/log

我变懒了因为我困了！！三种：exp、log、log2写在一起吧：

>>> import torch
>>> a = torch.exp(torch.ones(2,2))
>>> a,torch.log(a), torch.log2(a)
(tensor([[2.7183, 2.7183],[2.7183, 2.7183]]), tensor([[1., 1.],[1., 1.]]), tensor([[1.4427, 1.4427],[1.4427, 1.4427]]))

近似值

• .floor()——往下近似
• .ceil()——往上近似
• .trunc()——裁剪为整数部分
• .frac()——裁剪成小数部分

>>> import torch
>>> a = torch.tensor(3.1415926)
>>> a.floor(), a.ceil(), a.trunc(), a.frac()
(tensor(3.), tensor(4.), tensor(3.), tensor(0.1416))

• torch.round()——四舍五入

>>> import torch
>>> a = torch.tensor(3.499)
>>> b = torch.tensor(4.501)
>>> a.round(), b.round()
(tensor(3.), tensor(5.))

梯度裁剪

最大、最小值：

>>> import torch
>>> grad = torch.rand(2,3)*15
>>> grad.max(), grad.min()
(tensor(12.5018), tensor(4.8845))

.clamp( )

>>> import torch
>>> grad = torch.rand(2,3)*15
>>> grad, grad.clamp(10),grad.clamp(5,10)
(tensor([[11.7561,  0.7950,  7.1729],[11.2285,  5.1940, 14.5741]]), 
tensor([[11.7561, 10.0000, 10.0000],[11.2285, 10.0000, 14.5741]]),
tensor([[10.0000,  5.0000,  7.1729],[10.0000,  5.1940, 10.0000]]))

没睡午觉真的太困了，最后这点代码就放在一起打印吧。