Protobuf

Parameters

Parameters are Tensor subclasses, that have a very special property when used with Module s - when they’re assigned as Module attributes they are automatically added to the list of its parameters, and will appear e.g. in parameters() iterator.

简单来讲，nn.Parameters用于注册可供训练的参数。e.g:

1
2
3
4
5
6


In [6]: a = torch.nn.Parameter(torch.Tensor([0]))

In [7]: a
Out[7]: 
Parameter containing:
tensor([0.], requires_grad=True)

nn.Parameter默认requires_grad=True

一些常用参量（用于更新参数等）：

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40


>> class Net(torch.nn.Module):
    ...:     def __init__(self):
    ...:         super().__init__()
    ...:         self.linear_1 = torch.nn.Linear(3,2)
    ...:         self.linear_2 = torch.nn.Linear(2,1)
>> net = Net()
>> net.modules
>> <bound method Module.modules of Net(
  (linear_1): Linear(in_features=3, out_features=2, bias=True)
  (linear_2): Linear(in_features=2, out_features=1, bias=True)
)>
>> net.named_modules
>> <bound method Module.named_modules of Net(
  (linear_1): Linear(in_features=3, out_features=2, bias=True)
  (linear_2): Linear(in_features=2, out_features=1, bias=True)
)>
>> net.state_dict()
>> OrderedDict([('linear_1.weight',
              tensor([[-0.5300, -0.5476,  0.0943],
                      [-0.2907,  0.1068,  0.5465]])),
             ('linear_1.bias', tensor([ 0.3375, -0.4314])),
             ('linear_2.weight', tensor([[ 0.0114, -0.2185]])),
             ('linear_2.bias', tensor([0.1354]))])
>> net.parameters
>> <bound method Module.parameters of Net(
  (linear_1): Linear(in_features=3, out_features=2, bias=True)
  (linear_2): Linear(in_features=2, out_features=1, bias=True)
)>
>> net.lambda1 = torch.nn.Parameter(torch.randn((1,1)))
>> net.lambda1
>> Parameter containing:
tensor([[-0.0876]], requires_grad=True)
>> net.state_dict()
>> OrderedDict([('lambda1', tensor([[-0.0876]])),
             ('linear_1.weight',
              tensor([[-0.5300, -0.5476,  0.0943],
                      [-0.2907,  0.1068,  0.5465]])),
             ('linear_1.bias', tensor([ 0.3375, -0.4314])),
             ('linear_2.weight', tensor([[ 0.0114, -0.2185]])),
             ('linear_2.bias', tensor([0.1354]))])

optimizer

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11


>> optimizer = torch.optim.Adam(net.parameters(), lr=0.01)
>> optimizer.state_dict()
>> {'state': {},
 'param_groups': [{'lr': 0.01,
   'betas': (0.9, 0.999),
   'eps': 1e-08,
   'weight_decay': 0,
   'amsgrad': False,
   'params': [0, 1, 2, 3, 4]}]}
>> optimizer.state
>> defaultdict(dict, {})