current position：Home>Deepmind reinforcement learning boss launched pondernet, a neural network that can "think" like people
Deepmind reinforcement learning boss launched pondernet, a neural network that can "think" like people
2021-08-26 09:01:57 【Xinzhiyuan】
New Zhiyuan Report
【 Introduction to new wisdom 】 Do machines need thinking time ？ When making a neural network model , It may often be overlooked that the machine needs different calculations to solve different difficult problems .DeepMind Recently, a reinforcement learning model was introduced PonderNet, Can adaptively adjust the amount of calculation according to the difficulty of the problem .
When humans answer a question , If the problem is more difficult , Obviously, more time is needed to think .
But in a standard artificial neural network , The amount of computation used increases with the size of the input , It has nothing to do with the complexity of the problem learned .
But usually , The problem also has inherent complexity independent of the input size , For example, adding two numbers is faster than dividing .
Most machine learning algorithms do not adjust the computing budget according to the complexity of the task they are learning to solve , Or we can say , This adjustment is made by AI The creator of the model did it manually .
If this adaptation time works on people , It's called thinking .
Previous work such as adaptive computing time （Adaptive Computation Time, ACT） Automatically learn and estimate the required calculation time through standard probability .
This pause probability （halting probability） Adjust the number of calculation steps required for each input , be called 「 Thinking time 」. but ACT Very unstable , And you need to choose a super parameter very sensitively , Trade off accuracy and computational cost .
To overcome this limitation ,DeepMind A new model is proposed PonderNet, The amount of computation can be adjusted according to the complexity of the input problem .
PonderNet Learn the number of end-to-end calculation steps , To predict accuracy in training 、 Effective tradeoff between computational cost and generalization .
It includes a step function (step function), The outputs are the prediction of the network and in step n The probability of stopping . The step function can also be any neural network , Such as MLP、LSTM Or encoder - Decoder structure of the network , Such as Transformer. Apply this step function repeatedly , most N Time .
in application , Every problem requires a limited thinking step , Therefore, the step function can only be expanded in a finite number of iterations , And this must be normalized , Make the sum of probabilities 1.
It can be done in two ways ：
1、 Normalized probability , Make the sum of 1（ This is equivalent to adjusting the probability of stopping when you know the number of thoughts ）
2、 Assign all remaining pause probabilities to the last thought .
PonderNet The loss function used biases the network towards the expected number of previous steps . secondly , It provides an incentive , Make all possible steps have non-zero probability , Thus, it further promotes the exploration .
On a complex comprehensive problem ,PonderNet Compared with the previous adaptive calculation methods, the performance is greatly improved . As shown in the figure below ,PonderNet The parity check task is better than ACT Higher accuracy , And it makes more effective use of thinking time . Besides , If you consider the total calculation time during training , You can see , And ACT comparison ,PonderNet Take fewer calculations and get higher scores .
Another analysis is to observe the effect of a priori probability on the performance of parity check tasks . You can see PonderNet The only situation where a task cannot be solved is when prior（λp） Set to 0.9 when , That is, the average number of thinking steps is about 1（1/0.9） when .
The interesting phenomenon is , When a priori （λp） Set to 0.1 when , from 10 Step （1/0.1） The a priori average thinking time starts , The network can overcome this defect , And stabilize to about 3 Step is more effective, average thinking time . These results suggest that PonderNet More stable than previous methods , And with ACT There is obvious progress compared with , among τ Parameters are difficult to set , And it is the source of training instability .
Last , One advantage of setting a priori probability is , This parameter can easily be interpreted as “ Thinking steps ” Reciprocal , and ACT In the model τ Parameters have no direct explanation , So it becomes more difficult to define a priori .
In the test PonderNet Allow extrapolation (extrapolation) When . When in 96 When training a network on an input vector of elements , from 1 To 48 Start training with an integer of elements , And then in 49 To 96 Evaluate on integers between . Results show ,PonderNet Can achieve almost perfect accuracy in this extrapolation task , and ACT Keep at a random level .
Besides ,DeepMind The method matches the latest results of real-world question and answer data sets , Less computation is used . Include 20 Task bAbI When experimenting on a question and answer dataset , For standard neural network architecture without adaptive computing , It's hard to train .
PonderNet The model can match the most advanced results , Faster implementation , The average error is lower . And Universal transformerx comparison , It's used with PonderNet same Transformer framework , But use ACT The calculation time is optimized .
To solve 20 A mission ,Universal Transformer need 10161 A step , and PonderNet It only needs 1658, Therefore, it is confirmed that this method is better than ACT Use less computation .
also PonderNet It has achieved the most advanced results on a complex task designed to test the reasoning ability of neural network . In paired associative reasoning tasks （paired associative inference, PAI） I tested it. PonderNet. This task is considered to grasp the essence of reasoning , That is, the understanding of the distance relationship between elements distributed in multiple facts or memories , And it has been proved that it can benefit from the addition of adaptive computing .
PonderNet Able to match MEMO Result , Although this model uses UT The same architecture , But it can achieve higher accuracy .
PonderNet It is used to adapt to the computational complexity of neural networks . It optimizes a new objective function , This function combines the prediction accuracy with a regularization term , The regularization term stimulates exploration in thinking time .
Compared with the past ACT The method should be a progress .
Reference material ：
author[Xinzhiyuan],Please bring the original link to reprint, thank you.
The sidebar is recommended
- The Great Wall is the most aggressive SUV. The official map of tank 600 is released. The momentum is not inferior to that of Toyota Land patrol, with 3.0T power
- This car has the reputation of "Desert Fox"
- Cadillac needs to dismantle the engine 40 days after collecting the car, and the owner is depressed: the loss of dismantling is tens of thousands
- Carola, Chang'an Mazda 3, the ultimate duel of anksila: elegance and perseverance vs dynamic explosion
- The old driver came to teach you how to refit yueku 150
- Lexus es changed its model to market. I heard that the price has increased? Strength or gimmick?
- This year 58, if you want to buy a motorcycle within 200000, don't pedal or raise your ass, please recommend it
- Great Wall Fengjun takes you everywhere. It turns out that pickups are really not only suitable for home use
- Toyota fjcruiser reappears tjcruiser after it is discontinued. You can't miss the car
- The car is nearly 5 meters long, more domineering than the crown. The fuel version is sold from 210000, and Camry's number one rival
guess what you like
FAW doesn't lose BBA for standard replacement. The new car has a handsome appearance and fuel consumption is only 6.3l
Don't go away! Don't miss it! With a monthly salary of 5000, choose it without hesitation! Song Pro vs Rongwei I6
The man spent 20000 yuan to buy the Audi A6L, and then went to make money after it was repaired. Netizen: he is a talented man
How to compare with the new LX? Infiniti qx80 has survived another year
It's a big loss. I don't know how many people who bought Haval H6 will regret it as soon as this car goes on the market
2.3t diesel engine + 7at Nissan's new Tuda was exposed and sold from about 230000 yuan
Balance modification: the influence of angular weight on the vehicle. After the shock absorber is modified, the upper angular weight instrument needs to be adjusted
Self priming, fuel-efficient and durable, why do people shout "don't buy non-T"? Tell you the reason in three aspects
Whether the listing of Citroen c5x will affect Peugeot 508l?
The new Mercedes Benz S-class has a brand-new design style, which makes people bright and full of momentum
- Mysterious black coating, dreamy interior, new Mercedes Maybach s580 real car
- Shock! With an annual salary of 500000, there is no pressure to win the BMW Z4! Now the discount is 65000
- Don't regret buying it! Sure enough, I didn't wait in vain! Monthly salary 8000, Miss regret! Freedom Man vs yuelang
- Car market sales fell in July! Why do these three cars compete with each other?
- What are you waiting for? Don't regret buying it! Owner hematemesis recommended! Baojun 530 vs Ruicheng CC
- Shining and warm: the idea of celebrity theme match, cold and white skin as a necessary element, and old ink incarnates No.1
- Power · vision | watching GAC Mitsubishi Yige, how to conquer Chongqing's "autumn famous mountain"?
- "Change new clothes", the new car is highly praised, and the people can finally raise their heads
- Haval h6s appearance and interior exposure, pre-sale at the end of August
- Land Rover launched a "price war", reducing 500000 in one breath, challenging BMW X8 and Audi Q9
- Quality experience artifact! Don't worry about buying a car! Infiniti Q70 vs Jaguar XFL
- New Lexus es: comfort, performance and fuel consumption are never single choice questions
- Household travel artifact! A monthly salary of 8000 is easy to earn! Qichen T70 vs boyue
- Finally the trump card! Let's go! Don't see regret! Santana vs Baron HS
- The new favorite of female drivers, EULA good cat GT Mulan version, has been pre sold from 138000
- The two blockbuster MPVS to be unveiled at Chengdu auto show are worth a good look!
- The fuel consumption is 6.6l in 8.8 seconds, and the 1.7 ㎡ ultra wide-angle light sensing sky curtain large V SUV is equipped with open blind setting
- Necessary tools for home travel! Starting with a monthly salary of 5000, don't panic! Vision X6 vs Baojun 530
- What if your car is blocked? Traffic police: teach you three moves, and the owner will take the initiative to find you
- The new car hechuang z03 was officially launched for pre-sale, with a pre-sale price of 130000 yuan, which is defined as a compact SUV
- Class B car is "first class", with a wheelbase of nearly 2m 9. It is quiet when the door is closed with imported sound insulation cotton, and the fuel consumption is 7L
- What are you thinking? Don't you start with a monthly salary of 8000? Boyue vs Ruicheng CC
- After 10 years of continuous use, who is more durable than Toyota and Volkswagen?
- Shock! Finally conscience! Really fragrant series! Ruiji vs accord
- The new Roewe rx5 plus has amazing appearance and design!
- The price is 248800. The tank 300 Ranger version is on the market. Buy and pick up the car early!
- Hechuang z03: do you understand the happiness of "lying flat"?
- [how to choose the four Chery tiger 5x superhero cars?]
- In order to save fuel, start with the double engine version Rongfang, and then drive the new car bought by a friend. Owner: I regret it
- Zhao Liying's modeling is rare to overturn. Her green blouse and white pants appear thick waist and short legs
- The interior configuration is upgraded, and the new Infiniti qx80 is officially released!
- Latest news! The new Acura Integra rendering is exposed, and the luxury two door sedan is amazing in the market
- Another luxury car fell down. It was once a man's dream car. Now no one wants it
- Fiat 500 releases 165 pieces of performance version and draws lessons from formula F4 racing technology
- "Anti Mafia storm" borrow 5000 to repay 120000? Sisi wept about the inside of the naked loan. There are big men behind Sun Xing
- The future of electric smart is full of hope. The cooperation between the two brands is very optimistic about the Chinese market
- Is the "public exploration song" of "official fuel consumption 5.8" economical? Measured fuel consumption of the owner
- It is the "king of sexual price" and the "king of control". The strongest plus model in the same level is coming, with 252 horsepower
- Beautiful women's ancient clothes never disappoint people. Zhang Yuxi's ancient clothes are amazing, sweet and SA
- The configuration was upgraded again, and the Buick Regal GS sold 218800