ai人工智能的本質(zhì)和未來(lái)

The future looks towards running deep learning algorithms on more compact devices as any improvements in this space make for big leaps in the usability of AI.

未來(lái)的趨勢(shì)是在更緊湊的設(shè)備上運(yùn)行深度學(xué)習(xí)算法，因?yàn)樵擃I(lǐng)域的任何改進(jìn)都將使AI的可用性取得重大飛躍。

If a Raspberry Pi could run large neural networks, then artificial intelligence could be deployed in a lot more places.

如果Raspberry Pi可以運(yùn)行大型神經(jīng)網(wǎng)絡(luò)，那么人工智能可以部署在更多地方。

Recent research in the field of economising AI has led to a surprisingly easy solution to reduce the size of large neural networks. It’s so simple, it could fit in a tweet:

在節(jié)省AI領(lǐng)域中的最新研究已導(dǎo)致出乎意料的簡(jiǎn)單解決方案，以減小大型神經(jīng)網(wǎng)絡(luò)的大小。 它非常簡(jiǎn)單，可以在一條推文中顯示 ：

Train the Neural Network to Completion

訓(xùn)練神經(jīng)網(wǎng)絡(luò)完成

Globally prune the 20% of weights with the lowest magnitudes.

在全球范圍內(nèi)修剪最低重量的20％重量。

Retrain with learning rate rewinding for the original training time.

以原始訓(xùn)練時(shí)間的學(xué)習(xí)率倒帶進(jìn)行再訓(xùn)練。

Iteratively repeat steps 2 and 3 until the desired sparsity is reached.

反復(fù)重復(fù)步驟2和3，直到達(dá)到所需的稀疏度。

Further, if you keep repeating this procedure, you can get the model as tiny as you want. However, it’s pretty certain that you’ll lose some model accuracy along the way.

此外，如果繼續(xù)重復(fù)此過(guò)程，則可以根據(jù)需要獲得最小的模型。 但是，可以肯定的是，您將在此過(guò)程中損失一些模型精度。

This line of research grew out of the an ICLR paper last year (Frankle and Carbin’s Lottery Ticket Hypothesis) which showed that a DNN could perform with only 1/10th of the number of connections if the right subnetwork was found in training.

這項(xiàng)研究源于去年的ICLR論文(Frankle和Carbin的彩票假設(shè) )，該論文表明，如果在訓(xùn)練中找到正確的子網(wǎng)，則DNN只能執(zhí)行連接數(shù)量的1/10的操作。

The timing of this finding coincides well with reaching new limitations in computational requirements. Yes, you can send a model to train on the cloud but for seriously big networks, along with considerations of training time, infrastructure and energy usage — more efficient methods are desired because they’re just easier to handle and manage.

這一發(fā)現(xiàn)的時(shí)機(jī)恰好與在計(jì)算要求上達(dá)到新的限制相吻合。 是的，您可以發(fā)送模型在云上進(jìn)行訓(xùn)練，但對(duì)于大型網(wǎng)絡(luò)，需要考慮訓(xùn)練時(shí)間，基礎(chǔ)架構(gòu)和能源使用情況，因此需要更高效的方法，因?yàn)樗鼈兏子诓僮骱凸芾怼?

Bigger AI models are more difficult to train and to use, so smaller models are preferred.

較大的AI模型更難訓(xùn)練和使用，因此較小的模型是首選。

Following this desire for compression, pruning algorithms came back into the picture following the success of the ImageNet competition. Higher performing models were getting bigger and bigger but many researchers proposed techniques try keep them smaller.

隨著對(duì)壓縮的渴望，隨著ImageNet競(jìng)賽的成功，修剪算法重新出現(xiàn) 。 性能更高的模型變得越來(lái)越大，但是許多研究人員提出了一些技術(shù)，試圖將它們縮小。

Yuhan Du on 玉函杜上UnsplashUnsplash

Song Han of MIT, developed a pruning algorithm for neural networks called AMC (AutoML for model compression) which removed redundant neurons and connections, when then the model is retrained to retain its initial accuracy level. Frankle took this method and developed it further by rewinding the pruned model to its initial weights and retrained it at a faster initial rate. Finally, in the ICLR study above, the researchers found that the model could be rewound to its early training rate and without playing with any parameters or weights.

麻省理工學(xué)院的宋瀚 ( Song Han)開(kāi)發(fā)了一種稱為AMC( 用于模型壓縮的AutoML )的神經(jīng)網(wǎng)絡(luò)修剪算法，該算法刪除了多余的神經(jīng)元和連接，然后對(duì)其進(jìn)行了重新訓(xùn)練以保持其初始精度水平。 Frankle采用了這種方法，并通過(guò)將修剪后的模型重繞到其初始權(quán)重并以更快的初始速率對(duì)其進(jìn)行了重新訓(xùn)練來(lái)進(jìn)一步開(kāi)發(fā)了該方法。 最后，在上述ICLR研究中，研究人員發(fā)現(xiàn)該模型可以倒退至其早期訓(xùn)練速度，而無(wú)需使用任何參數(shù)或權(quán)重。

Generally as the model gets smaller, the accuracy gets worse however this proposed model performs better than both Han’s AMC and Frankle’s rewinding method.

通常，隨著模型變小，精度會(huì)變差，但是此提議的模型的性能優(yōu)于Han的AMC和Frankle的倒帶方法。

Now it’s unclear why this model works as well as it does, but the simplicity of it is easy to implement and also doesn’t require time-consuming tuning. Frankle says: “It’s clear, generic, and drop-dead simple.”

現(xiàn)在還不清楚為什么該模型能夠像它一樣運(yùn)作良好，但是它的簡(jiǎn)單性易于實(shí)現(xiàn)，并且不需要費(fèi)時(shí)的調(diào)整。 弗蘭克(Frankle)說(shuō)：“這很清楚，通用并且很簡(jiǎn)單。”

Model compression and the concept of economising machine learning algorithms is an important field that we can make further gains in. Leaving models too large reduces the applicability and usability of them (I mean, you can keep your algorithm sitting in an API in the cloud) but there are so many constraints in keeping them local.

模型壓縮和節(jié)省機(jī)器學(xué)習(xí)算法的概念是我們可以進(jìn)一步獲益的重要領(lǐng)域。模型過(guò)大會(huì)降低模型的適用性和可用性(我的意思是，您可以將算法保留在云中的API中)但是將它們保持在本地存在很多限制。

For most industries, models are often limited in their usability because they may be too big or too opaque. The ability to discern why a model works so well will not only enhance the ability to make better models, but also more efficient models.

對(duì)于大多數(shù)行業(yè)來(lái)說(shuō)，模型的可用性通常受到限制，因?yàn)槟Ｐ涂赡芴蠡蛱煌该鳌?辨別模型為何運(yùn)作良好的能力不僅可以增強(qiáng)制作更好模型的能力，而且可以提高效率。

For neural nets, the models are so big because you want the model to naturally develop connections, which are being driven by the data. It’s hard for a Human to understand these connections but regardless, the understanding the model can chop out useless connections.

對(duì)于神經(jīng)網(wǎng)絡(luò)，模型是如此之大，因?yàn)槟ＭＰ妥匀坏亟⒂蓴?shù)據(jù)驅(qū)動(dòng)的連接。 對(duì)于人類而言，很難理解這些連接，但是無(wú)論如何，對(duì)模型的理解都可以消除無(wú)用的連接。

The golden nugget would be to have a model that can reason — so a neural network which trains connections based on logic, thereby reducing the training time and final model size, however, we’re some time away from having an AI that controls the training of AI.

金塊將是擁有一個(gè)可以推理的模型-因此，一個(gè)基于邏輯來(lái)訓(xùn)練連接的神經(jīng)網(wǎng)絡(luò)，從而減少了訓(xùn)練時(shí)間和最終模型的大小，但是，我們距離控制訓(xùn)練的AI還有一段距離AI。

Thanks for reading, and please let me know if you have any questions!

感謝您的閱讀，如果您有任何疑問(wèn)，請(qǐng)告訴我！

Keep up to date with my latest articles here!

在這里了解我的最新文章！

翻譯自: https://towardsdatascience.com/the-future-of-ai-is-in-model-compression-145158df5d5e

ai人工智能的本質(zhì)和未來(lái)

總結(jié)

以上是生活随笔為你收集整理的ai人工智能的本质和未来_人工智能的未来在于模型压缩的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問(wèn)題。

如果覺(jué)得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。