yann.lecun.comyann.lecun.com/exdb/publis/pdf/lecun-99.pdf · 1)(+*!! 7\@ "! $ '

28

Transcript of yann.lecun.comyann.lecun.com/exdb/publis/pdf/lecun-99.pdf · 1)(+*!! 7\@ "! $ '

Page 1: yann.lecun.comyann.lecun.com/exdb/publis/pdf/lecun-99.pdf · 1)(+*!! 7\@ "! $ '

����������� ��������������������������� �"!$#%�������&(')!$*+��#, ��!$�-���.�/�

0214353�687:9<;538=5>"14?A@CBEDGFIHJ1:KL357.@M=N6PO7.QR3TSQ4?U?CQR;8=51R3NVW0-QYXAZ[;(1�S7\35]RBEQ^�_"`�_baRced.f[fYgMf�hid\jNk(lGmMmna4oAc4p[qsrut<vxwzyM{Mk[|2{�}�~�d\fY����$m��\�\mRl\k���aR^�k

�4�.���4�\�:�4�\�:�.�R�������.���8���M�C��:���M�������.�����8���:�4�\���\�R�������\���8���\�G�e�R���R�����

�n� �G¡G¢�£e¤\¡M¥$¦ wsfe}Rwsf[§<d\fJd\¨[¨RvxgM¨Rvxw©d�ªx{+«�{Cª8g.¬Y¬E{�d.ªxpRvx{G« ws«8d\f�{G«�«�{Gf:ªxwzd\q4¨Yvxg\j[qs{G­wsf$ªxc[{<}Y{G«�ws§\f�g.¬i«�c[d\¨�{2vx{GoGgM§\f[w®ªxwsgMf�«�¯R«�ªx{G­$«G°4_�c[ws«�¨[d\¨�{Cv�d.ª�ªx{G­$¨Rªx«�ªxg�«�cYg�±ªxc[d.ª�¬²g.vJvx{GoCgM§MfYwsrGwsf[§I«�ws­$¨[qs{PgMj4³�{CoCªx«�±-w®ªxc�cYwz§\c/«�ced.¨�{$yMd�vxwzd\j[wsqsw®ª�¯´«�p[oUc�d\«c[d\fe}4±"vxw®ª�ªx{GfµoUced�vUd\oCªx{Cvx«Gk\w®ª8ws«�¨�g\«�«�wsj[qs{MkMd\fe}�{Gy�{GfJdM}RyMd.f:ªUd\§M{CgMp[«Gk.ªxg<¬E{G{�}�ªxc[{«�¯R«�ªx{G­¶}Yw®vx{GoCªxq®¯P±-w®ªxcn­$wsf[ws­nd\qsq®¯P¨Rvxg:oG{G«�«�{�}nws­nd\§\{G«�d.fe}Pªxg�vx{Gq®¯$gMfnqs{�d.vxfYwsf[§ªxg�{C·4ª�vUd\oCªuªxcY{�vxws§\c4ª�«�{Aª�g\¬¸¬²{�d�ªxpYvx{G«G°i¹+gMf4y�g\qspYªxwsgMf[d\q��{GpRvUd\q �{Aª�±�g.vx�4«�d.vx{«�cYg�±-fJªxguj�{"¨ed�v�ªxwzoCp[qzd.vxq®¯�±�{Gqsq[«�pYw®ªx{�}JªxgªxcYwz«LªUd.«���°\ºT{-d\qs«�g�«�c[g�±%ªxced�ª�ªxcY{G«�{fY{Cª�±+g\vx�4«no�d\f»j�{�p[«�{�}%ªxgTvx{GoGg\§Mf[wsrG{I­Jp[q®ªxws¨[qs{�g\jR³�{GoCªx«n±-wsªxcYgMpRª�vx{G¼4pYw®vxwzfY§{C·R¨YqzwsoGw®ª2«�{G§\­${Gf:ªUd.ªxwsgMf�g\¬iªxcY{ugMj4³�{CoCªx«"¬EvxgM­½ªxcY{Gw®v-«�pYv�vxg\p[fe}Rwsf[§Y°Y_�cY{u«�{GoGg\fe}¨[d.v�ª�g.¬(ªxcY{-¨ed\¨�{Av�¨Yvx{C«�{Gf:ªx«¸ªxc[{<¾<vUd.¨[c�_(vUd\fY«�¬²g.vx­${Cv"�{Aª�±�g.vx�µ­$g4}R{Gq[±-c[wsoUc{C·4ªx{Gf[}Y«"ªxc[{�d.¨[¨Yqzwso�d.j[wsqsw®ª�¯�g.¬8§\vUdM}Rws{Gf:ª�¿�j[d\«�{�}Àqs{�d�vxf[wsf[§Pªxg$«�¯R«�ªx{G­$«"ªxc[d.ª<p[«�{§.vUd\¨[cY«"ªxgµvx{G¨Yvx{C«�{Gf:ªx«-¬E{�d.ªxpRvx{G«Gkeg\jR³�{GoCªx«Gked\fe}nªxc[{Gw®v2oGg\­µj[wsf[d.ªxwsgMfY«G°

Á Â�Ã8Ä�ÅNÆ�ÇAÆ�ÈÊÉ5Ë�ÃÍÌÎÇUÈ2Ë<ÉbÏ�Ã8Ä+É5Ð�Å5øÑÒuZ57WÓÀQYXU?�D.QRÓÀÓÀQR35ÔEÕÖ1YD.D.7.×�?C7\VØÓÀQ�V�7.ÔQ4Ù�×N14?U?C7.@C3Ú@A7MD�QY]R35B©?ABEQR38=�BEX�D�QYÓ�×iQYXA7\VÛQRÙ�1Ü�Ý�ÞRßÀÝ.àNá�Ý.â�ã Z5Q[XU7T@AQYÔ©7TB²X�?AQÛ7�äe?A@G1RD�?�QRå�æx7MD�?CX�QRÙJBE3Y?C7.@C7\XU?ÀÙç@AQYÓè?AZ57\B©@�å(1RDGFe]R@CQR;53(V =1WZN1R3NVeé�D�@G1:Ùê?C7\VIë ÝGì:á�í�âAÝIÝCîYá�âCìRï.á�ð4â ?AZN14?P][1:?CZ57.@GX�@C7.ÔE7.ñ:143[?PB©35ÙçQR@CÓÀ14?ABEQR3�Ùç@CQRÓ�?CZ57�B©3�é×5;�?�143NV´7.ÔEBEÓ�BE3N14?A7\XBE@A@C7.ÔE7.ñ:1R3Y?uñ:14@CBE1Rå5BEÔ©B©?ABE7\X\=�143NV´1 ï.ò©ì:ÜCÜ�ó ôÝ�âã Z5B²DGZ´D.14?A7\]RQR@CBEõ.7\X2?AZN7@A7MXU;5Ô©?ABE35]�Ùç7M1:?C;5@A7W@A7\×5@A7MXU7\3[?C1:?CB©QY3NXIö÷]R7.3N7.@G14ÔEÔ©Õ»ñR7\D�?AQY@CX$QY@�XU?A@CB©3N]YXnQRÙ�XUÕeÓ�åiQRÔ²XGøPB©3[?CQD.1:?C7.]YQR@CB©7MX.ù�ÒuZ57\@A7n1R@A7P?CZ5@A7\7$Ó�1:æxQR@�ÓÀ7.?AZ5Q�V5X�ÙçQR@�D.ÔE1YXAXABzúiD.1:?CB©QY38û á�Ý�ß�üiò©ì4á�Ý�ß�ì:á�ïCý�óêà[ÞÓÀ14?CDGZ57MXu?AZN7PÙç7\1:?C;5@C7$@A7\×5@A7MXU7\3[?C1:?CB©QY3I?CQI1�XU7.?�Q4Ù"D�Ô²1RXCX?A7.ÓÀ×5Ô²1:?C7\X\þ Þ[Ý�àLÝ�âAì4á�óêÿ:Ý�ßÀÝ.áçý��ð � Ü ;NXA7À1T×5@CQRå(14å5BEÔ©B©?xÕ�V�7.3(XUB©?xÕ%ÓÀQ�V�7.Ô�ÙçQY@µ7M1RDGZÖD�Ô²1RXCX.= 143(V ×5B²DGF/?AZN7ÀD.ÔE1YXAX ã B©?AZÖ?AZN7Z5B©]YZ57\XU?¸ÔEBEFR7.ÔEBEZ5QeQeV�Q4Ù5]Y7.357\@C14?ABE35]�?AZN72Ùç7M1:?A;N@A7@A7\×5@C7\XA7.3[?C14?ABEQR38þ � ó²Ü.ï�â�óêß�óêàiì4á�óêÿ:ÝJßÀð � Ý.ò®ÜD�QRÓÀ×5;5?A7I1�V�B²XAD.@ABEÓÀB©3(143[?PÙç;53ND�?CB©QY3Û?AZN14?�V�BE@A7MD�?AÔEÕ ×N@AQ�V�;ND.7\Xn1/XCD�QY@A7�ÙçQR@�7\1RDGZÛD�Ô²1RXCX\ù� 7.357\@C14?ABEñR7$143NV/V�B²XCD�@CB©ÓÀBE3N1:?CB©ñY7µÓÀQ�V�7\ÔEX�1R@A7PQ4Ùê?C7.3�7MXx?CB©Ó�1:?C7\VÛöçÔE7\1R@A3N7\VNø2Ùç@CQRÓ)?C@C1RB©35éB©35]ÀXC14ÓÀ×5ÔE7\X\ù���3T14ÔEÔLQ4Ù8?CZ57\XA7µ1R×5×5@CQY1YDGZ57\X\=4?CZ57JQ:ñY7.@G14ÔEÔN×i7.@AÙçQR@CÓ�143ND.7JQRÙ8?AZ57PXAÕeXU?A7\Ó BEXÔE1R@A]Y7.ÔEÕIV�7�?C7.@CÓ�BE357MV´åeÕ�?CZ57��[;N14ÔEB©?xÕIQ4Ù+?AZ57nXA7.]YÓÀ7.3[?A7\@u1R3NV´?AZ57PÙç7M1:?C;5@A7P7.ä[?C@C1YD�?CQR@MùS<7MD.1R;NXU7-?AZ57\ÕP1R@A72ZN143NV�é�D.@C14Ùê?A7\V8=.?CZ57XA7.]YÓÀ7.3[?A7\@¸143(VPÙç7\14?A;5@C727.ä[?C@C1YD�?CQR@8QRÙê?A7.3�@A7\Ô©Õ

QR3nXAB©ÓÀ×5ÔEBzÙçÕeBE35]�1YXAXA;5ÓÀ×�?ABEQR3(X�14åiQR;5?8?AZ57<B©3N×5;�?+VN1:?C1�143NV$D\143$@G14@C7.ÔEÕ�?C1RFR7-BE3[?AQ�1YD.D�QY;53[?14ÔEÔ ?AZ57�ñ�1R@AB²14åNB©ÔEBz?xÕ´Q4Ù+?AZ57�@A7M14Ô ã QR@CÔEV ù��3�B²V�7\1RÔ�XUQYÔ©;5?ABEQR3T?AQÀ?CZ5BEX�×5@CQRå5ÔE7.Ó BEXu?CQ�Ùç7.7MV?AZ57u7.3[?CB©@C7XAÕ�Xx?C7.Ó ã B©?AZ�ÓÀB©3NB©Ó�14ÔEÔEÕn×[email protected]\XCXU7MVnBE35×5;�?GX�öç7Yù ](ù�U@G1 ã � ×5B©äe7\Ô�B©Ó�1R]R7\XGø�=:1R3NV?A@G14BE3IBz?Ùç@CQRÓ V51:?G1�XAQ�1YX-?CQ�ÓÀB©3NB©ÓÀBEõ.7P143WQ:ñY7.@G14ÔEÔ5ÔEQYXCX-Ùç;53(D�?ABEQR3Öö ã Z5B²DGZIÓ�1:ä�BEÓÀB©õ\7\X1]RBEñR7.3�×i7.@AÙçQR@CÓ�143ND.7nÓÀ7\1YXU;5@C7Mø�ù��P7\7.×5BE35]�?CZ57�×5@A7\×[email protected]\XCXUBE35]À?CQ´1IÓÀBE35BEÓ�;5Ó 7\3NXU;N@A7MX?AZN14?35Q�;535@C7\1RÔ©B²Xx?CBEDJ1RXCXU;NÓ�×5?ABEQR3�B²X<Ó�1YV�7J1Rå(QY;�?2?CZ57µVN1:?C1Nù���3�ÙçQR@A?A;N3N1:?C7.ÔEÕR=Y?AZN14?u1RÔEXAQ

Page 2: yann.lecun.comyann.lecun.com/exdb/publis/pdf/lecun-99.pdf · 1)(+*!! 7\@ "! $ '

@A7 �Y;NB©@C7\XP?CQ%D�QYÓ�7W;5× ã Bz?CZÚ1%XA;5Bz?G14å5ÔE7IÔE7\1R@A35BE35]�14@GDGZ5B©?A7MD�?A;N@A7À?CZN1:?�D\143ØZN1R3NV�ÔE7�?AZN7Z5B©]YZÚV�BEÓ�7\3NXAB©QY3ØQ4Ù�?AZN7IB©3N×5;�?�öç3e;5Ó�åi7.@�Q4Ù�×5B©ä�7.Ô²XCø�=�?CZ57IZ5BE]RZbV57.][email protected]�QRÙ�ñ�1R@AB²14åNB©ÔEBz?xÕV�;57%?CQÚ×iQYXA7%ñ:14@CBE14?ABEQR3NX�QY@I]R7\QRÓÀ7�?C@AB²D%V�B²XU?AQR@A?ABEQR3(XI14ÓÀQR3N]ØQ4?AZN7.@I?CZ5BE35]YX\=<1R3NVÎ?AZN7357\D.7\XCXA1R@ABEÔ©ÕID.QRÓÀ×5ÔE7�äW3NQR3�é�Ô©BE357M14@u@A7\ÔE14?ABEQR3Tå(7.? ã 7\7.3T?AZN7$B©35×N;�?�143NVW?AZN7$QR;�?C×5;�?\ù

� âCì � ó÷Ý.àNá ���Jì�Ü.Ý ��� ÝGì4â�àNóêà[Þ ×5@CQ:ñeBEV57\X21nÙç@C1RÓÀ7 ã QR@CF�BE3 ã Z5B²DGZ�?CQ�å5;5BEÔ²V´XA;NDGZW1�XUÕ�XUé?A7.Ó/ù�ÒuZ57IÔE7\1R@A35BE35]�Ó�1RDGZNB©357´D�QRÓÀ×5;5?A7\X�1/Ùç;53ND�?ABEQR3���� ��ö�������� ø ã ZN7.@C7����/BEX?AZ57��Né�?CZ�BE35×5;5?-×N1:?A?A7.@C38=41R3NV�� @C7.×5@C7\XA7.3[?GX�?CZ57�D�QYÔ©ÔE7\D�?ABEQR3�Q4ÙL1RV�æx;NXU?C1Rå5ÔE7�×N14@G14ÓÀ7�?C7.@GXB©3´?AZ57$XAÕ�Xx?C7.Ó/ù�ÒuZ57PQR;�?C×5;�?�����D�QY3[?C14BE3NXuXCD�QR@C7\X<QR@×5@CQRåN1Rå5BEÔ©B©?ABE7\X<ÙçQR@u7M1RDGZ´D\1:?C7.]RQY@AÕYù� ÔEQYXCXuÙç;53ND�?ABEQR3�� � ��Tö! � �"��ö��#�$� � øUø�=iÓÀ7\1YXU;5@C7\X�?AZ57�V�BEXCD�@C7.×(143ND.ÕWåi7�? ã 7.7.3� � =?AZ57 CD�QR@C@C7\D�? � QY;�?A×N;�?$ÙçQR@n×(1:?U?C7.@C3�����=�143(V ?CZ57�QR;5?A×5;�?n×N@AQ�V�;ND.7\VÖåeÕ%?CZ57IXUÕ�XU?A7\ÓTùÒuZ57�1�ñR7\@C1R]R7uÔEQYXCX�Ùç;53ND�?ABEQR3%�'&)($*,+.-8ö�� ø-BEX-?AZ57µ1�ñR7\@C1R]R7Q4Ù ?AZN7�7.@C@AQY@CX��/�PQ:ñY7.@21nXA7�?<Q4ÙÔE1Rå(7\Ô©7MV�7.ä514ÓÀ×5ÔE7\X"D.1RÔ©ÔE7\V�?CZ57u?A@G14BE35BE35]$XU7.?'0[ö���12�� 31�ø4�65.575.5²ö��98��" :82ø,;Yù ��3�?CZ57�XUBEÓÀ×5ÔE7\XU?XU7.?U?ABE35](=5?AZN7nÔ©7M14@C35BE35]À×5@AQYå5ÔE7.Ó D.QR3NXAB²Xx?GX�B©3/ú(3NV�BE35]�?AZN7nñ:14ÔE;57nQ4Ù<� ?AZN14?�ÓÀB©3NB©ÓÀBEõ.7\X� &)(�*4+7- ö�� ø�ù

=%14FeBE35]�?AZN7´ÔEQYXCX$Ùç;53(D�?ABEQR3 � ó >�Ý.âAÝ.àNá�ó÷ì@?.ò©Ý�ã B©?AZb@C7\XA×i7\D�?�?AQ#��7.3NXA;5@C7\X$?CZN1:?À7�Ùêéú(D�BE7.3[?\=�]Y@C1YV�B©7\3[?Ué�åN1RXA7\VÖ35QY3�é�ÔEBE357\1R@�QY×�?ABEÓÀB©õM1:?ABEQR3bÓ�7.?AZ5Q�V5XÀD\143Úåi7´;(XU7MVØ?AQ úN3NV 1Ó�BE35BEÓ�;5Ó/ùRÒ¸QP7.3NXA;5@C7u]RÔEQRåN1RÔ[email protected][?AB²14åNB©ÔEBz?xÕY=:?AZN7uXAÕ�Xx?C7.Ó BEX�å5;NB©Ô©?21RX"1JÙç7.7MVeé�ÙçQR@ ã 1R@CV357�? ã QR@CF QRÙnÓÀQeV5;5Ô©7MX.ù ��3 ?CZ57ÖXABEÓ�×NÔ©7MXx?/D.1RXA7R=<7\1YDGZ Ó�Q�V�;NÔ©7ÛD�QYÓ�×N;�?A7MX´1ÛÙç;N3ND�?CB©QY3AB-CD�E-�ö��F-G�HA�-JI 1 ø�= ã Z57.@C7KA�-bB²XW1R3ÎQRå�æx7MD�? ö÷1ÚñR7\D�?AQY@�B©3Î?CZ57 XAB©ÓÀ×5ÔE7\XU?´D.1YXU7�ø@A7\×5@A7MXU7\3[?ABE35]�?AZN7�QY;�?A×5;5?µQRÙ"?AZ57�ÓÀQeV5;5Ô©7Y=L��-�B²XJ?AZ57�ñR7MD�?CQR@�QRÙ-?A;N3N14å5ÔE7´öç?A@G14BE3N14åNÔ©7�ø×N14@G14ÓÀ7�?C7.@GXB©3T?AZN7nÓÀQeV5;5Ô©7Iö�1�XU;NåNXU7.?�Q4Ù<� ø�=N1R3NV�A -JI 1 B²X?AZ57�Ó�Q�V�;NÔ©7NM X�BE35×5;5?�ö÷1RXã 7\Ô©Ô+1RX�?CZ57�×N@A7\ñ[BEQR;(X�ÓÀQ�V�;5ÔE7@M X�QY;�?A×5;5?Gø�ùLÒuZ57�B©35×N;�?OABPn?CQI?CZ57núN@GXU?�ÓÀQeV5;5Ô©7�BEX�?AZN7XUÕ�XU?A7.ÓQM XuB©3N×5;�?�×N14?U?C7.@C3K���eùÒuZ57�Ó�14BE3ÖB²V�7M1/Q4Ù � @C1YV�BE7.3[?Ué�Su1RXA7\V 6�7\1R@A35BE35](= ã Z5BEDGZØB²Xn1�XAB©ÓÀ×5ÔE7�7�äe?C7.3NXAB©QY3ÛQ4Ù?AZ57 ã 7\Ô©Ô©é�F[3NQ ã 3IåN1RDGF[é�×5@CQR×N1R]Y1:?CB©QY3�3N7.;5@G14Ôi357�? ã QR@CFÀÔ©7M14@C35BE35]�14ÔE]RQY@AB©?AZNÓT=[B²X2?CZN1:??AZN7QRå�æx7MD�?ABEñR7�Ùç;N3ND�?CB©QY3nD.143Påi7"74RID�BE7.3[?CÔ©ÕµÓÀB©35BEÓÀB©õ\7\Vµ?CZ5@AQY;5]RZP]Y@C1YV�B©7\3[? V57\XCD�7.3[?2ö÷QR@8QR?AZ57\@

Ó�QY@A7%XAQR×NZ5BEXU?AB²D.14?A7MV 3NQR3�é�Ô©BE357M14@�QR×5?ABEÓ�BEõ\14?ABEQR3 ÓÀ7�?CZ5Q�V5XGøÀå(7MD.1R;NXU7/?CZ57 ]Y@C1YV�BE7.3[?�Q4Ù� ã Bz?CZØ@A7MXU×i7\D�?$?AQ�� D.143Ûåi7I74RID.B©7\3Y?CÔ©ÕÛD�QYÓÀ×5;�?A7MV ã Bz?CZÚ1/åN1RDGF ã 14@GV @C7\D.;5@[email protected](D�7?AZ5@CQR;5]YZT?AZ57�357�? ã QR@CFIQ4Ù"ÓÀQ�V�;5ÔE7\X\ù ��Ù�?AZN7�×N1R@U?CBE1RÔ�V�7.@CBEñ�14?ABEñR7$QRÙ��/� ã B©?AZ�@C7\XA×(7MD�?�?CQA - BEXJFe35Q ã 38=i?AZ57\3%?CZ57�×(14@A?AB²14Ô�V�7.@CB©ñ:14?ABEñR7\XJQ4ÙS�/� ã Bz?CZ @C7\XA×(7MD�?�?CQQ� - 1R3NV�A -JI 1D.143Tåi7nD�QRÓÀ×5;5?A7\VT;NXABE35]�?AZ57PÙçQYÔ©ÔEQ ã B©3N]�å(1RDGF ã 1R@CVI@A7MD�;5@C@A7\3ND�7Yû

T �/�T � -

T �U-T � ö��F-V�HA�-JI 1 ø

T ���T A -

T ���T A�-JI 1

T �U-T A ö�� - �HA -JI 1 ø

T ���T A�- öXW�ø

ã Z57.@C7�Y[Z@\Y^] ö��F-G�HA�-JI 1 ø"B²X<?AZ57/_[1RD�QYå5B²143�Q4Ù`�U- ã B©?AZ´@A7MXU×i7\D�?2?CQa� 7\ñ�1RÔ©;(1:?A7MVI1:??AZN7×(QYB©3[?µö�� - �HA -JI 1 ø�=e143NVbYcZN\Y[d ö�� - �"A -�I 1 ø2BEX2?AZN7O_[1RD�QYå5B²143ÀQ4Ù`� - ã B©?AZI@C7\XA×(7MD�?<?AQ�AbùÒuZ57úN@CXU?"7 �[;N14?ABEQR3ÀD�QRÓÀ×5;5?A7\X"XAQRÓÀ7?A7\@AÓ�X�QRÙN?AZN7�]R@G1RV�BE7.3[?�Q4Ùe�/�5ö�� ø�= ã ZNB©ÔE7?AZ57�XA7\D�éQR3NVµ7 �[;N14?ABEQR3P×5@CQR×N1R]Y14?A7\Xi?CZ57-×N1R@U?CBE1RÔ:]R@G1RV�BE7.3[?GXLåN1RDGF ã 14@GV ù�ÒuZ57"B²V�7\1�D\143Påi7"?A@CB©ñeB²14ÔEÔ©Õ7�äe?A7\3NV�7\V»?CQ/1R3eÕ%357�? ã QR@CF�Q4Ù<Ùç;53ND�?ABEQR3N1RÔ"Ó�Q�V�;NÔ©7MX.ù � D.QRÓÀ×5ÔE7�?C7.ÔEÕ%@ABE]RQY@AQY;NXµV�7.@CB©ñ:14é?ABEQR3�Q4Ùi?AZN7�]R@G1RV�BE7.3[?"×5@CQR×N1R]Y14?ABEQR3�×N@AQ�D�7MV�;5@C7�B©3À?CZ57�]Y7.357\@C1RÔ5D.1RXA7�D.1R3�åi7�V�QY357�;NXAB©3N]6�14]Y@C1R35]R7�Ùç;53ND�?ABEQR3NXgf 687:9<;53#WchNikj6l�ù`f 6�7�9<;53mWchNi@i[l�=`f S<QR?U?CQR;�143NV � 14ÔEÔEB©3N1R@AB�W[h@hnW$l�ù

Page 3: yann.lecun.comyann.lecun.com/exdb/publis/pdf/lecun-99.pdf · 1)(+*!! 7\@ "! $ '

� ��Ë�Ä��$Ã�Ì Ã����-È2Æ�ÇxÉ5Ç��2Æ Ç�É5Ë�� �<Æ������UÐ�É�Ç��2Æ�Ä����¶Ã�Ð�Å5Ä���¶Ã É���-Å���Ñ

Ò�@G1RV�B©?ABEQR3(14Ô�Ó�;NÔz?CBzé�ÔE1�ÕY7.@�357\;5@C1RÔ5357.? ã QY@AF�X�14@C7�1$XU×i7\D.BE1RÔND.1YXU7uQ4Ùi?AZN7�14åiQ:ñR7 ã Z57.@C7u?AZN7Xx?G1:?A7MX�A - 14@C7�úNäe7MVeé�XUBEõ.7MV%ñR7MD�?CQR@GX.=L143NV ã Z57.@C7�?AZ57ÀÓÀQ�V�;5ÔE7\XP14@C7À14Ô©?A7.@C3N14?A7\V ÔE1�ÕY7.@GXQ4Ù�Ó�1:?A@CB©ä Ó�;5Ô©?ABE×5ÔEBED\1:?CB©QY3NXWöç?AZ57 ã 7.BE]RZ[?GXCøP1R3NVÛD�QRÓÀ×iQR357\3[?Ué ã BEXA7IXUBE]RÓÀQYBEV»Ùç;53ND�?ABEQR3NXöê?AZN7n;535B©?CXGø�ùiÒ�@G1RV�B©?ABEQR3(14ÔLÓ�;5Ô©?ABEÔ²1�ÕR7.@�3N7.;5@G14Ô83N7�?CX ã Z57.@C7$14ÔEÔ ?AZ57$;N35Bz?GX�BE3�1ÀÔE1�ÕY7.@�14@C7D�QR3N357\D�?A7\V�?AQ$1RÔ©Ôe?CZ57u;535B©?CX�BE3�?CZ57u357�äe?"Ô²1�ÕR7.@"D\143�åi7�;NXA7\V�?AQP@C7\D�QY]R35BEõ.7<@C1 ã öç@CQR;5]YZ5Ô©ÕXUBEõ.7.é�35QY@AÓ�1RÔ©BEõ.7MVW143(V´D.7.3[?A7\@A7MVNøB©Ó�1R]R7\X\=eå5;�?�?CZ57.@C7n14@C7J×N@AQYå5Ô©7\Ó�X.ù�¸BE@GXx?CÔ©ÕY=.?xÕe×5B²D.1RÔYBEÓ�14]R7MX�1R@A7-Ô²14@C]R7Y=.QRÙê?A7.3 ã Bz?CZ�XA7.ñY7.@G14Ô4Ze;53NV5@A7MVnñ:14@CBE1Rå5ÔE7\X2ö÷×5B©äe7\ÔEXGø�ù

� Ùç;5ÔEÔEÕYé�D�QY35357MD�?A7MV%357.? ã QY@AF ã B©?AZ8= XC1�Õ#W����W;535B©?CXµBE3 ?CZ57�ú(@CXU?µÔ²1�ÕR7\@\= ã QY;5Ô²V%14ÔE@A7M1RV�ÕD�QR3[?G14BE3WXU7\ñR7\@C1RÔ`W��N= ����� ã 7.BE]RZ[?CX\ù ��;NDGZ´1�ÔE1R@A]Y7�3e;5Ó�åi7.@Q4Ù�×(14@G14ÓÀ7�?C7.@GX-BE3ND�@C7\1YXU7MX"?AZN7D.14×(1RD�B©?xÕ�QRÙ ?AZ57µXUÕ�XU?A7\ÓÊ1R3NV�?AZ57\@A7.ÙçQR@C7�@C7 �[;5BE@A7MX-1$Ô²14@C]R7.@-?A@G14BE35B©3N]�XU7.?\ù[S<;5?2?AZN7�Ó�14BE3V�7�ú(D.B©7\3ND�Õ´Q4Ù�;53NXU?A@C;ND�?C;5@C7\VT357�?GX�B²Xu?AZN14?�?AZN7.ÕWZ(1�ñR7P35Q�å5;5BEÔ©?Ué�B©3/BE3[ñ:1R@AB²143ND.7 ã Bz?CZ�@A7.éXU×i7\D�?�?AQÛ?A@G143(XUÔ²1:?CB©QY3NX.=-XCD.1RÔ©7Y="QR@�]Y7.QRÓÀ7.?A@CBED/V�B²Xx?CQR@A?ABEQR3NXÀQRÙJ?CZ57%BE35×5;�?GX.ù ��ÓÀ1R]R7MXÀQ4ÙQRå�æx7MD�?CX<D.1R3Iå(7µ14×5×N@AQ�ä�BEÓÀ14?A7\Ô©Õ�XUBEõ.7.é�35QY@AÓ�1RÔ©BEõ.7MV�1R3NV�D�7\3[?A7.@C7\V8=Rå5;�?u35Q�XA;NDGZI×N@A7\×5@AQRéD�7\XCXAB©35]$D\143�å(7�×i7.@AÙç7\D�?\ùYÒuZNBEX\=RD�QYÓ�å5BE357MV ã B©?AZÀBE3Y?C@ABE3NXABED ã B©?AZNB©3�é�D�Ô²1RXCX�ñ:14@CBE1Rå5BEÔ©B©?xÕR= ã BEÔ©ÔD.14;(XU7$ñ:14@CB²1:?ABEQR3(XuB©3T?AZN7n×(Q[XUB©?ABEQR3/QRÙ"V5BEXU?ABE3ND�?CB©ñY7PÙç7\14?A;5@C7\X�BE3�BE35×5;�?�QYå�æx7\D�?CX\ù ��3/×5@CB©3�éD�BE×5Ô©7Y=�1$Ùç;5ÔEÔ©Õ[é�D�QY35357\D�?A7MVI357.? ã QY@AF�Q4Ù¸XA;�RID�BE7.3[?uXUBEõ.7PD�QY;5ÔEVIÔ©7M14@C3�?AQ�×5@AQ�V�;(D�7µQR;�?C×5;�?CX?AZN14?14@C7�BE3eñ�1R@AB²143[? ã B©?AZ´@A7MXU×i7\D�?-?AQ�XU;NDGZ�ñ:1R@AB²1:?CB©QY3NX.ùYH�Q ã 7.ñR7\@\=RÔ©7M14@C35B©3N]�XU;(DGZI1$?C1YXUFã QY;5ÔEVT×5@CQRå(14å5ÔEÕI@C7\XA;5Ôz?�BE3/Ó�;5Ô©?ABE×5ÔE7n;535B©?CX ã Bz?CZ%XUBEÓÀB©Ô²14@ ã 7.BE]RZ[?�×N1:?A?A7\@A3NX�×(Q[XUB©?ABEQR357MV1:?¸ñ:1R@ABEQR;NX¸ÔEQeD\1:?CB©QY3NX8BE3n?CZ57<BE35×5;5?�XAQJ1YX8?CQJV�7.?A7\D�?�V5BEXU?ABE3ND�?CB©ñY72Ùç7M1:?C;5@A7MX ã ZN7.@C7.ñR7\@8?CZ57.Õ14×5×i7\1R@�QR3T?AZ57�B©35×N;�?\ùi687\1R@A3NB©35]�?CZ57\XA7 ã 7\B©]YZ[?�D.QR3�ú(]R;5@G1:?CB©QY3NXu@A7 �[;5B©@C7\X�1�ñY7.@CÕIÔE1R@A]Y73[;NÓ�åi7.@nQRÙ<?C@C1RB©35BE35]�BE3NXU?C143(D�7\Xµ?CQ%D�Q:ñY7.@P?AZN7IXU×N1YD�7�QRÙ×iQYXCXUBEå5ÔE7�ñ:14@CBE14?ABEQR3NX\ù ��3ÚD�QR35éñRQRÔE;�?CB©QY3N14ÔL357.? ã QY@AF�X\=eV57\XCD�@CB©åi7\V´å(7\Ô©Q ã =5?AZ57$@CQRå5;(Xx?C357\XCX<?AQÀ]R7\QRÓÀ7�?C@AB²DPV�BEXU?AQY@U?CB©QY3NXBEX14;�?CQRÓ�1:?CBED\14ÔEÔ©ÕnQYå�?C1RB©3N7\V�åeÕ�Ùç[email protected]©35]µ?CZ57�@C7.×5ÔEBED\1:?CB©QY3�QRÙ ã 7.BE]RZ[?-D�QY3�úN]Y;5@C14?ABEQR3NX"1YD�@CQYXCXXU×N1YD�7Yù

�e7\D.QR3NV�ÔEÕR=+1�V�7�úiD�BE7.3ND.ÕÖQRÙuÙç;5Ô©ÔEÕ[é�D.QR353N7\D�?C7\VØ1R@CDGZ5B©?A7MD�?C;5@A7MX$B²X$?AZ(1:?�?CZ57�?CQR×iQRÔEQR]RÕQ4Ù�?CZ57/B©3N×5;�?IB²XÀ7.3[?ABE@A7\Ô©ÕØBE]R3NQR@C7\V ù"ÒuZ57/BE35×5;�?Iñ:14@CB²14å5ÔE7\X�D.1R3 åi7/×5@A7MXU7\3[?A7\V BE3 1R3[Õöêú5ä�7\V(ø�QR@GV�7.@ ã Bz?CZ5QR;5?W1:KL7\D�?CB©3N]»?CZ57/QR;�?GD�QYÓ�7/QRÙ�?AZN7´?C@C1RB©3NB©35](ù�!J3Ú?CZ57%D�QY3Y?C@C1R@AÕY=B©Ó�14]Y7\XÀZN1�ñY7%1ØXx?C@AQY35]#"�$ ÔEQ�D.1RÔ�Xx?C@A;(D�?A;N@A7Yû-ñ:14@CBE1Rå5Ô©7MX�öç×5B©ä�7.Ô²XCøÀ?AZN14?W14@C7%XU×N14?AB²14ÔEÔ©Õ357\1R@AåeÕ 1R@A7/Z5BE]RZNÔ©ÕÎD�QY@A@C7.Ô²1:?C7\V ù-6�QeD\14Ô�D.QR@C@A7\ÔE14?ABEQR3NX�1R@A7/?CZ57%@C7\1YXUQY3NX�ÙçQR@�?CZ57 ã 7.ÔEÔ©éF[3NQ ã 3´1RV5ñ�1R3[?C14]Y7\X<Q4Ù+7�äe?A@G1RD�?ABE35]�1R3NVWD.QRÓ�å5B©3NB©35] ò©ðMïGì:ò Ùç7\14?A;5@C7\Xåi7�ÙçQR@C7P@A7MD�QY]R35BEõ.BE35]XU×N14?AB²14ÔLQR@?C7.ÓÀ×(QY@C1RÔ QRå�æx7MD�?GX.=eåi7\D.1R;NXA7µD.QR3�úN]Y;5@G1:?ABEQR3(X2QRÙ¸357\B©]YZ[åiQR@CBE35]�ñ:1R@AB²14å5ÔE7\XD.1R3å(7ID.ÔE1YXAXABzú(7\VÖB©3[?CQ%1TXAÓ�14ÔEÔ-3e;5Ó�åi7.@�Q4Ùu@A7\Ô©7\ñ�1R3[?$D\1:?C7.]RQY@ABE7\XÀö÷7Rù ]Nù87MV�]R7MX.=8D.QR@C357.@GX.ùEùEù ø�ù% ð4àNÿ:ð:òsí�á�ó÷ð:àLì:ò'&�Ý�á)(2ð:â+*:Ü ÙçQR@GD�7$?CZ57�7.ä[?C@C1YD�?CB©QY3�Q4Ù-ÔEQeD\14Ô�Ùç7\1:?C;5@C7\X�åeÕ/@C7\XU?A@CBED�?ABE35]�?AZN7@A7MD�7.×5?ABEñR7µúN7.Ô²V5XuQRÙ+Z5B²V5V�7\3/;535B©?CXu?CQ�å(7$ÔEQ�D.14Ô�ù

,.-0/ 13254�67258:9�;�<:254>=�8�?A@ ;CBD27EGFIH

9<QR3eñRQYÔ©;�?CB©QY3N14ÔKJ�7.? ã QY@AF�X�D�QYÓ�å5BE357?AZN@A7\7u1R@CDGZNBz?C7\D�?C;5@G14ÔeBEV57\1RX+?CQP7.3NXA;5@C7XAQRÓÀ7�V�7.]Y@A7\7Q4Ù+XAZ5B©Ùê?\=5XCD.1RÔ©7Y=e1R3NVTV�BEXU?AQY@U?CB©QY3WBE3eñ�1R@AB²143(D�7Rû ò©ðMïGì:ò8âAÝGïGÝ�üiá�óêÿ�Ý<ô<Ý.ò � Ü = ÜAý5ì4âAÝ � (2Ý.ózÞMý�á÷Ü öçQY@ã 7\B©]YZY?n@C7.×NÔ©B²D.14?ABEQR3(ø�=�143NVÛXA×N1:?CBE1RÔ Ü�ín? ��Ü.ì4ß�üLòsóêà[Þ ù � ?xÕ[×NBED\14Ô2D.QR3eñRQYÔ©;�?CB©QY3N14Ô�357.? ã QY@AFÙçQR@$@C7\D�QY]R35BEõ.BE35]TXUZN1R×(7MX.= V�;Nå5å(7MVÛ687�J�7�?AéML5= BEX$XAZ5Q ã 3 BE3»úN]R;5@C7QWRù ÒuZ57ÀBE35×5;�?n×5Ô²143N7@A7MD�7.BEñR7MX2BEÓ�14]Y7\XQ4Ù�QYå�æx7\D�?CX?CZN1:?�14@C7J1R×5×5@CQ�ä�B©Ó�1:?C7.ÔEÕ�XUBEõ.7.é�35QY@AÓ�1RÔ©BEõ.7MVI143NVTD�7\3[?A7.@C7\V8ù

Page 4: yann.lecun.comyann.lecun.com/exdb/publis/pdf/lecun-99.pdf · 1)(+*!! 7\@ "! $ '

INPUT 32x32

Convolutions SubsamplingConvolutions

C1: feature maps 6@28x28

Subsampling

S2: f. maps6@14x14

S4: f. maps 16@5x5

C5: layer120

C3: f. maps 16@10x10

F6: layer 84

Full connectionFull connection

Gaussian connections

OUTPUT 10

����� ¥��e¥ ^2vxoUc[w®ªx{GoCªxpRvx{"g\¬5hN{G�{Aª�¿��:k�d�¹+g\f4y�gMqspYªxwsg\fed\qY�{GpRvUd\qR�{Cª�±�g.vx�5k\c[{Avx{�¬²g\v¸}Rwz§\w®ªx«�vx{GoCgM§\¿fYw®ªxwzg\f(°�d\oUc´¨Yq©d.f[{Jwz«�d$¬²{Gd.ªxpYvx{µ­nd\¨(kewç° {M°(d�«�{Cª�g\¬+p[fYwsªx«<±-c[gM«�{J±�{Gws§Mc:ªx«�d�vx{µoGg\f[«�ª�vUd\wsfY{�}ªxgµj�{uwz}Y{Gf:ªxwso�d\qê°

�21RDGZW;535B©?�B©3T1�Ô²1�ÕR7\@2@C7\D.7.BEñR7\XBE35×5;�?GXÙç@AQYÓ)1�XA7�?�QRÙ�;535B©?CXuÔEQ�D.1:?C7\VWB©3/1�XAÓ�14ÔEÔ 357.BE]RZ5éå(QY@AZ5QeQ�VÖB©3Û?CZ57I×5@C7.ñeBEQR;NXnÔE1�ÕY7.@Mù8ÒuZ57IB²V�7\1/QRÙ�D.QR353N7\D�?CB©3N]�;535B©?CX$?CQ%ÔEQeD\14Ô-@C7\D.7.×�?CB©ñY7úN7.Ô²V5X<QR3�?CZ57�BE35×5;�?]RQe7\X2åN1RDGF�?AQ�?AZ57J7\1R@AÔEÕ ���RX\=Y1R3NV ã 1RX-Ô²14@C]R7\Ô©Õ�B©3(XU×5BE@C7\V�åeÕÀH�;5åi7.Ô143NV�� BE7\XA7.Ô�M X�V�B²XAD.Q:ñR7.@CÕ%QRÙuÔEQ�D.1RÔ©ÔEÕ[é�XA7.3NXAB©?ABEñR7R=�QR@CB©7\3Y?G1:?CB©QY3�é�XA7.ÔE7\D�?ABEñR7À357\;5@AQY3NX$BE3Ø?AZN7D.1:?[M X�ñeBEXA;N14Ô�XAÕeXU?A7\Ó f H�;Nå(7\Ô¸143(V�� BE7\XA7.Ô<W[h�� "cl�ùL68Q�D.1RÔ¸D.QR35357MD�?CB©QY3NX�ZN1�ñY7nåi7.7.3 ;NXU7MVÓÀ1R3eÕn?ABEÓÀ7\X2B©3À357\;5@C1RÔ5ÓÀQ�V�7.Ô²X"Q4Ù ñeB²XU;N1RÔ�Ô©7M14@C35BE35]:f �N;5Fe;NXAZ5B©Ó�1�WchJj�Lcl�= f 687:9<;53#Wch@i��[l�=f 6�7�9<;53#WchNi@h[l�= f =�QRõ\7.@/WchNh W$l�ù�� B©?AZ�ÔEQ�D.14ÔN@A7MD�7.×5?ABEñR7uúN7\ÔEV5X\=R3N7.;5@CQR3NX"D\143ÀÔE7\14@C3�?AQn7.äeé?A@G1RD�?µ7.ÔE7.ÓÀ7.3[?G14@CÕ´ñeB²XU;N1RÔ�Ùç7M1:?C;5@A7MX�XA;NDGZ 1YX�[email protected][?A7MV/7\V�]Y7\X\=(7\3NVeé�×(QYB©3[?GX.=LD�QY@A357\@CX�öçQY@XUBEÓÀB©Ô²14@�Ùç7\1:?C;5@C7\X�B©3ÚQR?AZ57\@�XABE]R3N1RÔEX�XU;NDGZ 1RX�XA×i7.7\DGZ XU×i7\D�?A@CQR]R@G14Ó�XGø�ù+ÒuZ57\XA7IÙç7M1:?A;N@A7MX14@C7J?CZ57.3/D�QYÓ�å5BE357MV´åeÕ�?CZ57nXA;5åNXA7 �[;57.3[?�Ô²1�ÕR7.@GX<B©3/QR@GV�7\@<?CQIV�7�?C7\D�?uZNB©]YZ57.@Aé�QY@CV57.@<Ùç7\1:é?A;5@C7\X\ù ��X�Xx?G1:?A7MVW7M14@CÔ©BE7.@M=5V�B²Xx?CQR@A?ABEQR3NXuQY@�XUZNBzÙê?GX�Q4Ù+?AZ57$BE35×5;5?�D.143�D\14;NXA7µ?AZ57$×iQYXABz?CB©QY3Q4ÙµXC14ÔEBE7.3[?ÀÙç7\1:?C;5@C7\X�?AQÛñ:14@CÕRù ��3Î1RVNV�Bz?CB©QY38="7.ÔE7.ÓÀ7\3Y?G14@CÕØÙç7\14?A;5@C7�V�7�?C7\D�?AQR@GX�?AZN14?W14@C7;NXU7.Ùç;5Ô�QR3»QR357�×N14@A?JQRÙ-?AZN7�BEÓ�14]Y7�14@C7�ÔEBEFR7.ÔEÕ/?AQTåi7�;NXA7�Ùç;NÔ-1RD�@CQYXCX�?CZ57�7.3[?ABE@A7�BEÓÀ1R]R7YùÒuZ5BEXµFe35Q ã ÔE7\V�]Y7�D.1R3%åi7�1R×5×5ÔEB©7MV%åeÕ/ÙçQR@GD�BE35]W1TXU7.?JQRÙ2;535B©?CX\= ã Z5QYXA7�@C7\D.7.×�?CB©ñY7núN7\ÔEV5X14@C7´ÔEQ�D.1:?C7\V 1:?IV�B©Ki7\@A7\3[?À×NÔE1YD�7\XÀQY3b?CZ57/B©Ó�1R]R7R=�?CQÛZN1�ñR7TBEV57.3[?AB²D.1RÔ ã 7\B©]YZY?�ñY7\D�?CQR@GXf �N;5Fe;NXUZNB©Ó�1�143(V3=�BEÕY1RFR7aWchNi�"cl�=`f ��;5ÓÀ7.ÔEZN14@A?\=NH�B©3[?CQR3�143NV�� BEÔ©ÔEBE1RÓ�XWchNi��[l�=f 6�7�9<;53#WchNi@h[l�ù���35B©?CX<B©3W1nÔE1�ÕY7.@21R@A7�QY@A][1435BEõ.7MV�BE3I×5Ô²143N7\X ã B©?AZNB©3 ã Z5BEDGZI1RÔ©Ô(?AZN7�;535B©?CXXUZN1R@A7�?AZN7IXA1RÓ�7�XA7�?nQRÙ ã 7.BE]RZ[?CX\ù8ÒuZ57�XU7.?nQ4ÙQR;�?C×5;�?CXnQ4Ù2?CZ57�;535B©?CXnB©3ØXA;NDGZÖ1T×5Ô²143N7BEXnD\14ÔEÔ©7MVÛ1�ë ÝCì4á�í�âCÝIß�ìCü ù ��3NBz?GXnBE3Û1TÙç7M1:?C;5@A7�Ó�1R×Û1R@A7�1RÔ©Ô2D.QR3NXU?A@G14BE357MV%?CQ%×(7\@UÙçQY@AÓ?AZ57TXA1RÓ�7´QR×i7.@G1:?CB©QY3ØQR3ÚV�B©Ki7\@A7\3[?�×N1R@U?GXnQ4Ù�?CZ57TB©Ó�14]Y7Rù � D�QYÓÀ×5Ô©7.?A7TD�QR3eñYQRÔE;�?ABEQR3N1RÔÔE1�ÕY7.@"B²X-D�QYÓ�×iQYXA7\V�QRÙLXA7.ñY7.@G14ÔeÙç7\14?A;5@C7�Ó�14×NX�ö ã B©?AZIV�B©Ki7\@A7\3[? ã 7\B©]YZY?-ñY7\D�?AQR@GXGø�=4XAQP?AZN14?Ó�;5Ô©?ABE×5ÔE7$Ùç7\14?A;5@C7\XJD.143/åi7n7�äe?C@C1YD�?A7MV/1:?�7M1RDGZ/Ô©Q�D\1:?ABEQR3�ù � D�QY3ND�@C7�?C7$7�ä514ÓÀ×5ÔE7$Q4Ù�?CZ5BEXBEX?CZ57PúN@GXx?�Ô²1�ÕR7.@Q4Ù�687 J�7�?AéML�XAZ5Q ã 3WBE3A�+B©]Y;5@A7aWRù ��35B©?CX�BE3W?CZ57PúN@GXx?�Z5B²V5V�7\3/ÔE1�ÕY7.@Q4Ù687�J�7�?AéMLW14@C7�QR@C]Y1R35BEõ.7\VTBE3��W×5ÔE1R357\X\=i7\1RDGZ Q4Ù ã Z5B²DGZ%B²XP1�Ùç7\14?A;5@C7�Ó�1R×8ù�� ;535B©?$B©3»1Ùç7\1:?C;5@C7�Ó�1R×ÖZ(1RX "�LWBE35×5;�?GXnD�QY35357\D�?A7MV ?CQ�1 L´åeÕ LT14@C7\1´BE3»?CZ57�BE35×5;�?M=�D.1RÔ©ÔE7\V»?AZN7âAÝGïCÝ�üiá�óêÿ:Ý�ôÝ�ò � Q4Ù?AZ57�;53NBz?Mù��21YDGZ»;N35Bz?�ZN1RX "�LWBE35×5;5?CX\=¸143(V ?CZ57.@C7�ÙçQY@A7 "�LI?C@C1RB©3(14å5ÔE7D�Qe74RID�BE7.3[?GXn×5ÔE;NX�1/?A@G14BE3N14åNÔ©7IåNBE1YX.ù+ÒuZ57W@C7\D.7.×�?CB©ñY7ÀúN7.Ô²V5X�Q4Ù�D�QY3Y?CB©]Y;5QR;(Xn;535B©?CX�BE3Ú1Ùç7\1:?C;5@C7IÓ�14×[email protected][?A7\@A7MVÛQR3bD.QR@C@A7MXU×iQR3NV5B©35]YÔ©Õ»D�QR3[?CB©]Y;5QR;NXn;535B©?CX�B©3Ø?CZ57W×N@A7\ñ[BEQR;(XÔE1�ÕY7.@Mù+ÒuZ57\@A7.ÙçQR@C7I@C7\D�7\×�?ABEñR7´úN7\ÔEV5X�QRÙ�357.BE]RZeåiQR@CB©3N] ;535B©?CX�Q:ñY7.@CÔE1R×8ù��NQR@À7�ä514ÓÀ×5ÔE7R=�BE3?AZ57�úN@CXU?�Z5B²V5V�7\3%ÔE1�ÕY7.@�QRÙ"6�7�J�7�?Ué�L�=N?AZ57�@A7MD�7\×�?ABEñR7PúN7\ÔEVNX�Q4Ù"Z5QY@ABEõ.QY3[?C14ÔEÔEÕWD�QY3Y?CB©]Y;5QR;(X;535B©?CX�Q:ñR7\@AÔ²14×$åeÕ��PD�QYÔ©;5ÓÀ3NX�1R3NV3L�@AQ ã X\ù �JX"Xx?G1:?C7\Vn7M14@CÔ©BE7.@M=�14ÔEÔY?CZ57;535B©?CX�BE3�1�Ùç7\1:?C;5@C7ÓÀ1R×WXAZN14@C7u?AZ57µXA1RÓÀ7�XU7.?<QRÙ�"�L ã 7.BE]RZ[?CX2143NVÀ?CZ57JXC14ÓÀ7�åNBE1YX2XAQ$?AZ57\Õ�V�7�?C7\D�?<?AZ57PXC14ÓÀ7

Page 5: yann.lecun.comyann.lecun.com/exdb/publis/pdf/lecun-99.pdf · 1)(+*!! 7\@ "! $ '

Ùç7\1:?C;5@C7u14?-14ÔEÔ�×(Q[XAXAB©åNÔ©7uÔEQeD\1:?CB©QY3NX+QR3�?AZ57�BE35×5;�?MùRÒuZ57uQ4?CZ57.@�Ùç7M1:?C;5@A7uÓ�14×(X�B©3�?AZ57�Ô²1�ÕR7\@;[email protected][?�XA7�?CX�Q4Ù ã 7\B©]YZY?GX�143NV´å5B²1RXA7\X\=e?AZ57\@A7\åeÕW7�äe?C@C1YD�?ABE35]�V�B©Ki7\@A7\3[?�?xÕ[×i7\X�Q4Ù�Ô©QRéD.14ÔLÙç7M1:?C;5@A7MX.ù���3T?AZN7PD\1RXA7JQRÙ+6�7�J�7.?Ué�L�=514?u7M1RDGZTB©3N×5;�?�Ô©Q�D\1:?ABEQR3/XABzä´[email protected][?u?xÕe×(7MXuQ4ÙÙç7\1:?C;5@C7\X14@C7�7�äe?C@C1YD�?A7MV�åeÕ�XUB©ä�;535B©?CXuBE3IBEV57.3[?AB²D.1RÔiÔEQ�D.14?ABEQR3NX<B©3W?AZ57PXABzäÀÙç7\14?A;5@C7�Ó�1R×NX.ù�ÍXA7 �[;57\3[?AB²14ÔLB©ÓÀ×5ÔE7.ÓÀ7\3Y?G1:?CB©QY3´QRÙ+1�Ùç7\14?A;5@C7JÓ�1R× ã QR;5Ô²VWXCD.1R3W?CZ57PB©3N×5;�?�B©Ó�1R]R7 ã Bz?CZ1 XAB©35]YÔ©7W;535B©?�?CZN1:?�ZN1RX�1�ÔEQeD\14Ô@A7MD�7\×�?ABEñR7�ú(7.Ô²V =+1R3NVÚXU?AQY@A7�?CZ57´XU?C14?A7MX�QRÙ�?AZ5B²X�;N35Bz?1:?�D�QY@A@C7\XA×iQR3NV�BE35]TÔ©Q�D\1:?ABEQR3(XPB©3Ö?AZN7ÀÙç7\1:?C;5@C7�ÓÀ1R×8ù�ÒuZ5B²X$QY×(7\@C14?ABEQR3ÖBEXP7 �Y;NB©ñ:14ÔE7.3[?$?CQ1�D�QR3eñYQRÔE;�?ABEQR38=8ÙçQRÔEÔ©Q ã 7\V»åeÕÖ1R3Û1YV5V�B©?ABEñR7�å5B²1RX�143NVÛX �[;N1RXAZ5BE35]/Ùç;53ND�?ABEQR38=¸Z57.3ND.7�?AZN73N14ÓÀ7 ïGð:à(ÿ�ð4òsí�á�ó÷ð4àiì4ò�àiÝ.á)(2ð:â+* ùNÒuZ57PFR7\@A357\Ô Q4Ù+?AZ57nD.QR3eñRQYÔ©;5?ABEQR3WB²X?AZN7nXU7.?�Q4Ù"D�QR3N357\D�é?ABEQR3 ã 7.BE]RZ[?CX$;(XU7MVÛå[Õ ?AZ57´;535B©?CX�B©3Û?CZ57�Ùç7\14?A;5@C7IÓ�14×�ù ��3ØBE3[?A7\@A7MXx?CB©35]�×N@AQY×(7\@U?xÕ»Q4ÙD�QR3eñYQRÔE;�?ABEQR3N1RÔ5Ô²1�ÕR7.@GX"B²X-?CZN1:?BzÙ�?AZ57JB©35×N;�?BEÓ�14]Y7�B²X<XAZ5B©Ùê?A7\V8=R?CZ57�Ùç7M1:?C;5@A7JÓ�14×�QR;5?A×5;�?ã B©ÔEÔ�åi7�XAZ5BzÙê?C7\V�åeÕW?AZ57�XC14ÓÀ7�1RÓÀQR;53[?\=Nå5;�? ã BEÔ©Ô¸å(7�Ô©7.Ùê?�;53NDGZ(1435]Y7\VTQ4?CZ57.@ ã BEXA7Rù(ÒuZ5BEX×5@AQY×(7\@U?xÕ�BEX<1:?<?AZ57�å(1RXABEX2Q4ÙL?AZ57J@AQYå5;NXU?A357MXAX-Q4Ù8D�QY3[ñYQRÔE;�?ABEQR3(14Ô5357.? ã QY@AF�X+?CQ�XUZNBzÙê?GX<1R3NVV�BEXU?AQY@U?CB©QY3NXuQ4Ù¸?AZ57$BE35×5;5?\ù

!J3ND�7µ1PÙç7\14?A;5@C7JZ(1RX-åi7.7\3´V�7.?A7MD�?A7MV =eBz?GX7�ä51RD�?<ÔEQeD\1:?CB©QY3�å(7MD�QRÓÀ7MX2ÔE7\XCX2BEÓÀ×(QY@U?G143[?\ù!J35Ô©ÕÀB©?CXu14×N×5@AQ�ä�BEÓ�1:?A7�×(Q[XUB©?ABEQR3I@C7.Ô²1:?CB©ñY7�?AQ�Q4?CZ57.@<Ùç7\14?A;5@C7\X<B²X2@C7.ÔE7.ñ:143[?Mù��JXUBE35]�Z(143NVeéã @AB©?U?C7.3»V5B©]YBz?GXµ1YXP143 7.ä514ÓÀ×5ÔE7R=iQR3ND.7 ã 7�Fe35Q ã ?CZN1:?P?CZ57�BE35×5;5?PBEÓ�14]Y7�D�QY3[?C14BE3NXJ?AZN77.3NV�×iQRBE3[?�Q4Ù�1�@CQR;N]RZ5ÔEÕ�Z5QR@CBEõ.QR3[?G14Ô XU7\]RÓÀ7.3[?�BE3´?CZ57P;5×5×i7.@�ÔE7�Ùê?�1R@A7M15=�1ÀD�QY@A357\@uB©3T?AZN7;5×5×i7.@�@CBE]RZ[?�14@C7\1N=8143NV»?AZN7I7.3NV5×(QYB©3[?�QRÙ�1/@CQR;N]RZ5ÔEÕ%ñR7\@U?CBED\14Ô2XA7.]YÓÀ7.3[?nB©3Û?CZ57IÔEQ ã 7\@×(QY@U?CB©QY3´QRÙ¸?CZ57$B©Ó�1R]R7R= ã 7$D.1R3T?A7.ÔEÔ ?CZ57$B©3N×5;�?�BEÓ�14]R7PB²X�1%jeù�J�Q4?�QR3NÔ©Õ´BEXu?CZ57$×5@C7\D.BEXA7×(Q[XUB©?ABEQR3/Q4Ù�7\1YDGZ´QRÙ¸?CZ5QYXA7PÙç7\14?A;5@C7\XuBE@C@A7\Ô©7\ñ�1R3[?ÙçQY@�BEV57.3[?AB©ÙçÕeB©35]I?AZ57$×N14?U?C7.@C38=5B©?�BEX�×iQ4é?A7.3[?CBE1RÔ©ÔEÕ´Z(14@CÓ�Ùç;NÔ�åi7\D.1R;NXA7$?AZ57�×(Q[XUB©?ABEQR3NXJ14@C7$Ô©BEFR7\Ô©Õ´?AQWñ�1R@AÕWÙçQR@�[email protected][?JBE3NXU?C1R3ND�7MXQ4Ù5?CZ57�XUZN1R×(7Yù � XABEÓ�×NÔ©7 ã 1�ÕJ?CQJ@C7\V5;ND�7<?AZ57u×5@C7\D.BEXABEQR3 ã B©?AZ ã Z5B²DGZn?AZN7×iQYXAB©?ABEQR3�QRÙ(V�B²Xxé?ABE3ND�?CB©ñY7�Ùç7\14?A;5@C7\X14@C7�7\3ND�Q�V�7MV�B©3´1PÙç7\14?A;5@C7�Ó�14×WBEX2?AQ�@A7MV�;ND�7�?AZ57PXU×(1:?AB²14Ô(@A7MXUQYÔ©;5?ABEQR3Q4Ù?AZ57�Ùç7M1:?C;5@A7�Ó�1R×8ù�ÒuZ5B²XnD.1R3Ûå(7W1YDGZ5BE7.ñR7MV ã B©?AZÚ1/XAQ4é�D.1RÔ©ÔE7\V Ü�ín? ��Ü.ì:ß�üiòzóêà[Þ»ò©ì��RÝ.âGÜã Z5BEDGZØ×i7.@AÙçQR@CÓ�X�1�ÔEQ�D.1RÔ<1�ñR7\@C1R]RBE35]/143NVØ1%XA;5å�é�XC14ÓÀ×5ÔEB©35](=�@C7\V�;(D�BE35]/?AZ57´@A7MXUQYÔ©;5?ABEQR3Q4Ù�?CZ57PÙç7\14?A;5@C7$Ó�14×8=51R3NV´@A7MV�;ND�BE35]�?AZN7nXU7\3NXUB©?ABEñeBz?xÕWQ4Ù+?AZ57$QY;�?A×5;5?�?AQ�XUZNBzÙê?GX�143NVTV�B²Xxé?AQR@A?ABEQR3(X.ùeÒuZ57$XU7MD�QY3NV�Z5B²V5V�7\3´Ô²1�ÕR7\@2Q4Ù+687�J�7�?AéML�BEXu1�XU;Nå�é�XC14ÓÀ×5ÔEBE35]�Ô²1�ÕR7\@\ùeÒuZ5B²X<Ô²1�ÕR7\@D�QRÓÀ×5@CB²XU7MXXAB©ä�Ùç7\1:?C;5@C7JÓ�14×(X.=�QY357�ÙçQY@u7\1RDGZWÙç7\14?A;5@C7PÓÀ1R×WBE3W?CZ57P×5@C7.ñeB©QY;NXÔE1�ÕY7.@Mù[ÒuZN7@A7MD�7.×5?ABEñR7�ú(7.Ô²VIQ4Ù¸7\1RDGZW;535B©?uB²Xu13"�åeÕ "n14@C7\1�B©3W?AZ57µ×5@C7.ñeBEQR;NX<ÔE1�ÕY7.@[M X<D�QY@A@C7\XA×(QY3NV�BE35]Ùç7\1:?C;5@C7´Ó�14×�ù��21RDGZb;535B©?�D.QRÓÀ×5;�?C7\X�?CZ57 ì:ÿ:Ý�âAì\ÞYÝ Q4Ù�B©?CX�ÙçQY;5@ÀB©35×N;�?CX\=-Ó�;NÔz?CB©×5ÔEBE7\XÀB©?å[Õ/1�?A@G14BE3N1Rå5Ô©7�D�Qe74RID�BE7.3[?M=(1RVNV5X�1À?C@C1RB©3N1Rå5ÔE7$å5BE1YX.=i143NV/×(1RXCXU7MX?AZ57�@A7MXU;5Ô©?�?CZ5QR;5]YZ�1XUBE]RÓÀQRB²VÖÙç;53ND�?ABEQR38ù-9<QR3[?CB©]Y;5QR;NX�;535B©?CX�ZN1�ñR7W35QR3�é�Q:ñR7\@AÔ²14×N×5B©3N]TD.QR3[?ABE]R;5QY;NXn@C7\D.7.×�?CB©ñY7úN7.Ô²V5X\ù+9<QR3NXA7 �[;57\3[?AÔEÕR= 1/XU;5å5é�XC14ÓÀ×5ÔEB©3N]TÔ²1�ÕR7\@µÙç7\1:?C;5@C7�Ó�1R×»ZN1YXPZN14Ô©Ù?AZ57�3e;5Ó�å(7\@$Q4Ù@AQ ã X<143NV´D�QRÔE;5ÓÀ3NXu1RX2?AZN7�Ùç7M1:?A;N@A7µÓ�14×NX<B©3W?AZ57µ×5@C7.ñeBEQR;NX<ÔE1�ÕY7.@MùYÒuZ57J?A@G14BE3N1Rå5Ô©7µD�Qe7�Ùêéú(D�BE7.3[?µ1R3NV�å5B²1RXJD�QY3Y?C@AQYÔ8?CZ57�7.Ki7MD�?PQ4Ù�?AZ57ÀXAB©]YÓ�QYBEV/3NQR3�é�Ô©BE357M14@CBz?xÕYù ��Ù"?AZN7�D�Qe76R�D.B©7\3[?BEX�XUÓ�14ÔEÔ�=8?AZN7.3Ø?CZ57W;53NBz?�QR×i7.@G1:?A7MX$BE3b1 �[;N1RXAB©é�ÔEB©3N7\14@$ÓÀQ�V�7Y=+1R3NVÖ?AZ57TXU;Nå�é�XC14ÓÀ×5ÔEBE35]ÔE1�ÕY7.@"ÓÀ7.@C7.ÔEÕnå5ÔE;5@GX�?AZ57�B©35×N;�?\ù ��Ù(?CZ57�D�Qe76R�D.B©7\3[?-BEX"Ô²14@C]R7Y=4XU;Nå�é�XC14ÓÀ×5ÔEBE35]P;535B©?CX2D\143Àåi7XU7\7.3%1RX�×(7\@UÙçQY@AÓÀBE35]I1 U35QYBEXAÕ ! � � QR@�1 A35QRB²XUÕ � J $ � Ùç;N3ND�?CB©QY3%V�7.×i7.3NV5B©35]WQR3/?AZN7ñ�1RÔ©;N7�Q4Ù?AZ57�å5B²1RX\ù'�e;ND\D�7MXAXAB©ñY7ÀÔE1�ÕY7.@GXJQRÙuD.QR3eñRQYÔ©;5?ABEQR3NXP143(VÖXA;5å�é�XC14ÓÀ×5ÔEB©35]/14@C7�?xÕe×5BzéD.14ÔEÔEÕn14Ô©?A7.@C3N14?A7\V8=\@C7\XA;5Ô©?ABE35]JBE3�1 UåNBzé�×eÕ[@G14ÓÀB²V � ûM1:?"7\1YDGZnÔ²1�ÕR7.@M=\?AZ573e;5Ó�åi7.@�Q4ÙNÙç7\1:?C;5@C7ÓÀ1R×NXJBEXµBE3ND�@C7\1YXU7MV%1RX�?AZ57ÀXA×N1:?CBE1RÔ¸@C7\XAQRÔE;�?ABEQR3 BEXPV�7MD�@C7\1RXA7\V8ù�21YDGZ%;535B©?$B©3 ?AZ57�?AZNB©@GVZ5BEVNV�7.3»ÔE1�ÕY7.@�BE3�úN]Y;5@C7%W�Ó�1�Õ/ZN1�ñY7nB©3N×5;�?$D�QR3N357\D�?ABEQR3NX�Ùç@AQYÓ XU7\ñR7.@G14Ô8Ùç7\1:?C;5@C7�Ó�14×NXB©3Ø?CZ57W×N@A7\ñ[BEQR;(XnÔ²1�ÕR7.@Mù�ÒuZ57´D�QR3eñYQRÔE;�?ABEQR3��4XU;Nå�é�XC14ÓÀ×5ÔEBE35]�D�QRÓ�å5BE3N1:?CB©QY38=�BE3NXA×5B©@C7\VÛåeÕH�;5åi7.Ô-1R3NV � BE7\XA7.Ô�M Xµ35QR?ABEQR3NXJQ4Ù�CXUBEÓÀ×5Ô©7 � 1R3NV CD�QRÓÀ×5ÔE7�ä � D.7.ÔEÔEX\= ã 1YX�BEÓÀ×5Ô©7\ÓÀ7.3[?A7MV

Page 6: yann.lecun.comyann.lecun.com/exdb/publis/pdf/lecun-99.pdf · 1)(+*!! 7\@ "! $ '

B©3 �5;5Fe;NXAZ5BEÓÀ1nM X J�7.Q�D�QY]R35B©?A@CQR3 f �5;NF[;(XUZ5BEÓ�1�143NV�=�BEÕY1RFR7aWch@i "cl�=u?CZ5QR;N]RZ 35Q ]RÔEQRå5é14ÔEÔ©Õ�XA;5×(7\@AñeB²XU7MV�ÔE7\14@C35BE35]�×N@AQ�D�7MV�;5@C7�XA;NDGZW1RX<åN1YDGFYé�×5@CQR×N1R]Y14?ABEQR3 ã 1YX21�ñ:14BEÔ²14å5ÔE7�?AZN7.38ù� ÔE1R@A]Y7�V�7.]Y@A7\7�QRÙBE3eñ:14@CBE1R3ND�7�?AQ�]Y7.QYÓ�7.?A@CBED�?C@C1R3NXxÙçQY@AÓ�14?ABEQR3NXPQRÙ<?CZ57IBE35×5;�?�D\143Ûåi71RDGZ5BE7.ñY7\V ã Bz?CZ´?CZ5B²X�×5@CQR]R@C7\XCXAB©ñY7�@C7\V�;ND�?ABEQR3/Q4Ù�XU×(1:?AB²14Ô�@A7MXUQYÔ©;�?CB©QY3´D.QRÓÀ×(7\3NXC1:?A7MVTå[Õ´1×5@AQY]R@C7\XCXUBEñR7�B©3ND.@A7M1RXA7�QRÙ8?AZ57P@CBEDGZN357\XCX2QRÙ�?CZ57P@A7\×5@A7MXU7\3[?C1:?CB©QY3 öê?CZ57P3e;5Ó�åi7.@uQ4Ù�Ùç7\1:?C;5@C7ÓÀ1R×NXGø�ù

�eB©3(D�7À14ÔEÔ�?AZ57 ã 7.BE]RZ[?CXP14@C7�Ô©7M14@C357\V ã Bz?CZ»åN1YDGF[é�×5@CQR×(14]Y14?ABEQR38=(D�QR3eñYQRÔE;�?ABEQR3N1RÔ¸3N7�?Uéã QY@AF�X"D.1R3�å(7µXU7\7.3I1YX-XUÕe3[?AZN7\XAB©õ\B©35]P?CZ57.BE@2Q ã 3ÀÙç7M1:?C;5@A7�7�äe?A@G1RD�?AQR@GX\=4143NV�?C;535BE35]$?AZN7.Ó?AQ/?AZN7�?C1YXUF 14?nZN143(V ù¸ÒuZ57 ã 7.BE]RZ[?nXAZN14@CB©3N]´?C7\DGZ53NB �[;57�ZN1YXP?AZN7�B©3[?A7\@A7MXx?CB©3N]%XUB²V�7�7�ÙêéÙç7\D�?ÀQ4ÙJ@A7MV�;ND�BE35]�?CZ57T3[;NÓ�åi7.@ÀQ4Ù�Ùç@C7.7´×N14@G14ÓÀ7�?C7.@GX.=�?AZ57\@A7\åeÕÖ@C7\V5;ND�BE35]%?CZ57 AD.1R×N1RD�éBz?xÕ � Q4Ù�?CZ57�Ó�1RDGZNB©357n1R3NV/@C7\V�;ND.B©3N]À?AZ57�]Y14×/åi7�? ã 7.7\3/?A7\XU?�7\@A@CQR@�1R3NVT?A@G14BE35BE35]�7.@C@AQY@f 6�7�9<;53#WchNi@h[l�ù¸ÒuZ57W357�? ã QR@CF%BE3ØúN]R;[email protected][?C1RB©3NX�� � L5= ���@i/D�QY35357MD�?ABEQR3(X.=8åN;�?�QR3NÔ©Õ���5= �����$?C@C1RB©3N1Rå5ÔE7JÙç@C7.7P×N1R@C1RÓÀ7�?A7\@CXåi7\D\14;NXA7$Q4Ù¸?AZ57 ã 7.BE]RZ[?�XUZ(14@CB©35](ù

�¸B©ä�7\Veé�XAB©õ\7�9<QR3eñRQYÔ©;�?CB©QY3N14Ô.J�7�? ã QR@CFeXuZ(1�ñR7$åi7.7.3%1R×5×5ÔEB©7MV´?AQIÓ�143eÕT14×5×5ÔEB²D.1:?CB©QY3NX\=14ÓÀQR35]�Q4?CZ57.@GX.ûeZN1R3NV ã @CBz?CB©3N]�@C7\D�QY]R35B©?ABEQR3Ff 6�7�9<;53/7�?�1RÔ�ùGW[h@h��e= =%1R@U?CB©3mWchNh��[l�=51RX ã 7\Ô©Ô1RXnÓ�1YDGZ5B©3N7�é�×5@ABE3[?A7MVÖDGZ(14@G1RD�?C7.@$@C7\D.QR]R3NBz?CB©QY3 f �Û1435]I1R3NV3_Y7\1R3FW[h@h��[l�=+QR3�é�ÔEB©357�Z(143NVeéã @AB©?ABE35]À@A7MD�QY]R35B©?ABEQR3mf S7.3N]RBEQ�7.?�14Ô�ùGWchNh�Lcl�=N143NV´Ù÷1YD�7P@A7MD�QY]R35B©?ABEQR3f 6¸1 ã @A7\3ND�7µ7�?�1RÔ�ùGW[h@hkj6l�ù��+B©ä�7\Veé�XUBEõ.7-D.QR3eñRQYÔ©;5?ABEQR3N1RÔM3N7�? ã QR@CFeXi?CZN1:?+XUZN1R@A7 ã 7.BE]RZ[?CX 1RÔ©QY35]1�XUBE35]RÔE7�?A7\ÓÀ×(QY@C1RÔRV�BEÓ�7\3NXAB©QY3P1R@A7�Fe35Q ã 3$1RX�ÒuB©ÓÀ7�éM$J7.Ô²1�Õ J�7\;5@C1RÔ�J�7.? ã QY@AF�X"ö�Ò $ J J�XGø143NVÀ14×N×5Ô©BE7\V ã B²V�7\Ô©Õ�B©3IXA×(7\7\DGZÀ×5@CQ�D�7\XCXAB©35]$1R3NV�?ABEÓÀ7�é�XU7\@ABE7\X�×5@C7\V5BED�?ABEQR38ù��-14@CBE1Rå5Ô©7.é�XABEõ.7D�QR3eñYQRÔE;�?ABEQR3N1RÔY3N7�? ã QR@CFeX\= ã Z5B²DGZ�ZN1�ñY71R×5×5ÔEBED\1:?CB©QY3NX�B©3�QRå5æx7\D�?-V�7.?A7MD�?ABEQR3�1R3NV�ÔEQ�D.14?ABEQR314@C7PV57\XCD�@CB©åi7\V´B©3�XA7\D�?ABEQR3��5ù

,.- , ��@ ?A@ ;��ÒuZ5BEXÀXA7\D�?ABEQR3bV�7MXAD.@ABEåi7\X�B©3 Ó�QY@A7TV�7�?G14BEÔ?AZ57/14@GDGZ5B©?A7\D�?A;5@C7WQRÙ�687 J�7�?AéML5=+?AZ57%9<QR3eñRQRéÔ©;�?CB©QY3N14Ô J�7\;5@C1RÔ�J�7.? ã QY@AF»;NXA7\VÛBE3Û?AZ57I7.ä�×(7\@ABEÓÀ7.3[?CX\ù¸6�7�J�7.?Ué�L�D�QYÓÀ×5@AB²XA7\X�jTÔ²1�ÕR7\@CX\=35Q4?�D�QY;53[?ABE35]%?CZ57´QY;�?A×N;�?\="14ÔEÔQ4Ù ã Z5B²DGZ D.QR3[?C1RB©3Ú?A@G14BE3N14å5ÔE7I×N1R@C1RÓÀ7�?A7\@CX´ö ã 7.BE]RZ[?CXGø�ùÒuZ57�BE35×5;5?nBEXP1���"�ä�� "À×5B©äe7\Ô�B©Ó�14]Y7Rù ��35×5;5?$XAZN1R×(7MXµXAZ5QY;5ÔEV å(7�XAB©]Y35B©ú(D.1R3Y?CÔ©Õ�XAÓ�14ÔEÔ©7\@?AZN1R3%?AZ(1:?�öç7Rù ]NùLQY3%?AZN7�QY@CV�7\@�QRÙ "��4ä�"���×5B©ä�7.Ô²XCø�ùLÒuZN7�@C7\1YXUQY3�BEXJ?AZ(1:?PBz?PBEXPV�7MXUBE@G14å5ÔE7?AZN14?�×iQ4?C7.3[?AB²14ÔuV�B²Xx?CB©3ND�?ABEñR7´Ùç7\14?A;5@C7\X�XU;(DGZb1RX�7.3NV�é�×iQRBE3[?CX�QR@ÀD�QR@C357\@�D.1R3b14×N×(7M14@ óêàáêýNÝ$ïCÝ.àNá�Ý.â QRÙ ?AZ57J@A7MD�7\×�?ABEñR7�ú(7.Ô²V�Q4ÙL?AZN7�Z5BE]RZ57MXx?Aé�ÔE7.ñY7.ÔNÙç7\1:?C;5@C7�V�7�?C7\D�?AQR@GX\ù ��3W6�7�J�7.?Ué�L?AZ57ÀXU7.?JQRÙ2D�7.3[?C7.@GX�QRÙ"?AZ57�@A7MD�7\×�?ABEñR7$ú(7.Ô²V5X�QRÙ-?AZN7�Ô²1RXU?µD�QY3eñRQRÔE;�?CB©QY3N14Ô¸ÔE1�ÕY7.@�ö�9 �5=LXU7\7å(7\Ô©Q ã øµÙçQY@AÓ 1�"��4ä "��/14@C7\1/BE3Û?AZ57TD�7\3Y?C7.@nQRÙu?AZ57���"�ä�� "WBE35×5;5?\ù�ÒuZ57Iñ:14ÔE;57\X�Q4Ùu?AZN7B©35×N;�?�×5Bzä�7\ÔEXÀ14@C7W3NQR@CÓÀ1RÔ©BEõ.7MVÚXAQ%?AZ(1:?�?AZ57TåN1RDGFe]Y@AQY;53NVÛÔE7.ñR7\Ô$ö ã Z5B©?A7�ø�D.QR@C@A7MXU×iQR3(V5X?AQ´1Iñ�1RÔ©;N7�Q4Ù"éM�5ù.Wn143(V/?AZ57�ÙçQR@C7.]Y@AQY;53NVÛöçå5Ô²1RDGF5ø�D�QR@C@C7\XA×(QY3NV5Xu?CQ�WRù.W2j�L�ùNÒuZ5B²XJÓÀ1RFR7MX?AZ57�ÓÀ7\143»B©3N×5;�?$@AQY;5]RZNÔ©Õ �5= 143(V/?AZ57Àñ:14@CB²143ND.7n@AQY;5]RZNÔ©Õ#W ã ZNBEDGZÖ1RD.D.7.ÔE7.@G1:?C7\X�ÔE7\1R@A35éB©35] f 687:9<;538= �$1R3Y?C7.@�1R3NV ��QRÔEÔE1�WchNh W$l�ù ��3 ?AZ57/ÙçQYÔ©ÔEQ ã BE35]N=-D.QR3eñRQYÔ©;�?CB©QY3N14ÔÔ²1�ÕR7.@GXÀ14@C7ÔE1Rå(7\Ô©7MVb92äL=¸XU;5å5é�XC14ÓÀ×5ÔEB©3N]�ÔE1�ÕY7.@GX$14@C7�ÔE1Rå(7\Ô©7MV��eäL=+143NVÖÙç;NÔ©ÔEÕ[é�D.QR35357MD�?C7\VÖÔE1�ÕY7.@GX$14@C7ÔE1Rå(7\Ô©7MV �¸ä = ã Z57\@A7PäWB²X?AZN7$ÔE1�ÕY7.@uB©3(V�7�ä ù6�1�ÕR7\@À9OW/B²X�1»D.QR3eñRQYÔ©;�?CB©QY3N14ÔÔ²1�ÕR7.@ ã B©?AZ � Ùç7M1:?C;5@A7TÓ�14×NX\ù��21YDGZÚ;535B©?IBE3 7M1RDGZÙç7\1:?C;5@C7�ÓÀ1R×ÛBEXnD.QR353N7\D�?C7\V»?AQ%1�L:ä LW3N7.BE]RZeå(QY@AZNQ[Q�V»B©3Û?CZ57�BE35×5;�?Mù¸ÒuZ57WXABEõ.7ÀQ4Ùu?AZN7Ùç7\1:?C;5@C7JÓ�14×(X<B²X " i4ä�"@i ã Z5B²DGZW×5@C7.ñY7.3[?CXD�QY35357MD�?ABEQR3WÙç@AQYÓ ?AZ57µBE35×5;�?uÙç@CQRÓ Ù÷14ÔEÔEB©35]�Q4K?AZ57$åiQR;53(V514@CÕRù(9OW$D�QY3[?C14BE3NX/WGL ��?A@G14BE3N14å5ÔE7P×N14@G14ÓÀ7.?A7.@GX\=e1R3NVFW "�"5= ��� ��D�QR3N357\D�?ABEQR3NX\ù6�1�ÕR7\@ ��"JBEX-1PXA;5å�é�XA1RÓÀ×5Ô©BE35]PÔ²1�ÕR7.@ ã B©?AZ �JÙç7\14?A;5@C7�Ó�14×NX�QRÙLXABEõ.7W �RäGW �Nù��<1RDGZ�;N35Bz?B©3Ø7M1RDGZÖÙç7\14?A;5@C7�Ó�14×ÛBEX�D�QY35357\D�?A7MV ?CQ%1�"�ä "T357\B©]YZ[åiQR@CZ5QeQ�V BE3Û?AZN7WD�QY@A@C7\XA×(QY3NV�BE35]

Page 7: yann.lecun.comyann.lecun.com/exdb/publis/pdf/lecun-99.pdf · 1)(+*!! 7\@ "! $ '

Ùç7\1:?C;5@C7-Ó�14×nBE3�9OWRùMÒuZ57-ÙçQY;5@+B©35×N;�?CX�?AQµ1�;535B©?¸BE33��"�14@C721RV5V57\V =.?AZ57\3nÓ�;5Ô©?ABE×5ÔEB©7MVnåeÕP1?A@G14BE3N14å5ÔE7�D.Q[76RID�BE7.3[?\=[143NVI1YV5V�7MV�?AQ�1$?A@G14BE3N14å5ÔE7�å5B²1RX\ù[ÒuZN7�@A7MXU;NÔz?2B²X<×N1RXCXU7MV�?AZ5@CQR;N]RZ1bXAB©]YÓ�QYBEVN14ÔJÙç;53ND�?CB©QY38ù�ÒuZ57 "�ä "Û@C7\D�7\×�?ABEñR7 úN7.Ô²V5XT14@C7 35QY3�é�Q:ñR7.@CÔ²14×5×5BE35](="?AZ57\@A7.ÙçQR@C7Ùç7\1:?C;5@C7�Ó�14×NX<BE3 ��"nZN1�ñR7�Z(14Ô©Ù ?AZ57µ3e;5Ó�åi7.@Q4Ù�@CQ ã X-1R3NVWD.QRÔE;5ÓÀ3W1RX2Ùç7\14?A;5@C7�Ó�14×(X2BE39OWRù56�1�ÕY7.@ ��"�ZN1YX/W "�?A@G14BE3N14åNÔ©7P×N1R@C1RÓ�7.?A7\@CXu143(V L5= iNi���D�QR3N357\D�?ABEQR3NX\ù6�1�ÕR7\@�9 �/BEX�1�D�QR3eñYQRÔE;�?ABEQR3N1RÔ"ÔE1�ÕY7.@ ã B©?AZ W �TÙç7\1:?C;5@C7�ÓÀ1R×NX\ù �21YDGZÖ;N35Bz?�B©3Ø7M1RDGZÙç7\1:?C;5@C7$ÓÀ1R×�BEX�D�QR3N357\D�?A7\VT?CQIXU7\ñR7\@C1RÔ'L�ä L�357.BE]RZeåiQR@CZ5QeQeVNX�1:?�B²V�7.3[?CBED\14Ô8ÔEQ�D.1:?CB©QY3NX�BE3

1IXU;NåNXU7.?JQRÙ���"�M X�Ùç7M1:?A;N@A7nÓ�1R×NX.ù� ZeÕT3NQ4?µD.QR353N7\D�?J7.ñR7\@AÕA��"�Ùç7M1:?C;5@A7�ÓÀ1R×�?AQW7.ñR7\@AÕ9 ��Ùç7M1:?C;5@A7uÓ�14×���ÒuZ57�@C7\1RXAQR3�BEX+? ã Q4ÙçQYÔEV8ù��¸BE@GXx?M=41J3NQR3�é�D�QYÓ�×NÔ©7.?A7�D�QY35357\D�?ABEQR3�XADGZ57\ÓÀ7FR7.7\×NX+?AZN7�3[;NÓ�åi7.@"Q4Ù D�QY35357MD�?ABEQR3(X ã Bz?CZ5B©3À@C7\1YXUQY3N14å5ÔE7å(QY;53NV5X\ù@=�QY@A7B©ÓÀ×iQR@A?C1R3Y?CÔ©ÕY=4B©?ÙçQR@GD�7\Xu1�å5@C7\14F�QRÙ¸XAÕ[ÓÀÓÀ7�?C@AÕ�BE3´?AZ57P357.? ã QY@AFLù $�B©[email protected][?uÙç7\1:?C;5@C7JÓ�14×(X1R@A7JÙçQR@GD�7MV�?CQ7�äe?A@G1RD�?JV�B©Ki7\@A7\3Y?ÀöçZ5QY×(7.Ùç;5ÔEÔ©Õ%D.QRÓÀ×5ÔE7.ÓÀ7.3[?C1R@AÕ5øÙç7M1:?C;5@A7MX�åi7\D\14;NXA7$?AZN7.Õ/]R7.?µV�B©Ki7\@A7\3[?XU7.?CX<Q4Ù8BE35×5;�?GX.ù�ÒuZ57J@C14?ABEQR3N1RÔ©7�åi7.ZNB©3NVI?AZ57PD�QY35357MD�?ABEQR3IXCDGZ57\Ó�7JBEX2?AZN7�ÙçQYÔ©ÔEQ ã B©3N]NùYÒuZN7úN@CXU?WXABzä 9 �ÖÙç7\1:?C;5@C7�ÓÀ1R×NXÀ?G14FR7/BE35×5;�?GX�Ùç@CQRÓ 7.ñR7\@AÕ D�QR3[?CB©]Y;5QR;NX�XA;5åNXA7�?GX�Q4Ùµ?AZN@A7\7Ùç7\1:?C;5@C7´Ó�14×(X�BE3 ��"�ù"ÒuZ57T357�äe?IXABzäØ?G14FR7´B©3N×5;�?�Ùç@CQRÓè7.ñY7.@CÕØD�QR3[?CB©]Y;5QR;NX�XA;5åNXA7�?ÀQ4ÙÙçQR;5@MùYÒuZ57�357�ä[email protected]�?G14FR7�BE35×5;5?2Ùç@CQRÓ XUQYÓ�7JV�BEXCD�QY3[?ABE3[;NQR;NX-XA;5åNXA7�?GX-QRÙiÙçQY;5@Mù��+B©3(14ÔEÔ©Õ?AZ57%Ô²1RXU?IQR3N7´?G14FY7\X�BE35×5;�?WÙç@AQYÓ 14ÔEÔ ��"»Ùç7M1:?A;N@A7�Ó�14×(X.ù2ÒuZN7/Ùç;5Ô©ÔµD�QY35357\D�?ABEQR3 ?C1Rå5ÔE7BEX�]YB©ñY7.3 B©3 f 687:9<;53/7�?�14Ô�ùGWchNh@i[l�=26�1�ÕY7.@W9 �ÛZN1YXQWR= L�W � ?A@G14BE3N1Rå5Ô©7/×N1R@C1RÓÀ7�?A7\@CXÀ1R3NVW L �N= ������D�QR3N357\D�?ABEQR3NX\ù6�1�ÕR7\@ � ��BEXu1�XU;5å5é�XC14ÓÀ×5ÔEB©3N]�Ô²1�ÕR7.@ ã Bz?CZFW �$Ùç7M1:?A;N@A7JÓÀ1R×NXQ4Ù�XAB©õ\7 L:ä L�ù �21RDGZ�;N35Bz?B©3Ø7M1RDGZÖÙç7\14?A;5@C7�Ó�14×ÛBEX�D�QY35357\D�?A7MV ?CQ%1�"�ä "T357\B©]YZ[åiQR@CZ5QeQ�V BE3Û?AZN7WD�QY@A@C7\XA×(QY3NV�BE35]

Ùç7\1:?C;5@C7IÓ�14×ØBE3 9 �5=�BE3Ú1%XAB©ÓÀBEÔE1R@ ã 1�Õ»1RXÀ9OWI143NV#� �Nù¸6¸1�ÕR7\@D� �%ZN1YX���"´?C@C1RB©3(14å5ÔE7×N14@G14ÓÀ7�?C7.@GXu143NV "�= ������D.QR35357MD�?CB©QY3NX.ù6�1�ÕR7\@$9 LWB²X$1TD.QR3eñRQYÔ©;5?ABEQR3N1RÔ¸Ô²1�ÕR7\@ ã B©?AZ W "��IÙç7M1:?A;N@A7ÀÓ�14×NX\ù �21YDGZ ;535B©?nB²X$D�QR35é357\D�?A7\V´?CQI1 L:ä L�3N7.BE]RZeå(QY@AZNQ[Q�V´QR3�14ÔEÔ�W ��QRÙ�� � M XÙç7M1:?C;5@A7$Ó�1R×NX.ù(H�7.@C7R=�åi7\D\14;NXA7P?AZN7

XUBEõ.7IQRÙ � �%B²X�14Ô²XAQ L�ä L�= ?CZ57WXAB©õ\7IQ4ÙJ9 L M XPÙç7M1:?C;5@A7WÓÀ1R×NXnB²X:W.äGWRû ?CZ5BEX�14ÓÀQR;N3Y?GXP?AQ»1Ùç;5Ô©ÔD�QY35357\D�?ABEQR3Ûåi7�? ã 7.7.3#� �%1R3NVÚ9 L�ù+9 LTB²X$Ô²14åi7.ÔE7\VØ1YX$1�D�QR3eñYQRÔE;�?ABEQR3N1RÔ�ÔE1�ÕY7.@M= B©3�éXx?C7\1RVÀQRÙL1$Ùç;5Ô©ÔEÕ[é�D.QR353N7\D�?C7\VÀÔ²1�ÕR7\@\=:åi7\D\14;NXA7�BzÙ86�7�J�7.?Ué�LPBE35×5;�? ã 7.@C7�Ó�1RV�7�å5B©]Y]R7\@ ã Bz?CZ7.ñR7\@AÕ[?CZ5B©3N]�7.Ô²XU7nFY7.×�?µD�QR3(Xx?G143[?\=N?AZ57$Ùç7M1:?C;5@A7�ÓÀ1R×%V�BEÓ�7\3NXAB©QY3 ã QY;5ÔEV/åi7�Ô²14@C]R7\@u?AZN1R3W�äGWRùNÒuZNBEX�×5@AQ�D.7\XCX�Q4Ù"V�Õe3N14ÓÀB²D.1RÔ©ÔEÕ´BE3ND.@A7M1RXAB©35]�?CZ57�XAB©õ\7nQ4Ù-1�D�QY3eñRQRÔE;�?CB©QY3N14Ô 357.? ã QY@AFBEXV�7\XCD�@CBEå(7MV�B©3W?AZ57µXU7MD�?CB©QY3 �e7\D�?ABEQR3 �Nù[6¸1�ÕR7\@u9 L$Z(1RX��ki5=.W "��µ?A@G14BE3N14å5ÔE7JD.QR353N7\D�?CB©QY3NX\ù6�1�ÕR7\@'� �5=MD�QR3[?G14BE3NX`i���;535B©?CXuöê?CZ572@C7\1YXUQY3PÙçQR@�?AZ5B²X+3[;NÓ�åi7.@¸D.QRÓÀ7\X�Ùç@CQRÓ¶?AZ57V�7\XAB©]Y3

Q4Ù+?AZ57nQY;�?A×N;�?�ÔE1�ÕY7.@M=�7�ä�×5Ô²14BE357\V´Ô²1:?A7\@Gøu1R3NVTBEX�Ùç;5Ô©ÔEÕ/D�QR3N357\D�?A7\V´?CQ´9 L�ù ��?�ZN1YX/W��N=7W � �?A@G14BE3N14å5ÔE7P×N14@G14ÓÀ7.?A7.@GX\ù

��X BE3nD�Ô²1RXCXUB²D.1RÔ:357.;N@C1RÔ4357�? ã QR@CF�X.=�;N35Bz?GX BE3$ÔE1�ÕY7.@GXL;5×µ?AQ � ��D�QYÓÀ×5;�?A7<1�V�Q4?8×5@CQ�V�;ND�?å(7.? ã 7\7.3Ø?CZ57.BE@�BE35×5;5?�ñR7MD�?CQR@�1R3NVÛ?CZ57.BE@ ã 7.BE]RZ[?�ñY7\D�?CQR@M=8?CQ ã Z5B²DGZb1�å5B²1RX�BEX�1RVNV�7\V ùÒuZ5BEX ã 7\B©]YZY?C7\VTXU;NÓ)BEX�?AZ57\3T×(1RXCXU7MVI?CZ5@AQY;5]RZ/1ÀXAD\14ÔE7\V´ZeÕ[×i7.@CåiQRÔEBEDJ?C1R35]R7\3[?Ùç;N3ND�?CB©QY3?AQ�×5@CQ�V�;ND�7µ?CZ57nXx?G1:?C7µQRÙ¸?CZ57$;535B©?\ù

�¸BE3N1RÔ©ÔEÕR=�?CZ57�QR;�?C×5;�?IÔ²1�ÕR7\@�B²XID.QRÓÀ×(Q[XU7MVÚQRÙ �2;ND�ÔEBEV57\143 ��1YV�BE1RÔ�Su1RXABEX �N;53ND�?CB©QY3;535B©?CX/ö ��S �"ø�="QY357´ÙçQR@�7\1RDGZ D�Ô²1RXCX.= ã B©?AZbi�� BE35×5;�?GXÀ7\1YDGZ8ù �<1RDGZÚQY;�?A×N;�? ��S �½;N35Bz?D�QRÓÀ×5;5?A7\X�?AZ57 �2;ND�ÔEB²V�7\1R3ÎV5BEXU?C1R3ND�7�åi7�? ã 7.7\3 Bz?GX�B©3N×5;�?WñY7\D�?CQR@W1R3NV Bz?GX�×N1R@C1RÓ�7.?A7\@ñR7\D�?AQY@\ù�ÒuZN7IQR;�?C×5;�?�Q4Ù�1�×N1R@U?CBED.;5ÔE1R@ ��S �ÍD.143Øåi7WBE3[?A7\@A×5@C7�?C7\VØ1YXn1�×i7.3N1RÔz?xÕ»?A7\@AÓÓ�7M1RXA;5@CB©35]�?AZ57Jú5?uå(7.? ã 7\7.3W?AZ57PBE35×5;�?u×N1:?A?A7\@A3T143NV´1nÓÀQ�V�7.ÔLQ4Ù�?AZ57PD�Ô²1RXCX<1YXAXAQ�D�B²1:?C7\Vã Bz?CZ/?AZ57 ��S �<ù � B©ñY7.3�143�BE35×5;5?�×N1:?A?A7\@A38=N?AZ57�Ô©Q[XAXuÙç;N3ND�?CB©QY3 XUZNQR;5Ô²VTå(7�V57\XAB©]Y357\V�XAQ1RX$?CQ%]R7.?�?CZ57´D.QR3�úN]Y;5@G1:?ABEQR3ÛQRÙ � �»1RX�D.Ô©Q[XU7´1RX$×iQYXCXUBEå5ÔE7I?CQ�?AZ57´×N14@G14ÓÀ7.?A7.@�ñR7\D�?AQY@

Page 8: yann.lecun.comyann.lecun.com/exdb/publis/pdf/lecun-99.pdf · 1)(+*!! 7\@ "! $ '

Q4Ù+?AZ57 ��S �Ú?AZN14?�D�QR@C@C7\XA×(QY3NV5X2?AQÀ?AZN7$×N1:?A?A7.@C3LM X�V�7\XABE@A7MV´D.ÔE1YXAX\ù5ÒuZ57$×N1R@C1RÓ�7.?A7\@ñY7\D�é?AQR@GXJQ4Ù"?AZN7\XA7�;53NBz?GX ã 7.@C7�DGZ5Q[XU7\3%åeÕ/ZN143NV 143NV�FY7.×�?µú5ä�7\Vbö÷1:?PÔE7\1RXU?JBE35B©?AB²14ÔEÔ©Õ5ø�ùLÒuZN7D�QRÓÀ×iQR357\3[?CX�QRÙ�?AZ5Q[XU7�×N14@G14ÓÀ7�?C7.@GXuñR7MD�?AQY@CX ã 7\@A7�XA7�?�?AQIé$WnQY@�� W$?AQW×5@A7MV�7�?C7.@CÓÀB©357MVñ�1RÔ©;N7\X\ù¸ÒuZ57I×(14@G14ÓÀ7�?C7.@nñY7\D�?CQR@GXPQ4Ùu?AZN7���S ��Xn×5Ô²1�Õ%?CZ57W@CQRÔE7�Q4Ùu?G14@C]R7�?nñY7\D�?AQR@GXPÙçQY@ÔE1�ÕY7.@ � �5ùÒuZ57�XAB©ÓÀ×5ÔE7\XU?�QR;5?A×5;�?JÔ©Q[XAX�Ùç;53ND�?CB©QY3/?AZN14?µD\143/å(7�;NXA7\V ã B©?AZ/?AZN7�14åiQ:ñR7P357.? ã QY@AFBEX\û

�Iö�� øS W�

8����E1����

ö � � �$� ø ö0"Yø

ã Z57.@C7� � B²Xµ?AZ57ÀQY;�?A×5;5?nQ4Ù2?CZ57: � é�?AZ���S � ;N35Bz?M=8B�ù 7YùL?AZ57�QY357�?CZN1:?�D.QR@C@A7MXU×iQR3(V5X?AQ�?CZ57´D.QR@C@A7MD�?�D�Ô²1RXCX$QRÙuBE35×5;5?�×N14?U?A7\@A3b����ù+ÒuZ57´1RD�?C;N14Ô<Ô©Q[XAX$Ùç;53ND�?CB©QY3Ú;NXA7\VÛBE3ÚQR;N@

7�ä�×(7\@ABEÓÀ7.3[?CX¸ZN1YX¸1YV5V�B©?ABEQR3N1RÔR?C7.@CÓ½?AQ�Ó�1RFR72B©?�Ó�QY@A7<V�BEXCD�@CBEÓ�BE3N14?ABEñR7Rù2=�QY@A7V�7�?G14BEÔEX+14@C71�ñ�1RB©Ô²14åNÔ©7"BE3:f 6�7�9<;53/7.?�14Ô�ùGWch@hNi[l�ù49<QRÓÀ×5;5?ABE35]�?AZN72]R@G1RV5B©7\3Y?8QRÙe?AZ57-ÔEQYXCX Ùç;N3ND�?CB©QY3 ã Bz?CZ@A7MXU×i7\D�?�?AQT14ÔEÔ�?CZ57 ã 7.BE]RZ[?GX�BE3 14ÔEÔ¸?AZ57�ÔE1�ÕY7.@GX�Q4Ù"?CZ57�D.QR3eñRQYÔ©;5?ABEQR3N1RÔ8357�? ã QR@CFTBEXµV�QR3N7ã Bz?CZnåN1RDGF[é�×5@CQR×N1R]Y1:?CB©QY38ù�ÒuZ572XU?C1R3NV514@GVP14ÔE]RQY@AB©?AZ5Ó¶Ó�;NXU?¸åi72XAÔ©BE]RZ[?CÔ©ÕµÓÀQeV5BzúN7MVP?AQ�?C14FY71RD.D.QR;53[?�QRÙ+?CZ57 ã 7.BE]RZ[?�XAZN14@CB©3N]Nù ��3�7M1RXAÕ ã 1�Õ�?AQWB©ÓÀ×5ÔE7.ÓÀ7.3[?�B©?�B²X�?CQÀúN@CXU?JD.QRÓÀ×5;�?C7?AZ57I×(14@A?AB²14Ô-V�7\@ABEñ:1:?ABEñR7MXPQ4Ù?AZ57�ÔEQYXCXPÙç;53(D�?ABEQR3 ã B©?AZØ@C7\XA×(7MD�?$?CQ�7\1RDGZ ïGð:à(àiÝGï�á�ó÷ð:à =¸1YXBzÙ�?AZN7n357�? ã QR@CF ã 7.@C7n1�D�QY3eñR7.3[?CB©QY3N14Ô Ó�;5Ôz?CBzé�Ô²1�ÕR7.@�3N7�? ã QR@CF ã Bz?CZ5QR;5? ã 7\B©]YZ[?�XUZN1R@ABE35](ùÒuZ57.3T?AZN7µ×(14@A?AB²14Ô8V�7\@ABEñ:1:?ABEñR7MXQ4Ù+1RÔ©ÔL?CZ57nD�QY35357MD�?ABEQR3(X2?CZN1:?�XAZN1R@A7P1ÀXA1RÓ�7µ×N1R@C1RÓ�7.?A7\@14@C7P1YV5V�7MVI?CQÀÙçQR@CÓ ?AZ57nV57.@CB©ñ:1:?CB©ñY7 ã Bz?CZT@C7\XA×i7\D�?u?CQ�?CZN1:?�×(14@G14ÓÀ7�?C7.@Mù

,.- � � 4����I=����>80@���� @���2��74>< ��<:4!�#"A=�4�$�B E < ; ; @K4�% <&�7<:;�H��7\D.QR]R3NB©õ\B©35]ÛBE3NV�BEñeBEV�;(14ÔJV5B©]YBz?GX�BEXW143 7�ä5D.7.ÔEÔ©7\3Y?Iåi7.3(DGZ5Ó�14@CFØÙç[email protected]ÓÀ×N1R@ABE35]ØXUZ(14×i7@A7MD�QR]Y35B©?ABEQR3�Ó�7.?AZ5Q�V5X\ùYÒuZNBEX<D�QRÓÀ×N1R@C14?ABEñR7�XU?A;NV�Õ�D�QY3ND�7\3Y?C@C14?A7MX"QR3�1RVN14×�?CB©ñY7�Ó�7.?AZ5Q�V5X?AZN14?µQY×(7\@C14?A7�V5B©@C7\D�?AÔEÕ�QR3»XABEõ.7�é�35QY@AÓ�14ÔEBEõ.7\V/BEÓ�14]Y7\X\ùiHJ143NV ã @CB©?U?A7\3»V�BE]RB©?P@A7MD�QR]Y35B©?ABEQR3ÓÀ1�Õ XU7\7.Ó 1»Ô©B©?U?CÔ©7%XABEÓ�×NÔ©B²Xx?CBED ã ZN7.3 QR357NM XÀBE3[?A7.@C7\XU?�BEXW9<QYÓÀ×5;�?A7\@ ��B²XUBEQR3�="å5;�?�?AZN7XUBEÓÀ×5Ô©B²D�B©?xÕ�B²XJQY35ÔEÕ%14×5×N1R@A7\3[?\=L143NV%?AZ57�×5@CQRå5ÔE7.Ó�X�?AQ/XUQYÔ©ñY7�14@C7�7MXAXA7.3[?CBE1RÔ©ÔEÕT?AZ57�XC14ÓÀ71RX ã Bz?CZ�1R3eÕD"�$ÎXUZ(14×i7<@C7\D.QR]R3NBz?CB©QY38=�QR35ÔEÕP?AZ57\@A7B²X+1Rå5;53NVN143[?¸?C@C1RB©3NB©35]PV514?C1µ1�ñ�1RB©Ô²14åNÔ©7Y=143NV ?AZ57�BE3[?A@G1:é�D�Ô²1RXCXJXAZN1R×(7Àñ:14@CBE1Rå5BEÔ©B©?xÕ�BEX$D.QR3NXAB²V�7.@G14å5ÔEÕ�Ô²14@C]R7.@J?AZ(143 ã Bz?CZÛ1R3eÕ%@ABE]RB²VQRå�æx7MD�?�@A7MD�QY]R35B©?ABEQR3T×5@CQRå5ÔE7.Ó/ùÒuZ57µVN1:?C1RåN1RXA7�;(XU7MV�?AQ�?A@G14BE3W1R3NV�?C7\XU??CZ57PXUÕ�XU?A7.Ó�XuV�7MXAD.@ABEå(7MVIBE3I?CZ5BEX×N1R×(7\@ ã 1YXD�QR3(Xx?C@A;ND�?A7MV�Ùç@CQRÓ ?AZ57 J ����Ò/M X��e×i7\D.BE1RÔ7$µ1:?G14åN1YXU7 �$1R3NV�W�D�QY3Y?G14BE35BE35]På5B©3(14@CÕ�BEÓÀ1R]R7MX

Q4ÙLZN143NV ã @AB©?U?C7.3IV�BE]RB©?CX\ù��5@CQRÓ ?AZ57MXU7Y= ã 7�å5;5BEÔz?1$V51:?G14åN1YXU7�D.1RÔ©ÔE7\VB=�J ����Ò ã Z5B²DGZID�QR35é?C14BE3NX����5= �����I?A@G14BE35BE35]TXC14ÓÀ×5ÔE7\X�ö÷ZN14Ô©Ù2Ùç@CQRÓ � $�WY= ZN14Ô©ÙÙç@AQYÓ � $ �[ø�=�1R3NVbW��5= ������?C7\XU?B©Ó�14]Y7\XöçZN1RÔzÙNÙç@AQYÓ � $�Wu143(V$ZN14Ô©ÙNÙç@CQRÓ � $ �Yø�ù:ÒuZ57<QY@ABE]RBE3N1RÔYå5Ô²1RDGF$1R3NV ã Z5B©?A7$öçåNB©ÔE7.ñY7.ÔêøB©Ó�14]Y7\X ã 7.@C7�XUBEõ.7�3NQR@CÓÀ1RÔ©BEõ.7MV»?AQ/ú5?�BE3Ú1�"��4ä "��T×5B©äe7\Ô-åiQ�ä ã Z5BEÔ©7W×5@A7MXU7\@AñeBE35]T?AZ57\B©@1RXA×(7MD�?�@G1:?CB©Q(ù¸ÒuZ57´@A7MXU;NÔz?CB©35] B©Ó�14]Y7\X�D�QR3[?G14BE3Ø]R@C7.ÕÖÔ©7\ñR7.Ô²X�1YXn@C7\XA;5Ô©?�Q4Ù�1R3[?AB©é�1RÔ©B²1RXA7\V@A7MXA1RÓ�×NÔ©BE35]Nù Ò[email protected]ÀñY7.@GXUBEQR3NXµQRÙ2?AZN7IV51:?G14åN1YXU7 ã 7\@A7�;NXA7\V ù���3Ö?CZ57ÀúN@GXx?$ñR7\@CXABEQR38=i?AZN7B©Ó�14]Y7\X ã 7.@C7nD�7\3[?A7.@C7\V/BE3%1 "@i:ä " i�BEÓÀ1R]R7$åeÕ/D�QRÓÀ×5;5?ABE35]À?AZ57�D�7\3Y?C7.@�QRÙ"ÓÀ1YXAX�Q4Ù�?AZN7×5Bzä�7\ÔEX\=+1R3NVÖ?A@G143(XUÔ²1:?CB©35]%?AZ57´B©Ó�14]Y7´XAQ 1RXn?AQ»×(Q[XUB©?ABEQR3Û?CZ5BEX�×iQRBE3[?�1:?�?AZ57/D�7\3Y?C7.@�Q4Ù?AZ573"@i:ä " i�úN7\ÔEV8ù ��3%XAQRÓÀ7$B©3(Xx?G143ND.7\X\=�?AZ5B²X " i:ä "@inúN7\ÔEV ã 1RX�7�äe?A7\3NV�7\VT?CQ ��":ä���" ã Bz?CZåN1RDGFe]R@CQR;N3NV�×NBzä�7.Ô²X\ùYÒuZ5B²X-ñY7.@GXUBEQR3�Q4ÙL?AZN7�V51:?G14åN1YXU7 ã BEÔEÔNåi7�@C7�Ùç7.@C@C7\V�?AQ�1YX�?AZ57 âAÝ�Þ4í�òEì:âV51:?G14åN1YXU7Yù ��3Ú?CZ57/XU7MD�QR3(VÛñR7.@GXAB©QY3ØQ4Ù�?CZ57´V514?C1RåN1RXA7R=uöç@C7�Ùç7\@A@C7\VÖ?CQÖ1YX$?AZN7 � Ý�Ü�ò©ì4àNá�Ý �ñR7.@GXAB©QY3(ø�?AZ57�DGZ(14@G1RD�?C7.@�BEÓÀ1R]R7MX ã 7\@A7�V�7MXUÔ²143[?C7\V�;NXAB©35]P?CZ57�ÓÀQRÓÀ7\3Y?GX"Q4ÙLBE357.@A?AB²1PQ4ÙL?AZN7

Page 9: yann.lecun.comyann.lecun.com/exdb/publis/pdf/lecun-99.pdf · 1)(+*!! 7\@ "! $ '

����� ¥��N¥  ·Yd\­$¨[qs{G«<¬EvxgM­ ªxc[{�ªx{G«�ª�«�{Cª��êqs{C¬©ª��Ak(d\fe}I{C·Yd\­$¨Yqs{G«g.¬"}Yws«�ªxg.v�ªxwzg\f[«�g.¬+ªx{Gf�ª�vUd\wsfYwzfY§¨[d.ª�ªx{CvxfY«��²vxwz§\c:ª��A°

å5ÔE1YDGF$×5B©äe7\ÔEX"1R3NV�D�@CQR×5×i7\V�V�Q ã 3�?CQ "��:ä "���×5B©äe7\ÔEX�BEÓ�14]Y7\X\ù ��3�?AZN7<?CZ5BE@CV�ñR7\@CXAB©QY3�QRÙN?AZN7V51:?G14åN1YXU7Y=M;(XU7MVnBE3�XAQRÓÀ77\14@CÔEÕ$7�ä�×(7\@ABEÓÀ7.3[?CX\=M?AZ57uBEÓ�14]R7MX ã 7\@A7@A7MV�;ND.7\V$?CQBW �:äGW ��×5B©ä[é7.Ô²X.ù5ÒuZ57P@C7.]Y;5ÔE1R@VN1:?C1RåN1RXA7µBEX�1�ñ:14BEÔ²14å5ÔE7J14?��� ����������������������� ������� ���� ��!��"�#$�&% '���( ( ù

,.-*) � @ H 9>8 ;�H =�4!$ 132�� ��= E <:H 254 B < ;,+ 2 ;,+�@ E 138:= H�H <&-�@ EGH

�e7.ñY7.@G14ÔiñR7.@GXAB©QY3NX-QRÙ¸687 J�7�?AéML ã 7\@A7�?A@G14BE357\VIQR3I?CZ57µ@A7\]R;5Ô²14@V51:?G14åN1YXU7Y= ã B©?AZ´?xÕe×5BED\14ÔEÔ©Õ"���B©?A7\@C14?ABEQR3NX�?AZ5@CQR;5]YZ/?AZ57�7.3[?CB©@C7$?A@G14BE35BE35]WV514?C1NùiÒuZ57�?A7MXx?µ7.@C@AQY@�@C14?A7�XU?C1Rå5BEÔ©BEõ.7MX�1:Ùêé?A7.@n1R@AQY;53NV�W��T×N1RXCXA7\XJ?AZ5@CQR;N]RZ%?CZ57À?A@G14BE35BE35]/XU7.?$14?D�5ù h�L .Iù ÒuZ57À7\@A@CQR@µ@C14?A7ÀQR3»?AZN7?A@G14BE35B©3N]ÀXA7�?�@C7\1YDGZ57\X �5ù ��L .Ê1:Ùê?C7.@Wch�×N1YXAXA7\X\ù�ÒuZ57PB©3�/N;57.3(D�7$Q4Ù¸?AZ57P?C@C1RB©3NB©35]�XA7�?�XAB©õ\7ã 1YX$ÓÀ7\1RXA;5@C7\VÛåeÕ»?A@G14BE35B©3N]/?AZ57´357�? ã QR@CF ã B©?AZ WGL�= �����5= ���N= �����N=8143NV ���5= �����T7.ä�1RÓ�é×5Ô©7MX.ù[ÒuZ57�@C7\XA;5Ô©?CX2ÓÀ1YV�7�B©?D.Ô©7M14@-?CZN1:?1RV5V�B©?ABEQR3(14Ô5?C@C1RB©3NB©35]�V51:?G1 ã QY;5Ô²VÀå(7Jå(7\357�ú(D.BE1RÔ�ù��3%1435QR?AZ57\@�XU7.?JQRÙ�7.ä�×(7\@ABEÓÀ7.3[?CX\= ã 7�14@A?AB©ú(D�B²14ÔEÔEÕW]R7\357.@G1:?C7\VTÓÀQR@C7P?A@G14BE35BE35]I7�ä51RÓ�×NÔ©7MXå[ÕT@G143NV�QYÓÀÔ©ÕTV�BEXU?AQY@U?CB©3N]�?AZ57�QR@CB©]YB©3N1RÔ ?A@G14BE35BE35]IBEÓ�14]R7MX.ù(ÒuZ57nBE3ND�@C7\1YXU7MVW?C@C1RB©3NB©35]´XU7.?ã 1YX2D�QYÓÀ×(Q[XU7MVÀQ4ÙL?AZ57 ���N= �����PQR@CBE]RBE3N14Ô5×(1:?U?C7.@C3NX-×NÔ©;NX L�� �5= �����JBE3NXU?C1R3ND�7MX-Q4Ù8V�B²XU?AQR@A?A7MV×N1:?A?A7.@C3NX ã B©?AZ�@G143NV5QRÓÀÔ©Õ$×NBEDGFY7\V�V�BEXU?AQY@U?CB©QY3�×(14@G14ÓÀ7�?C7.@GX.ù:ÒuZ57�V�B²XU?AQR@A?ABEQR3(X ã 7\@A7�D�QYÓ�éå5B©3(1:?ABEQR3(X�Q4Ù�?CZ57$ÙçQRÔEÔEQ ã B©35]�×NÔE1R3N14@�1 R�357$?A@G143(XxÙçQY@AÓ�1:?CB©QY3NX\û�Z5QR@CB©õ\QR3[?C1RÔ8143NV/ñY7.@A?AB²D.1RÔ?A@G143NXAÔE14?ABEQR3NX\=�XAD\14ÔEB©3N]N=�X �[;57\7.õ\B©35]/ö÷XABEÓ�;5Ô©?C1R357.QY;NXuZ5QR@CBEõ.QR3[?G14ÔLD�QRÓÀ×5@C7\XCXAB©QY3´143(VWñY7.@A?AB©éD.14Ô�7\Ô©QY35]Y14?ABEQR38=:QY@�?AZ57�@C7.ñY7.@GXU7�ø�=41R3NV�ZNQR@CB©õ\QR3[?C1RÔ5XUZ57M14@CB©3N]Nù �+B©]Y;5@A7 "PXAZ5Q ã X�7�ä51RÓ�×NÔ©7MXQ4Ù+V5BEXU?AQY@U?C7\V´×N1:?A?A7.@C3NX;NXA7\VWÙçQR@?A@G14BE35BE35]Nù � ZN7.3/V�BEXU?AQY@U?C7\VTV51:?G1 ã 1RX;NXA7\VWÙçQR@u?C@C1RB©35éB©35](=M?AZ57<?A7MXx?+7.@C@AQY@�@G1:?A7V�@CQR×5×i7\VP?CQ �Nù i�. öêÙç@CQRÓ �5ù h�L . ã B©?AZNQR;�?"V�7�ÙçQY@AÓ�14?ABEQR3(ø�ùG�eQYÓÀ7Q4Ù�?CZ57PÓÀBEXCD�Ô²1RXCXUB©úN7MVI7�ä51RÓ�×NÔ©7MX1R@A7J]R7\3[;NB©357\Ô©ÕI1RÓ�å5BE]R;NQR;NX\=eå5;�?�XU7\ñR7\@C1RÔi1R@A7µ×(7\@UÙç7MD�?CÔ©ÕBEV�7\3[?AB©ú(14å5ÔE7�åeÕ�Ze;5Ó�143NX\=Y1RÔz?CZ5QR;5]YZÀ?AZ57\Õ�1R@A7 ã @AB©?U?C7.3IBE3I143I;N3NV�7.@Aé�@A7\×5@A7MXU7\3[?A7\V�XU?xÕ[ÔE7RùÒuZ5BEX�XAZ5Q ã X$?CZN1:?�Ùç;5@U?CZ57.@�BEÓ�×N@AQ:ñY7.ÓÀ7.3[?CX�14@C7À?AQ»å(7T7�ä�×i7\D�?C7\V ã B©?AZ ÓÀQR@C7�?A@G14BE35BE35]V51:?G15ù

�5QR@+?CZ57�XA1RFR7Q4Ù D�QYÓ�×(14@CBEXAQR38=:1µñ:14@CB©7.?xÕ$Q4ÙLQ4?CZ57.@+?A@G14BE3N1Rå5Ô©7�D.ÔE1YXAXABzú(7.@GX ã 1RX¸?C@C1RB©357MV143NVÖ?C7\XU?A7\VÚQR3Û?AZN7´XC14ÓÀ7IV51:?G14åN1YXU7Yù¸ÒuZ57W7.@C@AQY@$@C14?A7\X�QR3Û?CZ57I?C7\XU?�XU7.?nÙçQR@�?AZ57´ñ:14@AéB©QY;NXIÓÀ7�?CZ5Q�V5XT14@C7 XUZNQ ã 3 B©3 ú(]R;5@C7 �5ùÒuZ57 7.ä�×(7\@ABEÓÀ7.3[?CXWB©3ND.Ô©;(V�7\VÎ?CZ57%ÙçQYÔ©ÔEQ ã BE35]Ó�7.?AZ5Q�V5X\û òzóêàiÝGì:âÀï�ò©ì:ÜGÜGó ôïGì:á�ó÷ð:àWã B©?AZ W ��? ã QRé ã 1�ÕID�Ô²1RXCXABzúN7\@CX?C@C1RB©3N7\VT?AQID.ÔE1YXAXAB©ÙçÕWQR3N7

Page 10: yann.lecun.comyann.lecun.com/exdb/publis/pdf/lecun-99.pdf · 1)(+*!! 7\@ "! $ '

K−NN Euclidean

[deslant] K−NN Euclidean

40 PCA + quadratic

1000 RBF + linear

SVM poly 4

RS−SVM poly 5

28x28−300−10

28x28−1000−10

28x28−300−100−10

28x28−500−150−10

LeNet−4 / Local

LeNet−4 / K−NN

LeNet−5

−−−− 12.0 −−−−>

−−−− 8.4 −−−−>

−−−− 7.6 −−−−>

5

2.4

3.3

3.6

1.1

1.1

1

0.8

4.7

3.6

1.6

4.5

3.8

3.05

2.5

2.95

2.45

1.7

1.1

1.1

1.1

0.95

0.8

0.7

0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5

[dist] V−SVM poly 9

[dist] 28x28−300−10

[dist] 28x28−1000−10

[dist] 28x28−300−100−10

[dist] 28x28−500−150−10

[16x16] LeNet−1

[dist] LeNet−5

[dist] Boosted LeNet−4

LeNet−4

[16x16] Tangent Distance

[deslant] 20x20−300−10

Linear

[deslant] Linear

Pairwise

����� ¥��N¥  v�vxg\v"vUd.ªx{<gMf�ªxc[{-ªx{G«�ª-«�{Cª ��� �8¬²g\v-y\d.vxwsg\p[«"oGqzd\«�«�w��[o�d.ªxwsgMf�­${Aªxc[g4}Y«G°�� }R{G«�qzd\f:ª�Nwsfe}Yw®¿oGd.ªx{G« ªxced�ªLªxc[{+oGqzd\«�«�w��e{Cv8±�d\« ª�vUd\wsf[{G}µd\f[}�ªx{C«�ªx{�}Jg\fJªxc[{�}Y{G«�qzd\f:ªx{�}�y�{Cvx«�wsgMfJg\¬Yªxc[{"}Yd.ªUd\j[d\«�{M°� }Rwz«�ª�Rwsfe}Rwso�d.ªx{G«Lªxc[d.ªLªxcY{¸ª�vUd.wzfYwsf[§«�{Aª ±"d\«8d.p[§\­${Gf:ªx{�}u±-wsªxcµd.v�ªxw��eoCw©d.qsqs¯µ}Yws«�ªxg.v�ªx{�}�{A·[d.­$¨[qs{G«G°�®l�.·5l� �Lwsfe}RwzoGd.ªx{G«"ªxced�ª-ªxc[{u«�¯R«�ªx{G­½pY«�{�}�ªxcY{Pl��·Nl�µ¨[w®·R{Gq5ws­nd\§\{G«G°�_�cY{up[fYoG{Cv�ªUd\wsf:ª�¯�wsf�ªxcY{¼:pYg\ªx{�}�{Cv�vxg\v-vUd.ªx{C«<ws«2d\j�g\pYª2mR°sl���°

D�Ô²1RXCXPÙç@AQYÓ ?CZ57IQ4?CZ57.@�35B©3N7Rþ üNì:óêâ ("ó²Ü.ÝTòsóêàLÝCì4â´ï.ò©ì�ÜGÜ�ó ô<Ý.â$ã Bz?CZ � L´? ã Q4é ã 1�Õ%D.ÔE1YXAXAB©úN7.@GX?A@G14BE357\V ?AQ%D.ÔE1YXAXABzÙçÕ QR357�D�Ô²1RXCXµñR7\@CXA;NXPQR3N7�QR?AZ57\@PÙçQRÔEÔ©Q ã 7\V åeÕ»1TñRQR?ABE35]´ÓÀ7MDGZN1435B²XAÓTþ� �:&�ÝGì:âAÝ�Ü�á�&�Ý�ózÞ�ý ?Gð:âPï�òEì�ÜGÜ�ó ô<Ý.âGÜ�ã B©?AZ�1$XAB©ÓÀ×5ÔE7 �2;ND�ÔEBEV57\143ÀV�B²Xx?G143ND.7�QR3À×5B©ä�7.Ô5BEÓ�14]Y7\X\þ��� � � óêßÀÝ.à5Ü�ó÷ð4à�üiâ�óêàiï.ó üNì:ò ïCð4ß�üNð:àLÝ�à(á-ì:àiì4ò ��Ü�ó²Ü ë ð:òêòEð (2Ý � ? � � Ý�ÞRâAÝGÝ���üNð:ò �:àLð:ß�ó÷ì:ò8ï�òEì�Ü �ÜGó ôÝ�â þ âAì � ó÷ì:ò'?Gì�Ü�ó²Ü ë í�àLï�á�ó÷ð:à àiÝ.á0(2ð4â+*Àã B©?AZ W ����� � 1R;NXCXUB²143 ��S � ?A@G14BE357\V ã B©?AZ �µéÓ�7M143NX�×(7\@�D�Ô²1RXCX.=(143NV´ÙçQRÔEÔEQ ã 7MV´åeÕT1ÀÔ©BE357\1R@�D�Ô²1RXCXABzúN7\@\þ�� ì:àeÞYÝ.àNá���ó²ÜGá�ì:àLïGÝ�ï.ò©ì:ÜCÜ�ó ôÝ�â =1I357M14@C7\XU?Ué�357.BE]RZeåiQR@�D�Ô²1RXCXABzúN7\@ ã Z57\@A7n?AZ57�V�BEXU?C1R3ND�7�BEXJÓ�1RV�7�B©3eñ:14@CBE1R3[?�?AQTXUÓ�14ÔEÔ�]Y7.QRéÓ�7.?A@CBED�V5BEXU?AQY@U?CB©QY3NXµå[Õ ×5@AQRæx7\D�?ABE35]T?AZ57�×N14?U?C7.@C3»QY3Y?CQ/Ô©BE357\1R@n14×5×N@AQ�ä�BEÓÀ14?ABEQR3NXJQ4Ù?AZN7ÓÀ1R35B©ÙçQRÔ²V5X$]R7\357.@G1:?C7\V»åeÕ»V�B²Xx?CQR@A?ABE35]T?AZ57�×N@AQR?AQ4?xÕe×i7\X\þ�� íMüYü5ð4â�á��(ÝGï.á�ð4â��%ìYïCý�óêàiÝ�Ü Q4Ùñ�1R@ABEQR;(X�?xÕe×(7MX�ö÷@A7\]R;5Ô²14@ � �O=Ø=L@C7\V�;(D�7\V�é�XA7�? � �O=Ø=LñeB©@A?A;(14Ô>� �O= ø�;NXAB©35]´×(QYÔ©Õe35QYÓÀBE1RÔFR7.@C357\ÔEX\þ5ë í�òêò �»ïGð:à(àiÝGï.á�Ý � àLÝ�í�âCì:ò-àLÝ�á�ÜJã B©?AZØQY357�QR@P? ã Q�Z5B²V5V�7.3ØÔ²1�ÕR7\@CXP1R3NVÖñ�1R@ABEQR;(X3[;NÓ�åi7.@GXnQ4Ù�ZNBEV5V57.3Ú;53NBz?GX.þ � Ý &�Ý.á ��� =¸1 XUÓ�14ÔEÔ<D.QR3eñRQYÔ©;5?ABEQR3N1RÔ-357.;N@C1RÔ-357�? ã Bz?CZÚQR3NÔ©Õ" �����ÀÙç@C7.7�×(14@G14ÓÀ7�?C7.@GX�143NV�W����5= �����ÀD.QR35357MD�?CB©QY3NX.þ � Ý+&�Ý�á � � =L1WD.QR3eñRQYÔ©;�?CB©QY3N14Ô¸357.;N@C1RÔ357�? ã Bz?CZbW^je= ������Ùç@A7\7�×N1R@C1RÓÀ7�?A7\@CXJ143NV " ���5= ������D.QR353N7\D�?CB©QY3 XAB©ÓÀBEÔE1R@�?CQ´å5;5?$XAÔEB©]YZY?CÔ©ÕV�[email protected][?IÙç@CQRÓ 687 J�7�?AéML5þ �Jð\ð:ÜGá�Ý �F� Ý+&�Ý�á � =1ØD�Ô²1RXCXUB©úN7\@�QRå�?G14BE357\V åeÕbñYQ4?ABE35]Ö?AZN@A7\7B©3NXU?C1R3ND�7MXÀQ4ÙP687�J�7�?Aé �Ö?A@G14BE357MVÚQR3ÎV�B©Ki7\@A7\3[?WXU;5å(XU7.?CX�Q4Ùµ?AZ57�V514?C1RåN1RXA7Rþ"143(VÚúN3(14ÔEÔ©Õ� Ý &�Ý.á �� ù9<QR3ND.7.@C35B©3N]�Ùç;5ÔEÔEÕYé�D�QY35357MD�?A7MVT357.;5@G14Ô 3N7�? ã QR@CFeX\=eBz?�@C7.Ó�1RB©3NX�XAQRÓÀ7 ã ZN1:?�Q4Ù"1�Ó�Õ�XUé

?A7.@CÕ�?CZN1:?�;53(Xx?C@A;ND�?A;5@C7\V´357\;5@C1RÔ 357�?GX ã Bz?CZ�XU;NDGZ/1�ÔE1R@A]Y7J3e;5Ó�å(7\@�Q4Ù+Ùç@A7\7P×N14@G14ÓÀ7�?C7.@GXÓÀ1R3N14]Y7I?CQÖ1YDGZ5B©7\ñR7´@A7M1RXAQR3N1Rå5Ô©7W×(7\@UÙçQY@AÓ�143(D�7Rù �Ö7TD.QR3:æx7MD�?C;5@A7´?AZ(1:?À?AZ57/V�Õe3N1RÓ�B²D.X

Page 11: yann.lecun.comyann.lecun.com/exdb/publis/pdf/lecun-99.pdf · 1)(+*!! 7\@ "! $ '

Q4Ù¸]Y@C1YV�BE7.3[?�V�7\XCD�7\3Y?�ÔE7\1R@A35BE35]�B©3/Ó�;5Ôz?CB©Ô²1�ÕR7\@u357�?GX�ZN1RX�1 CXU7\ÔzÙêé�@A7\]R;5Ô²14@CBEõ\1:?CB©QY3 � 7�KL7\D�?\ùS<7MD.1R;NXU7T?CZ57%QR@CBE]RBE3 QRÙ ã 7.BE]RZ[?WXA×N1RD.7�BEXW1ØXA1YV5V�ÔE7�×(QYB©3[?I?CZN1:?IB²XW14?U?A@G1RD�?ABEñR7/BE3 14Ô©éÓ�Q[Xx?J7.ñR7\@AÕ/V�BE@A7MD�?CB©QY38=5?CZ57 ã 7.BE]RZ[?CX�B©3eñ:14@CBE1Rå5ÔEÕ´XAZ5@ABE35F/V�;5@CB©3N]�?AZ57�úN@GXx?�Ùç7 ã 7.×iQ�DGZNX.ù�eÓ�14ÔEÔ ã 7.BE]RZ[?CXµD.1R;NXA7n?AZN7�XAB©]YÓ�QYBEVNX�?CQWQR×i7.@G1:?C7�BE3�?AZN7 �[;N1YXUB©é�ÔEBE357\1R@�@C7.]RBEQR3�=NÓ�14FeBE35]?AZ57�3N7�? ã QR@CFn7\XCXU7\3[?AB²14ÔEÔ©Õn7 �[;5B©ñ:1RÔ©7\3Y?"?CQn1PÔ©Q ã é�D.1R×N1RD.Bz?xÕY=4XUBE35]YÔ©7.é�Ô²1�ÕR7\@¸357.? ã QY@AFLù �JX"?AZN7Ô©7M14@C35B©3N]I×[email protected].=(?AZN7 ã 7.BE]RZ[?CX�]R@CQ ã = ã Z5B²DGZ%×5@CQR]Y@A7MXAXABEñR7.ÔEÕIBE3ND�@C7\1YXU7MX�?CZ57�7.Ki7MD�?CB©ñY7D.14×(1RD�B©?xÕnQRÙi?CZ57�357.? ã QY@AFLù4ÒuZ5B²X2XA7.7\Ó�X�?AQ$åi7�143I1RÔ©ÓÀQ[Xx?-×i7.@AÙç7\D�?\=RB©ÙLÙçQR@A?A;5B©?AQY;NX\=4B©ÓÀ×5ÔE7�éÓ�7\3[?C1:?CB©QY3´QRÙ �-1R×535BEFVM X C�[?A@C;ND�?C;5@G14Ô ��B²XUF3=�B©3NB©ÓÀBEõ\1:?CB©QY3 � ×5@CB©3(D�BE×5Ô©7�f �-14×53NB©FFWch@h Lcl�ùÒuZ57D�e;5×5×iQR@A? �"7\D�?AQY@9=%1RDGZNB©357�f �-14×53NB©FFWch@h Lcl�ZN1YX�7�ä5D�7\Ô©ÔE7.3[?�1YD.D�;N@C1YD�ÕR= ã Z5B²DGZ/BEXÓ�Q[Xx?µ@A7\Ó�14@CF�1Rå5ÔE7R=Nåi7\D\14;NXA7n;53NÔ©BEFR7�?AZ57�Q4?CZ57.@JZ5B©]YZ%×i7.@AÙçQR@CÓÀ1R3ND�7�D�Ô²1RXCXUB©úN7.@GX\=NB©?µV�Qe7\X

35Q4?�BE3ND�ÔE;NV�7 ìµüLâ�ó÷ð:â�ó F[3NQ ã Ô©7MV�]R7µ14åiQR;�?u?AZN7J×5@CQRåNÔ©7\Ó f S;5@C]R7MX�143NV ��DGZ��QYÔ©FYQR×�Ù�Wch@hJj6l�ù��3»Ù÷1YD�?M= ?AZ5B²XnD.ÔE1YXAXABzú(7.@ ã QR;5Ô²V»V5QÀæx;NXx?�1YX ã 7.ÔEÔ"BzÙu?AZN7�B©Ó�14]Y7À×5Bzä�7\ÔEX ã 7.@C7À×(7\@AÓ�;�?A7MVã Bz?CZ´1$úNäe7MV�Ó�14×5×NB©35]�143NVWÔ©Q[Xx?<?AZ57\B©@×5B²D�?CQR@CBE1RÔiXU?A@C;ND�?C;5@C7RùYH�Q ã 7\ñR7\@\=Y@A7M1RDGZ5BE35]nÔE7.ñY7.Ô²XQ4Ù2×i7.@AÙçQR@CÓ�143ND.7�D�QYÓÀ×N14@G14å5ÔE7n?CQW?CZ57W9<QR3eñRQYÔ©;5?ABEQR3N1RÔ�J�7\;5@G14Ô�J�7.? ã QY@AF�XµD.143»QR35ÔEÕ�åi7V�QR357�1:?JD.QR3NXABEV57.@G14å5ÔE7P7�ä�×(7\3NXA7nB©3 Ó�7\ÓÀQR@CÕ´143(V�D�QRÓÀ×5;5?C1:?CB©QY3N14Ô�@A7 �Y;NB©@C7.ÓÀ7.3[?GX.ùNÒuZN7D�QRÓÀ×5;5?C1:?CB©QY3N14ÔL@C7 �[;5BE@A7\ÓÀ7.3[?CX<Q4Ù�S<;N@A]Y7\XcM X<@A7MV�;ND.7\Veé�XU7.? � �O= 14@C7 ã B©?AZ5BE3/1nÙ÷1YD�?AQY@uQ4Ù? ã Q´Q4Ù2687 J�7.?Ué�L�=L143NV%?AZ57�7.@C@AQY@�@G1:?A7�BEXJñR7\@AÕ/D�ÔEQYXA7Rù���ÓÀ×5@CQ:ñR7.ÓÀ7\3Y?GX�QRÙ"?AZ5Q[XU7�@A7MXU;NÔz?GX14@C7µ7.ä�×(7MD�?A7MV =N1YX?AZ57P?C7\DGZ535B �[;57PBEX�@A7\ÔE14?ABEñR7.ÔEÕ�357 ã ùS<QeQ[Xx?C7\VT687�J�7�?Aé �À×i7.@AÙçQR@CÓ�7MVWåi7\XU?\=(1RDGZ5BE7.ñeBE35]�1�XCD�QY@A7µQ4Ù��5ù j�.W=5D.Ô©Q[XU7\Ô©Õ�ÙçQYÔ©ÔEQ ã 7\Vå[Õ 687�J�7�?AéMLÖ1:? �Nù i�.Iù-SQeQYXU?A7MV 6�7�J�7�?Ué�� f $J@C;NDGFR7\@\=���DGZN1R×5BE@A7$143(V �eBEÓ�14@GVmWch@h��[l�BEX

åN1RXA7\VÖQR3Û?CZ57.QY@A7.?AB²D.1RÔ ã QY@AF å[Õ �$ù>�5DGZN14×5BE@C7�f ��DGZ(14×5BE@A7BW[h@h��[l�ù¸ÒuZN@A7\7I687�J�7�?Aé �[Xn14@C7D�QRÓ�å5BE357\V û�?AZ57IúN@CXU?nQR3N7IBEX$?A@G14BE357\VÖ?CZ57I;NXA;N14Ô ã 1�ÕYù ?AZ57´XU7MD�QR3(V»QY357�B²X$?A@G14BE357MV»QY3×N1:?A?A7.@C3NX?CZN1:?�14@C7JúNÔz?C7.@C7\V´åeÕ�?AZ57µúN@GXU?u3N7�?�XUQ�?AZ(1:?�?AZN7PXA7\D.QR3NVWÓ�1RDGZ5BE357$XU7\7\Xu1�ÓÀBzäQ4Ù8×N14?U?A7\@A3(X.=KL�� . Q4Ù ã ZNBEDGZ�?CZ57�ú(@CXU?<357.?<]YQ4?2@CB©]YZ[?\=Y1R3NV L�� . Q4Ù ã Z5B²DGZIB©?<]RQR? ã @CQR35](ù�¸BE3N1RÔ©ÔEÕR=M?AZN72?CZ5B©@GV$357.?+B²X¸?A@G14BE357\VnQR3n357 ã ×N1:?A?A7\@A3NX+QY3 ã Z5B²DGZ$?AZ57<úN@GXx?�143NVn?AZ57uXA7\D�QY3NV357�?GX�V�B²XA1R][email protected]ù $J;5@CB©35]�?C7\XU?ABE35]N=�?CZ57$QR;5?A×5;�?GX�Q4Ù¸[email protected]$357.?CX�14@C7$XUBEÓÀ×5ÔEÕ´1RVNV�7\V ù

� Z57.3»×5Ô©7\3[?xÕ�Q4Ù<V514?C1WBEXP1�ñ:14BEÔE1Rå5ÔE7R=iÓÀ1R3eÕTÓÀ7.?AZ5Q�V5XPD.1R3»14?U?C1RB©3»@A7MXU×i7\D�?C1Rå5Ô©7�1RD�éD�;5@G1RD.ÕRù29<QRÓÀ×N1R@A7MVØ?AQÛQR?AZ57\@�ÓÀ7.?AZ5Q�V5X\=2D�QY3eñRQRÔE;�?CB©QY3N14Ô357.;N@C1RÔ3N7�?CXÀQRKi7\@À35Q4?�QR3NÔ©Õ?AZ57Jå(7MXx?u1RD.D.;5@G1RD�ÕY=:å5;�?u14Ô²XUQ$]YQ[Q�V�XA×(7\7\V =[ÔEQ ã ÓÀ7.ÓÀQY@AÕ�@A7 �[;5B©@C7.ÓÀ7\3Y?GX.=Y143NVÀ7.ä�D.7.ÔEÔ©7\3[?@AQYå5;NXU?A357MXAX�1YXV5BEXCD�;NXCXA7\VTå(7\Ô©Q ã ù

,.- � 4�6�=�EG<0=�4!��@ = 4�$ ?A25<:H @ � @KH�<0H ; =�4!��@� Z5B©ÔE7TÙç;5ÔEÔ©ÕÚBE3eñ:14@CBE1R3Y?�@A7MD�QY]R35B©?ABEQR3ÚQRÙJD�QYÓÀ×5Ô©7.äbXUZ(14×i7\X�BEXÀXU?ABEÔ©Ô�1R3b7\Ô©;NXABEñR7T]RQY1RÔ�=+B©?XU7\7.Ó�Xu?AZ(1:?�D�QY3eñRQRÔE;�?CB©QY3N14ÔL357.? ã QY@AF�X\=�å(7MD.14;(XU7PQ4Ù�?CZ57.BE@�14@GDGZ5B©?A7MD�?A;N@A7Y=�Q4KL7.@�1À×N1R@U?CBE1RÔ143NX ã 7\@P?AQ/?CZ57�×5@CQRå5ÔE7.Ó Q4ÙuB©3eñ:14@CBE1R3ND�7ÀQY@$@AQYå5;NXU?A357MXAX ã B©?AZØ@C7\XA×(7MD�?$?CQ%V�BEXU?AQY@U?CB©QY3NX\=ñ�1R@AÕeBE35]�×(Q[XUB©?ABEQR38=5XCD.1RÔ©[email protected][?G1:?ABEQR3�=e1YX ã 7\Ô©Ô81YXuB©3[?A@CBE3NXUB²DPD�Ô²1RXCXñ�1R@AB²14åNB©ÔEBz?xÕYù �¸BE]4é;5@A7 �´XAZ5Q ã X�XA7.ñY7.@G14Ô¸7�ä514ÓÀ×5ÔE7\XJQ4Ù-;53e;NXA;N1RÔ+1R3NV%V�B²Xx?CQR@A?A7MV%DGZN14@G1RD�?A7\@CXu?CZN1:?$14@C7�D.QR@Aé@A7MD�?AÔEÕW@C7\D.QR]Y35B©õ\7\V´åeÕ´687 J�7.?Ué�L�ù �NQR@u?AZN7\XA7n7�ä�×i7.@CB©ÓÀ7.3[?GX.=�?CZ57$?C@C1RB©35BE35]�XC14ÓÀ×5ÔE7\X ã 7.@C714@A?AB©ú(D�B²14ÔEÔ©Õ´V�BEXU?AQY@U?C7\V´;NXAB©35]�@G143(V�QRÓ)×NÔE1R3N14@�1 RÀ3N7J?C@C1R3NXUÙçQR@CÓÀ14?ABEQR3NX\=�143NV´?CZ57$×5B©äe7\ÔEXB©3Ø?CZ57´?A@G14BE35B©3N]%B©Ó�1R]R7\X ã 7\@A7W@C1R3NV�QRÓÀÔEÕ /(B©×5×i7\V ã Bz?CZb×5@CQRå(14å5BEÔ©B©?xÕ��5ù.W�?AQ»B©3ND.@A7M1RXA7?AZ57$35QYBEXA7P@A7MXUB²XU?C143(D�7Rù�ÒuZ57P?CQR×T@AQ ã B©3´?AZN7PúN]R;5@C7$XUZNQ ã X2?CZ57$@CQRå5;NXU?A3N7\XCX?AQ�XUBEõ.7$1R3NVQR@CB©7\3Y?G1:?CB©QY3´ñ:14@CB²1:?ABEQR3(X.ù ��?�B²X�7MXx?CB©Ó�1:?C7\V´?AZ(1:?J1YD.D�;N@C14?A7P@A7MD�QY]R35B©?ABEQR3TQ�D.D�;N@CXÙçQY@�XAD\14ÔE7ñ�1R@AB²1:?CB©QY3NX$;5×Ú?AQ 1Rå(QY;�?�1/Ù÷1YD�?CQR@�QRÙ "�=�ñY7.@A?AB²D.1RÔ<XAZ5BzÙê?�ñ:14@CBE14?ABEQR3NX$QRÙ�×5ÔE;NXnQY@�ÓÀB©3e;NX14åiQR;�?�ZN1RÔzÙ�?CZ57/Z57.BE]RZ[?�Q4Ù�?AZ57�DGZN1R@C1YD�?A7\@\=�143(VÚ@CQ4?C14?ABEQR3NX�;5× ?AQÛ×5ÔE;NXÀQR@�ÓÀB©3e;NX ���

Page 12: yann.lecun.comyann.lecun.com/exdb/publis/pdf/lecun-99.pdf · 1)(+*!! 7\@ "! $ '

3

4 4 4

4

34

8

3

C1 S2 C3 S4 C5

F6

Output

����� ¥��i¥ 8·Yd.­$¨[qs{G«�g\¬�p[f4p[«�p[d\qêk(}Yws«�ªxg\v�ªx{�}5kLd.fe}Wf[gMws«�¯WoUced.vUd.oCªx{Cvx«�oGg\v�vx{CoCªxq®¯´vx{GoCgM§MfYwsrG{�}Tj:¯hN{G�{Aª�¿��:°N_�c[{P§.vx{C¯4¿÷qz{Cy�{Gq g\¬¸ªxc[{�gMpRªx¨[pYªqzd\j�{Gq8vx{G¨Yvx{C«�{Gf:ªx«ªxc[{J¨�{Gfed.qsª�¯ �êqsws§Mc:ªx{Cv�¬²g.v�c[ws§McY{Cv¨�{Cfed\q®ªxws{G«��A°

V�7.]Y@A7\7\X\ù�� Z5B©ÔE7�?CZ57�DGZN1R@C1YD�?C7.@GX+1R@A7�V5BEXU?AQY@U?C7\VÀV�;5@CB©3N]$?A@G14BE35BE35]N=4B©?<XA7.7\ÓÀX"?AZN14?-?AZ57�@AQRéå5;NXU?A357MXAX<Q4ÙL?AZN7�357.? ã QY@AFÀXU;NåNXUB²XU?CX"ÙçQY@<V�B²XU?AQR@A?ABEQR3(X"?AZ(1:?1R@A7�XABE]R35B©ú(D.1R3[?AÔEÕ�Ô²14@C]R7\@"?AZN1R3?AZ57$QY357\X�;(XU7MVTV5;5@ABE35]�?A@G14BE35BE35]Nù��+BE]R;5@C7��ÀB©3(D�ÔE;NV�7\X�7�ä514ÓÀ×5ÔE7\XuQRÙ�DGZN1R@C1YD�?C7.@GX ã @CB©?U?A7\3B©3�ñR7\@AÕP;53e;NXA;N14Ô�XU?xÕeÔ©7MX.ùGJ�7\7\V�ÔE7\XCX¸?AQPXA1�ÕY=\?CZ57.@C7u14@C7235QPXU;(DGZ�7�ä51RÓ�×NÔ©7MX¸BE3n?CZ57?A@G14BE35BE35]XU7.?\ù�J�7.ñY7.@A?AZ57\Ô©7MXAX\=5?CZ57�3N7�? ã QR@CFTD.ÔE1YXAXABzú(7\X�?CZ57.Ó D.QR@C@A7MD�?AÔEÕR= ã Z5B²DGZ XU7\7.Ó�X�?CQ´XA;5]R]Y7\XU??AZN14?�?AZ57µÙç7\14?A;5@C7\X?CZN1:?�ZN1�ñY7Jåi7.7\3/Ô©7M14@C357\V´ZN1�ñR7PXAQRÓÀ7µV57.][email protected]µQ4Ù�]R7\357.@G14ÔEB©?xÕRùe6�1RXU?AÔEÕR=úN]R;5@C7 ��B©3(D�ÔE;NV�7\X<7�ä51RÓ�×NÔ©7MX-?CZN1:?uV�7.ÓÀQR3(Xx?C@C14?A7\X<687 J�7�?AéML M X<@CQRå5;NXU?A3N7\XCX"?AQ�7�äe?C@A7\Ó�7\Ô©ÕZ5B©]YZ�Ô©7\ñR7\ÔEX�QRÙ-Xx?C@A;ND�?A;5@C7\V/3NQRB²XU7YùNH�1R3NV�ÔEB©35]I?AZ57MXU7nBEÓ�14]Y7\X ã B©?AZ�?C@C1YV�B©?ABEQR3N1RÔ�XU7\]RÓÀ7.35é?C1:?CB©QY3»1R3NV�Ùç7M1:?A;N@A7�7�äe?A@G1RD�?ABEQR3�?C7\DGZ535B �[;57\X ã QR;5Ô²V�×iQYXA7�BE3NXA;5@CÓ�QY;53[?C1Rå5Ô©7�×5@CQRå5ÔE7.Ó�X\ù�-ñR7\3%?CZ5QR;5]YZ%?CZ57ÀQR35ÔEÕ%35QYBEXA7�;(XU7MV V�;5@CBE35]T?A@G14BE35B©3N] ã 1RXJ@C1R3NV�QYÓ ×5Bzä�7\Ô /NB©×N×5B©3N]N= B©?XU7\7.Ó�XÀ?AZ(1:?I?CZ57 357.? ã QY@AF D.143Î7\Ô©BEÓÀB©3N14?A7�?CZ57 1YV�ñR7\@CXA7/7�KL7\D�?GXIQRÙP3NQR3�é�XU7\3NXABED\14Ô�å5;�?Xx?C@A;ND�?A;5@C7\V%ÓÀ1R@AF�X�Ùç@AQYÓ BEÓ�14]R7MX�XA;NDGZ 1YX�?AZN7 �I1R3NV/?CZ57aiIB©3�?CZ57ÀXU7MD�QR3(V/@AQ ã ùiÒuZ5BEXV�7.ÓÀQR3(Xx?C@C14?A7\X$1/XAQRÓÀ7 ã Z(1:?$×5;5õ\õ.ÔEB©35]�1Rå5B©ÔEB©?xÕ Q4ÙuXU;(DGZÖ3N7�? ã QR@CFeXJ?CQT×i7.@AÙçQR@CÓ ö÷BzÙuBEÓ�é×5Ô©B²D�B©?AÔEÕ5ø�1ÀFeB©3(V/Q4Ù�7.ÔE7.ÓÀ7.3[?C1R@AÕ�ë ÝGì:á�í�âAÝ ?.óêà � óêà[Þ XUQYÔ©7\Ô©ÕI?AZ5@CQR;5]YZTÙç7.7\V�é�ÙçQY@ ã 14@GVWÔEBE357\1R@D�QRÓ�å5BE3N1:?CB©QY3NX�143NVTXAB©]YÓ�QYBEVWÙç;53ND�?ABEQR3NX\ù

Page 13: yann.lecun.comyann.lecun.com/exdb/publis/pdf/lecun-99.pdf · 1)(+*!! 7\@ "! $ '

��35BEÓ�1:?A7MVÎ7�ä514ÓÀ×5ÔE7\XWQ4Ùn687 J�7.?Ué�LÖB©3 1YD�?CB©QY3Î1R@A7%1�ñ:1RB©Ô²14å5ÔE7�QR3Î?CZ57 ��3[?A7\@A3N7�?´14?��� ����� ��������� ��������������� ����� �!��"�#$����'��(�(iù

� � Ð ��É5Ç�� �xÃ�����[Ã'�iÉÎÌ Ã����-È2Æ�ÇxÉ5Ç��2Æ Ç�É5Ë � �JÄ��LÃ� ÇUÑ � �UÄ��LÃ��ÊÃ�Æ<É�¶Ã�Ð�ÅNÄ�� �¶Ã É���-Å���Ñ

�ÚÓ�1:æxQY@�D�QY3ND�7\×�?A;N1RÔR×5@CQRåNÔ©7\Ó BE3nñeBEXAB©QY3n143NVP×(1:?U?C7.@C3$@A7MD�QY]R35B©?ABEQR3PB²X8Z5Q ã ?AQJ@A7MD�QR]Y35BEõ.7B©3NV5B©ñeB²V�;N14Ô2QRå�æx7MD�?CX ã Z57.3Ö?AZNQYXA7�QYå�æx7\D�?CXnD\14353NQ4?$å(7W7\1RXABEÔ©Õ XA7.]YÓÀ7.3[?A7MV QY;�?nQ4Ù?AZ57\B©@XU;5@C@CQR;53NV5B©35](ù ��3$]R7\357.@G14Ô�=G?AZ5B²X8×iQYXA7\Xi?CZ57-×5@CQRåNÔ©7\Ó Q4ÙeÙç7M1:?A;N@A7"å5BE3NV�BE35](û�Z5Q ã ?AQ�B²V�7.3[?AB©ÙçÕ143NV åNB©3NV ?AQR]Y7�?CZ57.@´Ùç7\14?A;5@C7\XI?AZN14?´åi7.ÔEQR35]b?AQ 1 XUBE35]YÔ©7%QYå�æx7\D�?\= ã Z5BEÔ©7ÖXU;5×N×5@A7MXAXABE35]Ùç7\1:?C;5@C7\X¸?AZN14?"å(7\Ô©QY35]µ?AQJ?AZ57uåN1YDGF[]Y@AQY;53NV$QY@+?AQPQ4?CZ57.@�QRå5æx7\D�?GX.ù:ÒuZ57�D.QRÓÀÓÀQR3 ã B²XCV�QRÓBEXµ?CZN1:?M= 7�ä5D�7\×�?µBE3»?AZ57�XAB©ÓÀ×5ÔE7\XU?nD.1RXA7R=LQY357�D\14353NQ4?PBEV�7\3[?AB©ÙçÕ 143NV»å5BE3NV ?AQR]Y7�?CZ57.@µ?AZN7Ùç7\1:?C;5@C7\XuQ4Ù"143TQRå5æx7\D�?�;53NÔ©7MXAX�QY357PFe35Q ã X ã ZN1:?�QYå�æx7\D�??CQ�Ô©QeQYF�ÙçQR@Mù

��3 ZN1R3NV ã @CBz?CB©35]T@C7\D�QY]R35B©?ABEQR3�=N?CZ57�×N@AQYå5Ô©7\Ó BEXJ?CQTXA7.×N1R@C14?A7�1´DGZN1R@C1YD�?C7.@�Ùç@CQRÓ B©?CX357.BE]RZeåiQR@GX.= ]YB©ñY7.3Ö?AZN14?$?AZ57W357.BE]RZeåiQR@GX$D.143Ö?CQR;NDGZÛB©?�QR@nQ:ñY7.@CÔE1R× ã B©?AZØB©?\ù+ÒuZ57�ÓÀQYXU?D�QRÓÀÓÀQR3/XAQRÔE;�?ABEQR3/B²X�D.14ÔEÔE7\V UZ57\;5@CBEXU?AB²DµQ:ñR7.@Aé�XU7\]RÓÀ7.3[?C14?ABEQR3 � ù ��?�D.QR3NXABEXU?CXuBE3�]R7\357.@G1:?AéB©35]%1/Ô²14@C]R7�3e;5Ó�å(7\@nQ4Ùu×iQ4?A7\3[?AB²14Ô<D.;�?CX�å(7.? ã 7\7.3ØDGZN1R@C1YD�?A7\@CXµ;NXABE35]�Z57.;N@AB²Xx?CBEDIB©Ó�14]Y7143N1RÔ©Õ�XABEX�?A7MDGZ535B �Y;N7\X\ù 91R3NV�B²V51:?C7�DGZN1R@C1YD�?C7.@GX�14@C7nÙçQY@AÓÀ7\V åeÕ/D.QRÓ�å5B©3NB©35]TD�QY3Y?CB©]Y;5QR;(XXU7\]RÓÀ7.3[?CX<BE3WÓ�;NÔz?CB©×5ÔE7 ã 1�Õ�X\ùYÒuZ57µD.143(V�BEVN1:?A7µDGZN14@G1RD�?A7\@CX<14@C7�?AZ57\3´XA7.3[??AQn?CZ57µ@A7MD�QR]Ré35B©õ\7.@JÙçQR@PD�Ô²1RXCXUB©ú(D\1:?ABEQR3%1R3NV XCD�QY@ABE35](ù�ÊXUBEÓÀ×5Ô©7�]R@G14×NZ�é�XA7\1R@CDGZ´?A7MDGZ535B �[;57�?CZ57.3 úN3NV5X?AZ57nD.QR3NXABEXU?A7\3[?�XA7 �[;57\3ND�7PQRÙ�DGZN1R@C1YD�[email protected](V�BEVN1:?A7MX ã Bz?CZT?AZ57$åi7\XU?�Q:ñR7\@C1RÔ©Ô XCD�QR@C7RùÒuZ57.@C7´B²X�1ÛXUBEÓÀ×5ÔE7/1RÔz?C7.@C3N1:?CB©ñY7W?CQÛ7�ä�×5ÔEBED.Bz?CÔ©Õ XU7\]RÓÀ7.3[?ABE35]ÖB©Ó�1R]R7\X�QRÙµDGZN1R@C1YD�?A7\@Xx?C@ABE35]YXÀ;(XUBE35]ØZ57\;5@AB²XU?AB²D.X\ù2ÒuZ57�B²V�7\1ÛB²XÀ?AQÚX ã 7.7.×Î1Û@C7\D�QY]R35BEõ.7\@�1RD�@CQYXCXÀ14ÔEÔ�×iQYXCXUBEå5ÔE7Ô©Q�D.14?ABEQR3NXµQY3Ö1R3 BEÓÀ1R]R7ÀQ4Ù2?AZ57�7\3Y?CB©@C7 ã QY@CV QR@$Xx?C@ABE35] ã Z5QYXA7ÀZ57.BE]RZ[?PZN1RXPåi7.7\3»3NQR@AéÓÀ1RÔ©BEõ.7MV ù � Bz?CZ%?CZ5BEXJ?C7\DGZ535B �[;57R=L35Q/XA7.]RÓÀ7\3Y?G1:?CB©QY3�Z57.;5@CB²Xx?CBED\X�B²Xµ@A7 �Y;NB©@C7\V ùLH�Q ã 7.ñY7.@M=?AZ57\@A7%1R@A7/×5@CQRå5ÔE7.Ó�X ã B©?AZ ?AZNBEX´14×5×5@CQY1YDGZ8ù��+B©@GXx?M=�?AZ57 Ó�7.?AZ5Q�V B²X�B©3Î]Y7.357\@C1RÔ �Y;NBz?C77�ä�×(7\3NXAB©ñY7RùNÒuZ57$@C7\D.QR]Y35B©õ\7.@uÓ�;(Xx?�åi7n14×N×5Ô©BE7\V�14?�7.ñY7.@CÕI×(Q[XAXABEå5Ô©7$ÔEQ�D.14?ABEQR3TQR3T?AZ57�B©3�é×5;�?\=�QR@$1:?PÔ©7M1RXU?n1:?n1WÔ²14@C]R7�7.3NQR;5]YZ»XA;5åNXA7�?$Q4ÙÔ©Q�D.14?ABEQR3NXµXAQ´?CZN1:?$ÓÀB²XA1RÔ©BE]R35ÓÀ7\3Y?GXµQ4ÙDGZN14@G1RD�?A7.@GXPBE3Ø?AZ57WúN7.Ô²VÛQ4Ù�ñeBE7 ã Q4Ùu?CZ57W@C7\D.QR]R3NB©õ\7.@GX$14@C7WXUÓ�1RÔ©Ô<7.35QY;5]RZÛ?CQ%ZN1�ñR7�3NQ7�KL7\D�?JQR3/?AZN7n7.@C@AQY@�@C14?A7Yù5�e7MD�QR3(V = ã Z57\3�?AZ57�@A7MD�QR]Y35BEõ.7.@�B²X�D.7.3[?A7\@A7MVTQR3%1�DGZN1R@C1YD�?A7\@?AQ åi7T@A7MD�QR]Y35BEõ.7\V8=8?AZN7´357\B©]YZeå(QY@CX�Q4Ù�?CZ57TD.7.3[?A7\@�DGZ(14@G1RD�?C7.@ ã BEÔEÔåi7T×5@A7MXU7\3[?�BE3Ú?AZN7úN7.Ô²VIQRÙ�ñeB©7 ã Q4Ù ?CZ57P@A7MD�QR]Y35BEõ.7.@M=4×iQYXCXUBEå5ÔEÕÀ?AQY;NDGZ5BE35]n?CZ57µD.7.3[?A7\@DGZ(14@G1RD�?C7.@MùYÒuZ57\@A7.ÙçQR@C7?AZ57´@A7MD�QR]Y35BEõ.7.@$Ó�;NXU?�åi7´14åNÔ©7�?CQ D�QY@A@C7\D�?AÔEÕ @C7\D.QR]R3NB©õ\7À?AZ57TDGZN14@G1RD�?A7.@$BE3Ø?CZ57´D.7.3[?A7\@Q4Ù�Bz?GX$BE35×5;�?�úN7.Ô²V =87\ñR7.3ÛB©Ù�357.BE]RZeåiQR@CB©3N]/DGZ(14@G1RD�?C7.@GXP14@C7�ñY7.@CÕ»D.Ô©Q[XU7�?AQ(=�QY@$?AQY;NDGZ5BE35]?AZ57$D�7\3[?A@G14ÔLDGZN14@G1RD�?A7.@MùeÒuZ5B©@GV =�1 ã QY@CVWQR@uDGZN14@G1RD�?A7.@uXU?A@CB©35]ÀD.1R3535QR?åi7P×(7\@UÙç7MD�?AÔEÕIXAB©õ\735QR@CÓ�14ÔEB©õ\7\V ù���3NV�BEñeBEV5;N14Ô DGZN1R@C1YD�?A7\@CX ã Bz?CZ5BE3/1�XU?A@CB©35]�Ó�1�ÕÀZN1�ñR7 ã BEV57.ÔEÕ�ñ�1R@AÕeBE35]ÀXUBEõ.7MX143NVÛåN1YXU7\Ô©BE357�×iQYXABz?CB©QY3NX\ù�ÒuZ57.@C7�ÙçQY@A7À?CZ57I@C7\D.QR]R3NB©õ\7.@$Ó�;(Xx?�åi7IñY7.@CÕ @CQRå5;NXU?$?AQ»XUZ5B©Ùê?CX143NVTXUBEõ.7$ñ:1R@AB²1:?CB©QY3NX.ùÒuZ57\[email protected]$×N@AQYå5Ô©7\Ó�X�14@C7$7.ÔE7.][143[?AÔEÕWD�BE@GD�;5Ó�ñR7.3[?C7\VTBzÙ-1�D.QR3eñRQYÔ©;�?CB©QY3N14ÔL357.? ã QY@AFBEX´@A7\×5ÔEBED\1:?A7MV Q:ñR7\@�?AZN7»BE35×5;�?TúN7\ÔEV ù �+B©@GXU?´Q4Ù�14ÔEÔ�=u1RX´XUZ5Q ã 3 B©3 ?CZ57Ö×5@A7\ñeB©QY;NXWXA7\D�é?ABEQR38=2D.QR3eñRQYÔ©;5?ABEQR3N1RÔ<357\;5@G14Ô�357.? ã QY@AF�XÀ14@C7´ñY7.@CÕØ@AQYå5;NXU?À?AQÚXAZ5BzÙê?GX�143NVÎXCD.1RÔ©7Tñ:14@CBE14é?ABEQR3NXuQRÙ�?AZN7µBE35×5;5?�B©Ó�14]Y7R=�1RX ã 7.ÔEÔ81RX<?CQ�3NQRB²XU7P143NV´7�ä[email protected];NXÓÀ1R@AF�X<B©3T?CZ57PB©35×N;�?\ù

ÒuZ57\XA7�×5@CQR×i7.@A?ABE7\X-?C1RFR7JD\14@C7�Q4Ù�?AZ57JÔE14?U?A7\@2? ã Q�×N@AQYå5Ô©7\Ó�X-ÓÀ7.3[?ABEQR3N7\VIBE3�?CZ57J×N@A7\ñ[BEQR;(X×N14@G14]Y@C1R×5Z8ù.�e7\D.QR3NV =8D.QR3eñRQYÔ©;5?ABEQR3N1RÔ+3N7�? ã QR@CFeXJ×5@CQ:ñeBEV�7À1/V�@G1RXU?AB²D�XC1�ñeB©3N]´BE3ÖD.QRÓÀ×5;�é?C1:?CB©QY3N14Ô"@C7 �[;5BE@C7.ÓÀ7.3[? ã Z57.3Û@C7.×5ÔEBED\1:?C7\V»Q:ñR7.@µÔ²14@C]R7�B©35×N;�?núN7\ÔEV5X\ù � @C7.×5ÔEBED\1:?C7\VÖD�QR35é

Page 14: yann.lecun.comyann.lecun.com/exdb/publis/pdf/lecun-99.pdf · 1)(+*!! 7\@ "! $ '

$����� ¥��N¥ ^»aR¨ed.oG{+twz«�¨Yqzd\oG{G­${Cf4ªL�{GpRvUd\q4�{Cª�±�g.vx��wz«�d-oGgMf4y�g\qspYªxwsgMf[d\qRf[{Cª�±�g\vx�uªxc[d.ª c[d\« j�{G{Cfvx{C¨[qswso�d.ªx{�}�g.yM{Cv<dJ±-wz}Y{�wsf[¨YpYª��e{Cq©}5°

ñRQRÔE;�?CB©QY3N14Ô83N7�? ã QR@CFi=(14Ô²XUQID\14ÔEÔ©7MVT1 � üNìRïGÝ �nó²Ü�üLò©ìRïGÝ.ßÀÝ.àNá &�Ý.í�âAì4ò5&�Ý.á)(2ð:â+* QR@ � $ J Jf =%14?C1R3´7.?�14Ô�ùGWch@h "[l�=�B²X�XAZ5Q ã 3ÛB©3 �+BE]R;5@C7 L�ù�� Z5BEÔ©7TXAD\143535BE35]%1�@C7\D.QR]Y35B©õ\7.@nD\143Øåi7×5@AQYZ5BEå5Bz?CB©ñY7.ÔEÕb7.äe×i7.3(XUBEñR7%BE3 ]R7\357.@G14Ô�=2D.QR3eñRQYÔ©;�?CB©QY3N14Ô�3N7�? ã QR@CFeX�D\143 åi7 XCD.143N357\VÎQY@@A7\×5Ô©B²D.14?A7MV/ñR7.@CÕW76RID�BE7.3[?AÔEÕ´Q:ñY7.@�Ô²14@C]R7Y=5ñ:14@CBE1Rå5ÔE7�é�XUBEõ.7PBE35×5;�?JúN7.Ô²V5X\ù 9<QY3NXUB²V�7\@�QR357�B©3�éXx?G143ND.7nQ4Ù21´D�QR3eñYQRÔE;�?ABEQR3N1RÔ8357.?µ143(V/Bz?GX ì4òsá�Ý�âÀÝ�Þ[ð 14?µ1�3N7\14@CåeÕ´ÔEQ�D.14?ABEQR38ùiS<7MD.1R;NXU7�Q4Ù?AZ57/D�QY3[ñYQRÔE;�?ABEQR3(14Ô23(1:?A;N@A7´Q4Ù�?CZ57´3N7�? ã QR@CFi=¸;535B©?CX�BE3Ú?AZN7W? ã Q BE3NXU?C143(D�7\X�?AZ(1:?ÀÔ©QeQRF1:?�B²V�7.3[?AB²D.1RÔ8ÔEQeD\1:?CB©QY3NX�QR3T?CZ57nBE35×5;�?�Z(1�ñR7PBEV57.3[?AB²D.1RÔ8QR;�?C×5;�?GX.=N?AZ57\@A7.ÙçQR@C7J?CZ57.BE@�XU?C14?A7\XV�Q�35Q4?n3N7.7\VÖ?CQ�å(7´D�QRÓÀ×5;5?A7\V»? ã B²D�7Yù>!J35ÔEÕ»1T?AZNB©3 CXUÔEBED.7 � Q4Ùu357 ã XU?C14?A7MXµ?AZN14?�14@C735Q4?�XAZN14@C7\VWå[ÕI?AZ57P? ã Q�3N7�? ã QR@CF�B©3NXU?C1R3ND�7MXu357.7MV5X?AQÀåi7P@A7MD�QRÓÀ×5;5?A7\V8ù � Z57.3�14ÔEÔL?AZN7XUÔEBED.7\X-1R@A7�×N;�?2?CQR]R7.?AZ57\@\=4?AZ57�@C7\XA;5Ô©?2B²X2XAB©ÓÀ×5ÔEÕ�1PÔ²14@C]R7\@-D�QY3[ñYQRÔE;�?ABEQR3(14Ô�357.? ã QY@AF ã Z5QYXA7Xx?C@A;ND�?A;5@C7�B²XJBEV�7\3[?AB²D.14Ô¸?AQ´?AZN7�QR@CBE]RBE3N14Ô¸357�? ã QR@CFL=N7�ä5D.7.×�?µ?AZN14?�?CZ57�Ùç7M1:?C;5@A7�Ó�14×NXµ14@C7ÔE1R@A]Y7.@PB©3Û?CZ57IZ5QY@ABEõ.QY3[?C14Ô"V�BEÓÀ7.3NXABEQR38ù ��3ÖQR?AZ57\@ ã QR@GV5X.= @C7.×NÔ©B²D.14?ABE35]�1�D�QR3eñYQRÔE;�?ABEQR3N1RÔ357�? ã QR@CF�D.143»å(7�V�QY357ÀXUBEÓÀ×5Ô©Õ�åeÕ�BE3ND�@C7\1YXUBE35]W?AZ57�XAB©õ\7�QRÙ-?AZ57�úN7\ÔEVNXJQ:ñY7.@ ã Z5BEDGZ»?AZN7D�QR3eñYQRÔE;�?ABEQR3NX+14@C7×(7\@UÙçQY@AÓÀ7\V8=�1R3NV�å[Õ$@C7.×NÔ©B²D.14?ABE35]�?CZ57�QR;5?A×5;�?"Ô²1�ÕR7\@+1YD.D�QY@CV5B©35]YÔ©ÕYùMÒuZN7QR;�?C×5;�?JÔ²1�ÕR7\@�7�KL7\D�?ABEñR7\Ô©ÕTå(7MD�QYÓ�7MX�1ID.QR3eñRQYÔ©;5?ABEQR3N1RÔ8ÔE1�ÕY7.@Mù ��3%QY;�?A×N;�? ã Z5Q[XU7�@A7MD�7.×5é?ABEñR7<úN7.Ô²VnB²X�D�7.3[?C7.@C7\V$QY3�143�7.ÔE7.ÓÀ7.3[?C1R@AÕPQYå�æx7\D�? ã BEÔ©Ôe×5@CQ�V�;ND.72?CZ57D.ÔE1YXAX¸Q4ÙN?AZ5B²X+QRå�æx7MD�?\=ã Z5B©ÔE7�1R3%BE3�é�åi7�? ã 7.7\3%QR;5?A×5;�?PÓ�1�Õ´BE3NV�B²D.14?A7�35Q´DGZN1R@C1YD�?C7.@�QY@JD�QY3[?C14BE3%@C;5å5å5B²XAZ8ùiÒuZN7QR;�?C×5;�?CX2D.143�åi7�B©3[?A7\@A×N@A7.?A7\V�1RX+7\ñ[B²V�7\3ND�7MX+ÙçQR@�?AZN7�×5@A7MXU7\3ND�7uQRÙ(QRå5æx7\D�?GX"1:?-14ÔEÔ�×iQYXCXUBEå5ÔE7×(Q[XUB©?ABEQR3NXuBE3T?AZ57$BE35×5;5?�úN7.Ô²V ùÒuZ57 ��$ J J 14@GDGZ5Bz?C7\D�?A;[email protected]\ÓÀX×N1R@U?CBED.;5Ô²14@CÔ©Õ�1:?A?A@G1RD�?ABEñR7�ÙçQY@u@A7MD�QR]Y35BEõ.BE35]�D�;N@CXAB©ñY7

ZN143NV ã @AB©?ABE35] ã Z57\@A7$35Q�QYåeñ[BEQR;(X�XU7\]RÓÀ7.3[?C14?ABEQR3/Z57\;5@AB²XU?AB²D.X�7.äeB²XU?\ù ��Ôz?CZ5QR;5]YZT?AZ57nB²V�7M1

Page 15: yann.lecun.comyann.lecun.com/exdb/publis/pdf/lecun-99.pdf · 1)(+*!! 7\@ "! $ '

2 3 3 4 5

2345C1 C3 C5

F6

Input

SDNNOutput

Compose + Viterbi

Answer

����� ¥��N¥ ^<f%{C·Yd.­$¨[qs{�g.¬u­µpYq®ªxwz¨Yqs{�oUced�vUd\oCªx{Cv$vx{CoGgM§\f[w®ªxwsgMfÖ±-w®ªxcØa4tu���°LºØw®ªxcÖaRtu�<�Jk fYg{A·Y¨YqswzoCwsª2«�{G§\­${Gf:ªUd.ªxwsgMf�wz«-¨�{Cv�¬Eg\vx­${�}5°

Q4Ù7� $ J J B²X �[;5B©?A7<QRÔ²V:f �P7\7.ÔE7.@M= ��;5ÓÀ7.ÔEZN1R@U?�143(VT6�7.Q ã W[h@hnW:= =%14?C1R3T7.?�14Ô�ùLWchNh�"cl�=M1R3NVñR7.@CÕ�14?U?C@C1YD�?ABEñR7�å[Õ�B©?CXXUBEÓÀ×5Ô©B²D�B©?xÕR=[B©?<ZN1YX-35QR?2]R7\357.@G1:?C7\V ã B²V�7�BE3[?A7.@C7\XU?2;53[?CB©Ôi@A7MD�7\3Y?CÔ©Õå(7MD.14;(XU7PQ4Ù+?AZN7$7.35QY@AÓÀQR;(XV57.Ó�143NVNXuBz?�×5;5?CX�QY3´?CZ57$@C7\D�QY]R35BEõ.7\@\ù

�I-0/ � 4�; @ E �>E @ ;�<:4!��;�+>@�� 9�;���9�; 2�� =�4�� % ?A?

ÒuZ57WQY;�?A×5;5?�QRÙ�1%Z5QY@ABEõ.QY3Y?G14ÔEÔ©Õ»@A7\×5ÔEBED\1:?A7MV � $ J J BEX�1 XA7 �[;57\3ND�7WQ4Ù�ñR7MD�?CQR@GX ã Z5B²DGZ7.3ND.QeV57À?AZ57IÔEBEFR7.ÔEBEZ5QeQeVNX.=8×i7.3(14Ô©?ABE7\X\=8QR@nXCD�QY@A7MXPQ4ÙúN3NV�BE35] DGZN1R@C1YD�?C7.@PQ4Ù�1/×N14@A?AB²D�;NÔE1R@D�Ô²1RXCX ÔE1Rå(7\ÔR1:? ?CZ572D.QR@C@A7MXU×iQR3NV5B©35]uÔEQ�D.1:?CB©QY3PB©3P?CZ57"B©3N×5;�?\ù �Ø×iQYXU?Ué�×[email protected]\XCXUQY@LBEX @C7 �[;5BE@C7\V?AQ�×5;5ÔEÔRQY;�?8?CZ57"å(7MXx?¸×(Q[XAXAB©åNÔ©7"Ô²14åi7.Ô[XU7 �Y;N7.3ND.7�Ùç@AQYÓ ?AZNBEX ñY7\D�?AQR@¸XU7 �Y;N7.3ND.7Rù ��3µ7.ä514ÓÀ×5ÔE7Q4ÙD� $ J J QR;�?C×5;�?´B²XWXAZ5Q ã 3 BE3 �+B©]Y;5@C7 �5ù �"7\@AÕÎQRÙê?A7.3�=2BE3NV�BEñ[B²V�;N1RÔµDGZ(14@G1RD�?C7.@GX�14@C7XU×iQ4?A?A7\V åeÕÎXA7.ñY7.@G14ÔJ357.BE]RZeåiQR@CB©35]bB©3NXU?C1R3ND�7MXWQRÙn?AZN7»@C7\D.QR]R3NB©õ\7.@M=21bD.QR3NXA7 �[;57\3ND�7»Q4Ù?AZ57Ö@AQYå5;NXU?A357MXAX´Q4Ù$?CZ57Û@A7MD�QY]R35BEõ.7\@�?AQ Z5QR@CBEõ.QR3[?G14Ô�?C@C1R3NXAÔE14?ABEQR3NX\ù ��ÔEXAQ �[;5B©?A7ÖQ4Ùê?C7.38=DGZN14@G1RD�?A7.@GX"14@C7�7\@A@CQR357\QR;NXAÔEÕ�V�7�?C7\D�?A7\VIå[Õ�@C7\D�QY]R35BEõ.7\@"B©3(Xx?G143ND.7\X"?CZN1:?uXU7\7�QR3NÔ©ÕÀ1$×5BE7\D.7Q4Ùi1PDGZN1R@C1YD�?C7.@Mù �NQR@"7�ä51RÓ�×NÔ©7�1µ@A7MD�QY]R35BEõ.7\@¸BE3NXU?C143(D�7?AZN14?-QR35ÔEÕ�XA7.7MX�?CZ57�@CB©]YZY?�?CZ5B©@GV�Q4Ù1 � � ÓÀBE]RZ[?�QY;�?A×5;5?�?AZN7-ÔE1Rå(7\ÔnWYù\H�Q ã D\143 ã 727.ÔEB©ÓÀBE3N1:?C7"?AZ5Q[XU727�äe?A@G14357\QR;NX DGZ(14@G1RD�?C7.@GXÙç@AQYÓ ?AZ57�QY;�?A×N;�?2XU7 �[;57.3ND.7�143NV�×5;5ÔEÔzé�QR;5?+?CZ57�åi7\XU?-B©3[?A7\@A×N@A7.?C1:?CB©QY3 �uÒuZ5B²X-D.1R3�åi7�V�QR3N7ã Bz?CZ�1�XABEÓ�×NÔ©7 ã 7\B©]YZ[?A7\Vµú(35Bz?C7XU?C14?A7Ó�1RDGZ5BE357Rù�ÒuZ57XU7 �Y;N7.3ND.72Q4Ù5ñY7\D�?AQR@GX8×N@AQ�V�;ND.7\V$åeÕ?AZ57���$ J JbB²X úN@GXU? ?A;5@C357\VnB©3[?AQ�1�Ô©BE357\1R@8]R@G14×NZPD.QR3NXU?A@C;ND�?A7\V$1YXLÙçQRÔEÔEQ ã X.ù �<1RDGZPñR7MD�?CQR@8BE3?AZ57PQR;5?A×5;�?�XA7 �[;57.3(D�7µBEX<?A@G143(XxÙçQY@AÓÀ7\VIB©3[?AQÀ1�å5;53NV�ÔE7PQ4Ù¸1R@CD\X ã B©?AZ/1�D�QRÓÀÓÀQR3´XUQY;[email protected]�V�7´1R3NVÛ?C1R@A]Y7�?�3NQeV57Rù �21RDGZØ[email protected][?C1RB©3(XnQR3N7WQ4Ù�?CZ57W×iQYXCXAB©å5ÔE7WDGZN1R@C1YD�?C7.@nÔ²14åi7.Ô²X.=?AQR]Y7�?CZ57.@ ã Bz?CZ B©?CX/D�QY@A@C7\XA×iQR3NV�BE35]Ú×i7.3(14Ô©?xÕRù �21YDGZ å5;53NV�ÔE7ÛD�QR3[?G14BE3NX´1R3¶1YV5V�B©?ABEQR3N1RÔ14@GD�å(7M14@CB©3N]Ø?AZ57 U35QY357%QRÙ$?AZ57Ö14åiQ:ñR7 � ÔE1Rå(7\Ô ã B©?AZ¶1Ú×(7\3N14Ô©?xÕRùuÒuZ57\XA7 åN;53NV�ÔE7\XT14@C7D�QR3(D.1:?C7.3N14?A7MV BE3 ?CZ57�QR@GV�7\@µQRÙ2?CZ57�ñR7MD�?AQY@$XU7 �Y;N7.3ND.7/öç?AZ57À?G14@C]R7.?P3NQeV57�QRÙ1Tå5;53(V�Ô©7å(7MD�QRÓÀ7MX ?AZN72XUQY;5@GD�7"35Q�V�7-QRÙe?AZ57-3N7�äe?�å5;53(V�Ô©7�ø�ù �21YDGZµ×(1:?AZ$BE3P?AZNBEX�]R@G14×5ZPB²X�1u×iQYXCXUBEå5ÔE7

Page 16: yann.lecun.comyann.lecun.com/exdb/publis/pdf/lecun-99.pdf · 1)(+*!! 7\@ "! $ '

6 7 7 7 8 8678

3 5 5 1 1 43514

1 1 1 4 4 11114

5 5 4 0540

Input

F6

SDNNoutput

Answer

����� ¥��5¥ ^2f aRtu�<� d\¨Y¨[qsws{�}/ªxgWdIfYgMws«�¯�ws­nd\§\{�g.¬<}Yws§Mw®ª$«�ª�vxwsf[§R°8_�c[{�}Yws§\wsªx«µ«�c[g�±-f%wzf/ªxcY{a4tu�� g\pYªx¨YpYªµvx{C¨Yvx{G«�{Gf:ªPªxcY{�±-wsfYf[wsf[§WoGqzd\«�«$qzd\j�{Cqz«Gk8±-w®ªxc»d´qsws§Mc:ªx{Cv$§.vx{C¯%qs{Gy�{Gq�¬Eg\v$cYwz§\cY¿¨�{Cfed\q®ª�¯�d.f[«�±+{Cvx«G°

B©3[?A7\@A×N@A7.?C1:?CB©QY3IQ4Ù ?CZ57µB©35×N;�?\ù � ]R@G14ÓÀÓ�14@2B²X<?AZ57\3´D.QR3NXU?A@C;ND�?C7\VI1YX<1 ã 7\B©]YZ[?A7\V�úN35B©?A7�éXx?G1:?A7�Ó�1RDGZ5BE357�?AZN14?nD�QR3[?G14BE3NXJ1´ÓÀQ�V�7.Ô�ÙçQY@µ7M1RDGZÖDGZN14@G1RD�?A7.@Mù(ÒuZN7�]Y@C1RÓ�Ó�1R@J7\3NXU;N@A7MXÙçQR@n7.ä�1RÓÀ×5Ô©7I?AZN14?�357.BE]RZeåiQR@CB©3N]/DGZ(14@G1RD�?C7.@GXPÓ�;NXU?�åi7WXA7.×N1R@C14?A7\VÖåeÕ»1 U35QY357�Q4Ùu?AZN714åiQ:ñR7 � Ô²14åi7.ÔPö ã ZNBz?C7´XA×N1RD.7Mø�=�143(VÖ?AZN14?�XA;ND.D.7\XCXUBEñR7IQ�D\D�;5@C@A7\3ND�7MX$Q4Ù�?CZ57´XC14ÓÀ7IÔ²14åi7.Ô14@C7�×5@CQRåN1Rå5ÔEÕ�×5@CQ�V�;ND.7\V�åeÕI1�XABE35]RÔE7�BE35×5;�?�DGZN1R@C1YD�?A7\@\ù[ÒuZ57µ]R@G14ÓÀÓ�14@21R3NV�?CZ57JÔEBE357\1R@]R@G14×5Z 1R@A7�?AZ57\3 ïGð4ß�üNð�Ü.Ý � ö÷1´]R@G14×5Z QR×i7.@G1:?ABEQR3»XUBEÓÀB©Ô²14@µ?AQ/1W?A7.3(XUQY@J×5@CQ�V�;ND�?Gø�ù8ÒuZN7D�QRÓÀ×iQYXA7\VÛ]Y@C1R×5ZÚD�QY3Y?G14BE3NXn1RÔ©Ô<?AZ57´×N1:?CZNXnQRÙ�?AZ57´Ô©BE357M14@�]Y@C1R×5ZÖ?AZN14?�ZN1R×5×i7.3Ø?CQ åi7]R@G14ÓÀÓ�1:?AB²D.1RÔ©ÔEÕ%D�QY@A@C7\D�?\ù�� �JBz?C7.@Cå5B214ÔE]RQY@AB©?AZNÓ D.143Ö?CZ57.3Ûåi7�;NXU7MV»?AQTúN3NVÖ?CZ57�×N1:?CZã Bz?CZ´?CZ57nXAÓÀ1RÔ©ÔE7\XU?�Q:ñR7\@C1RÔ©ÔL×i7.3N1RÔz?xÕYù

�I- , � � ��@ E < �#@K4�;�HDB < ;�+�� % ?A?��3W1nXA7.@CB©7MX"Q4Ù87.äe×i7.@CBEÓ�7\3[?CX\=Y687 J�7.?Ué�L ã 1YX"?C@C1RB©357MV ã Bz?CZ�?AZN7�]RQ[14Ô5QRÙ8å(7\B©3N]�@C7.×5ÔEBED\1:?C7\VB©3[?AQ$1R3 � $ J J XAQ$1RX¸?AQP@C7\D�QY]R35BEõ.7<Ó�;5Ô©?ABE×5ÔE7�DGZN14@G1RD�?A7.@GX ã Bz?CZ5QR;�?-XA7.]YÓ�7\3[?C1:?CB©QY3NX\ù�ÒuZN7V51:?G1 ã 1YX8]Y7.357\@C14?A7\VµÙç@CQRÓ¶?AZN7<×5@C7.ñeBEQR;NXAÔEÕJV�7MXAD.@ABEå(7MVg=�J ����ÒØXU7.?+1YX ÙçQRÔEÔEQ ã X.ùMÒ�@G14BE35BE35]B©Ó�14]Y7\X ã 7.@C7�D�QYÓ�×iQYXA7\V QRÙ$1ØD.7.3[?A@G14Ô�DGZ(14@G1RD�?C7.@M= /(143NFR7\V åeÕÚ? ã QÚXABEV�7%DGZ(14@G1RD�?C7.@GX×5BEDGFY7\V 14?´@G143NV5QRÓ B©3 ?CZ57»?A@G14BE35B©3N]bXU7.?\ùuÒuZ57ÛXU7\×N14@G1:?CB©QY3 åi7�? ã 7.7\3 ?AZN7»åiQR;53(V�B©3N]å(Q�ä�7\X2Q4Ù ?CZ57JDGZN1R@C1YD�?C7.@GX ã 7\@A7�DGZNQYXA7.3�1:?@C1R3NV�QRÓÊåi7�? ã 7.7.3Àé$W�1R3NV �n×5B©ä�7.Ô²X.ù���3IQR?AZ57\@B©3NXU?C1R3ND�7MX.=�35QÛD�7\3[?A@G14Ô�DGZN14@G1RD�?A7\@ ã 1RX�×5@C7\XA7.3[?\=�BE3 ã Z5B²DGZÎD.1RXA7I?CZ57�V�7\XABE@A7MVÚQR;5?A×5;�?Q4Ù$?AZN7»357.? ã QY@AF ã 1RX�?CZ57Öå5ÔE1R35F XU×(1RD�7ÖD�Ô²1RXCX.ù ��3¶1YV5V�B©?ABEQR38=<?A@G14BE35BE35]bBEÓ�14]R7MX ã 7.@C7V�7.]Y@C1YV�7\VWå[Õ´@C1R3NV�QYÓ�ÔEÕ /NBE×5×5BE35]À?AZN7$×5Bzä�7\ÔEX ã B©?AZ�×5@CQRå(14å5BEÔ©B©?xÕ �5ù.WRù

Page 17: yann.lecun.comyann.lecun.com/exdb/publis/pdf/lecun-99.pdf · 1)(+*!! 7\@ "! $ '

�¸BE]R;N@A7MX �Ú143NV jÖXAZ5Q ã 1ÖÙç7 ã 7�ä514ÓÀ×5ÔE7\XIQ4ÙnXU;(D.D�7MXAXUÙç;5Ô�@C7\D.QR]Y35Bz?CB©QY3NXÀQ4Ù$Ó�;5Ôz?CBzé×5Ô©7IDGZ(14@G1RD�?C7.@GXµå[Õ ?AZ57I6�7�J�7�?Ué�L � $ J Jnù��[?G143NV51R@CV ?A7MDGZ535B �[;57\X$å(1RXA7\V»QR3ØH�7\;5@AB²XU?AB²D!JñR7.@Aé��e7.]YÓ�7\3[?C1:?CB©QY3 ã QR;5Ô²VIÔEB©FY7.ÔEÕ�Ù÷14BEÔ QR3TÓÀQYXU?�Q4Ù�?CZ5QYXA7P7�ä514ÓÀ×5ÔE7\X\ù5ÒuZ57P@CQRå5;NXU?A3N7\XCXQ4Ù�?CZ57/357�? ã QR@CFÖ?AQØXCD.14ÔE7/143NVbñR7.@A?AB²D.1RÔu×(Q[XUB©?ABEQR3 ñ�1R@AB²1:?CB©QY3NXÀ14ÔEÔ©Q ã X�B©?�?CQÛ@A7MD�QR]Y35BEõ.7DGZN14@G1RD�?A7.@GX�BE3 XA;NDGZ XU?A@CB©35][X.ùV=�QY@A7nBEÓÀ×(QY@U?G143[?AÔEÕR=iB©?PXA7.7\ÓÀX�?AZN14?J?CZ57�3N7�? ã QR@CF´B²XJ1Rå5ÔE7?AQÀB©3(V�B©ñeB²V�;N1RÔ©ÔEÕW@C7\D.QR]R3NB©õ\7�?CZ57nDGZN14@G1RD�?A7\@CX<7.ñY7.3 ã Z57.3T?AZN7.@C7µB²X�1ÀXUBE]R3NBzú(D\143[?�Q:ñR7\@AÔ²14×ã Bz?CZ�?AZ57�357.BE]RZeåiQR@GX.ù ��?JBEXJ14Ô²XUQI1Rå5Ô©7n?AQWD.QR@C@A7MD�?AÔEÕI]Y@AQY;5×�V�BEXCD�QY35357MD�?A7MV/×5BE7\D�7MX�QRÙ�BE35F?AZN14?µÙçQY@AÓ DGZN1R@C1YD�?C7.@GX.=i1RXµ7�ä�7.ÓÀ×5ÔEB©úN7\V B©3»?AZ57�;5×5×i7.@$Z(14Ô©Ù2Q4Ù-?CZ57�úN]R;N@A7Yù��3 ?AZ57�?AQY×Ô©7.Ùê?n7�ä514ÓÀ×5ÔE7R=L?CZ57 ��143NV ?AZ57 �/1R@A7�ÓÀQR@C7ÀD.QR353N7\D�?C7\V ?AQ/7\1YDGZ»QR?AZ57\@P?AZN1R3 ?CZ57.ÕÖ14@C7D�QR3N357\D�?A7\V ã B©?AZI?AZ57\ÓÀXA7.ÔEñR7MX.=YÕR7.?-?AZ57JXUÕ�XU?A7.Ó D�QY@A@C7\D�?AÔEÕ�B²V�7.3[?CBzúN7MX-?CZ57 ��1R3NV�?AZN7 �n1YXXU7\×N14@G1:?C7$QRå�æx7MD�?CX\ù(ÒuZN7n?AQY×�@ABE]RZ[?�7.ä514ÓÀ×5ÔE7nBEX�B©3[?C7.@C7\XU?ABE35]�ÙçQR@µXU7\ñR7\@C1RÔ @A7M1RXAQR3NX\ù �+B©@GXU??AZ57µXUÕ�XU?A7.Ó D�QY@A@C7\D�?AÔEÕ�B²V�7.3[?CBzúN7MX"?CZ57�?CZ5@A7\7�BE3NV�BEñ[B²V�;N1RÔ cW � ù �e7\D.QR3NV =R?AZ57JÔ©7.Ùê?<ZN1RÔzÙ81R3NV@ABE]RZ[?nZN1RÔzÙ�QRÙ?CZ57WV�B²XCD�QR3N357\D�?A7\V��»14@C7ID�QR@C@C7\D�?CÔ©Õ ]R@CQR;5×i7\V =¸7.ñY7.3Ö?AZ5QY;5]RZØ3NQ XUBEÓÀ×5ÔE7×5@AQ�ä�BEÓÀBz?xÕ�D�@CB©?A7.@CBEQR3ID�QY;5Ô²VWV�7MD�B²V�7�?CQ�1YXAXAQ�D�B²1:?A7�?AZ57µÔE7�Ùê?uZN14Ô©Ù�Q4Ù�?AZ57 �n?CQ�?CZ57JñY7.@A?AB²D.1RÔåN14@"QY3�B©?CX"ÔE7�Ùê?-QY@+QY3�B©?CX-@ABE]RZ[?\ù:ÒuZ57�@CBE]RZ[?+Z(14Ô©Ù(Q4Ùi?AZN7 �$V�Qe7MX"D.14;(XU7?AZN7u1R×5×i7\14@G143(D�7Q4Ù143�7.@C@CQR357\QR;NX cW � QY3�?AZ57 � $ J J QY;�?A×N;�?\=eå5;�?<?AZ5B²X 6W � B²X<@A7\Ó�Q:ñY7\V�åeÕ�?CZ57�]Y@C1RÓÀÓÀ1R@ã Z5BEDGZ�×5@C7.ñR7\3[?CX+V�[email protected][?"35QY3�é�åNÔE1R35F$DGZN14@G1RD�?A7.@GX Ùç@CQRÓ 1R×5×i7\14@CBE35]�QY3�D�QY3[?ABE]R;5QY;NX�QY;�?Ué×5;�?CX\ùeÒuZ57�åiQ4?A?AQRÓ ÔE7�Ùê?7�ä514ÓÀ×5ÔE7JV57.ÓÀQR3NXU?A@G1:?C7\X-?AZN14?<7�ä[email protected];NX"Ó�14@CF�X"?AZ(1:?V5Q�35QR?å(7\Ô©QY35]I?CQWB²V�7.3[?AB©ú(1Rå5Ô©7�DGZN14@G1RD�?A7\@CX�1R@A7�XA;5×5×N@A7MXAXA7\V/7\ñR7.3%?AZ5QY;5]RZ�?CZ57.Õ/Ó�1�Õ/D�QR3N357\D�?]R7.3e;5BE357�DGZ(14@G1RD�?C7.@GX?AQI7M1RDGZ/Q4?CZ57.@Mù(ÒuZN7nÔ©Q ã 7.@�@ABE]RZ[?�7�ä51RÓ�×NÔ©7�XAZ5Q ã Xu?CZ57�D.QRÓ�åNB©357MV@AQYå5;NXU?A357MXAX?CQÀDGZ(14@G1RD�?C7.@uQ:ñR7\@AÔ²14×(X.=[ñR7\@U?CBED\14Ô XUZNBzÙê?GX.=NXABEõ.7$ñ:14@CBE14?ABEQR3NX\=�143NV´35QYBEXA7Rù

�e7.ñY7.@G14Ô"14;�?CZ5QR@GXµZN1�ñR7À14@C]R;N7\V�?CZN1:?nBE3eñ:14@CBE1R3ND�7À143NV Ùç7\14?A;5@C7Àå5BE3NV�BE35]/ÙçQR@$Ó�;5Ôz?CBzé×5Ô©7�QRå5æx7\D�?µ@A7MD�QY]R35B©?ABEQR3�@C7 �[;5BE@A7MXJXA×i7\D�B©ú(D�ÓÀ7\DGZN1R35BEXAÓ�X�BE3eñRQRÔEñeB©3N]IÙç7.7MV�åN1YDGFi=i7�ä�×5ÔEB²D�B©?X ã B©?CDGZ5BE35]ÀV�7.ñeB²D�7MXµö �:é ã 1�ÕÀÓ�;5Ôz?CB©×NÔ©B²D.14?ABEñR7$D�QY35357\D�?ABEQR3NXGøOf 6¸1RV�7MXu7�?�14Ô�ùLWchNh��[l�=�QRå�æx7MD�?AéD�7.3[?C7.@C7\V$@C7.×5@C7\XA7.3[?G1:?ABEQR3(X.=�]Y@C1R×5ZnÓ�1:?GDGZ5BE35]�ÓÀ7\DGZN1R35B²XUÓ�X\=\QY@¸]Y7.357\@C14?ABEñR7"ÓÀQ�V�7.Ô²X¸?AZN14?1:?U?C7.ÓÀ×�?µ?AQ/XUBEÓ�;NÔz?G14357\QR;NXAÔ©Õ/7.ä[?C@C1YD�?J?AZ57�×(Q[XU7À143(V/?AZ57�D\1:?A7\]RQY@AÕ´Q4Ù-?CZ57�QYå�æx7\D�?CX\ù ��?BEXÀXAQRÓÀ7 ã ZN14?�V�B²XCD�QR3(D�7.@A?ABE35] ?AQÖQRåNXA7.@CñR7W?AZN14?�?CZ57�14åiQ:ñR7 � $ J J XA7.7\Ó�Xn?CQ CXUQYÔ©ñY7 �?AZ57�Ùç7M1:?C;5@A7Wå5B©3(V�B©3N]%×5@CQRå5ÔE7.Ó/=�1RÔ©åi7.B©?�×N1R@U?CBE1RÔ©ÔEÕ 1R3NVÛB©3Ú1%@A7MXx?C@AB²D�?C7\VÛD�QY3Y?C7�äe?\=�7.ñR7\3?AZ5QY;5]RZÎB©?I×(Q[XAXA7\XCXU7MXÀ35QÛå5;5BEÔz?´B©3ÎÓ�1RDGZ5BE357\@AÕØ?CQÚV�QÛBz?´7�ä�×5ÔEBED.Bz?CÔ©ÕYù ��Ùµ3NQ4?AZNB©35]Ø7\ÔEXA7R=?AZ57MXU7T7�ä�×i7.@CB©ÓÀ7.3[?GX�XAZ5Q ã ?AZ(1:?À×5;5@C7.ÔEÕ»Ùç7\7\Veé�ÙçQR@ ã 14@GV U3e;5ÓÀ7.@CBED\14Ô � Ó�;NÔz?CBzé�ÔE1�ÕY7.@�XUÕ�XUé?A7.Ó�X ã Bz?CZW1Pú5ä�7MVI14@GDGZ5B©?A7\D�?A;5@C7�D.1R3�7.Ó�;5ÔE14?A7�Ùç;N3ND�?CB©QY3NX-?CZN1:?u14×5×i7\1R@<D.QRÓ�åNB©3N14?AQY@AB²14Ô�=143NV»14@C7 �[;N1RÔ©B©?C14?ABEñR7.ÔEÕ�Ó�;(DGZ ÓÀQR@C7�D.QRÓÀ×5ÔE7�ä�?CZN143Û143[?AB²D�BE×N14?A7\V åeÕ�Ó�Q[Xx?Iö÷B©3ND.Ô©;(V�B©3N]?AZ57n1R;�?AZ5QY@CXGø�ù

�e7.ñY7.@G14ÔNXAZ5QR@A?1R35BEÓÀ14?ABEQR3NX2Q4Ù8?CZ57J6�7�J�7�?Ué�LD��$ J Jn=eB©3(D�ÔE;NV�BE35]�XAQRÓÀ7 ã B©?AZTDGZN14@G1RD�é?A7.@GX?AZ(1:?�ÓÀQ:ñR7PQR3´?AQY×´QRÙ+7M1RDGZTQ4?CZ57.@M=5D.1R3´åi7ñ[BE7 ã 7\VT1:? �� ����������������������� � ������� � � � "�# � �,'��( ( ù

�I- � ��=���@ % @ ;�@���; <:254 =�4!$�� ��27; ; <:4!� B <:;�+ � % ?A?��3ÖB©3[?A7\@A7MXx?CB©3N]T1R×5×5ÔEBED\1:?ABEQR3ÖQ4Ù � $ J J�X$B²XPQRå5æx7\D�?nV�7.?A7MD�?ABEQR3Û143(V XA×(QR?U?ABE35](ù8ÒuZ57ÀB©3�éñ�1R@AB²143(D�7Ö×5@AQY×(7\@U?CB©7MXTQ4ÙW9<QR3eñRQYÔ©;�?CB©QY3N14Ô J�7.? ã QY@AF�X\=�D�QRÓ�å5BE357\V ã Bz?CZÍ?AZ57Û74RID.B©7\3ND�Õã Bz?CZ ã ZNBEDGZ ?AZN7.Õ D\143 å(7�@C7.×NÔ©B²D.14?A7\V Q:ñR7\@ÀÔE1R@A]Y7WúN7\ÔEVNXIXU;N]R]R7MXx?À?CZN1:?À?AZN7.Õ D\143Îåi7;NXU7MVTÙçQR@ Aå5@C;�?A7$Ùç[email protected] � QRå5æx7\D�?µXU×iQ4?A?ABE35]W143(V�V�7�?C7\D�?CB©QY3�B©3�Ô²14@C]R7PBEÓ�14]R7MX.ù(ÒuZ57nÓ�14BE3BEV�7M1ÚB²XI?CQØ?A@G14BE3 1 XUBE35]RÔE7Ö9<QY3[ñYQRÔE;�?ABEQR3(14Ô J�7�? ã QR@CFb?CQ V�B²Xx?CB©3N]R;5B²XUZ B©Ó�14]Y7\XWQ4Ù$?AZN7QRå�æx7MD�?WQRÙnB©3[?A7\@A7MXx?WÙç@AQYÓ BEÓ�14]Y7\X�×5@C7\XA7.3[?WBE3 ?CZ57 åN1YDGFe]R@CQR;53NV8ù !J3ND�7%?A@G14BE357\V =<?AZN7357�? ã QR@CFÛBEX�@C7.×5ÔEBED\1:?C7\VÚXAQ»1YX�?CQÖD.Q:ñR7\@n?AZN7´7\3Y?CB©@C7TB©Ó�14]Y7I?CQ»åi7T1R3N14ÔEÕeõ.7MV =+?AZ57\@A7\å[Õ

Page 18: yann.lecun.comyann.lecun.com/exdb/publis/pdf/lecun-99.pdf · 1)(+*!! 7\@ "! $ '

ÙçQR@CÓ�BE35]I1�? ã Q4é�V�BEÓ�7\3NXAB©QY3N14Ô'�e×(1RD�7 $JB²XU×5Ô²1RD.7.ÓÀ7.3[? J�7.;5@G14Ô'J�7�? ã QR@CFLù5ÒuZ57nQR;5?A×5;�?JQ4Ù?AZ57 � $ J J BEXI1»? ã Q4é�V�BEÓ�7\3NXAB©QY3N14Ô�×5Ô²143N7TBE3 ã Z5B²DGZ ?CZ57�ÓÀQYXU?W1RD�?ABEñ�14?A7�;53NBz?GX�B©3(V�BzéD.1:?C7�?CZ57�×5@C7\XA7.3ND.7�QRÙ2?AZN7�QYå�æx7\D�?$Q4Ù<BE3[?A7\@A7MXx?PB©3»?AZN7ÀD.QR@C@A7MXU×iQR3(V�B©3N]W@C7\D�7\×�?ABEñR7�úN7\ÔEV ù�eB©3(D�7»?AZ57ÖXUBEõ.7»Q4Ùn?CZ57»QYå�æx7\D�?CXW?AQ å(7ÛV�7�?C7\D�?A7\V ã Bz?CZ5B©3¶?AZN7 BEÓÀ1R]R7Ö14@C7%;535Fe35Q ã 38=?AZ57ÖB©Ó�1R]R7»D\143¶å(7Ö×5@C7\XA7.3[?A7MVÎ?AQ ?AZ57Ö357.? ã QY@AFÎ1:?/Ó�;5Ô©?ABE×5ÔE7»@C7\XAQRÔE;�?CB©QY3NX.=u143(V ?AZN7@A7MXU;5Ô©?CX<1:?-Ó�;5Ôz?CB©×NÔ©7J@A7MXUQYÔ©;�?CB©QY3NX-D.QRÓ�åNB©357MV ùYÒuZN7�BEV57\1PZN1YX-åi7.7.3I1R×5×5ÔEB©7MV�?AQ$Ù÷1YD�7�ÔEQ�D.14é?ABEQR38=<f �-14BEÔ©Ô²143[?M= =�QR35@CQ�D �I1R3NVT687�9<;N3mW[h@h��2l�=81YV5V�@C7\XCX�å5ÔEQ�DGF�Ô©Q�D\1:?ABEQR3»QR3»7.3eñR7\Ô©QY×(7MXf �ÖQRÔ©Ù�1R3NV/>-Ô²1:?A?/W[h@h��2l�=N1R3NV´ZN143NV´?C@C1YDGF[BE35]�B©3/ñeB²V�7.QKf J�Q ã Ô²143/143NVT>-Ô²1:?A?/W[h@h Lcl�ùÒ�QTB©ÔEÔE;NXx?C@C14?A7�?AZ57�ÓÀ7.?AZ5Q�V = ã 7 ã B©ÔEÔ-D�QY3NXUB²V�7\@�?CZ57ID.1YXU7�Q4Ù-Ù÷1YD�7�V�7.?A7\D�?ABEQR3ÖB©3ÖBEÓ�é14]R7MX"1RX"V�7MXAD.@ABEåi7\V�B©3�f �-14BEÔEÔE1R3Y?M= =�QR3N@AQ�D ��1R3NV/687:9<;53#Wch@h��2l�ù��¸BE@GXx?M=4B©Ó�1R]R7\X"D.QR3[?C1RB©35é

B©35]ÛÙ÷1YD�7MXI1:?Iñ:14@CBEQR;NX�XCD.1RÔ©7MXI14@C7�D�QRÔEÔE7\D�?C7\V ù<ÒuZ5QYXA7�B©Ó�1R]R7\X�1R@A7TúNÔ©?A7\@A7MV ?AZ5@CQR;N]RZÎ1õ.7.@CQ4é�ÓÀ7\1R3´6�1R×5Ô²1RD�B²143WúNÔ©?A7.@�XAQÀ1RX?AQÀ@C7.ÓÀQ:ñR7Jñ:14@CBE14?ABEQR3NXBE3T]YÔ©QYåN14ÔLBEÔ©ÔE;5ÓÀBE3N1:?CB©QY3T1R3NVÔE1R@A]Y7�é�XAD\14ÔE7BEÔEÔ©;5ÓÀBE3N1:?CB©QY3W]Y@C1YV�B©7\3[?CX\ùRÒuZ57.3�=Y?C@C1RB©3NB©35]�XA1RÓ�×NÔ©7MX2QRÙ Ù÷1RD�7MX21R3NV�35QY3�é�Ù÷1YD�7MX14@C7WÓ�143e;N1RÔ©ÔEÕÛ7�äe?A@G1RD�?A7\VØÙç@AQYÓ ?CZ57\XA7TB©Ó�14]Y7\X\ù+ÒuZN7WÙ÷1YD�7/XU;5å5é�BEÓ�14]R7MX�1R@A7W?AZ57\3 XAB©õ\735QR@CÓ�14ÔEB©õ\7\VIXAQn?CZN1:??AZ57PZ57\B©]YZ[?QRÙ ?AZ57P7.3[?CB©@C7�Ù÷1YD�7µBEXu14×N×5@AQ�ä�BEÓ�1:?A7\Ô©Õ "���×5B©äe7\ÔEX ã Z5BEÔE7FR7.7\×5BE35]�Ù÷1RB©@CÔEÕIÔE1R@A]Y7�ñ:14@CB²1:?ABEQR3(Xµö ã Bz?CZ5BE3/1�Ù÷1RD�?AQY@QRÙ�? ã Q[ø�ù5ÒuZ57$XAD\14ÔE7JQRÙ¸åN1YDGF[]Y@AQY;53NVXU;5å5é�BEÓ�14]R7MX�1R@A7n×NBEDGFY7\V%14?J@G143NV�QYÓ/ù�ÊXUBE35]RÔE7�D.QR3eñRQYÔ©;�?CB©QY3N14Ô¸357�? ã QR@CFWB²X�?A@G14BE357\V�QY3?AZ5Q[XU7 XC14ÓÀ×5ÔE7\X�?AQbD.ÔE1YXAXABzÙçÕbÙ÷1RD.7 XU;Nå�é�BEÓ�14]Y7\XÀÙç@CQRÓ 35QR35é�Ù÷1YD�7%XA;5å�é�B©Ó�14]Y7\X\ù�� Z57\3 1XAD.7.357�BEÓ�14]Y7�BEX-?AQnåi7�143(14ÔEÕ[õ\7\V =RBz?2B²X"ú(@CXU?-úNÔ©?A7\@A7MV�?AZ5@CQR;5]YZ�?AZ57�6¸14×5Ô²1RD.BE1R3�úNÔz?C7.@M=Y1R3NVXU;5å5é�XC14ÓÀ×5ÔE7\VÖåeÕ @G1:?CB©Q[Xµ?AZN14?�14@C7�XU;ND\D�7MXAXAB©ñY7À×(Q ã 7.@GXPQ4Ù?AZ57´X �[;N1R@A7À@CQ[QR?nQ4Ù "�ù�ÒuZN7357�? ã QR@CF�B²X-@C7.×5ÔEB²D.1:?C7\V�Q:ñR7\@-7\1YDGZÀQ4ÙL?AZ57JB©Ó�1R]R7\X21:?27M1RDGZÀ@A7MXUQYÔ©;5?ABEQR38ù �¶XUBEÓÀ×5ÔE7�ñRQR?ABE35]?A7\DGZN35B �[;57$B²Xu;NXA7\V´?AQ�D.QRÓ�åNB©357µ?CZ57$@A7MXU;NÔz?GXÙç@AQYÓ Ó�;NÔz?CB©×5ÔE7$@C7\XAQRÔE;�?ABEQR3NX\ù

=�QR@C7%@C7\D�7\3[?AÔEÕR=uXUQYÓ�7Ö14;�?CZ5QR@GXIZ(1�ñR7%;NXA7\V J�7\;5@C1RÔ J�7�? ã QR@CF�X.=<QR@WQR?AZ57\@TD.ÔE1YXAXAB©éúN7.@GX/XU;NDGZ½1RXA�e;N×5×(QY@U? �"7MD�?AQY@K=%1RDGZ5BE357MXWÙçQY@TÙ÷1RD�7ØV�7.?A7MD�?ABEQR3 ã B©?AZÍ]R@C7\1:?�XA;ND.D.7\XCXf ��Q ã ÔE7.ÕY=�7�?�1RÔ�ùGW[h@h��e= !µXA;53N1�7�?�1RÔ�ùGW[h@hkj6l�ù�ÒuZ57\B©@�XUÕ�XU?A7.Ó�X$1R@A7�XAQRÓÀ7 ã ZN14?nXUBEÓÀB©Ô²14@P?CQ?AZ57nQY357�V�7MXAD.@ABEåi7\V�14åiQ:ñR7Y=5BE3ND�ÔE;NV�BE35]�?CZ57nB²V�7\1�QRÙ"×5@A7MXU7\3[?ABE35]À?AZ57�B©Ó�14]Y7$?AQI?AZ57n3N7�?Uéã QY@AFW14?�Ó�;5Ô©?ABE×5ÔE7nXAD\14ÔE7\X\ù5S;�?JXAB©3ND.7P?AZ5Q[XU7$XAÕ�Xx?C7.Ó�X�V�Q�35QR?�;NXA7�9<QR3eñRQYÔ©;�?CB©QY3N14Ô.J�7�?Uéã QY@AF�X\=5?AZN7.Õ%D.1R3535QR?�?C1RFR7�1RV�ñ:143[?G14]R7�Q4Ù-?CZ57�XA×i7.7\V5;5×»V�7MXAD.@ABEåi7\V�Z57\@A7Y=i1R3NV%Z(1�ñR7$?CQ@A7\Ô©Õ�QR3ÀQ4?CZ57.@"?C7\DGZ53NB �[;57MX.=RXA;NDGZ�1RX-×5@A7.é�ú(Ôz?C7.@CB©35]$1R3NVÀ@A7M14Ô©é�?CB©ÓÀ7?A@G1RDGFeBE35]N=�?CQnFR7\7.×�?AZN7D�QRÓÀ×5;5?C1:?CB©QY3N14Ô @C7 �[;5BE@C7.ÓÀ7.3[? ã Bz?CZ5BE3T@C7\1YXUQY3N14åNÔ©7µÔ©BEÓÀBz?GX.ù ��3/1YV5V�B©?ABEQR38=Nå(7MD.14;(XU7µ?AZNQYXA7D�Ô²1RXCXUB©úN7.@GX�14@C7WÓ�;(DGZbÔE7\XCX�BE3eñ:14@CBE1R3Y?�?AQÛXAD\14ÔE7´ñ:1R@AB²1:?CB©QY3NXn?CZN143 9<QR3eñRQYÔ©;5?ABEQR3N1RÔ J�7�?Uéã QY@AF�X\=:Bz?2B²X-3N7\D�7MXAXC14@CÕ$?CQ$;NXU7J1PÔE1R@A]Y73e;5Ó�å(7\@2Ó�;5Ôz?CBEXCD.1RÔ©7�BEÓ�14]Y7\X ã B©?AZÀúN3N7.ÔEÕYé�XA×N1RD.7\VXAD\14ÔE7\X\ù

� � Å5Ä��JË���Å5Ä"Æ�Ñ��C�-Å �ÊÃ8Å �¶Ã É���-Å���Ñ

$�7MXU×5B©?A7I?AZ57W1R×5×N1R@A7\3Y?�1Rå5BEÔ©B©?xÕ QRÙ?CZ57WXAÕ�Xx?C7.Ó�XnV�7MXAD.@ABEå(7MVÛB©3Û?CZ57I×5@C7.ñeBEQR;NX�XU7MD�?ABEQR3(X?AQÛXUQYÔ©ñY7TD.QRÓ�åNB©3N14?AQY@AB²14Ô<×5@AQYå5ÔE7.Ó�X ã Bz?CZ 35QY3�é�D�QRÓ�å5BE3N1:?CQR@CBE1RÔ2ÓÀ7\1R3NX.=+?CZ57.@C7T1R@A7TXUB©?Ué;N1:?CB©QY3NX ã Z57.@C7W?CZ57/357.7MVØÙçQY@ ïGð4ß�üNð�Ü�óêá�ó÷ð:àiì4òsóêá � 1R3NV D�QRÓ�å5BE3N1:?CQR@CBE1RÔ<XA7\14@GDGZ57MX�B²X�B©3�é7\XCD.14×(14å5ÔE7Rù �Ú]RQeQ�VP7�ä51RÓ�×NÔ©7-B²X¸ÔE1R35]R;N1R]R7"ÓÀQ�V�7.ÔEBE35]J1R3NV$ÓÀQR@C7-]R7\357.@G14ÔEÔEÕR=�ÓÀQ�V�7.Ô²X�?AZN14?B©3eñRQYÔ©ñY7Wú(35Bz?C7�é�Xx?G1:?C7T]Y@C1RÓÀÓÀ1R@CX\= ã 7\B©]YZ[?A7\VbúN35B©?A7.é�XU?C14?A7/Ó�1RDGZ5BE357MX.="QY@�Q4?AZN7.@À]R@G14×5Z5éåN1RXA7\V$Fe35Q ã ÔE7\V5]R7-@C7.×5@C7\XA7.3[?G1:?ABEQR3(X8XU;(DGZ�1RX�úN35B©?A7.é�XU?C14?A7-?C@C1R3NXCV�;ND�7\@CX\ù\ÒuZN7<Ó�14BE3n×iQRBE3[?Q4Ù�?AZNBEX+XU7MD�?CB©QY3$BEX�?AQ�XAZ5Q ã ?AZ(1:?¸]Y@C1YV�BE7.3[?Ué�åN1RXA7\VµÔE7\1R@A35BE35]�?C7\DGZ53NB �[;57MX�D.1R3nåi727�äe?C7.3NV�7MV?AQIXABz?C;N1:?CB©QY3NX ã Z57.@C7µ?AZ5Q[XU7PÓÀQ�V�7.Ô²X�14@C7P;NXA7\V ù

Page 19: yann.lecun.comyann.lecun.com/exdb/publis/pdf/lecun-99.pdf · 1)(+*!! 7\@ "! $ '

��?<B²X27M1RXAÕ�?CQ�XAZ5Q ã ?AZ(1:?2?CZ57µÓ�Q�V�;NÔE1R@<]Y@C1YV�B©7\3[?Ué�åN1RXA7\V�ÔE7\14@C35BE35]nÓÀQ�V�7.ÔL×5@C7\XA7.3[?C7\VB©3TXU7MD�?ABEQR3KWµD.1R3Iå(7P14×N×5Ô©BE7\V�?AQ�3N7�? ã QR@CFeX2Q4Ù8ÓÀQ�V�;5ÔE7\X ã Z5QYXA7JXU?C14?A7Jñ�1R@AB²14åNÔ©7MX<A�-�14@C7]R@G14×5ZNX ã B©?AZÛ3e;5ÓÀ7.@CBED\14Ô"BE3�ÙçQR@CÓ�1:?CB©QY3»1:?A?C1YDGZ57\V ?AQ/?CZ57�[email protected]�ö÷XCD.1RÔE1R@CX\=LñR7MD�?AQY@CX\=i7�?CDMø�=@C14?AZ57\@�?AZN1R3Îú5ä�7\Veé�XAB©õ\7�ñR7\D�?AQY@CX\ù2ÒuZ57\@A7%1R@A7/? ã QÚÓ�14BE3 D�QY3NV�B©?ABEQR3NXÀÙçQY@I?CZ5BEX\ù��+BE@CXU?\=?AZ57»Ó�Q�V�;NÔ©7MXWÓ�;NXx?´×5@AQ�V�;(D�7�?CZ57 ñ:14ÔE;57MXWQY3Î?AZ57»QR;�?C×5;�?´]Y@C1R×5ZNXÀÙç@CQRÓ ?AZN7 ñ:14ÔE;57\XQR3 ?AZ57�BE35×5;�?I]Y@C1R×5ZNX�?AZN@AQY;5]RZÎV�B©Ki7\@A7\3Y?CBE1Rå5ÔE7´Ùç;N3ND�?CB©QY3NX.ù �e7\D.QR3NV =�?CZ57�Q:ñR7\@C1RÔ©ÔÔEQYXCXÙç;53ND�?CB©QY3WXAZ5QR;5Ô²VÀåi7JD�QY3[?ABE3[;NQR;NX-1R3NVIV�B©[email protected][?AB²14å5ÔE7 ì4òsßÀð:Ü�á2Ý�ÿ:Ý�â � (¸ýNÝ�âCÝ2ã B©?AZW@A7MXU×i7\D�??AQ�?CZ57W×N1R@C1RÓÀ7�?A7\@CX\ù'J�7�? ã QR@CFeXPQRÙ�]R@G14×5Z5é�Ó�143NB©×5;NÔE14?ABE35]/Ó�Q�V�;NÔ©7MX�14@C7WD\14ÔEÔ©7MV � âCìCü5ý� âAì4à5Ü ë ð:â�ß�Ý.â &�Ý.á0(2ð4â+*�Ü f SQ4?A?AQY;8=N687:9<;53/143NV/S7.35]YB©QQW[h@hkjY= 687:9<;53T7�?�14Ô�ùGWchNh@i[l�ù

)'-0/ � 27E $ � @���2��74>< ; <:254 B <:;�+ =�� EG=�� +���E =�4>H�� 27E � @�E ?A@ ;CBD27EGF

ÒuZ5QR;5]YZ ?AZN7��e×(1RD�7 $�B²XA×5ÔE1YD�7\Ó�7\3[? J�7\;5@G14Ô J�7�?/ÓÀ7�?CZ5Q�V ×5@C7\XA7.3[?A7MVÎB©3 ?AZ57Ö×5@C7.ñeB©éQR;NX�XU7MD�?ABEQR3ØB²XnñY7.@CÕ»×N@AQYÓ�B²XAB©35]/ÙçQY@ ã QR@GVÛ@A7MD�QY]R35B©?ABEQR3Ø14×N×5Ô©B²D.14?ABEQR3NX\= ?CZ57WÓÀQR@C7�?C@C14éV�Bz?CB©QY3N14ÔÓÀ7�?CZ5Q�V ö÷1R3NVbXAQ�Ù÷1R@�XU?ABEÔ©Ô?CZ57WÓÀQYXU?�V�7.ñY7.ÔEQR×i7\VNønBEX�D.1RÔ©ÔE7\VØZN7.;5@CBEXU?AB²DIQ:ñY7.@AéXU7\]RÓÀ7.3[?C14?ABEQR38ù � B©?AZÛ?CZ5B²X$ÓÀ7�?AZNQeVNX.=�?AZ57 ã QR@GV B²XnXA7.]YÓ�7\3[?A7\V»BE3Y?CQ%D.1R3NV�B²V51:?C7IDGZN14@Aé1RD�?C7.@GX;NXUBE35]ÀZ57\;5@AB²XU?AB²DJBEÓ�14]Y7P1R3N14ÔEÕ�XUB²X<?A7MDGZ535B �Y;N7\X\ù ��3�ÙçQY@U?C;53N1:?C7.ÔEÕR=�B©?�BEX�1RÔ©ÓÀQ[Xx?�BEÓ�é×(Q[XAXAB©åNÔ©7?AQnV57.ñeBEXA7u?A7MDGZ535B �[;57\X+?CZN1:? ã BEÔ©ÔNB©3�Ù÷1RÔ©ÔEBEå5Ô©Õ�XU7\]RÓÀ7.3[?"3N14?A;5@G14ÔEÔ©Õ ã @AB©?U?C7.3 ã QY@CVNXB©3[?AQ ã 7\Ô©Ô�ÙçQR@CÓ�7MV/DGZN14@G1RD�?A7.@GX\ù5ÒuZ5BEXJXU7MD�?CB©QY3%143NVT?CZ57n357.ä[?µV�7\XCD�@CB©åi7nBE3 V�7.?C14BEÔ¸1�XABEÓ�é×5Ô©7$7.ä514ÓÀ×5ÔE7JQRÙ � Ò J¶ÙçQR@�@C7\1YV�BE35] ã QR@GV5X\ù�ÒuZ57$ÓÀ7�?AZNQeV/D\143W@C7.ÔEÕIQY3´]Y@C1YV�B©7\3[?Ué�åN1RXA7\VÔ©7M14@C35B©3N]J?CQn1�ñRQYBEV5X¸?CZ57�7�ä�×i7.3NXABEñR7�143NV�;53N@A7\Ô©B²14å5ÔE7u?C1YXUF�Q4ÙLÓ�143e;N14ÔEÔEÕ�XU7\]RÓÀ7.3[?CB©35]$1R3NVZN143NV�é�?C@A;�?CZ5BE35]�1�V514?C1RåN1RXA7µXAQÀ1RX<?AQ�?A@G14BE3W?CZ57P@A7MD�QY]R35BEõ.7\@<QY3WBE3NV�BEñ[B²V�;N1RÔ8DGZN14@G1RD�?A7\@CX\ù

�I@����#@K4�;�= ; <:254�- � B©ñY7.3�1 ã QR@GV =N1À3e;5Ó�åi7.@�Q4Ù-D.1R3NV�B²V51:?C7nD�;�?GX�14@C7P]R7\357.@G1:?C7\V ã Bz?CZZ57.;5@CB²Xx?CBED´ÓÀ7�?AZNQeVNX.ù-ÒuZ57�D.;�?�]R7.3N7.@G1:?ABEQR3bZ57.;5@CB²Xx?CBED´BEX�V57\XAB©]Y357\V XUQÛ1RX�?AQÖ]R7\357.@G1:?C7Ó�QY@A7%D.;�?CXI?AZN1R3 357\D.7\XCXA1R@AÕY="B©3 ?CZ57 Z5QY×(7/?CZN1:?W?AZ57 CD�QY@A@C7\D�? � XU7.?IQ4ÙnD�;5?CX ã BEÔEÔ�åi7B©3ND.Ô©;(V�7\V ù�!J3ND.7u?AZ57µD�;�?GX-ZN1�ñY7�å(7\7.3�]R7\357.@G1:?C7\V =Y14Ô©?A7.@C3N14?ABEñR7�XA7.]RÓÀ7\3Y?G1:?CB©QY3NX-14@C7�åi7\XU?@A7\×5@A7MXU7\3[?A7\VIå[ÕI1�]R@G14×NZ8=[D\14ÔEÔ©7MV�?CZ57 Ü�Ý�ÞRßÀÝ.àNá�ì:á�ó÷ð:àTÞ4âAìGü5ý ù5ÒuZ57$XU7\]RÓÀ7.3[?C14?ABEQR3I]Y@C1R×5ZBEX�1 �nóêâCÝCï.á�Ý ��� ï �Rï.òsó÷ï � âAìCüNý ö0$ � � ø ã B©?AZ 1%Xx?G14@A?�35Q�V�7W1R3NVØ143Ø7\3NVØ35Q�V�7Yù��<1RDGZB©3[?A7\@A3(14Ô�3NQeV57nBEXJ1RXCXUQ�D.BE14?A7\V ã B©?AZ 1�D\143NV5BEV514?A7�D.;�?J×N@AQ�V�;ND.7\V/åeÕ´?AZ57�XA7.]YÓÀ7.3[?C14?ABEQR314ÔE]RQR@CB©?AZ5Ó/ù �21RDGZI1R@CD�å(7.? ã 7\7.3T1�XAQR;5@GD�7J35Q�V�7µ1R3NVW1�V�7\XU?ABE3N14?ABEQR3W3NQeV57JB²X<1YXAXAQ�D�B²1:?C7\Vã Bz?CZÎ1R3 BEÓ�14]Y7´?CZN1:?ID.QR3[?C1RB©3NX�1RÔ©Ôu?CZ57/B©3NFbåi7�? ã 7.7.3 ?AZN7%D�;�?W1YXAXAQ�D�B²1:?A7MV ã B©?AZ ?AZN7XUQY;[email protected]�35Q�V�7P143(VÀ?AZ57$D.;�?u1YXAXAQ�D�B²1:?C7\V ã B©?AZ´?AZ57PV�7\XU?ABE3N14?ABEQR3W3NQeV57Rù ��3T14@GD�B²XD.@A7M1:?C7\Vå(7.? ã 7\7.3�? ã Qn35Q�V�7MX-B©Ù8?AZ57µXU7\]RÓÀ7.3[?C7.@2V�7MD�B²V�7\V�?AZN14?2?AZN7�×5BE7\D.7Yö÷XGø"QRÙ8B©3NF�åi7�? ã 7.7\3�?AZN7D�QR@C@C7\XA×(QY3NV�BE35]�D.;�?CXuD�QY;5Ô²VIÙçQY@AÓ)1�D.1R3NV�B²V51:?C7µDGZN1R@C1YD�?C7.@MùYÒ<Õ[×NBED\14ÔEÔ©ÕY=[7M1RDGZ´B©3NV5B©ñeB²V�;N14Ô×5B©7MD�7IQRÙBE35F ã QY;5Ô²V»åi7W1YXAXAQ�D�B²1:?A7MV ã B©?AZÚ1R3Ú14@GD4ù�>"1RB©@GXµQ4Ù�XA;ND.D.7\XCXUBEñR7À×5BE7\D.7\X$QRÙuBE35Fã QY;5ÔEVÀ1RÔEXAQ$å(7�B©3ND.Ô©;(V�7\V =Y;535ÔE7\XCX�?AZN7.ÕÀ14@C7�XU7\×N14@G1:?C7\V�å[ÕÀ1 ã B²V�7�]Y1R×8= ã Z5B²DGZÀBEX<1$D�ÔE7\1R@B©3NV5BED\1:?ABEQR3I?AZN14?<?CZ57.Õ�åi7.ÔEQR35]�?AQ�V�[email protected][?uDGZN14@G1RD�?A7.@GX\ù��21YDGZID�QYÓ�×NÔ©7.?A7�×(1:?AZW?AZ5@CQR;N]RZ?AZ57µ]R@G14×5Z´D�QR3[?G14BE3NX-7M1RDGZI×5BE7\D.7JQRÙ�BE35FÀQR3ND.7µ1R3NV�QR3NÔ©ÕÀQR3(D�7Rù �<1RDGZ�×N14?AZ´D.QR@C@A7MXU×iQR3(V5X?AQI1ÀV�B©Ki7\@A7\3Y? ã 1�ÕIQ4Ù�1RXCXUQ�D.BE14?ABE35]�×NB©7MD�7\X�Q4Ù¸BE35FW?AQR]Y7�?CZ57.@�XAQÀ1YX?AQÀÙçQY@AÓ DGZ(14@G1RD�?C7.@GX.ù

� @���2��74>< ; <:254���E =�4>H�� 27E � @�E#= 4�$� <:;�@ E�><���EG= 4�H � 2 E �#@ EK- � XAB©ÓÀ×5ÔE7 � Ò J)?CQ@A7MD�QR]Y35BEõ.7uDGZN14@G1RD�?A7.@�XU?A@CB©35][X+BEX"XAZ5Q ã 3�BE3 �¸BE]R;N@A7'i5ù�!J3NÔ©ÕP?CZ57�@ABE]RZ[?�å5@G143NDGZ�Q4Ùi?AZ57?AQY×ZN14Ô©Ù<B²X$;NXA7\V ÙçQR@P@C7\D�QY]R35B©?ABEQR3�ùLÒuZN7�Ô©7.Ùê?nå5@C1R3NDGZ B²XP;NXA7\V ÙçQR@P?CZ57À?A@G14BE35BE35]´×N@AQ�D�7MV�;5@C7V�7\XCD�@CB©åi7\V BE3 ?CZ57�357.ä[?IXA;5å�é�XU7MD�?CB©QY38ù-ÒuZ57 � Ò J�B²X�D�QRÓÀ×iQYXA7\V Q4ÙJ? ã QÛÓ�14BE3 ]Y@C1R×5Z

Page 20: yann.lecun.comyann.lecun.com/exdb/publis/pdf/lecun-99.pdf · 1)(+*!! 7\@ "! $ '

Path Selector

Viterbi Tansformer

GcvitGvit

Viterbi Transformer

Σ

Segmentation Graph

Gseg

RecognitionTransfomer

recT

Interpretation Graph

Gint

++

[0.6](−1)[0.7](+1)

1 [0.1](−1)4 [2.4](0)

4 [0.4](−1)2 [1.3](0)

3 [0.1](0)5 [2.3](0)

3 [3.4](0)4 [4.4](0)

4 [0.6](+1)9 [1.2](0)

[0.1](+1)

+ −

"34"DesiredAnswer

WNeural NetWeights

NN NN NN NN NN

4 4 1

(−1) (+1) (−1)

4 [0.6](+1)

3 [0.1](+1)

4 [0.4](−1)3 [0.1](−1) 1 [0.1](−1)

Gc3 [3.4](0) 4 [0.6](+1)

4 [2.4](0)3 [0.1](+1)

Loss Function

Segmenter

����� ¥��N¥ ^ ¾u_�� ^2vxoUc[w®ªx{GoAªxpYvx{ ¬²g.vʱ+g\vU} vx{GoCgM§MfYw®ªxwzg\f jed\«�{G} gMf��<{GpYvxws«�ªxwso��y�{Cv�¿a4{G§M­${Cf4ªUd�ªxwsgMf(°itupYvxwsfY§�vx{GoGg\§Mf[w®ªxwsgMfNk8gMfYqs¯Wªxc[{$vxws§Mc:ª�¿÷ced.fe}T¨[d.ªxc/g.¬-ªxc[{PªxgM¨�¨[d.v�ª�ws«µpY«�{�}N°¦ g\v2ª�vUd\wsf[wsf[§$±-w®ªxc��w®ªx{CvxjYw(ª�vUd.wzfYwsf[§YkegMf[q®¯�ªxc[{�qs{C¬Eª�¿÷ced.fe}À¨[d.ªxcÀws«pY«�{�}N° ¦ g\v}Yws«�oCvxws­$wsfed�ªxwsy�{�<w®ªx{Cvxj[w[ª�vUd.wsf[wsf[§RkYj�g.ªxcn¨ed.ªxcY«�d�vx{p[«�{G}N°�p[d\f:ªxw®ªxwz{C«�wsf�«x¼:p[d.vx{2jYvUd\oU��{Cªx«�d.vx{2¨�{Gfed.qsªxws{G«"oCgM­P¿¨YpYªx{�}J}YpRvxwzfY§<ªxcY{+¬²g.v�±"d.vU}P¨RvxgM¨[d\§�d�ªxwsgMf(°�ped.f4ªxw®ªxws{G«8wsfµ¨[d.vx{Gf:ªxc[{C«�{G«�d.vx{"¨ed�v�ªxwzd\qe}R{CvxwsyMd�ªxwsy�{G«oCgM­$¨[pRªx{�}�}YpYvxwsfY§µªxc[{uj[d\oU�:±"d.vU}�¨Yvxg\¨ed\§Md.ªxwsgMfN°

Page 21: yann.lecun.comyann.lecun.com/exdb/publis/pdf/lecun-99.pdf · 1)(+*!! 7\@ "! $ '

?A@G143NXUÙçQR@CÓÀ7.@GX�D.1RÔ©ÔE7\V$?AZ57 âAÝGïGðAÞRàNóêá�ó÷ð:àIá�âAì4à5Ü ë ð:â�ß�Ý�â�� (����\=:143NV$?AZ57 ��óêá�Ý.â6?�óLá�âAì:àNÜ ë ð4â�ßÀÝ.â��� +7&�ùLÒuZ57�]YQY14Ô¸Q4Ù-?CZ57�@C7\D.QR]Y35Bz?CB©QY3/?A@G143NXUÙçQR@CÓÀ7.@JBEX�?AQ´]R7.3N7.@G1:?A7�1I]R@G14×NZ8=iD\14ÔEÔ©7MV/?AZN7óêàNá�Ý�â�üLâAÝ�á�ì:á�ó÷ð:à ÞRâAìCüNý QR@ âAÝGïCðCÞRàNóêá�ó÷ð:à Þ4âCìCü5ý� +.-^&�=?AZN14?/D.QR3[?C1RB©3(X´1RÔ©ÔJ?AZ57Ö×iQYXCXUBEå5ÔE7B©3[?A7\@A×N@A7.?C1:?CB©QY3NXÀÙçQY@T1RÔ©Ô�?AZ57»×(Q[XAXAB©åNÔ©7ÖXU7\]RÓÀ7.3[?C14?ABEQR3NXIQ4Ù$?AZN7 BE35×5;�?Mù �<1RDGZ ×N14?AZ BE3� +7-^& @C7.×5@C7\XA7.3[?GX"QR357�×(Q[XAXAB©åNÔ©7�BE3[?A7\@A×5@C7�?G1:?CB©QY3�QRÙ QR357�×(14@A?AB²D�;5Ô²14@-XA7.]YÓ�7\3[?C1:?CB©QY3�Q4ÙL?AZN7B©35×N;�?\ù8ÒuZ57�@CQRÔE7�Q4Ù2?AZ57 ��B©?A7\@AåNB¸?C@C1R3NXUÙçQR@CÓ�7\@JB²XJ?AQT7�äe?A@G1RD�?�?CZ57�åi7\XU?$B©3[?C7.@C×5@A7.?C14?ABEQR3Ùç@AQYÓ ?AZ57$BE3[?A7\@A×5@C7�?G1:?CB©QY3´]Y@C1R×5Z8ùÒuZ57À@A7MD�QR]Y35B©?ABEQR3�?C@C1R3NXxÙçQY@AÓÀ7\@ � (���� ?C1RFR7\XJ?CZ57IXU7\]RÓÀ7.3[?G1:?ABEQR3 ]R@G14×5Z �� � � 1YXµB©3�é×5;�?\=-1R3NVb1R×5×5ÔEB©7MX�?AZ57T@A7MD�QY]R35BEõ.7\@nÙçQR@�XAB©3N]RÔE7TDGZN1R@C1YD�?C7.@GX$?AQ»?AZN7´BEÓ�14]R7MX�1YXAXAQ�D�B²1:?C7\V

ã Bz?CZT7M1RDGZTQ4Ù¸?AZ57n1R@CD\XuB©3´?AZ57nXA7.]YÓÀ7.3[?C14?ABEQR3T]R@G14×5Z8ù�ÒuZ57$BE3[?A7\@A×5@C7�?G1:?CB©QY3W]R@G14×NZ � +.-^&ZN1RX�1RÔ©ÓÀQ[Xx?�?CZ57�XC14ÓÀ7nXx?C@A;(D�?A;N@A7n1YXu?AZ57�XU7\]RÓÀ7.3[?C14?ABEQR3T]R@G14×5Z�=57�ä5D�7\×�?�?AZ(1:?�7M1RDGZ�14@GDBEXu@C7.×5Ô²1RD.7\VWå[Õ´1�XA7�?�Q4Ù�[email protected]<Ùç@CQRÓ)143(VI?CQ�?CZ57$XA1RÓÀ7J35Q�V�7Yù ��3W?CZ5B²X�XU7.?�Q4Ù¸1R@CD\X.=[?AZN7.@C7BEXJQR357�14@GDµÙçQR@µ7\1RDGZ/×iQYXCXUBEå5ÔE7�D.ÔE1YXAXuÙçQY@�?CZ57�BEÓ�14]Y7n1RXCXUQ�D�B²1:?C7\V ã B©?AZ�?CZ57�D.QR@C@A7MXU×iQR3(VeéB©35]T14@GDPB©3 � � �eùLÒ�Q´7\1RDGZ%1R@CDnBEXJ1:?U?G1RDGZ57MV%1ID.ÔE1YXAX�ÔE1Rå(7\Ô�=L143NV�?AZ57�×(7\3N14Ô©?xÕW?CZN1:?µ?AZN7B©Ó�14]Y7$å(7\Ô©QY35]YX�?AQÀ?AZNBEXJD�Ô²1RXCX�1RX�×5@CQ�V�;ND.7\VTåeÕI?AZN7n@A7MD�QY]R35BEõ.7\@\ù ��Ù+?AZ57�XU7\]RÓÀ7.3[?A7\@�ZN1YXD�QRÓÀ×5;5?A7\V ×i7.3N1RÔz?CB©7MX�ÙçQR@�?AZ57�D\143NV�B²V514?A7/XU7\]RÓÀ7.3[?CX\=+?AZ57MXU7/×i7.3N1RÔz?CB©7MX�14@C7´D.QRÓ�åNB©357MVã Bz?CZ»?CZ57�×i7.3N1RÔz?CB©7MXnD�QYÓ�×N;�?A7MV»åeÕ�?CZ57IDGZN1R@C1YD�?A7\@P@A7MD�QR]Y35BEõ.7.@M=(?AQ�QRå5?C14BE3»?AZ57�×i7.3N1RÔzé?ABE7\X�QR3b?AZ57�[email protected]�Q4ÙJ?AZ57TB©3[?C7.@C×5@A7.?C14?ABEQR3Ú]Y@C1R×5Z8ù ��Ô©?AZ5QY;5]RZ D�QRÓ�å5BE35B©3N]»×i7.3N1RÔz?CB©7MX�Q4ÙV�[email protected][?�3(1:?A;[email protected]\Ó�XnZ5BE]RZ5ÔEÕÖZ57.;5@CB²Xx?CBEDR=�?CZ57 � Ò J ?C@C1RB©3NB©35] ×5@AQ�D.7\V�;5@C7 ã BEÔ©Ô2?A;5357?AZ57$×i7.3N1RÔz?CB©7MX�143(VW?G14FY7P1YV�ñ:143[?C1R]R7µQ4Ù¸?AZ5B²X�D.QRÓ�åNB©3N14?ABEQR3T143eÕ ã 1�ÕYù �<1RDGZT×N1:?CZ/B©3T?AZN7B©3[?A7\@A×N@A7.?C1:?CB©QY3W]Y@C1R×5ZWD�QY@A@C7\XA×iQR3NV5X2?AQ�1�×(Q[XAXABEå5Ô©7µBE3Y?C7.@C×5@C7�?C14?ABEQR3´Q4Ù�?CZ57PB©3N×5;�? ã QY@CV8ùÒuZ57J×i7.3(14Ô©?xÕ�Q4Ù�1�×N14@A?AB²D�;5Ô²14@<B©3[?C7.@C×5@A7.?C14?ABEQR3�ÙçQY@1�×N14@A?AB²D�;NÔE1R@<XA7.]RÓÀ7\3Y?G1:?CB©QY3IBEX]YB©ñY7.3å[Õ ?AZ57TXU;NÓ Q4Ùu?CZ57W14@GDÀ×i7.3N1RÔz?CB©7MX�14ÔEQR3N]´?CZ57WD.QR@C@A7MXU×iQR3NV5B©35]/×N14?AZØBE3Û?AZ57WB©3[?A7\@A×N@A7.é?C1:?CB©QY3�]R@G14×5Z�ùL9<QRÓÀ×5;5?ABE35]�?AZN7�×i7.3N1RÔz?xÕ/Q4Ù-1R3�B©3[?A7\@A×N@A7.?C1:?CB©QY3�B©3(V�7.×i7.3NV57.3[?AÔEÕ/Q4Ù"?AZN7XU7\]RÓÀ7.3[?C14?ABEQR3Ú@C7 �[;5BE@C7\X�?AQ»D.QRÓ�å5B©3N7W?CZ57´×i7.3(14Ô©?ABE7\X�Q4ÙJ1RÔ©Ô?CZ57´×(1:?AZ(X ã B©?AZ ?AZ(1:?ÀB©3�é?A7.@C×5@C7�?G1:?ABEQR3�ù�ÒuZNBEXÀD.1R3båi7�V�QR357T;NXAB©3N]»?CZ57 xÙçQY@ ã 14@GV � 14ÔE]RQY@AB©?AZNÓ ã BEV�7\Ô©ÕØ;(XU7MVbBE3H�B²V5V�7.3K=%1R@AFYQ:ñ:=�Q�V�7.Ô²X\ùÒuZ57 ��B©?A7\@AåNB5?A@G143(XxÙçQY@AÓÀ7.@"×N@AQ�V�;ND.7\X-1n]R@G14×5Z ��� + & ã B©?AZI1�XUBE35]RÔE7�×N14?AZ8ù[ÒuZ5BEX2×N1:?CZBEX<?AZN7J×N14?AZ´Q4Ù8ÔE7\1RXU?�D�;5Ó�;5ÔE14?A7MVI×i7.3N1RÔz?xÕÀBE3I?AZN7JBE3[?A7.@C×5@C7�?G1:?ABEQR3W]R@G14×5Z8ùeÒuZ57µ@A7MXU;NÔz?Q4Ù

?AZ57�@A7MD�QR]Y35B©?ABEQR3%D.1R3�å(7�×5@CQ�V�;ND�7MV/åeÕT@C7\1YV�BE35]�Q4KÖ?AZ57�Ô²14åi7.Ô²X�Q4Ù"?CZ57�1R@CD\X�1RÔ©QY35]�?AZN7]R@G14×5Z � � +7& 7.äe?A@G1RD�?C7\VTåeÕI?CZ57 ��B©?A7\@AåNB8?A@G143(XxÙçQY@AÓÀ7.@Mù5ÒuZ57 �JBz?C7.@Cå5B8?C@C1R3NXUÙçQR@CÓ�7\@uQ ã 7MXBz?GX�3N14ÓÀ7µ?AQÀ?CZ57PÙ÷14ÓÀQY;NX ��óêá�Ý�â6?.ó2ì:ò ÞYð4â�óêáêý�ß ?AQÀú(3NV´?AZ57nXAZ5QR@A?A7MXx?�×N14?AZTB©3�1�]R@G14×NZ8ù

)'- , � E =�$><:@K4�; ���3= H�@�$ ��EG= <:4><:4!� 2�� =�� � ?ÒuZ57u×5@A7\ñeB©QY;NX�XU7MD�?ABEQR3ÀV�7MXAD.@ABEå(7MX+?AZ57u×5@CQ�D�7MXAX+Q4ÙL@C7\D�QY]R35BEõ.BE35]µ1µXU?A@CBE35]J;(XUBE35]$H�7\;5@AB²XU?AB²D!JñR7.@Aé��e7.]YÓ�7\3[?C1:?CB©QY38=51RXCXA;5ÓÀB©35]À?CZN1:?J?AZ57�@A7MD�QR]Y35BEõ.7.@�B²X�?A@G14BE357\V/XAQW1RX�?AQW1YXAXAB©]Y3/Ô©Q ã×(7\3N14Ô©?ABE7\X�?CQP?AZ57�D.QR@C@A7MD�?"D�Ô²1RXCX�ÔE1Rå(7\Ô5Q4Ù D�QY@A@C7\D�?AÔEÕnXU7\]RÓÀ7.3[?C7\V�DGZN14@G1RD�?A7.@GX\=MZNB©]YZ�×i7.3N1RÔzé?ABE7\X+?AQP7\@A@CQR357\QR;NX¸D.14?A7.]YQR@CB©7MX�QRÙiD.QR@C@A7MD�?AÔEÕ$XU7\]RÓÀ7.3[?C7\V�DGZ(14@G1RD�?C7.@GX.=�143(VnZ5BE]RZ�×(7\3N14Ô©?ABE7\X?AQ�14ÔEÔ D.1:?C7.]YQR@CB©7MX-ÙçQY@å(1RV�ÔEÕ�ÙçQR@CÓÀ7\V´DGZN14@G1RD�?A7.@GX\ù[ÒuZ5B²X�XA7\D�?CB©QY3W7.äe×NÔE1RB©3NXuZNQ ã ?AQ�?A@G14BE3?AZ57µXUÕ�XU?A7.Ó 1:?<?AZ57µXx?C@ABE35]$ÔE7.ñY7.Ô5?CQ�V�Qn?AZ57µ14åiQ:ñR7 ã B©?AZ5QY;�?<@C7 �[;5BE@ABE35]$Ó�143e;N1RÔNÔ²14åi7.ÔEB©3N]Q4Ù"DGZN14@G1RD�?A7.@uXA7.]RÓÀ7\3Y?GX.ù

��3%Ó�143eÕT1R×5×5ÔEBED\1:?CB©QY3NX.=N?AZ57\@A7�BEX�7.35QY;5]RZ%1W×5@CB©QY@AB¸F[3NQ ã Ô©7MV�]R7n1Rå(QY;�? ã ZN14?JB²X�7.äeé×(7MD�?A7MVÀÙç@CQRÓÊ7\1RDGZ�QRÙL?AZ57JÓ�Q�V�;NÔ©7MX-BE3IQR@GV�7.@-?AQn?C@C1RB©3À?CZ57.Ó XA7.×N1R@C14?A7\Ô©ÕYù��5QY@-7�ä514ÓÀ×5ÔE7R=ã Bz?CZ»H�7\;5@CBEXU?AB²D !JñY7.@AéM��7.]RÓÀ7\3Y?G1:?CB©QY3´QY357�D.QR;5Ô²V�BE3NV�BEñeBEV�;(14ÔEÔ©Õ/Ô²14åi7.Ô"XUBE35]YÔ©7.é�DGZN1R@C1YD�?C7.@B©Ó�14]Y7\X�143NVØ?A@G14BE3 1 DGZ(14@G1RD�?C7.@�@A7MD�QR]Y35BEõ.7.@�QR3Ú?CZ57.Ó/=�å5;�?�B©?�ÓÀBE]RZ[?�åi7�V�B7R�D.;5Ô©?�?CQ

Page 22: yann.lecun.comyann.lecun.com/exdb/publis/pdf/lecun-99.pdf · 1)(+*!! 7\@ "! $ '

QRå�?G14BE3 143 1R×5×5@CQR×5@CBE14?A7%XA7�?TQ4Ùn35QY3�é�DGZ(14@G1RD�?C7.@�BEÓ�14]R7MX�?CQÚ?A@G14BE3Î?AZN7 ÓÀQ�V�7.ÔJ?AQ @A7.éæx7\D�? ã @AQY35]RÔEÕWXA7.]YÓ�7\3[?A7\V�D\143NV5BEV514?A7MX.ù ��Ôz?CZ5QR;N]RZ%XA7.×N1R@C14?A7P?A@G14BE35BE35]�B²X�XAB©ÓÀ×5ÔE7R=(Bz?µ@A7.é�Y;NB©@C7\X�1RVNV�Bz?CB©QY3N14Ô�XA;5×i7.@Cñ[B²XAB©QY3nB©35ÙçQR@CÓÀ14?ABEQR3�?AZN14?�B²X+Q4Ùê?C7.3�ÔE1YDGF[BE35]JQR@�B©3(D�QRÓÀ×5ÔE7�?C7µöç?AZ57D�QR@C@C7\D�?$XA7.]RÓÀ7\3Y?G1:?CB©QY3»143(V%?CZ57ÀÔE1Rå(7\ÔEXPQRÙ<BE3ND�QY@A@C7\D�?PD\143NV�B²V514?A7�XU7\]RÓÀ7.3[?GXCø�ùLÒuZN7�ÙçQYÔzéÔ©Q ã BE35]%XA7\D�?CB©QY3ÚV�7\XCD�@CBEå(7MXP? ã Q Q4Ùu?AZ57WÓÀ1R3eÕ»]Y@C1YV�B©7\3[?Ué�åN1RXA7\V»ÓÀ7�?AZNQeVNX$ÙçQR@�?A@G14BE35BE35]� Ò J�é�åN1RXA7\V�ZN1R3NV ã @CBz?CB©35]�@A7MD�QR]Y35BEõ.7.@GX�1:?-?CZ57�XU?A@CB©3N]nÔ©7\ñR7\Ô�û �JBz?C7.@Cå5B5?C@C1RB©3NB©35]�143NVÀV�B²XxéD�@CB©ÓÀBE3N1:?CB©ñY7 �JBz?C7.@Cå5B ?C@C1RB©35BE35](ù ��35ÔEB©FY7nXUBEÓÀB©Ô²14@�1R×5×5@CQY1YDGZ57\X<B©3T?CZ57nD�QY3Y?C7�äe?�Q4Ù�XU×i7.7MDGZ@A7MD�QR]Y35B©?ABEQR38= ã 7%Ó�1RFR7%35Q @A7MD�QY;5@CXA7/?AQ 1Ø×5@CQRåN1Rå5BEÔ©B²Xx?CBED%B©3[?A7\@A×N@A7.?C1:?CB©QY38=2åN;�?/XAZ5Q ã?AZN14?\= ã Bz?CZ5BE3I?AZN7 � @G1RV�BE7.3[?Aé�Su1RXA7\V�6�7\1R@A35BE35]�1R×5×5@CQY1YDGZ8=YV�B²XCD�@CB©ÓÀBE3N1:?CB©ñY7�?A@G14BE35BE35]�B²X1XUBEÓÀ×5Ô©7$BE3NXU?C1R3ND�7PQ4Ù+?AZN7$×(7\@Añ:1RXABEñR7µ×[email protected]©×5ÔE7$Q4Ù+7.@C@AQY@�D�QR@C@C7\D�?CB©3N]�ÔE7\1R@A35BE35](ù

� < ; @ E �< ��EG= <:4><:4!�'- $J;5@CB©35]�@A7MD�QR]Y35B©?ABEQR38= ã 7�XA7.ÔE7\D�?-?AZ57�×N1:?CZIB©3À?CZ57 ��3[?C7.@C×5@A7.?C14?ABEQR3� @C1R×5ZÀ?AZN14?<ZN1YX"?CZ57�ÔEQ ã 7MXx?-×i7.3(14Ô©?xÕ ã B©?AZ�?CZ57 �JBz?C7.@Cå5BL14ÔE]RQR@CB©?AZ5Ó/ù �xV�7M14ÔEÔ©ÕY= ã 7 ã QR;5Ô²VÔ©BEFR7µ?AZNBEX�×(1:?AZTQ4Ù+Ô©Q ã 7\XU?�×(7\3N14Ô©?xÕ�?CQ�åi7n1RXCXAQeD.BE14?A7MV ã B©?AZT?AZ57$D.QR@C@A7MD�?uÔE1Rå(7\Ô�XU7 �[;57.3ND.71RX¸Q4Ùê?A7\3�1YX�×iQYXCXUBEå5ÔE7Rù ��3�QRåeñeB©QY;NX¸Ô©Q[XAX�Ùç;53ND�?ABEQR3n?CQJÓÀBE35BEÓ�BEõ.7B²X�?CZ57.@C7�ÙçQY@A72?AZ57u1�ñR7\@C1R]R7Q:ñR7.@u?CZ57$?C@C1RB©35BE35]IXA7�?�QRÙ�?AZ57n×i7.3(14Ô©?xÕ´QRÙ¸?CZ57�×N14?AZ ì�ÜGÜ.ðMï�ó÷ì4á�Ý � ("óêáçý�áçý5ÝÀïGð:â�âAÝGï.áuò©ì@?GÝ.òÜ�Ý��.íNÝ�àLïGÝ ?CZN1:?�ZN1RX�?AZ57´Ô©Q ã 7\XU?�×i7.3N1RÔz?xÕYù"ÒuZ57W]YQY14Ô<Q4Ù�?C@C1RB©35BE35] ã BEÔEÔ<åi7W?CQ%úN3(VØ?AZN7XU7.?�Q4Ù�@A7MD�QR]Y35BEõ.7.@×N1R@C1RÓ�7.?A7\@CXPöê?CZ57 ã 7.BE]RZ[?CX\=�B©Ù¸?CZ57$@A7MD�QY]R35BEõ.7\@B²X�1�357\;5@C1RÔ 357�? ã QR@CF5ø?AZN14?�ÓÀBE35B©ÓÀBEõ.7´?AZ57T1�ñR7\@C1R]R7�×i7.3N1RÔz?xÕÛQ4Ù�?CZ5B²X CD�QY@A@C7\D�? � ÔEQ ã 7MXx?�×i7.3(14Ô©?xÕÖ×(1:?AZ�ù�ÒuZN7]R@G1RV�BE7.3[?2QRÙi?CZ5B²X<ÔEQYXCX"Ùç;N3ND�?CB©QY3WD.1R3�å(7PD�QYÓ�×N;�?A7MV�åeÕ�åN1YDGF[é�×5@CQR×(14]Y14?ABEQR3�?AZ5@CQR;N]RZÀ?AZN7� Ò J 14@GDGZ5B©?A7MD�?A;N@A7IXAZ5Q ã 3ØBE3ØúN]R;N@A7:iN=�;NXAB©3N] QR3NÔ©Õ»?AZ57WÔ©7.Ùê?Ué�ZN143NVÚ×N1:?CZÚQ4Ùu?AZN7I?AQY××N14@A?\="143(VÚBE]R35QY@ABE35]%?CZ57/@ABE]RZ[?ÀZN14Ô©Ùxù-ÒuZ5B²X�?A@G14BE35B©3N]Ö1R@CDGZ5B©?A7MD�?C;5@A7TD�QY3Y?G14BE3NXÀ1 ]Y@C1R×5Z?A@G143NXUÙçQR@CÓÀ7.@ÀD.14ÔEÔE7\Vb1 üNì:áçý Ü.Ý�ò©ÝGï.á�ð4â ="B©3(XU7\@U?C7\VÚåi7�? ã 7.7\3b?CZ57 ��3[?A7.@C×5@C7�?G1:?ABEQR3 � @C1R×5Z143NVÛ?CZ57 �JBz?C7.@Cå5B�Ò�@G143(XxÙçQY@AÓÀ7.@Mù�ÒuZ5B²X�?A@G143NXUÙçQR@CÓÀ7.@$?G14FR7MX$?CZ57WBE3[?A7.@C×5@C7�?G1:?ABEQR3Ø]Y@C1R×5Z143NVn?AZ57uV�7\XABE@A7MVnÔ²14åi7.Ô�XU7 �[;57.3ND.71YX�BE35×5;5?\ù ��?"7�äe?A@G1RD�?CX�Ùç@AQYÓ½?AZ57BE3Y?C7.@C×5@C7�?C14?ABEQR3n]Y@C1R×5Z?AZ5Q[XU7�×N1:?CZNX�?CZN1:?PD�QY3Y?G14BE3�?AZN7�D�QY@A@C7\D�?�ö÷V57\XAB©@C7\VNø�ÔE1Rå(7\Ô+XA7 �[;57\3ND�7Yù��?GX�QY;�?A×N;�?µ]Y@C1R×5Z��� B²X2D\14ÔEÔ©7MV�?AZ57 ïCð4à5Ü�á�âAì:óêàLÝ � óêà(á�Ý.â�üiâAÝ.á�ì:á�ó÷ð:àIÞ4âCìCü5ý =�143NV�D.QR3[?C1RB©3NX214ÔEÔ5?AZN7�×N14?AZNX-?AZN14?D�QR@C@C7\XA×(QY3NV´?AQI?CZ57�D.QR@C@A7MD�?�Ô²14åi7.Ô�XU7 �[;57.3ND.7RùiÒuZ57�D.QR3NXU?A@G14BE357\VTBE3[?A7.@C×5@C7�?G1:?ABEQR3�]Y@C1R×5ZBEX�?CZ57.3 XA7.3[?�?AQÖ?AZN7 ��B©?A7.@Cå5B�?C@C1R3NXUÙçQR@CÓ�7\@ ã Z5BEDGZ ×5@CQ�V�;ND�7MX�1Ö]R@G14×5Z ������� ��ã Bz?CZ 1XUBE35]RÔE7�×(1:?AZ�ùiÒuZ5B²Xµ×N1:?CZ%B²X�?CZ57 CD�QY@A@C7\D�? � ×N14?AZ ã B©?AZ ?AZ57�Ô©Q ã 7\XU?�×i7.3N1RÔz?xÕYù5�+BE3N14ÔEÔEÕR=L1×N1:?CZWXAD.QR@C7.@�?C@C1R3NXxÙçQY@AÓÀ7\@"?C1RFR7\X � ����� � =e143(V�XUBEÓÀ×5Ô©Õ�D.QRÓÀ×5;�?C7\X-B©?CXD�;NÓ�;5Ô²1:?C7\VÀ×i7.3N1RÔz?xÕ ����� � å[ÕI1YV5V�BE35]�;5×T?AZ57P×i7.3N1RÔz?CB©7MX�14ÔEQR35]�?AZ57$×N14?AZ8ù5ÒuZN7JQY;�?A×5;5?�Q4Ù�?CZ5B²X � Ò J¶B²X?AZN7Ô©Q[XAXÙç;53(D�?ABEQR3TÙçQY@u?AZ57nD.;5@[email protected][?�×N14?U?A7\@A3�û

� ��� � ����� � ö �[øÒuZ57JQY35ÔEÕ�Ô²14åi7.ÔLBE3�ÙçQR@CÓ�1:?CB©QY3I?AZ(1:?uBEX@C7 �[;5BE@A7MVÀå[ÕÀ?CZ57P14åiQ:ñR7JXAÕ�Xx?C7.Ó B²X2?CZ57$XU7 �[;57.3ND.7Q4Ù V�7\XAB©@C7\VÀDGZN1R@C1YD�?C7.@�ÔE1Rå(7\ÔEX\ù�J�Q$F[3NQ ã Ô©7MV�]R7uQ4Ùi?AZN7�D�QY@A@C7\D�?"XA7.]YÓÀ7.3[?C14?ABEQR3�BEX"@C7 �[;5BE@C7\VQR3�?AZ57�×(14@A?-Q4Ùi?AZ57�XA;5×i7.@Cñ[B²XAQR@M=4XUBE3ND�7u?CZ57�XAÕeXU?A7\Ó DGZ5QeQ[XU7MX"14ÓÀQR35]µ?CZ57�XU7\]RÓÀ7.3[?G1:?ABEQR3(XB©3T?AZN7$B©3[?A7\@A×N@A7.?C1:?CB©QY3W]Y@C1R×5ZW?CZ57$QR3N7J?CZN1:?�ÕeBE7.Ô²V5X?AZN7$Ô©Q ã 7\XU?�×(7\3N14Ô©?xÕRùÒuZ57<×N@AQ�D�7MXAX+QRÙ(åN1YDGFYé�×5@CQR×N1R]Y14?ABE35]�]R@G1RV�BE7.3[?GX8?CZ5@AQY;5]RZ�?AZ57 �JBz?C7.@Cå5B�?A@G14BE35BE35] � Ò JBEXP35Q ã V�7\XCD�@CB©åi7\V8ù �JXP7�ä�×5Ô²14BE357\V B©3ÛXA7\D�?CB©QY3�WY=L?AZ57À]R@G1RV5B©7\3Y?GX�Ó�;NXU?Påi7À×5@CQR×N1R]Y1:?C7\VåN1RDGF ã 14@GV5XJ?AZ5@CQR;5]YZÖ1RÔ©Ô2Ó�Q�V�;NÔ©7MXnQ4Ù?AZN7 � Ò Jn=�BE3ØQR@GV�7\@P?AQ%D�QYÓÀ×5;�?A7�]Y@C1YV�BE7.3[?CXPBE3×5@A7MD�7MV�B©3N]�ÓÀQ�V�;5ÔE7\X�143(V�?AZN7.@C7\1:Ùê?C7.@<?A;53N7J?CZ57.BE@u×N14@G14ÓÀ7.?A7.@GX\ùYSu1RDGF[é�×5@AQY×N14][1:?CB©35]$]Y@C14éV�B©7\3[?CX�?AZ5@CQR;5]YZ/?AZ57�×N1:?CZ XCD�QR@C7.@�BEX �[;5Bz?C7�XU?A@G14BE]RZ[?UÙçQY@ ã 14@GV ù5ÒuZ57�×N14@A?AB²14Ô+V�7.@CBEñ�14?ABEñR7MXQ4Ù"?AZN7�ÔEQYXCX�Ùç;53(D�?ABEQR3 ã Bz?CZ @C7\XA×(7MD�?�?CQI?CZ57�BE3NV�BEñeBEV5;N14Ô+×(7\3N14Ô©?ABE7\XµQY3�?AZ57�D�QY3NXx?C@C1RB©3N7\V

Page 23: yann.lecun.comyann.lecun.com/exdb/publis/pdf/lecun-99.pdf · 1)(+*!! 7\@ "! $ '

��B©?A7.@Cå5B[×N1:?CZ � ����� � 1R@A7"7 �[;N14Ô:?CQ�WY=.XUBE3ND.7"?AZ57<Ô©Q[XAX Ùç;53ND�?CB©QY3nB²X�XAB©ÓÀ×5ÔEÕ�?CZ57<XA;5ÓÍQ4Ùe?AZNQYXA7×(7\3N14Ô©?ABE7\X\ù(Su1RDGF[é�×5@CQR×N1R]Y1:?CB©3N]$?AZ5@CQR;N]RZT?AZ57���B©?A7.@Cå5B�Ò¸@G143NXUÙçQR@CÓÀ7.@�B²X�7 �[;N1RÔ©ÔEÕ´XAB©ÓÀ×5ÔE7RùÒuZ57À×N14@A?AB²14Ô"V�7\@ABEñ:1:?ABEñR7MX�QRÙ�� ��� � ã B©?AZÛ@C7\XA×(7MD�?µ?AQ´?AZ57�×i7.3(14Ô©?ABE7\XµQR3»?AZ57�1R@CD\X�QRÙ2?AZN7D�QR3(Xx?C@C1RB©357MV�]R@G14×5Z ��� 14@C7gW�ÙçQY@"?AZ5Q[XU7µ[email protected]�?CZN1:?14×5×i7\1R@2BE3�?AZ57JD�QR3(Xx?C@C1RB©357MV ��B©?A7\@Aå5B×N1:?CZ ������� � =N1R3NV ��ÙçQR@�?CZ5QYXA7P?AZ(1:?�V�Q�35QR?\ù � ZeÕ´BEX�Bz?�ÔE7.]YBz?CB©Ó�1:?C7P?AQ�åN1YDGF[é�×5@CQR×(14]Y14?A7?AZ5@CQR;5]YZÖ1R3Û7\XCXU7\3Y?CBE1RÔ©ÔEÕ V�B²XCD�@C7�?A7ÀÙç;N3ND�?CB©QY3ÚXU;NDGZÛ1RX$?AZ57 �JBz?C7.@Cå5B<Ò¸@C1R3NXUÙçQR@CÓ�7\@ �ÀÒuZN7143NX ã 7\@WB²XI?CZN1:?´?AZ57 ��B©?A7\@AåNBµÒ¸@C1R3NXUÙçQR@CÓ�7\@IB²XW35QR?AZ5BE35] Ó�QY@A7 ?AZN1R3 1bD.QRÔEÔ©7MD�?CB©QY3 Q4Ù#��,(ÖÙç;53ND�?ABEQR3NXÀ143(VÚ1RV5V57.@GXn×5;�?�?AQY]R7.?AZ57\@\ù ��?ÀD\143Øåi7TXAZ5Q ã 3Ú7M1RXAB©ÔEÕ ?CZN1:?�]R@G1RV�BE7.3[?GXD.143bå(7TåN1RDGF[é�×5@CQR×N1R]Y1:?C7\V»?AZN@AQY;5]RZ #��,(ÖÙç;53ND�?CB©QY3NX ã Bz?CZ5QR;5?�1YV�ñR7\@CXA7I7.Ki7MD�?CX\ù"Su1RDGF[é×5@AQY×N14][1:?CB©QY3J?CZ5@CQR;5]YZ$?AZ57<×N1:?CZ�XA7.ÔE7\D�?CQR@�?A@G143NXUÙçQR@CÓÀ7.@�B²X+XUBEÓÀB©Ô²14@�?AQ�å(1RDGF[é�×N@AQY×N14][1:?ABEQR3?AZ5@CQR;5]YZT?AZ57 �JBz?C7.@Cå5B8?C@C1R3NXxÙçQY@AÓÀ7\@\ù ��@GD.X�B©3 � � ��� ?AZN14?J14×5×i7\1R@�B©3 � � ZN1�ñR7µ?CZ57�XC14ÓÀ7]R@G1RV�BE7.3[?µ1YX�?CZ57�D�QY@A@C7\XA×(QY3NV�BE35]W1R@CD�B©3 � � =LB�ù 7Yù W�QY@ �N=LV57.×i7.3NV�BE35]TQR3 ã Z57�?CZ57.@µ?AZN714@GD�14×N×(7M14@µB©3 � ����� � ù ÒuZ57�Q4?CZ57.@$1R@CD\X.=(B�ù 7RùL?CZ5QYXA7n?CZN1:?nV�Q´35Q4?PZN1�ñR7�143 ì:òsá�Ý�â�Ý�ÞYð BE3� � å(7MD.1R;NXU7T?CZ57.Õ V�QÛ35Q4?ID�QY3[?C14BE3Ú?CZ57�@CB©]YZY?�Ô²14åi7.Ô�ZN1�ñY7T1Ö]Y@C1YV�B©7\3[?�QRÙ �5ù $J;5@CB©3N]?AZ57´ÙçQR@ ã 1R@CVÖ×5@CQR×(14]Y14?ABEQR3Û?CZ5@AQY;5]RZÛ?CZ57T@A7MD�QR]Y35B©?ABEQR3Û?A@G143(XxÙçQY@AÓÀ7.@M=�QY357WBE3NXU?C1R3ND�7´Q4Ù?AZ57$@C7\D.QR]R3NB©õ\7.@ÙçQR@�XABE35]RÔE7$DGZN14@G1RD�?A7.@ ã 1RX�D.@A7M1:?A7MV´ÙçQR@�7\1YDGZT1R@CDµBE3/?AZ57nXA7.]YÓÀ7.3[?C14?ABEQR3]R@G14×5Z8ù8ÒuZN7IXx?G1:?A7�QRÙ@C7\D.QR]Y35B©õ\7.@PBE3NXx?G143ND.7\X ã 1RX$XU?AQY@A7MV ù'�eBE3ND�7I7M1RDGZÛ14@GD�×(7\3N14Ô©?xÕ BE3��� ��� B²X�×5@CQeV5;ND�7MV´åeÕW1R3/B©3NV5B©ñeB²V�;N14Ô�QR;�?C×5;�?�QRÙ�1�@C7\D�QY]R35BEõ.7\@uB©3NXU?C1R3ND�7Y= ã 7$35Q ã ZN1�ñY71P]R@G1RV�BE7.3[?�öXW�QY@��Yø�ÙçQR@-7M1RDGZÀQR;�?C×5;�?2QRÙ 7\1RDGZÀBE3NXU?C143(D�7�Q4ÙL?CZ57�@C7\D�QY]R35BEõ.7\@\ù���7MD�QY]R35BEõ.7\@QR;�?C×5;�?CXµ?CZN1:?PZN1�ñY7�1´35QR3»õ.7.@CQT]R@G1RV�BE7.3[?µ1R@A7�×N14@A?PQ4Ù-?CZ57�D�QR@C@C7\D�?P143NX ã 7.@M=i1R3NV ã BEÔ©Ô?AZ57\@A7.ÙçQR@C7�ZN1�ñR7�?CZ57.BE@2ñ:1RÔ©;57�×5;NXAZ57\VIV5Q ã 38ùRÒuZ57J]R@G1RV�BE7.3[?CX-×5@A7MXU7\3[?-QR3À?AZN7�@C7\D�QY]R35BEõ.7\@QR;�?C×5;�?CXD.1R3Iåi7JåN1YDGF[é�×5@CQR×(14]Y14?A7MVn?CZ5@AQY;5]RZ�7M1RDGZ�@C7\D.QR]R3NB©õ\7.@-BE3NXU?C143(D�7RùK�5QY@<7M1RDGZ�@C7\D�éQR]R3NB©õ\7.@nBE3NXU?C143(D�7R= ã 7WQRå�?G14BE3Ú1�ñR7MD�?AQY@nQ4Ù�×(14@A?AB²14Ô2V�7\@ABEñ:1:?CB©ñY7\XnQRÙu?AZ57´Ô©Q[XAXPÙç;N3ND�?CB©QY3ã Bz?CZT@C7\XA×i7\D�?u?CQ�?CZ57P@A7MD�QY]R35BEõ.7\@<BE3NXU?C143(D�7P×N14@G14ÓÀ7.?A7.@GX\ù���Ô©Ô ?CZ57P@A7MD�QY]R35BEõ.7\@BE3NXU?C1R3ND�7MXXUZN1R@A7W?AZ57�XC14ÓÀ7T×N14@G14ÓÀ7�?C7.@�ñR7MD�?AQY@\=�XAB©3ND.7T?AZ57\ÕÚ14@C7´ÓÀ7.@C7.ÔEÕÚD�ÔEQR3N7\X�Q4ÙJ7M1RDGZ Q4?AZN7.@M=?AZ57\@A7.ÙçQR@C7�?CZ57ÀÙç;5ÔEÔ-]R@G1RV5B©7\3Y?$QRÙ2?AZN7�Ô©Q[XAXµÙç;53(D�?ABEQR3 ã B©?AZÛ@C7\XA×(7MD�?$?AQT?CZ57�@A7MD�QY]R35BEõ.7\@cM X×N14@G14ÓÀ7�?C7.@nñY7\D�?AQR@�BEX�XUBEÓÀ×5ÔEÕ»?CZ57´XA;5Ó Q4Ù�?CZ57W]Y@C1YV�BE7.3[?�ñR7MD�?CQR@GX$×5@CQeV5;ND�7MVÛå[ÕÛ7M1RDGZ@A7MD�QR]Y35BEõ.7.@uBE3NXU?C143(D�7Rù �JBz?C7.@Cå5B ?C@C1RB©35BE35](=e?AZ5QY;5]RZTÙçQR@CÓ�;NÔE14?A7\VTV�B©Ki7\@A7\3[?AÔEÕR=5B²XuQ4Ùê?C7.3/;NXA7B©3�H9=�=/é�å(1RXA7\VTXU×i7.7MDGZ´@C7\D.QR]R3NBz?CB©QY3TXAÕ�Xx?C7.Ó�Xf ��14å5BE357.@W[h@iNh[l�ù

� Z5B©ÔE7WB©?�XU7\7.Ó�X�XABEÓ�×NÔ©7/143NVbXA14?AB²XxÙçÕeBE35]N=¸?AZ5B²X�?A@G14BE35B©3N]»14@GDGZ5B©?A7MD�?A;N@A7IZ(1RX�1 /(1 ã?AZN14?PD\143 ×iQ4?C7.3[?AB²14ÔEÔEÕ�å(7�Ù÷1:?G14Ô�ù���Ù2?AZ57�@A7MD�QY]R35BEõ.7\@�B²Xµ1TXAB©ÓÀ×5ÔE7�3N7.;5@G14Ô+357�? ã QR@CF ã Bz?CZXUBE]RÓÀQRB²VÖQR;�?C×5;�?n;53NBz?GX.=�?AZ57IÓÀBE35BEÓ�;5Ó Q4Ù?AZ57WÔ©Q[XAXµÙç;53(D�?ABEQR3ØB²Xn14?U?C1RB©3N7\V =¸35Q4? ã Z57\3?AZ57µ@A7MD�QY]R35BEõ.7\@<14Ô ã 1�ÕeX<]RBEñR7MX"?CZ57P@ABE]RZ[?1R3NX ã 7\@\=[å5;�? ã Z57\3WB©?BE]R35QY@A7MX-?CZ57µB©35×N;�?\=51R3NVXU7.?CXnB©?CX�QR;�?C×5;�?�?AQ 1%D.QR3NXU?C1R3[?nñR7\D�?AQY@ ã Bz?CZbXAÓÀ1RÔ©Ô<ñ:14ÔE;57\XPÙçQY@�1RÔ©Ô2?AZ57WD.QRÓÀ×iQR357\3Y?GX.ùÒuZ5BEX<B²X-Fe35Q ã 3I1RX áêýNÝ$ïGð:òêò©ìGü(Ü.ÝüLâAð@?.ò©Ý.ß ù�ÒuZ57µD�QRÔEÔ²14×NXA7�QR35ÔEÕ�QeD\D�;5@GX-B©Ù ?AZN7�@C7\D�QY]R35BEõ.7\@QR;�?C×5;�?CX�D\143/XUBEÓ�;NÔz?G14357\QR;NXAÔ©ÕÀ?G14FY7�?CZ57.BE@�ÓÀB©35BEÓ�;NÓ ñ:14ÔE;57Rù ��Ù+QR3´?AZ57PQR?AZ57\@�ZN143NVW?AZN7@A7MD�QR]Y35BEõ.7.@[M X<QY;�?A×N;�?�Ô²1�ÕR7.@�D.QR3[?C1RB©3(X ��S �b;53NBz?GX ã B©?AZ/ú5ä�7\VT×N1R@C1RÓÀ7�?A7\@CX\=e?AZN7.3/?AZN7.@C7BEXJ35QWXA;NDGZ�?C@ABEñeBE1RÔ¸XAQRÔE;�?ABEQR38ùiÒuZ5B²X�B²X�V�;N7n?AQW?AZ57�Ù÷1RD�?�?AZ(1:?µ1WXU7.?JQ4Ù ��S � ã B©?AZ�úNäe7MVV�BEXU?ABE3ND�?/×(14@G14ÓÀ7�?C7.@WñY7\D�[email protected]?/XUBEÓ�;5Ô©?C1R357.QY;NXAÔ©Õ ?C1RFR7 ?AZ57\B©@/ÓÀB©3NB©Ó�;5Ó ñ:14ÔE;57Rù��3b?CZ5BEX�D\1RXA7R=+?AZN7/D.QRÓÀ×5ÔE7�?C7TD.QRÔEÔE1R×NXU7/V�7MXAD.@ABEå(7MVb1Rå(Q:ñY7TV�Qe7MX�3NQ4?�Q�D.D�;N@\ù"H�Q ã 7.ñY7.@M=?AZ5B²X�V�Qe7\XÀ35QR?À?AQ4?G14ÔEÔ©ÕØ×N@A7\ñR7.3[?�?AZ57�Q�D\D�;5@C@A7\3ND�7´Q4ÙP1 ÓÀBEÔEV�7\@ID�QYÔ©Ô²14×NXA7Tå(7MD.14;(XU7T?AZN7Ô©Q[XAX-Ùç;53ND�?CB©QY3WXU?ABEÔ©ÔiZN1RX<1 /i1:?<XA×(QR? � ÙçQY@<1P?C@ABEñeBE1RÔ(XAQRÔE;�?ABEQR3 ã B©?AZ´D�QR3(Xx?G143[?2@C7\D�QY]R35BEõ.7\@QR;�?C×5;�?\ù:ÒuZ5B²X /(1:?"XU×iQ4?+BEX+1�XC1RVNV�Ô©7<×(QYB©3[?M=MåN;�?+B©?�BEX+1:?A?A@G1RD�?CB©ñY7-BE3�14ÔEÓÀQYXU?+1RÔ©ÔeV�BE@A7MD�?ABEQR3(X143NV�B²XñR7\@AÕÀV�B7R�D.;5Ô©??CQ�]Y7�?uQR;�?Q4Ù�;NXABE35]�]Y@C1YV�B©7\3[?Ué�åN1RXA7\V�ÓÀB©3NB©ÓÀBEõ\1:?CB©QY3W×5@CQ�D�7MV�;5@C7\X\ù��Ù�?AZN7$×N14@G14ÓÀ7�?C7.@GX2QRÙ�?CZ57���S �+X�1R@A7P14ÔEÔEQ ã 7MVÀ?AQ�1RVN14×�?M=Y?CZ57.3T?CZ57$D�QRÔEÔ²14×NXA7J×N@AQYå5Ô©7\Ó�X

Page 24: yann.lecun.comyann.lecun.com/exdb/publis/pdf/lecun-99.pdf · 1)(+*!! 7\@ "! $ '

@A7M14×5×i7\1R@CXuåi7\D\14;NXA7P?AZ57 ��S � D.7.3[?A7\@CX�D.143�1RÔ©Ô+D�QR3eñY7.@C]R7J?AQW1�XAB©3N]RÔE7$ñR7\D�?AQY@\=N1R3NVT?AZN7;53NV�7\@AÔEÕeB©35]�357.;5@G14ÔR357�? ã QR@CFJD\143PÔ©7M14@C3P?AQ�×N@AQ�V�;ND.7"?AZ(1:?�ñY7\D�?CQR@M=.143(VPB©]Y35QR@C7�?AZ572B©35×N;�?\ù� V�B©Ki7\@A7\3Y?JF[BE3NV/QRÙ�D.QRÔEÔE1R×NXU7$Q�D\D�;5@GX�B©Ù+?CZ57 ã B²Ve?AZ�QRÙ�?AZ57 ��S ��XJ14@C7P1RÔEXAQI14ÔEÔEQ ã 7MVI?CQ1RV51R×�?\ùNÒuZN7�D�QYÔ©Ô²14×NXA7$QR3NÔ©Õ´QeD\D�;5@GX�B©Ù-1À?A@G14BE3N14åNÔ©7$ÓÀQ�V�;5ÔE7�XA;NDGZ�1RX�1À357\;5@G14Ô8357.? ã QY@AFÙç7.7\VNX+?AZ57 ��S ��X.ù ��35Q4?CZ57.@"×N@AQYå5Ô©7\Ó ã B©?AZ ��B©?A7.@Cå5Be?C@C1RB©35BE35]µBEX+?CZN1:?"?CZ57u×(7\3N14Ô©?xÕnQRÙN?AZN7143NX ã 7\@µD\14353NQ4?Jåi7À;NXA7\V%@C7.ÔEB²14å5ÔEÕ�1RXP1WÓÀ7M1RXA;5@A7�Q4Ù<D.QR3�úiV�7.3ND.7�åi7\D\14;NXA7�Bz?$V�Qe7\Xµ35QR??C14FY7-ÔEQ ã é�×i7.3N1RÔz?xÕÀöçQY@�Z5BE]RZ5é�XCD�QY@ABE35][øLD.QRÓÀ×i7�?ABE35]µ143NX ã 7\@CX8B©3[?AQµ1RD\D�QR;N3Y?Mù � XUBEÓÀ×5Ô©7 ã 1�Õ?AQ/1RV5V�@C7\XCX�?AZ5B²XP×5@CQRå5ÔE7.Ó 143(V�?AQ/1�ñRQYBEV�?AZ57�D�QYÔ©Ô²14×(XU7�BEXJ?AQ´?A@G14BE3%?CZ57 ã Z5QRÔE7�XAÕ�Xx?C7.Óã Bz?CZ/1ÀV�B²XCD�@CB©ÓÀBE3N1:?CB©ñY7µÔEQYXCXÙç;53ND�?ABEQR3�1RX�V57\XCD�@CB©åi7\V´B©3T?CZ57$357�äe?�XA7\D�?ABEQR38ù

% <0H ��EG<&�#<:4>= ; < 67@ � <:;�@ E�>< ��E =�<:4><:4!��- ÒuZ57uB²V�7\1JQ4Ù(V5BEXCD�@CB©ÓÀBE3N1:?CB©ñY7 �JBz?C7.@Cå5Be?A@G14BE35BE35]BEX�?AQI3NQ4?JQY35ÔEÕ´ÓÀBE35B©ÓÀBEõ.7�?AZ57�D�;5Ó�;5Ô²1:?A7MV�×i7.3N1RÔz?xÕTQ4Ù"?CZ57�ÔEQ ã 7MXx?J×(7\3N14Ô©?xÕ/×N1:?CZ ã Bz?CZ?AZ57D�QR@C@C7\D�?8BE3[?A7\@A×5@C7�?G1:?CB©QY38=.å5;5?+14Ô²XAQu?AQJXAQRÓÀ7\Z5Q ã BE3ND.@A7M1RXA7�?AZ57<×(7\3N14Ô©?xÕPQ4Ù5D�QYÓ�×i7�?CB©3N]143NV/×iQYXCXUBEå5ÔEÕ´BE3ND.QR@C@A7MD�?�×N14?AZNX�?AZN14?�ZN1�ñY7�1IV51R35]R7\@AQY;NXUÔEÕIÔEQ ã ×(7\3N14Ô©?xÕRùiÒuZ5BEX�?xÕe×(7�Q4ÙD�@CBz?C7.@CB©QY3´B²X�D\14ÔEÔ©7MV � ó²Ü.ï�â�óêß�óêàiì4á�óêÿ:Ý =Låi7\D\14;NXA7$Bz?J×5ÔE1�Õ�X?CZ57n]RQeQ�V/143NX ã 7.@GX1R]Y1RB©3NXU?�?AZN7åN1RVÀQY357\X\ù�$JBEXCD�@CBEÓ�BE3N14?ABEñR7�?C@C1RB©35BE35]n×N@AQ�D�7MV�;5@C7\X-D\143�åi7JXA7.7\3I1RX-14?U?A7\ÓÀ×�?ABE35]n?CQnå5;5BEÔ²V14×5×5@CQR×N@AB²1:?C7�XU7\×N14@G1:?CB©3N]�XU;N@UÙ÷1YD�7\X2å(7.? ã 7\7.3TD�Ô²1RXCXU7MX-@G1:?AZN7.@-?CZN143W?AQ�Ó�Q�V�7\ÔLB©3NV5B©ñeB²V�;N14ÔD�Ô²1RXCXU7MX<BE3NV�7\×(7\3NV�7\3Y?CÔ©ÕTQ4Ù+7\1YDGZ´QR?AZ57\@\ù

!J357�7�ä514ÓÀ×5ÔE7�QRÙLV�B²XCD�@CB©ÓÀBE3N1:?CB©ñY7�D�@CBz?C7.@CB©QY3�B²X"?CZ57�V�B©[email protected]�å(7.? ã 7\7.3À?CZ57�×i7.3N1RÔz?xÕQ4Ùµ?AZ57���B©?A7\@AåNB�×N1:?CZÎB©3 ?AZN7%D�QR3(Xx?C@C1RB©357MVÚ]Y@C1R×5Z8=-143(VÚ?CZ57�×(7\3N14Ô©?xÕÚQRÙJ?CZ57 ��B©?A7\@Aå5B×N1:?CZIB©3W?AZ57Àöç;N3ND�QY3NXx?C@C1RB©3N7\VNø-B©3[?A7\@A×N@A7.?C1:?CB©QY3�]R@G14×5Z�=RB�ù 7Yù[email protected]�åi7�? ã 7.7\3I?AZN7×(7\3N14Ô©?xÕ QRÙ?CZ57�åi7\XU?�D�QY@A@C7\D�?P×(1:?AZ�=�143NVÖ?CZ57�×i7.3N1RÔz?xÕÖQ4Ù?AZ57�åi7\XU?�×N14?AZ ö�D�QR@C@C7\D�?$QY@B©3ND.QR@C@A7MD�?�ø�ù�ÒuZ57ID.QR@C@A7MXU×iQR3(V�B©3N] � Ò J ?A@G14BE35B©3N]%14@GDGZ5Bz?C7\D�?A;5@C7ÀBEX�XUZ5Q ã 3ÛBE3ÛúN]R;N@A7�i5ùÒuZ57JÔE7�Ùê?�XABEV�7µQ4Ù ?CZ57$V�B²14]R@G14ÓÊB²X2B²V�7.3[?CBED\14Ôi?AQ�?AZ57 � Ò JÍ;NXU7MVÀÙçQR@35QY3�é�V5BEXCD�@CB©ÓÀBE3N1:?CB©ñY7��B©?A7.@Cå5BL?C@C1RB©35BE35](ùeÒuZNBEXuÔEQYXCX<Ùç;53ND�?CB©QY3´@C7\V5;ND�7MX2?CZ57P@AB²XAF�Q4Ù+D.QRÔEÔE1R×NXA7�åi7\D.1R;NXA7µB©?uÙçQR@GD�7MX?AZ57$@C7\D.QR]R3NB©õ\7.@<?AQ óêàiï.âAÝGì�Ü.Ý�Ü ?AZN7$×(7\3N14Ô©?xÕIQ4Ù ã @CQR35]YÔ©ÕÀ@C7\D�QY]R35BEõ.7MVIQYå�æx7\D�?CX\ù�$�B²XAD.@ABEÓÀBzé3N1:?CB©ñY72?C@C1RB©35BE35]µD.143�14Ô²XUQJå(7�XA7.7\3�1RX�1R35Q4?CZ57.@+7�ä514ÓÀ×5ÔE7Q4Ù Ý.â�âAð:â�ïGð:â�âCÝCï.á�ó÷ð:à$üLâAðMïCÝ � í�âAÝ =ã Z5BEDGZW?A7\3NV5X<?AQ�ÓÀB©35BEÓÀB©õ\7�?CZ57$V�B©Ki7\@A7\3ND�7µå(7.? ã 7\7.3�?CZ57$V�7\XABE@A7MV�QR;�?C×5;�?�D�QYÓÀ×5;�?A7MVIBE3?AZ57JÔ©7.Ùê?ZN1RÔzÙ8QRÙ ?AZ57 � Ò J BE3Iú(]R;5@C79in1R3NVÀ?AZN7µ1YD�?A;(14ÔNQY;�?A×N;�?uD.QRÓÀ×5;�?C7\V�BE3�?CZ57J@CBE]RZ[?ZN14Ô©Ù+QRÙ�úN]Y;5@C7/iNù687�?u?CZ57$V�BEXCD�@CBEÓ�BE3N14?ABEñR7 ��B©?A7.@Cå5B ÔEQYXCX<Ùç;53ND�?CB©QY3/å(7$V�7\35Q4?C7\V���� ��� � =5143(VIÔE7�?�;NXuD.1RÔ©Ô

����� � ?AZ57�×i7.3(14Ô©?xÕbQ4Ùµ?AZN7 ��B©?A7\@Aå5B�×N1:?CZ B©3 ?AZN7%D�QR3(Xx?C@C1RB©357MVb]Y@C1R×5Z8=-1R3NV ��� � ?AZN7×(7\3N14Ô©?xÕWQRÙ�?AZN7 ��B©?A7.@Cå5B8×(1:?AZ/BE3T?AZ57$;53(D�QR3(Xx?C@C1RB©357MVIBE3[?A7.@C×5@C7�?G1:?ABEQR3T]R@G14×NZ8û��� ��� � ����� ��� ��� � ö �eø

��� ��� � B²XP14Ô ã 1�Õ�X�×iQYXAB©?ABEñR7ÀXUBE3ND�7�?AZN7ÀD.QR3NXU?A@G14BE357MV�]R@G14×5Z BEXP1TXU;5å(XU7.?µQRÙ-?AZ57À×N14?AZNXµBE3?AZ57TB©3[?C7.@C×5@A7.?C14?ABEQR3Ø]R@G14×NZ8=¸1R3NVÛ?AZN7 �JBz?C7.@Cå5B�14ÔE]RQR@CB©?AZ5Ó XU7\Ô©7MD�?GXn?CZ57W×N14?AZ ã Bz?CZÚ?AZN7Ô©Q ã 7\XU?�?AQ4?G14Ô(×(7\3N14Ô©?xÕRù ��3À?CZ57�B²V�7\1RÔND.1YXU7Y=:?AZN7�? ã QP×N14?AZNX ����� � 143(V ��� � D�QYB©3ND.BEV57R=R1R3NV� � ��� � B²X�õ\7.@CQNùS1YDGF[é�×5@CQR×(14]Y14?ABE35]%]Y@C1YV�B©7\3[?CX�?AZN@AQY;5]RZ ?AZ57%V5BEXCD�@CB©ÓÀBE3N1:?CB©ñY7 ��B©?A7\@Aå5B � Ò J 1RV5V5XXUQYÓ�7 U357\]Y1:?CB©ñY7 � ?A@G14BE35BE35]�?CQ�?AZ57<×5@C7.ñeB©QY;NXAÔ©ÕµV�7\XCD�@CB©åi7\VP3NQR3�é�V�B²XAD.@ABEÓÀB©3N14?ABEñR7�?C@C1RB©35BE35](ù�¸BE]R;N@A7�i�XAZ5Q ã X-ZNQ ã ?AZ57µ]R@G1RV5B©7\3Y?GX21R@A7JåN1RDGF[é�×5@AQY×N14][1:?C7\V ùRÒuZN7JÔE7�Ùê?uZN14Ô©Ù�B²XBEV�7\3[?AB²D.14Ô?AQ�?AZ57µ35QR35é�V�B²XCD�@CB©ÓÀBE3N1:?CB©ñY7 ��B©?A7\@Aå5Bi?C@C1RB©35BE35] � Ò Jn=[?AZ57\@A7.ÙçQR@C7�?CZ57Jå(1RDGF[é�×N@AQY×N14][1:?ABEQR3BEXµB²V�7.3[?AB²D.1RÔ�ù ÒuZ57�]R@G1RV5B©7\3Y?GX�å(1RDGF[é�×N@AQY×N14][1:?A7MV´?AZ5@CQR;5]YZ�?AZN7�@CB©]YZ[?µZN1RÔzÙ<Q4Ù-?CZ57 � Ò J14@C7�Ó�;5Ô©?ABE×5ÔEB©7MV åeÕbé$WR=<XUBE3ND�7 ��� � D.QR3[?A@CB©åN;�?A7MX�?AQÚ?AZ57»Ô©Q[XAX ã B©?AZ¶1Ø357.][1:?CB©ñY7%XUBE]R38ù

Page 25: yann.lecun.comyann.lecun.com/exdb/publis/pdf/lecun-99.pdf · 1)(+*!! 7\@ "! $ '

!�?AZ57\@ ã B²XU7�?AZN7�×5@AQ�D.7\XCXJB²X$XAB©ÓÀBEÔE1R@P?AQT?AZ57ÀÔE7�Ùê?nZN1RÔzÙxù¸ÒuZ57À]R@G1RV�BE7.3[?CXµQY3Ö1R@CD\XJQRÙ � � ���]R7�?×(Q[XUB©?ABEñR7µD�QY3Y?C@ABEå5;�?CB©QY3NX2Ùç@AQYÓ ?CZ57µÔ©7.Ùê?ZN1RÔzÙ�1R3NVI3N7.]Y14?ABEñR7�D.QR3[?A@CBEå5;�?ABEQR3(X-Ùç@CQRÓÊ?AZN7@ABE]RZ[?�ZN14Ô©Ùxù-ÒuZ57T? ã QÛD.QR3[?A@CBEå5;�?ABEQR3(X�Ó�;NXU?Àåi7%1RVNV�7\V =-XABE3ND�7T?AZN7�×(7\3N14Ô©?ABE7\XÀQY3 � � ���[email protected]µ14@C7�XA7.3[?µ?AQT?AZ57�? ã Q´Z(14ÔEñR7\X�?AZN@AQY;5]RZÖ1 U0 � D�QR3N357\D�?ABEQR3 BE3»?AZ57�ÙçQR@ ã 14@GV�×N1RXCX\ù��@GD.XnBE3 ��� ��� ?AZ(1:?�1R×5×(7M14@�357.B©?AZ57\@�BE3 � ��� � 35QR@�B©3 ������� � ZN1�ñY7I1�]R@G1RV5B©7\3Y?�Q4Ù�õ.7\@AQ(ùÒuZ57.ÕØV�QÖ35Q4?ÀD�QY3Y?C@ABEå5;�?C7I?CQ ?CZ57´D.QYXU?\ù ��@GD.X�?AZN14?À1R×5×(7M14@�B©3 å(QR?AZ � ��� � 1R3NV ������� �14Ô²XUQ�ZN1�ñY7Põ.7.@CQ�]R@G1RV5B©7\3Y?MùNÒuZ57$é$WnD�QY3[?A@CB©å5;5?ABEQR3TÙç@AQYÓ)?CZ57n@CB©]YZY?�Z(14Ô©Ù"D\143ND.7.Ô²Xu?AZN7n?AZN7� W/D.QR3[?A@CBEå5;�?ABEQR3 Ùç@CQRÓ ?CZ57�ÔE7�Ùê?IZN1RÔzÙxù ��3ÎQR?AZ57\@ ã QY@CVNX.= ã Z57\3Î1R3 14@GD´B²XÀ@CB©]YZY?AÙç;5ÔEÔ©Õ×N14@A?nQ4Ù?AZ57´143NX ã 7\@\=i?CZ57.@C7�BEXn35Q�]R@G1RV�BE7.3[?Mù ��Ùu1R3Ø14@GD�1R×5×i7\14@GXPBE3 � ����� � å5;�?�35Q4?nBE3� ��� � =e?CZ57P]R@G1RV�BE7.3[?uBEX�� WYù�ÒuZ57$14@GDJXAZ5QR;NÔEV´ZN1�ñR7JZN1RVT1�Ô©Q ã 7.@×i7.3N1RÔz?xÕ�?CQÀÓÀ1RFR7µBz?u?CQ� ��� � ù��Ù2143%1R@CDPB²X�BE3 � ��� � åN;�?J35QR?�BE3 � ����� � =N?AZ57�]R@G1RV�BE7.3[?�B²X�é$WRù(ÒuZN7�14@GDPZN1RV�1IÔ©Q ã×(7\3N14Ô©?xÕR=�å5;5?�XUZ5QY;5Ô²VIZN1�ñY7JZN1YV´1�Z5BE]RZ57\@u×(7\3N14Ô©?xÕWXAB©3ND.7PBz?�B²Xu35Q4?�×N1R@U?uQ4Ù¸?AZ57nV�7MXUBE@C7\V143NX ã 7\@\ù �-14@CBE14?ABEQR3NX�QRÙ"?AZ5B²X�?A7\DGZN35B �[;57�ZN1�ñR7nåi7.7\3%;NXA7\VTÙçQY@�?AZN7�XA×(7\7\DGZ�@C7\D�QY]R35B©?ABEQR3�ù$�@CBE1R3ND�QY;5@A?143(VISQ4?U?CQR;�f $J@AB²143ND.QR;5@A?�143NVTSQ4?A?AQR;mWchNh W$l ;NXA7\VI1�ñR7\@CXAB©QY3�Q4Ù�B©? ã Z57.@C7?AZ57$ÔEQYXCXÙç;53ND�?ABEQR3/BEX�XC1:?C;5@G1:?A7MVI?CQI1�ú5ä�7\V´ñ�1RÔ©;N7Rù

��3�BEÓ�×iQR@A?C1R3[?J1RV�ñ:1R3Y?G14]Y7$Q4Ù"]RÔEQRåN1RÔ�143(V�V�BEXCD�@CBEÓ�BE3N14?ABEñR7P?A@G14BE35BE35]IB²X�?CZN1:?�ÔE7\1R@A35éB©35]�ÙçQ�D�;NXA7\X2QR3À?AZN7�ÓÀQYXU?<BEÓÀ×(QY@U?G143[?27.@C@CQR@GX.=R1R3NV�?AZN7JXAÕeXU?A7\Ó Ô©7M14@C3NX�?AQ�B©3[?C7.]R@G1:?C7�?AZN714Ó�åNB©]Y;5Bz?CB©7MX+Ùç@AQYÓ ?AZN7uXA7.]YÓÀ7.3[?C14?ABEQR3�1RÔ©]YQR@CBz?CZ5Ó ã B©?AZ�?AZ57�1RÓ�å5BE]R;NBz?CB©7MX¸QRÙN?CZ57�DGZN14@G1RD�é?A7.@"@C7\D.QR]Y35B©õ\7.@Mù:ÒuZ57.@C7�14@C7<QR?AZ57\@�?A@G14BE35B©3N]µ×N@AQ�D�7MV�;5@C7\X+?CZN143�?AZ57�QY357\X"V57\XCD�@CB©åi7\V�Z57.@C7R=XUQYÓ�7Q4Ù ã Z5BEDGZ�14@C7V57\XCD�@CB©åi7\V�B©3Qf 6�7�9<;53T7�?�1RÔ�ù`Wch@hNi[l�ùR9<QYÓ�×NÔ©7.ä � @C1R×5Z�Ò¸@C1R3NXUÙçQR@CÓ�7\@Ó�Q�V�;NÔ©7MX�?AZ(1:?JD.QRÓ�åNB©357�B©3[?A7\@A×N@A7.?C1:?CB©QY3´]Y@C1R×5ZNX ã B©?AZ�Ô²143N]R;N1R]R7PÓÀQeV57.Ô²X�D.1R3�å(7�;NXU7MV?AQÀ?C1RFR7µÔ©BE35]Y;5BEXU?AB²D$D�QY3NXU?A@G14BE3Y?GXB©3[?AQI1YD.D.QR;53[?f 6�7�9<;53/7�?�1RÔ�ùGW[h@hNi[l�ù

� � �<Æ �.�UÐJÑeÇ��2Æ

ÒuZ572ÓÀ7�?CZ5Q�V5X�V�7\XCD�@CBEå(7MV$B©3n?AZ5B²X�×(14×i7.@�D�QR35úN@AÓ�X ã ZN1:?¸?AZ57Z5B²Xx?CQR@CÕJQRÙN>"1:?U?C7.@C3 ��7\D.QR]Ré35Bz?CB©QY3ÀZN1RX�14ÔE@C7\1RV5ÕnXUZ5Q ã 3�@C7.×i7\1:?C7\V�ÔEÕLûMúN3NV�BE35] ã 1�ÕeX¸?CQµBE3ND.@A7M1RXA72?CZ57�@AQYÔ©7Q4ÙiÔ©7M14@C35BE35]143NV/XU?C1:?CBEXU?AB²D.1RÔ 7\XU?ABEÓ�1:?ABEQR3�1RÔ©ÓÀQYXU?�B©3eñ:14@CB²14å5ÔEÕIBEÓ�×N@AQ:ñY7\X<?AZ57$×i7.@AÙçQR@CÓ�143ND.7PQ4Ù�@A7MD�QR]Ré35Bz?CB©QY3%XUÕ�XU?A7\ÓÀX\ù7�NQR@ "�$ XAZN14×i7n@C7\D.QR]R3NBz?CB©QY38=(9<QR3eñRQYÔ©;5?ABEQR3N1RÔIJ�7.;N@C1RÔ�J�7.? ã QY@AF�XuZN1�ñY7å(7\7.3�XUZNQ ã 3n?AQP7\Ô©BEÓÀB©3N14?A7?AZN7�357.7MVnÙçQY@"ZN143NV�é�D.@C14Ùê?A7\VnÙç7\14?A;5@C7u7�äe?A@G1RD�?AQR@GX\ù ��7\×5ÔEBED\1:?A7MV9<QR3eñRQYÔ©;�?CB©QY3N14ÔIJ�7�? ã QR@CF�XuZN1�ñR7Påi7.7.3»XUZ5Q ã 3´?AQWZN143NV5Ô©7$Ù÷1RB©@CÔEÕ´D�QYÓÀ×5Ô©7.ä´BE3NXU?C143(D�7\X�Q4Ù?AZ57WÙç7\1:?C;5@C7Iå5BE3NV�BE35] ×5@CQRåNÔ©7\Ó ã B©?AZb1 D�QRÓÀ×5ÔE7�?C7.ÔEÕ Ùç7\7\Veé�ÙçQR@ ã 14@GV = ?A@G14BE357MVØ14@GDGZ5Bz?C7\D�é?A;5@C7�BE3NXU?A7M1RV�QRÙ-?AZ57�ÓÀQR@C7n?C@C1YV�Bz?CB©QY3N14Ô�D�QYÓ�å5BE3N14?AQR@CB²14Ô�ZeÕe×iQ4?AZN7\XABEX�?A7MXx?CB©35]TÓÀ7�?CZ5Q�V5X.ù��3ÖXABz?C;N1:?CB©QY3 ã Z57\@A7ÀÓ�;5Ôz?CB©×NÔ©7ÀZeÕe×(QR?AZ57MXUB²XJ?A7\XU?ABE35]/B²XJ;N3N1�ñRQYBEV51Rå5ÔE7R=i?A@G14BE3N14åNÔ©7 � @C1R×5ZÒ�@G143NXUÙçQR@CÓÀ7.@ J�7�? ã QR@CFeXIZN1�ñR7%åi7.7\3 XAZ5Q ã 3 ?AQb@C7\V5;ND�7/?CZ57 357\7\VÎÙçQY@WZN1R3NVeé�D�@G1:Ùê?C7\VZ57.;5@CB²Xx?CBED\X.=[Ó�143e;N14ÔiÔ²14åi7.ÔEB©35](=[1R3NVIÓ�1R3[;(14ÔN×(14@G14ÓÀ7�?C7.@-?C;535BE35]�B©3TV�Q�D�;5ÓÀ7.3[?u@A7MD�QY]R35B©é?ABEQR3�XUÕ�XU?A7\ÓÀX\ù

� Ô©QYåN14ÔEÔ©Õ[é�?A@G14BE357\V � @G14×5ZnÒ¸@G143NXUÙçQR@CÓÀ7.@'J�7�? ã QR@CFeX�ZN1�ñR7"åi7.7\3�1R×5×5ÔEB©7MV$XU;ND\D�7MXAXUÙç;5ÔEÔ©Õ?AQÚQY3�é�ÔEBE357%ZN1R3NV ã @CBz?CB©35]b@A7MD�QR]Y35B©?ABEQR3 143NV DGZN7\DGF @A7MD�QR]Y35B©?ABEQR3 f 6�7�9<;53/7.?�1RÔ�ù`Wch@hNi[l�ùÒuZ57 DGZ57MDGFb@C7\D�QY]R35B©?ABEQR3ÎXAÕeXU?A7\Ó å(1RXA7\V QR3 ?AZNBEX´D�QR3(D�7.×5?WBEXW;NXA7\VÎD�QYÓ�ÓÀ7\@CD.BE1RÔ©ÔEÕbBE3XU7\ñR7.@G14Ô å(1435F�X�1RD.@AQ[XAX<?CZ57 � �/1R3NVT@A7M1RV5XuÓÀBEÔ©ÔEBEQR3NX�QRÙ�DGZN7\DGF�Xu×(7\@JV51�ÕYù5ÒuZ57nD�QY3ND�7\×�?CX143NV�@C7\XA;5Ô©?CX<B©3W?AZ5B²X×N14×i7.@Z57\Ô©×I7MXx?G14å5ÔEB²XUZ�?CZ57µ;NXU7.Ùç;5ÔE357\XCX<1R3NVI@C7.ÔE7.ñ:1R3ND�7�QRÙ�]R@G1RV5B©7\3Y?AéåN1RXA7\V ÓÀB©35BEÓÀB©õM1:?CB©QY3 ÓÀ7�?CZ5Q�V5XI1YX�1»]Y7.357\@C1RÔuQR@C]Y143NB©õ\B©35]»×5@CB©3ND.B©×NÔ©7/ÙçQY@�ÔE7\1R@A3NB©35]ÛBE3ÔE1R@A]Y7ÀXAÕ�Xx?C7.Ó�X\ù ��?�B²XnD�ÔE7\1R@$?AZN14? � @C1R×5ZØÒ�@G143(XxÙçQY@AÓÀ7.@ J�7�? ã QR@CF�X$D.143Ûåi7W14×N×5Ô©BE7\V»?CQÓÀ1R3eÕ»XABz?C;N1:?CB©QY3NX ã Z57.@C7À?AZN7WV�QRÓ�1RB©3ÖFe35Q ã Ô©7MV�]R7ÀQY@$?AZ57IXU?C14?A7�BE3�ÙçQR@CÓ�1:?CB©QY3ÛD\143Ûåi7

Page 26: yann.lecun.comyann.lecun.com/exdb/publis/pdf/lecun-99.pdf · 1)(+*!! 7\@ "! $ '

@A7\×5@A7MXU7\3[?A7\V/åeÕ�]Y@C1R×5ZNX\ù(ÒuZ5B²X�B²XJ?AZ57�D.1YXU7�B©3»ÓÀ1R3eÕTñeB²XU;(14Ô¸?C1RXAF�X ã Z57\@A7�]R@G14×5Z(X�D.1R3@A7\×5@A7MXU7\3[?�1RÔz?C7.@C3N1:?CB©ñY7PB©3[?A7\@A×N@A7.?C1:?CB©QY3NXuQ4Ù"1ÀXAD.7.357Y=5Ó�;NÔz?CB©×5ÔE7nBE3NXU?C143(D�7\XuQRÙ�1R3/QRå�æx7MD�?\=QR@�@C7.Ô²1:?ABEQR3(XUZ5BE×Wåi7�? ã 7.7\3/QRå�æx7MD�?CX\ù

Ì Ã �CÃ�Å5Ã�Æ �LøÑ� ~+{Cf[§Mwsg${Cª<d.qç°(l���� � �$~+{Gf[§\wsgYk��J°zk:h({�¹+pYf(k��J°sk��g\c[qêk4¹<°sk4d.fe}P~¸pYvx§M{C«GkR¹<°��l���� ���A°Nh({G|2{Go���^���� �� �<¯RjYvxwz}µ¬Eg\v �fR¿�hNwsf[{ �ud.fe}R±"vxw®ªxwsf[§�|2{GoGg\§Mf[w®ªxwsgMfN° ������������� �������������! "��#Yk[��� �,�A°

� ~+g.ª�ªxgMpId.fe}À¾ud.qsqzwsf[d.vxw+l$���Rl �n~+g.ª�ªxgMp(k"h�°"d.fe}Ú¾ud.qzqswsfed�vxwêk&% °���l����Rl �A°¶^ ¦ vUd\­${C±+g\vx�»¬Eg\vªxc[{µ¹+g:gM¨�{CvUd�ªxwsgMf/g.¬"h({Gd.vxf[wsfY§�^2qz§\g\vxw®ªxc[­$«G°�'�fW_igMpRvx{CªxrG�:¯:k�tµ°5d.fe}Wh(ws¨[¨Y­nd\fYf(ke|�°sk5{�}Yw®¿ªxg\vx«Gk�(&)�*+��#�,-��.& /#0�����������21-#$3$���-�4���! !��#657�8��,���.9.9 :#<;>=@?�.9�A���B.Uk�y�gMqsp[­${DCRk�t{Gf4y�{Cv�°��g\vx§�d.fE d\pR¬²­nd\fYf(°

� ~+g.ª�ªxgMpNk�h({�¹+p[fÀd.fe}À~¸{Gf[§\wzgÀl$���M� �n~+g.ª�ªxgMpNk�h�°sknh({�¹+p[f(k6�µ°sk$d.fe} ~+{GfY§MwsgYk4�J° ��l��������A°¾qsgMjed.q�_(vUd.wzfYwsf[§%g\¬�tug:oGpY­${Gf:ªF%8vxg:oG{C«�«�wzfY§Ûa:¯R«�ªx{G­$«np[«�wsf[§ ¾<vUd.¨[cÚ_(vUd.f[«�¬²g.vx­${CvÀ�<{Cª�¿±�g.vx�R«G° '�f�5D����,HGI�83>� ���J�����A���LKM :.9 "��#N��#�)65O���!�A���-#LPO��,-�9;�#@ :�! "��#YkQ%8p[{Cv�ªxg.¿�|2wsoGgY°@'� -°

� ~+pRvx§M{G«<d\f[}Àa4oAcSRgMqs��gM¨R¬�l��������n~+pYvx§\{G«Gk�¹<°L�Y°�¹<° d.fe}�a4oAcYg:{Gqs��gM¨R¬�k8~2° ��l��������A°�'�­$¨Rvxg.y4wsf[§ªxc[{2d\oCoGpYvUd.oC¯$d.fe}P«�¨�{G{�}Pg.¬N«�p[¨[¨�g.v�ª+yM{GoCªxg\v¸­nd\oUc[wsfY{G«G°Q'�f0´°��gMrG{Cv�k@´°:�Y°:d.fe}4%L{Cªx«�oUc[{Mk_°sk5{�}Yw®ªxg\vx«GkT(O)�*+��#�,���.B :#��U���������S1-#�3$���-�4�+�! "��#L5D����,-��.9.9 :#�;V=W?+.9�A���6.6X\°N_�c[{6Y'�_Z%8vx{G«�«Gk¹�d\­JjYvxwz}Y§\{M°

� tvxwzd.f[oGg\pYv�ª<d\f[}�~+g.ª�ªxgMp/l����Rl �$tvxwzd\fYoGgMpRv�ª�k\[J°ed\fe}�~¸g\ª�ªxgMpNkNh�°���l����4l �A°�ÀhT%Lk5h � � d\f[}t%7�.¹+gM­$¨[d.vxws«�gMfJ`Ö¹+g:g\¨�{CvUd.ªxwsg\f(°�'�fB5D����,-�9��)� :#�;�.D�83]�/^W�]1-#W�A���-#\�+�! "��#\���2_��+ :#W�I� ��#$3����9��#�,����#V�U�����8���Q�����!`]���9a+.xkNa4{�d.ª�ªxqs{M°

� tvxpYoA�M{Cv�k[aRoUced\¨Yw®vx{�d\fe}�aRws­nd.vU}Il�����C �Ptvxp[oU��{Cv�k ��°sk\aRoUced\¨Yw®vx{Mk�|�°sk�d.fe}JaRws­nd.vU}5kH%L° ��l�����C,�A°'�­$¨Rvxg.y4wsf[§$¨�{Av�¬²g\vx­nd.f[oG{Jwsf´fY{GpYvUd.q f[{Cª�±�g\vx�4«upY«�wsf[§Àdnj�g:gM«�ªxwsfY§Àd\qs§\g\vxw®ªxc[­À°�'�f �d\f[«�g\f(ka5°e�Y°sk(¹+g�±�d\fNkN�Y°�tµ°sk5d.fe}I¾wsqs{G«Gki¹<°5h�°sk�{�}Rw®ªxg\vx«GkT(&)+*+��#\,-��.U :#L�����������S1-#$3$���-�4���! !��#�5D����b,-��.-.9 :#<;c=W?+.9�A���6.Ud\kY¨ed.§M{G«Oe�f�g<e��RkNaRd\fLÀd�ªx{GgYk�¹�^�°��g\vx§Md\f E d\pR¬²­nd.f[f(°

� ¦ pY�4p[«�c[ws­nd$l��M� � � ¦ p[�4p[«�cYwz­nd4k E ° ��l��M� �,�A°n¹+gM§MfYw®ª�vxgMfS�L^Ía4{Gq®¬E¿ �<vx§�d.f[wsrGwsf[§��p[q®ªxwsq©dG¯:{Avx{�}�{GpRvUd\qN�{Cª�±�g.vx�5°ih7 "���j�9;� ",9���I�k?�l-���-#Q���! ",m.xkMf.m2�slHf:l�g[l�C 4°

� ¦ pY�4p[«�c[ws­nd�d\f[}��w®¯4d\��{Jl���n�f � ¦ pY�4p[«�c[ws­nd4k E °Md\f[}4�w®¯4d\�M{Mk4a�°���l���n�f��A°��{Gg:oCgM§MfYw®ª�vxgMfS�:^f[{C±Öd\qs§\g\vxw®ªxc[­ ¬²g.v�¨ed�ª�ªx{Cvxfµvx{GoCgM§MfYw®ªxwzg\f$ªxgMqs{CvUd\f:ª+g\¬N}Y{A¬²g\vx­nd�ªxwsgMf[«+d\f[}µ«�c[w®¬Eªx«¸wsf$¨�gM«�w®ªxwsgMf(°5&�+�!�A����#�P&��,9�9;�#W /�! "��#[kil ��� e � ��g�e ��R°

� �<p[j�{Gq(d\f[}�ºØws{G«�{Gq�l����f � �p[j�{Cqçk�tµ° ��°�d\fe}%ºØws{G«�{Gqêk+_°+��° ��l�� �f,�A°Ú|2{GoG{G¨Rªxwsy�{ ¦ ws{Gqz}Y«Gk~+wsf[g:oGpYqzd.vD'�f:ªx{CvUd\oCªxwsg\f(k:d.fe} ¦ p[f[oAªxwzg\fed.q[^-vxoAcYwsªx{CoCªxpYvx{"wsfPªxc[{2¹�d�ª�o « �<wz«�p[d\qe¹+g\v�ªx{C·5°I_�������b#\���S�83&5]^�?�.9 "���j�9;�?Np/qT�+#\)���#�r�kLl�\m2�sl�m�Hgel �+eR°

� E {G{Cqz{Av�ke|2p[­${Gqsced�v�ª-d\f[}�hN{Gg�±¶l$���Rl � E {G{Gqs{Cv�k��Y°sk�|<pY­${Gqsced.v�ª�kntJ°zk�d\f[} hN{Gg�±�k�ºÎ° E °��l����Rl �A°�'�f:ªx{G§\vUd�ªx{�}n«�{G§M­${Gf:ªUd�ªxwzg\fÀd.fe}Pvx{GoGg\§Mf[w®ªxwsgMf�g\¬Lc[d\f[}R¿÷¨Yvxwsf:ªx{�}$f4p[­${CvUd.qz«G°S'�f�h(ws¨Y¨Y¿­nd\fYf(kY|�°Q%L°skM�g:g4}4¯:k5�Y°M´°sk5d.fe}�_ig\pYvx{CªxrG�:¯:k[tµ°Na5°sk[{�}Yw®ªxg\vx«GkT�����������S1-#$3$���-�4���! !��#�5D����b,-��.-.9 :#<;=@?�.9�A���B.Uk\y�g\qzpY­${iC4k.¨ed\§\{G« � �\��g ���C4°��g.vx§�d\f E d\pR¬²­nd\fYfU%�pYj[qsws«�c[{Cvx«GkMaYd.fBÀd.ªx{GgRk¹�^�°

� h(dM}Y{C«2{Aªd.qê°il�����C �nhid\}Y{G«Gk2´°sk �¸g.vxjYv�RpY§M§M{Cf(k4�Y°:¹<°sk4~+p[cY­nd\fYf(k\�Y°sk4d\f[}µy�g\f$}Y{CviÀd.qz«�jYpYvx§Rk¹<° ��l�����C,�A°�tuws«�ªxg\v�ªxwsg\fU'�f4yMd�vxw©d.f:ª �j4³�{CoCªi|2{GoGg\§MfYwsªxwsg\fJwsfuªxc[{8t¯Rf[d\­$wso h(wsfY�u^2vxoUc[w®ªx{GoAªxpYvx{M°18s]s sutW�8��#W.mGD� ���J�QGzkWe�f��"C,��� C\mMm�g�CRl\lM°

� h(d�±"vx{Cf[oG{�{Cª2d.qç°(l������ ��h(dG±"vx{Gf[oG{\kYa5°skY¾wsqz{C«Gke¹<°Yh8°zkR_�«�g\wêk[^�°R¹<°skYd\fe}n~�d.oA��kR^�°4tµ°��l��������A°¦ d\oC{P|<{CoGgM§\f[w®ªxwsgMfS�L^͹+g\f4y�gMqspYªxwsg\fed\q��{CpYvUd\q��{Aª�±�g.vx�I^<¨Y¨Yvxg�d.oAcN°c18s s]svtW����#@.���,m�! "��#@.��#V�U�����8���Q�����!`]���9a+.xkQn���l ��� ��n�gel\l�CR°

� hN{�¹+p[fTl$��n� �nhN{�¹+p[fNk��J° ��l$��n ,�A°�hN{�d.vxfYwzfY§% vxg:oG{G«�«�{G«¸wzfµd\f$^2«�¯R­$­${Cª�vxwso�_�cYvx{C«�c[gMqz}µ�<{Cª�¿±�g.vx�5°W'�f$~¸wz{Cf[{GfY«�ªxg:oA��k�"°sk ¦ g\§M{Gqs­nd.fY¿�aRg\p[qsw-w{Mk ¦ °sk�d\fe}�º/{Gws«�j[pYoAcNk:¾J°skM{�}Yw®ªxg\vx«Gk�x :.����8)<���9��).9?�.9�A���6.4��#\)Llm "���j�9;� !,-���]�+�";���#@ zy����! "��#Yki¨[d\§M{C«6f+C�C�g2f+eMmRk8hN{G« �g\p[oUc[{G«Gk ¦ vUd\f[oC{M°iaR¨YvxwsfY§M{Cv�¿�¸{Avxq©d.§Y°

Page 27: yann.lecun.comyann.lecun.com/exdb/publis/pdf/lecun-99.pdf · 1)(+*!! 7\@ "! $ '

� hN{�¹+p[fTl$��nM� �nhN{�¹+p[fNkW�µ°���l���n��,�A°�� ��)<�m�z��.B,9��#W#\���� "��#W#@ :.9�A��.B)<�B��� �������9��#@�! :.9.m�$;<�VpA,9��#@#Q��,$b�! "��#@ :.9�D�z�����-#W :#<;0�4��)��m� .Ar�°]%8c[t ªxcY{G«�ws«Gke�fYwsy�{Cvx«�w®ª2w{%L°[{Aª�´°e¹+pYvxws{��"% d�vxwz«���A°

� hN{�¹+p[fTl$��n�n �nhN{�¹+p[fNkT�µ° ��l���n�n,�A°�^¶ªxcY{Gg\vx{Cªxwso�d.q¸¬©vUd\­${C±+g\vx�´¬Eg\vµ~�d\oU�:¿A%8vxg\¨ed\§Md.ªxwsgMfN°�'�f_igMpRvx{CªxrG�:¯:kYtµ°sk �<wzf:ªxg\f(ke¾J°zk�d\fe}ÀaR{�³�f[g�±-«��4wçk�_°zk[{�}Yw®ªxg.vx«GkM57�8��,-�-��)� /#�;�.B�83�/^W����X���� � ��#Wb#Q��,m�! "��#@ :.9�� ��)<�m� .D=@���6�0���O=�,-^@����� k�¨[d\§\{G«Df:l�g�f�nRk4¹7��Jk<%8w®ª�ªx«�j[pRvx§Mc(k�% dR°<�g\vx§Md\f E d\pR¬E¿­nd\fYf(°

� hN{�¹+p[fTl$��n�� �nhN{�¹+p[fNk��µ° ��l���n����A°T¾{GfY{CvUd\qswsr�d.ªxwsg\fÖd.fe}T�<{Cª�±+g\vx�Tt{G«�ws§Mf»a4ª�vUd�ªx{G§Mws{G«G°N'�f%8¬E{Gw®¬²{Cv�k2|�°sk2aRoUcYvx{Cªx{Av�k� °sk ¦ g\§M{Gqs­nd.f(k ¦ °sk2d\f[}ba4ªx{G{Gqs«Gk2h8°zk�{�}Yw®ªxg\vx«Gk4� �+#W#Q��,m�! "�+#W :.9� /#5����-."�@��,m�! :*��Ak��pYvxwsoAcNk�a:±-w®ªxrG{Cvxqzd\f[}N° 8qs«�{Gy4ws{Cv�°

� hN{�¹+p[fÀ{Aªd.qê°(l$��n�� ��h({�¹+pYf(kH�µ°sk.~+g\«�{Cv�k:~2°zk.tu{Cf[��{Cv�k.�R°:a�°zk �<{Gfe}R{Cvx«�gMfNk�tµ°sk��<g�±"d.vU}5kM|�° "°zk�pYj[jed�vU}Nk5ºÎ°skid\fe}´��d\oU��{GqêkLh�°(tµ° ��l���n����A°�~�d\oU�4¨Yvxg\¨ed\§Md.ªxwsgMf%^2¨[¨Yqzws{�}Iªxg �ud.fe}4±"vxwsª�ªx{Cf5ws¨À¹+g4}Y{�|<{CoGgM§\f[w®ªxwsgMf(°D����������� � ���J���������! !��#[k(l��!e ��� �HeYl�g � �:lM°

� hN{�¹+p[fÀ{Aªd.qê°(l$���Mm ��h({�¹+pYf(kH�µ°sk.~+g\«�{Cv�k:~2°zk.tu{Cf[��{Cv�k.�R°:a�°zk �<{Gfe}R{Cvx«�gMfNk�tµ°sk��<g�±"d.vU}5kM|�° "°zk�pYj[jed�vU}NkRºÎ°sk�d.fe}À��d.oA�M{Gqêk5h�°[tµ°���l$���\m��A° �ud.fe}4±"vxwsª�ªx{CfI}Yws§Mw®ª2vx{GoGgM§\f[w®ªxwsgMf�±-w®ªxcIdPjed.oA�:¿¨Yvxg\¨ed\§Md.ªxwsgMf�f[{Aª�±�g.vx��°Q'�f�_igMpRvx{CªxrG�:¯:k�tJ°zk�{�}Yw®ªxg\v�k2(&)�*+��#�,-��.J :#4�U�����8���W1-#�3$���-�4�+�! "��#05D����b,-��.-.9 :#<;c=W?+.9�A���6. �Lp:��185D=����<Xmr�ketu{CfRyM{Cv�k[¹ �J°\�g.vx§�d\f E d.pY¬E­nd\f(°

� hN{�¹+p[fÀ{Aªd.qê°(l$����n ��h({�¹+pYf(kc�µ°sk�~¸g\ª�ªxgMpNkIh�°skÀ~+{Gf[§\wsgYkV�J°zkÀd\f[} �ud��5f[{Cv�kV% ° ��l�����n,�A°¾<vUdM}Rwz{Cf4ª�¿�~�d.«�{�}ÎhN{�d.vxfYwzfY§Ø^<¨Y¨[qsws{�}bªxg»tug:oGpY­${Gf:ª�|<{CoGgM§\f[w®ªxwsgMf(° 5D����,-�9��)� :#<;�. �83 �/^W�18s]s s�k �"n���Al\l�� f�f\��nHg�f�C�f+eY°

� hN{�¹+p[fNk E d.f4ªx{Avd.fe}�aRgMqsqzd�l����Rl �nh({�¹+pYf(k<�J°zk E d.f:ªx{Cv�k<'A°skRd.fe}�aRgMqsqzdRk[a�°��l����Rl �A° �ws§M{CfRy\d\q®¿p[{G«<g\¬�oGg�y\d.vxwzd\fYoG{µ­nd�ª�vxwzoC{G«$�5d\¨Y¨[qswso�d.ªxwsg\fWªxgnf[{CpYvUd\q®¿÷f[{Aª�±�g.vx��qs{�d�vxf[wsf[§R°�5]^�?�.9 ",9���SPO��*� ���`q����!�A���-.Uk ��l$n���� f�C�� �g2f�C����4°

� Àd�v�ªxwzf/l$����C �cÀd�v�ªxwzfNk¾J°h8°���l�����C��A°Ê¹+{Gf:ªx{Cvx{G}R¿÷gMj4³�{GoAªIwzf:ªx{G§.vUd.ªx{�}Ú«�{G§\­${Gf:ªUd.ªxwsgMf d\f[}vx{GoGg\§Mf[w®ªxwsgMf�g\¬Lg�y�{Avxq©d.¨[¨YwzfY§Jc[d\f[}R¿÷¨Yvxwsf:ªx{�}$oAc[d.vUd.oCªx{Cvx«G°]�����������I� ���J�W�2���+�! "��#[k ��� eRl��Hg�e<f+�R°

� Àd�ªUd\f�{Cª2d.qç°(l�����f �0Àd.ªUd\fNk �J°zkR~¸pYvx§M{C«Gke¹<°Y�Y°Y¹<°skYh({�¹+p[f(k<�J°skRd\fe}ntu{GfY��{Cv�k4�R°ea�°��l�����f��A°�p[q®ªxw®¿÷tuws§Mw®ª$|2{GoGgM§\f[w®ªxwsgMf �«�wsf[§´dWaR¨[d\oG{�tuws«�¨[qzd\oG{C­${Gf:ªµ�<{GpYvUd.q"�{Cª�±�g\vx��° '�f �g:g4}R¯:k�Y°]´°sk �ud.f[«�gMfNk�a5° �Y°sk+d.fe}%h(ws¨Y¨[­nd\fNkL|�°I%L°sk {�}Yw®ªxg.vx«Gki�U���������]1-#�3$���-�4�+�! "��# 5D����,-��.9.9 :#�;=W?+.9�A���6.xkey�g\qsp[­${�eR°\�g.vx§�d\f E d.pY¬E­nd\f[fY%�pYj[qsws«�c[{Cvx«GkeaYd.fLÀd.ªx{CgYk5¹�^�°

� �g\rG{Cv�l����4l �F�g\rG{Cv�k@´°R¹<°���l$���Rl �A°JtS^@�D�@���8,-�A���! "��#N�83O�6�2� �! �\�z����l��H��,m�!.��Q( ,9��#W#\��,m�! "��#W /.-��������8����,-^4°DY'�_ % vx{G«�«�¿�~¸vUdM}4¬²g\vU}I~+g:g\�4«Gk5¹�d\­JjYvxwz}Y§\{Mk�À^�°

� �<g�±-qzd.fId\fe}F%�qzd�ª�ª�l$��� � �n�g�±-qzd\fNk-a�°"d\f[} %8q©d�ª�ª�k-�Y°���l���� �,�A° ^ ¹+gMf4y�gMqspRªxwzg\fed.qu�{GpRvUd\q�{Cª�±�g.vx� �ud.fe}�_(vUd.oA�M{Cv�°�'�f�_i{G«xd.pYvxgRk\¾J°zk�_igMpRvx{CªxrG�:¯:kGtµ°skGd\fe}�hN{G{Gf(kG_°sk�{�}Yw®ªxg\vx«Gk+(&)�*+��#�,-��. :#6�����2�8����1-#$3$���-�4���! !��#657�8��,���.9.9 :#<;U=W?+.9�A���6.���k\¨ed.§M{G« �Mm4l�g<�Mm�n4kRaYd.f4Àd�ªx{GgYk:¹�^�°��g\vx§�d.fE d\pR¬²­nd\fYf(°

� �«�pYfedµ{Cª2d.qê°il�������� �«�p[f[dRk "°sk ¦ vx{GpYfe}Nki|�°sk8d.fe}%¾w®vxgM«�wêk ¦ ° ��l$���M�,�A°T_(vUd\wsf[wsfY§/a4p[¨[¨�g.v�ª�¸{CoCªxg\vcÀd.oAcYwsf[{G«$�+d\fÖ^<¨[¨YqswzoGd.ªxwsgMf ªxg ¦ d.oG{Àtu{Cªx{CoCªxwsgMf(° '�f 5D����,-�9�9)+ :#�;�.L� 3 �K\5]P�� X��.k¨ed.§M{G«�l�C\mHg[l�C 4°Q'� �Ö¹+gM­$¨YpYªx{Cv<a4g:oGws{Cª�¯Y%8vx{C«�«G°

� |<d\jYwzfY{Cv�l���n�� �P|d.j[wsf[{Cv�k�h�°+|�° ��l���n����A° ^ _ipYªxg.vxwzd\q �f �wz}[}R{Gf Àd.vx�Mg.y �g4}Y{Cqz«�d\f[}aR{Gqs{GoAªx{�}�^<¨Y¨[qswso�d.ªxwsgMfY«Jwsf a4¨�{G{GoUc/|<{CoGgM§\f[w®ªxwsgMf(°L57�8��,��9��)� :#<;�.V� 34�/^@�418s]s s�k8�M����f,��� f �\��gf�n�R°

� |2g�±-qz{A¯:k[{Cª<d.qç°Nl���� �n|<g�±-qs{C¯:k �J°µ^�°skP~+d\qsp\³�dRk�a�°zkµd\f[} E d\f[dM}R{Mk$_° ��l���� ��A° �{GpRvUd\q�{Cª�±�g.vx�4¿�~�d.«�{�} ¦ d.oG{$tu{Cªx{CoCªxwsgMf(°F'�f 57�8��,-�-��)� /#�;�.F�83V�K\5]P�� X��.k(¨ed.§M{G«Bf.m�CHg�f\m�nR°I'� ¹+gM­$¨YpYªx{Cv<a4g:oGws{Cª�¯F% vx{G«�«G°

� |2p[­${Gqsc[d.v�ª�k�wsf:ªxgMf�d\fe}nºØwsqsqzwzd.­$«�l$��n �$|2p[­${Gqsced�v�ª�k tJ° "°zk �wsf:ªxgMf(k ¾J° "°sk d\f[}ºØwsqsqzwzd.­$«Gki|�°N�Y° ��l���n���A°nhN{�d.vxfYwsf[§�wsf:ªx{Cvxf[d\q vx{G¨Yvx{C«�{Gf:ªUd.ªxwsgMfY«�j:¯�{Av�vxg\v�¨Yvxg\¨ed\§Md.ªxwsgMfN°4'�f5&�+�����:�z�m� )� /.-�!�- "lm���A��)c���8��,-��.-.9 :#<;��]s����\�j���8���! "��#@.> :# �/^@�4�B ",m����.9�!�-�@,m�!�2���V�834,9�9;�#W :�! !��#[k(y�gMq®¿p[­${�'AkY¨ed\§\{G«&CRl$nHg�C��f:°N~¸vUdM}4¬²g\vU}�~¸g:gM�4«Gk5¹�d.­µjYvxwz}R§M{Mk�À^�°

� a4oAc[d\¨[w®vx{Pl$���Mm ��aRoUced.¨[w®vx{Mki|�° -° ��l����\m��A°W_�cY{�«�ª�vx{Cf[§\ªxcTg\¬-±�{�d.�Tqs{�d�vxfed\jYwsqzw®ª�¯:°�� ��,-^� /#Q�q������-#W :#<;\k ����f����sl����$g2f�fM�4°

Page 28: yann.lecun.comyann.lecun.com/exdb/publis/pdf/lecun-99.pdf · 1)(+*!! 7\@ "! $ '

� �¸d\wsqsqzd\f:ª�kQ�g\fYvxg:o�¼�d\f[}�h({�¹+p[fTl$����e � �¸d\wsqsq©d.f:ª�k\|�°sk��gMfRvxg:o�¼ek:¹<°sk\d\f[}JhN{�¹+p[fNk��µ°���l�����e��A°�<vxws§Mwsfed.qd.¨[¨YvxgMd\oUc»¬Eg\v$ªxc[{�qsg:o�d\qsws«xd.ªxwsgMfÚg.¬ugMj4³�{CoCªx«nwzf»ws­nd\§M{C«G° 18s su5D����,N��# KM :.9 "��#��1-�4��;<���]��#�)0=@ z;�#\����5D����,-��.9.9 :#�;\kLlmeYl,�!e ��� f+e ��g2f �\m4°

� �¸d\¨Yf[ws��l���� � � �¸d\¨Yf[ws��k �J°u�J° ��l���� ���A° tS^W� ����!���9� �83 =W�����! :.9�! ",9���UqT�9�+��#@ :#�; tS^W�����-?.°aR¨RvxwzfY§M{Cv�kY�{A±"¿��¸g\vx��°

� º/d\f[§Pd\fe}��M{�d.f/l�����C �Pº/d\fY§Yk:�R°4d\f[}P�M{�d.f(k4�Y°���l�����C��A°S�pYqsªxw®¿çvx{G«�g\qzpRªxwsgMf�f[{GpRvUd\q[fY{Cª�±+g\vx�4«¬²g.vPgM­$fYw®¬²gMf:ªµoAc[d.vUd\oAªx{Cvµvx{GoCgM§MfYw®ªxwzg\f(° '�f 5D����,-�9��)� :#�;�.V�8361-#W�A���-#\���! "�+#\����� ��#$3����9��#�,-�L�+#�U�����8���Q�����!`]���9a+.xk�yMgMqsp[­${�'�'�'AkY¨ed\§\{G«�l �+n�nHg[l �+��CR°

� ºTgMq®¬�d\f[}Y%8q©d�ª�ª�l����+e��$ºTgMq®¬�k¸|�°+d\f[} %8qzd.ª�ª�k"�R°���l$����e��A° %Lg\«�ªUd\qdM}[}4vx{G«�«nj[qsg:oU� qsg:o�d�ªxwsgMfp[«�wsf[§µdJoGg\f4y�gMqspYªxwsg\fed\qNqsg:o�d.ªxg.v2f[{Cª�±�g\vx��°k'�f�¹+g�±"d.f(k[�Y°Rtµ°ske_({G«xd\pRvxgYke¾J°zkYd.fe}�^2qs«�¨�{GoCªxg\v�k�Y°ske{�}Rw®ªxg\vx«Gk\(&)�*+��#�,-��.U /#Y�U���������Q1-#�3m�����>���! "��#F5D����,-��.9.9 :#�;F=@?�.9�A���B. �.k[¨[d\§\{G«u�He��$gR� ��f4°