Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-....

117
/HDUQLQJ%D\HVLDQ1HWZRUNVIURP’DWD 1LU)ULHGPDQ 8&%HUNHOH\ ZZZFVEHUNHOH\HGXaQLU 0RLVHV*ROGV]PLGW 65,,QWHUQDWLRQDO ZZZHUJVULFRPSHRSOHPRLVHV )RUFXUUHQWVOLGHVDGGLWLRQDOPDWHULDODQGUHDGLQJOLVWVHH KWWSZZZFVEHUNHOH\HGXaQLU7XWRULDO 2XWOLQH “,QWURGXFWLRQ %D\HVLDQQHWZRUNVDUHYLHZ 3DUDPHWHUOHDUQLQJ&RPSOHWHGDWD 3DUDPHWHUOHDUQLQJ,QFRPSOHWHGDWD 6WUXFWXUHOHDUQLQJ&RPSOHWHGDWD $SSOLFDWLRQFODVVLILFDWLRQ /HDUQLQJFDXVDOUHODWLRQVKLSV 6WUXFWXUHOHDUQLQJ,QFRPSOHWHGDWD &RQFOXVLRQ

Transcript of Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-....

Page 1: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � .

/HDUQLQJ�%D\HVLDQ�1HWZRUNV�IURP�'DWD

1LU�)ULHGPDQ

8�&��%HUNHOH\

ZZZ�FV�EHUNHOH\�HGX�aQLU

0RLVHV�*ROGV]PLGW

65,�,QWHUQDWLRQDO

ZZZ�HUJ�VUL�FRP�SHRSOH�PRLVHV

)RU�FXUUHQW�VOLGHV��DGGLWLRQDO�PDWHULDO��DQG�UHDGLQJ�OLVW�VHH

KWWS���ZZZ�FV�EHUNHOH\�HGX�aQLU�7XWRULDO

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254�6

2XWOLQH

ª,QWURGXFWLRQ7

%D\HVLDQ�QHWZRUNV��D�UHYLHZ7

3DUDPHWHU�OHDUQLQJ��&RPSOHWH�GDWD7

3DUDPHWHU�OHDUQLQJ��,QFRPSOHWH�GDWD7

6WUXFWXUH�OHDUQLQJ��&RPSOHWH�GDWD7

$SSOLFDWLRQ��FODVVLILFDWLRQ7

/HDUQLQJ�FDXVDO�UHODWLRQVKLSV7

6WUXFWXUH�OHDUQLQJ��,QFRPSOHWH�GDWD7

&RQFOXVLRQ

Page 2: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254��

/HDUQLQJ��LQ�WKLV�FRQWH[W�

73URFHVV

� ,QSXW��GDWDVHW�DQG�SULRU�LQIRUPDWLRQ� 2XWSXW��%D\HVLDQ�QHWZRUN

73ULRU�LQIRUPDWLRQ��EDFNJURXQG�NQRZOHGJH

� D�%D\HVLDQ�QHWZRUN��RU�IUDJPHQWV�RI�LW�� WLPH�RUGHULQJ� SULRU�SUREDELOLWLHV� ���

������� ���������

��������� ��� � � ��� ������ �! ��!�����! �����" �

� ��#$����%&���$���

�� #!'$��#�( ) �+*,QGXFHU'DWD���

3ULRU�LQIRUPDWLRQ

,

-

.

/

0

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /10325421

:K\�OHDUQLQJ"

7)HDVLELOLW\�RI�OHDUQLQJ

� $YDLODELOLW\�RI�GDWD�DQG�FRPSXWDWLRQDO�SRZHU7

1HHG�IRU�OHDUQLQJ� &KDUDFWHULVWLFV�RI�FXUUHQW�V\VWHPV�DQG�SURFHVVHV

� 'HI\�FORVHG�IRUP�DQDO\VLV

×QHHG�GDWD�GULYHQ�DSSURDFK�IRU�FKDUDFWHUL]DWLRQ

� 6FDOH�DQG�FKDQJH�IDVW

×QHHG�FRQWLQXRXV�DXWRPDWLF�DGDSWDWLRQ7

([DPSOHV�� FRPPXQLFDWLRQ�QHWZRUNV��HFRQRPLF�PDUNHWV��LOOHJDO

DFWLYLWLHV��WKH�EUDLQ���

Page 3: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254��

:K\�OHDUQ�D�%D\HVLDQ�QHWZRUN"

7&RPELQH�NQRZOHGJH�HQJLQHHULQJ�DQG�VWDWLVWLFDO

LQGXFWLRQ� &RYHUV�WKH�ZKROH�VSHFWUXP�IURP�NQRZOHGJH�LQWHQVLYH

PRGHO�FRQVWUXFWLRQ�WR�GDWD�LQWHQVLYH�PRGHO�LQGXFWLRQ7

0RUH�WKDQ�D�OHDUQLQJ�EODFN�ER[� ([SODQDWLRQ�RI�RXWSXWV� ,QWHUSUHWDELOLW\�DQG�PRGLILDELOLW\� $OJRULWKPV�IRU�GHFLVLRQ�PDNLQJ��YDOXH�RI�LQIRUPDWLRQ�

GLDJQRVLV�DQG�UHSDLU7

&DXVDO�UHSUHVHQWDWLRQ��UHDVRQLQJ��DQG�GLVFRYHU\� 'RHV�VPRNLQJ�FDXVH�FDQFHU"

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254��

:KDW�ZLOO�,�JHW�RXW�RI�WKLV�WXWRULDO"

7$Q�XQGHUVWDQGLQJ�RI�WKH�EDVLF�FRQFHSWV�EHKLQG�WKH

SURFHVV�RI�OHDUQLQJ�%D\HVLDQ�QHWZRUNV�IURP�GDWD�VR�WKDW

\RX�FDQ� 5HDG�DGYDQFHG�SDSHUV�RQ�WKH�VXEMHFW� -XPS�VWDUW�SRVVLEOH�DSSOLFDWLRQV� ,PSOHPHQW�WKH�QHFHVVDU\�DOJRULWKPV� $GYDQFH�WKH�VWDWH�RI�WKH�DUW

Page 4: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254��

2XWOLQH

7,QWURGXFWLRQ

ª%D\HVLDQ�QHWZRUNV��D�UHYLHZ� 3UREDELOLW\����� :KDW�DUH�%D\HVLDQ�QHWZRUNV"� :KDW�FDQ�ZH�GR�ZLWK�%D\HVLDQ�QHWZRUNV"� 7KH�OHDUQLQJ�SUREOHP���

73DUDPHWHU�OHDUQLQJ��&RPSOHWH�GDWD

73DUDPHWHU�OHDUQLQJ��,QFRPSOHWH�GDWD

76WUXFWXUH�OHDUQLQJ��&RPSOHWH�GDWD

7$SSOLFDWLRQ��FODVVLILFDWLRQ

7/HDUQLQJ�FDXVDO�UHODWLRQVKLSV

76WUXFWXUH�OHDUQLQJ��,QFRPSOHWH�GDWD

7&RQFOXVLRQ

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254��

3UREDELOLW\����

7%D\HV�UXOH

7&KDLQ�UXOH

7,QWURGXFWLRQ�RI�D�YDULDEOH��UHDVRQLQJ�E\�FDVHV�

��

���_��_�

<3

;3;<3<;3

⋅=

∑ ⋅==

<=3<=;3<;3 �_���_��_�

���_��_������� ������ −=QQQ ;;;3;;3;3;;3 KLK

Page 5: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254��

5HSUHVHQWLQJ�WKH�8QFHUWDLQW\�LQ�D�'RPDLQ

7$�VWRU\�ZLWK�ILYH�UDQGRP�YDULDEOHV�

� %XUJODU\��(DUWKTXDNH��$ODUP��1HLJKERU�&DOO�

5DGLR�$QQRXQFHPHQW� 6SHFLI\�D�MRLQW�GLVWULEXWLRQ�ZLWK��

���� ���SDUDPHWHUV

�����������������������������������������������������������PD\EH«

�$Q�H[SHUW�V\VWHP�IRU�PRQLWRULQJ�LQWHQVLYH�FDUH�SDWLHQWV

� 6SHFLI\�D�MRLQW�GLVWULEXWLRQ�RYHU����YDULDEOHV�ZLWK

��DW�OHDVW������

�SDUDPHWHUV

�����������������������������������������������������������QR�ZD\���

� � � �� ��� ��� � � � �������� ��� � � !���� " ��# � $� ��� �&%(' � ) � )+*�' # � ) , �&� � - .�/�0 0 � - � � � � - � '�� ��# � 1�# # � � 2�3 - )+� � ) � � 4 � � 5 687:9<;(9>=

3UREDELOLVWLF�,QGHSHQGHQFH��D�.H\�IRU

5HSUHVHQWDWLRQ�DQG�5HDVRQLQJ

�5HFDOO�WKDW�LI�;�DQG�<�DUH�LQGHSHQGHQW�JLYHQ�=�WKHQ

�,Q�RXU�VWRU\«LI

? EXUJODU\�DQG�HDUWKTXDNH�DUH�LQGHSHQGHQW? EXUJODU\�DQG�UDGLR�DUH�LQGHSHQGHQW�JLYHQ�HDUWKTXDNH

�WKHQ�ZH�FDQ�UHGXFH�WKH�QXPEHU�RI�SUREDELOLWLHV�QHHGHG

�_���_� =;3<=;3 =

Page 6: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

� � � �� ��� ��� � � � �������� ��� � � !���� " ��# � $� ��� �&%(' � ) � )+*�' # � ) , �&� � - .�/�0 0 � - � � � � - � '�� ��# � 1�# # � � 2�3 - )+� � ) � � 4 � � 5 687:9<;(9�9

3UREDELOLVWLF�,QGHSHQGHQFH��D�.H\�IRU

5HSUHVHQWDWLRQ�DQG�5HDVRQLQJ�

,Q�RXU�VWRU\«LI? EXUJODU\�DQG�HDUWKTXDNH�DUH�LQGHSHQGHQW? EXUJODU\�DQG�UDGLR�DUH�LQGHSHQGHQW�JLYHQ�HDUWKTXDNH

�WKHQ�LQVWHDG�RI����SDUDPHWHUV�ZH�QHHG��

���_���_����_������ %3%(3%(53%(5$3%(5$3 ⋅⋅⋅=

�����_���_������ %3(3(53%($3%(5$3 ⋅⋅⋅=

YHUVXV

1HHG�D�ODQJXDJH�WR�UHSUHVHQW�LQGHSHQGHQFH�VWDWHPHQWV

� � � �� ��� ��� � � � �������� ��� � � !���� " ��# � $� ��� �&%(' � ) � )+*�' # � ) , �&� � - .�/�0 0 � - � � � � - � '�� ��# � 1�# # � � 2�3 - )+� � ) � � 4 � � 5 687:9<;(9��

%D\HVLDQ�1HWZRUNV

&RPSXWHU�HIILFLHQW�UHSUHVHQWDWLRQ�RI�SUREDELOLW\

GLVWULEXWLRQV�YLD�FRQGLWLRQDO�LQGHSHQGHQFH

���������

H

E

H

����� �����

�����������������������

EH

E

E

H

%( 3�$�_�(�%�������������� �"!�#

$%��&�' (

)*�+��,%- �"�.

/ - �"�"0

1 ��- -

Page 7: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

� � � �� ��� ��� � � � �������� ��� � � !���� " ��# � $� ��� �&%(' � ) � )+*�' # � ) , �&� � - .�/�0 0 � - � � � � - � '�� ��# � 1�# # � � 2�3 - )+� � ) � � 4 � � 5 687:9<;(9 �

%D\HVLDQ�1HWZRUNV

4XDOLWDWLYH�SDUW��VWDWLVWLFDO

LQGHSHQGHQFH�VWDWHPHQWV

�FDXVDOLW\���

'LUHFWHG�DF\FOLF�JUDSK

�'$*�? 1RGHV���UDQGRP

YDULDEOHV�RI�LQWHUHVW

�H[KDXVWLYH�DQG�PXWXDOO\

H[FOXVLYH�VWDWHV�? (GJHV���GLUHFW��FDXVDO�

LQIOXHQFH

�4XDQWLWDWLYH�SDUW��/RFDO

SUREDELOLW\�PRGHOV��6HW

RI�FRQGLWLRQDO�SUREDELOLW\

GLVWULEXWLRQV�

���������

H

E

H

����� �����

�����������������������

EH

E

E

H

%( 3�$�_�(�%�������������� �"!�#

$%��&�' (

)*�+��,%- �"�.

/ - �"�"0

1 ��- -

� � � �� ��� ��� � � � �������� ��� � � !���� " ��# � $� ��� �&%(' � ) � )+*�' # � ) , �&� � - .�/�0 0 � - � � � � - � '�� ��# � 1�# # � � 2�3 - )+� � ) � � 4 � � 5 687:9<;(9��

%D\HVLDQ�1HWZRUN�6HPDQWLFV

�&RPSDFW��HIILFLHQW�UHSUHVHQWDWLRQ�

� QRGHV�KDYH�≤�N�SDUHQWV� ⇒�2���

Q��YV��2���

��SDUDPV� SDUDPHWHUV�SHUWDLQ�WR�ORFDO�LQWHUDFWLRQV

4XDOLWDWLYH�SDUWFRQGLWLRQDOLQGHSHQGHQFH�VWDWHPHQWVLQ�%1�VWUXFWXUH

4XDQWLWDWLYH�SDUW

ORFDOSUREDELOLW\

PRGHOV

8QLTXH�MRLQWGLVWULEXWLRQRYHU�GRPDLQ

3�&�$�5�(�%�� �3�%� 3�(_%� 3�5_(�%� 3�$_5�%�(� 3�&_$�5�%�(�

��������������������������������������������������YHUVXV

3�&�$�5�(�%�� �3�%� 3�(��� �3�5_(��� �3�$_%�(��� �3�&_$�

Page 8: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254 276

0RQLWRULQJ�,QWHQVLYH�&DUH�3DWLHQWV

7KH�³DODUP´�QHWZRUN

�������YDULDEOHV������SDUDPHWHUV��LQVWHDG�RI���8:9

; �(< ; ��=> '�� ;

> '�?�@(" > '&�*�A

?�'�'�(*��(A�?�'> '> ) &(A�=#'�B

�(*(A?�� > =#C

&�*�=�D ?�E ; �=�D

*�'�A(�=�D

F ?�(A *�C F

F ?�(A�C ��" F ?(��) A����(?

G ) &(��=#����?��A

�H) � F =#C &�?�A

F ?�(A���*�� >@() �@(? G A(����?) �(A(����*(A() =#�; ��C �H?��H��=#C �&

; * ; & > ����A

*���* ;�> B�C *E�) &

�H) �= F C

;�F &�*A

��) =�D ; '�?(&�&

) �(&������ *���?�&�A >A ; '

C F � *�) C �'�?

?�'�'��C =�<I=#�(A ; �(A&�A('=#? F = C ���?C F ? G�F =#C �� ?

> B ; = F =#C ?��H) *

� F;

� ;

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254 25J

4XDOLWDWLYH�SDUW

K1RGHV�DUH�LQGHSHQGHQW�RI�QRQ�GHVFHQGDQWV�JLYHQ�WKHLU

SDUHQWVL 3�5_( \�$�� �3�5_( \��IRU�DOO�YDOXHV�RI�5�$�(

*LYHQ�WKDW�WKHUH�LV�DQG�HDUWKTXDNH�

,�FDQ�SUHGLFW�D�UDGLR�DQQRXQFHPHQW

UHJDUGOHVV�RI�ZKHWKHU�WKH�DODUP�VRXQGV

KG�VHSDUDWLRQ��D�JUDSK�WKHRUHWLF�FULWHULRQ

IRU�UHDGLQJ�LQGHSHQGHQFH�VWDWHPHQWV

&DQ�EH�FRPSXWHG�LQ�OLQHDU�WLPH

�RQ�WKH�QXPEHU�RI�HGJHV�

�NM5OIP(Q5RTS�MVU5W

�XM7YTZ [

�XS#O�\^] MVO _

� ] MVOV`

M5] ]

Page 9: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254 2 9

%ORFNHG 8QEORFNHG

( %

$

&

( %

$

&

( %

$

&

(

&

$

(

&

$

(

5 $

(

5 $

G�VHSDUDWLRQ

K7ZR�YDULDEOHV�DUH�LQGHSHQGHQW

LI�DOO�SDWKV�EHWZHHQ�WKHP�DUH

EORFNHG�E\�HYLGHQFH

7KUHH�FDVHV�L &RPPRQ�FDXVH

L ,QWHUPHGLDWH�FDXVH

L &RPPRQ�(IIHFW

%ORFNHG 8QEORFNHG

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254 2��

K ,�;�<_=��GHQRWHV�;�DQG�<�DUH�LQGHSHQGHQW�JLYHQ�=

L ,�5�%�L a,�5�%_$�L ,�5�%_(�$�L a,�5�&_%�

([DPSOH

( %

$

&

5

Page 10: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254 2��

,�(TXLYDOHQW�%D\HVLDQ�1HWZRUNV

K1HWZRUNV�DUH�,�HTXLYDOHQW�LI

�WKHLU�VWUXFWXUHV�HQFRGH�WKH�VDPH�LQGHSHQGHQFH

VWDWHPHQWV

K7KHRUHP��1HWZRUNV�DUH�,�HTXLYDOHQW�LII

WKH\�KDYH�WKH�VDPH�VNHOHWRQ�DQG�WKH�VDPH�³9´�VWUXFWXUHV

(

$5

(

$5

(

$5

,�5�$_(�

6 &

(

'

) *

6 &

(

'

) *

127�,�HTXLYDOHQW

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254����

K$VVRFLDWHG�ZLWK�HDFK�QRGH�; � �WKHUH�LV�D�VHW�RI�FRQGLWLRQDO

SUREDELOLW\�GLVWULEXWLRQV�3�; � _3D � ��L ,I�YDULDEOHV�DUH�GLVFUHWH���LV�XVXDOO\�PXOWLQRPLDO

L 9DULDEOHV�FDQ�EH�FRQWLQXRXV���FDQ�EH�D�OLQHDU�*DXVVLDQL &RPELQDWLRQV�RI�GLVFUHWH�DQG�FRQWLQXRXV�DUH�RQO\

FRQVWUDLQHG�E\�DYDLODEOH�LQIHUHQFH�PHFKDQLVPV

�NM OTP�Q7RIS�M U5W �:S#O(\ ] M O _

� ] M OH`

����� ���

H

E

H

������ ���

�������� �������� ���

EH

E

E

H

%( 3�$�_�(�%�

4XDQWLWDWLYH�3DUW

Page 11: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254�� 2

:KDW�&DQ�:H�'R�ZLWK�%D\HVLDQ

1HWZRUNV"K

3UREDELOLVWLF�LQIHUHQFH��EHOLHI�XSGDWH� 3�(� <_�5� �<��&� �<�

K3UREDELOLVWLF�LQIHUHQFH��EHOLHI�UHYLVLRQ

� $UJPD[ ����� ��� �3�H��E�_�& <�K

4XDOLWDWLYH�LQIHUHQFH� ,�5�&_�$�

K&RPSOH[�LQIHUHQFH

� UDWLRQDO�GHFLVLRQ�PDNLQJ

�����LQIOXHQFH�GLDJUDPV�� YDOXH�RI�LQIRUPDWLRQ� VHQVLWLYLW\�DQDO\VLV

K&DXVDOLW\��DQDO\VLV�XQGHU�LQWHUYHQWLRQV�

�NM5OIP(Q7RIS�MVU5W

�XM7YTZ [

�XS#O�\^] MVO _

� ] MVOV`

M5] ]

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254�� �

%D\HVLDQ�1HWZRUNV��6XPPDU\

K%D\HVLDQ�QHWZRUNV�

DQ�HIILFLHQW�DQG�HIIHFWLYH�UHSUHVHQWDWLRQ�RI�SUREDELOLW\

GLVWULEXWLRQVK

(IILFLHQW�L /RFDO�PRGHOVL ,QGHSHQGHQFH��G�VHSDUDWLRQ�

K(IIHFWLYH��$OJRULWKPV�WDNH�DGYDQWDJH�RI�VWUXFWXUH�WR

L &RPSXWH�SRVWHULRU�SUREDELOLWLHVL &RPSXWH�PRVW�SUREDEOH�LQVWDQWLDWLRQL 'HFLVLRQ�PDNLQJ

K%XW�WKHUH�LV�PRUH��VWDWLVWLFDO�LQGXFWLRQ�Á�/($51,1*

Page 12: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254(687

/HDUQLQJ�%D\HVLDQ�QHWZRUNV��UHPLQGHU�

,QGXFHU'DWD���

3ULRU�LQIRUPDWLRQ

9

:

;

<

= >�?@> AB

CB

>�DE>�F

>�?G?H>�I3A>(JK>�L

CBCC

B

MN OQP#RTSUNWV�MYX

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254(6YZ

7KH�/HDUQLQJ�3UREOHP

.QRZQ�6WUXFWXUH 8QNQRZQ�6WUXFWXUH

&RPSOHWH�'DWD 6WDWLVWLFDOSDUDPHWULFHVWLPDWLRQ[ \^] _8`ba8cYd�e�_gf�hia^jgk l

'LVFUHWH�RSWLPL]DWLRQRYHU�VWUXFWXUHV[ cgm `Q\nf�ano�ap`Qa^qrf�\nsQl

,QFRPSOHWH�'DWD 3DUDPHWULFRSWLPL]DWLRQ[t�upvxwgf q^cgm aryxo

c8a^`Q\bagybok k k l

&RPELQHG[�zGof�{Q\5o{^f�qr]Qt�upvYh|m }Qo{^f ah~_8c^ag] `8��l

Page 13: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254(6��

/HDUQLQJ�3UREOHP��������

� ��� ��������� �� � ����������� ��� ��������� ��

� ������� ����� "!$#�!&% '(!&% )*#�+ � , #�-.#�/102!&-&% )0�'(!&% /1#2!&% 3�45 687 98:<;�=(>@?A92B@C ;(D E F

G % '*)2-.0�!.0 � 3 , !�% /H% I2#�!&% 3J4 � 3�K20�-'8!&-&L*)(!&L�-.0M'5@=�N :O68B ;8P ;Q:R;�S�B 6(TRF

U@� � ���V��� ����� W #�-.#�/X0�!&-&% ) � 3 , !�% /H% I2#�!&% 3�45ZY�[]\_^�B S8=�N ;�`RP�=(;�:O6a;�`_P$E E E Fb 3�/dc�% 4*0�e5@f�P$BhgO6iP$g�B S*7<Y�[]\<C]N jRP.g(B ;Ck98=�;�7 :*ldF

m mB

CB

m m

m mm m

CBCC

B

MN OQP#R S�NrV M^X

n�o2p�oOqr<s oAt"o t�ur<s o s o s ur t"oAt�o s ur t"o s o s u

vvr t"o s o s u

,QGXFHU

9 ;

<>(?@>#A

B

CB

>�DE>F

>(? ?H>�I3A>JK>�L

CBCC

B

MN OQP#R S�NrV M^X

9 ;

<

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254(6(w

/HDUQLQJ�3UREOHP��������

� ��� ��������� �� � ����������� ��� ��������� ��

� ������� ����� "!$#�!&% '(!&% )*#�+ � , #�-.#�/102!&-&% )0�'(!&% /1#2!&% 3�45@687 98:<;�=(>@?A92B@Cx;(D�E F

G % '*)2-.0�!.0 � 3 , !�% /H% I2#�!&% 3J4 � 3�K20�-'8!&-&L*)(!&L�-.0M'5@=�N :O68B ;8P ;Q:R;�S�B 6(TRF

U@� � ���V��� ����� W #�-.#�/X0�!&-&% ) � 3 , !�% /H% I2#�!&% 3�45ZY�[]\_^�B S8=�N ;�`RP�=(;�:O6a;�`_P$E E E Fb 3�/dc�% 4*0�e5@f�P$BhgO6iP$g�B S*7<Y�[]\<C]N jRP.g(B ;Ck98=�;�7 :*ldF

m mB

CB

m m

m mm m

CBCC

B

MN OQP#R S�NrV M^X

n�o2p�oOqrOs oAt"oAt�urOs o yMo s ur t�oAt"o s ur t�o s o y�u

vvr yMo s o s u

,QGXFHU

9 ;

<>(?@>#A

B

CB

>�DE>F

>(? ?H>�I3A>JK>�L

CBCC

B

MN OQP#R S�NrV M^X

9 ;

<

Page 14: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254(6��

/HDUQLQJ�3UREOHP��������

� ��� ��������� �� � ����������� ��� ��������� ��

� ������� ����� "!$#�!&% '(!&% )*#�+ � , #�-.#�/102!&-&% )0�'(!&% /1#2!&% 3�45@687 98:<;�=(>@?A92B@Cx;(D�E F

G % '*)2-.0�!.0 � 3 , !�% /H% I2#�!&% 3J4 � 3�K20�-'8!&-&L*)(!&L�-.0M'5@=�N :O68B ;8P ;Q:R;�S�B 6(TRF

U@� � ���V��� ����� W #�-.#�/X0�!&-&% ) � 3 , !�% /H% I2#�!&% 3�45ZY�[]\_^�B S8=�N ;�`RP�=(;�:O6a;�`_P$E E E Fb 3�/dc�% 4*0�e5@f�P$BhgO6iP$g�B S*7<Y�[]\<C]N jRP.g(B ;Ck98=�;�7 :*ldF

m mB

CB

m m

m mm m

CBCC

B

MN OQP#R S�NrV M^X

n�o2p�oOqrOs oAt"oAt�urOs o s o s ur t�oAt"o s ur t�o s o s u

vvr t�o s o s u

,QGXFHU

9 ;

<>(?@>#A

B

CB

>�DE>F

>(? ?H>�I3A>JK>�L

CBCC

B

MN OQP#R S�NrV M^X

9 ;

<

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254(6��

/HDUQLQJ�3UREOHP��������

� ��� ��������� �� � ����������� ��� ��������� ��

� ������� ����� "!$#�!&% '(!&% )*#�+ � , #�-.#�/102!&-&% )0�'(!&% /1#2!&% 3�45@687 98:<;�=(>@?A92B@Cx;(D�E F

G % '*)2-.0�!.0 � 3 , !�% /H% I2#�!&% 3J4 � 3�K20�-'8!&-&L*)(!&L�-.0M'5@=�N :O68B ;8P ;Q:R;�S�B 6(TRF

U@� � ���V��� ����� W #�-.#�/X0�!&-&% ) � 3 , !�% /H% I2#�!&% 3�45ZY�[]\_^�B S8=�N ;�`RP�=(;�:O6a;�`_P$E E E Fb 3�/dc�% 4*0�e5@f�P$BhgO6iP$g�B S*7<Y�[]\<C]N jRP.g(B ;Ck98=�;�7 :*ldF

m mB

CB

m m

m mm m

CBCC

B

MN OQP#R S�NrV M^X

n�o2p�oOqrOs oAt"oAt�urOs o yMo s ur t�oAt"o s ur yMo s o s u

vvr t�o s o8y�u

,QGXFHU

9 ;

<>(?@>#A

B

CB

>�DE>F

>(? ?H>�I3A>JK>�L

CBCC

B

MN OQP#R S�NrV M^X

9 ;

<

Page 15: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254(6��

2XWOLQH

�,QWURGXFWLRQ

�%D\HVLDQ�QHWZRUNV��D�UHYLHZ

ª3DUDPHWHU�OHDUQLQJ��&RPSOHWH�GDWD� 6WDWLVWLFDO�SDUDPHWULF�ILWWLQJ� 0D[LPXP�OLNHOLKRRG�HVWLPDWLRQ� %D\HVLDQ�LQIHUHQFH

�3DUDPHWHU�OHDUQLQJ��,QFRPSOHWH�GDWD

�6WUXFWXUH�OHDUQLQJ��&RPSOHWH�GDWD

�$SSOLFDWLRQ��FODVVLILFDWLRQ

�/HDUQLQJ�FDXVDO�UHODWLRQVKLSV

�6WUXFWXUH�OHDUQLQJ��,QFRPSOHWH�GDWD

�&RQFOXVLRQ

������� ��� ������� ��� ������������������ � ����� ��� �

� ���� �! ��� ��"�#�� #

$ �������� %! ��� �&"�#�� #

')( * * +-, .0/ 132 1 / 4 5�687�9�, :<; =>; ?34�1 @ 4�A 4 B�, 7�9 5�CED / F 4 FHG8D A 5 F I 6�/ 5 J , KML<N N 9 J 4 1 9 7 J / D�9 7�A ; O<A A 1 / P�Q J FH1 4 F 4 1 R 4 5 S TVU W�X0Y�Z

([DPSOH��%LQRPLDO�([SHULPHQW

�6WDWLVWLFV�����

[:KHQ�WRVVHG��LW�FDQ�ODQG�LQ�RQH�RI�WZR�SRVLWLRQV��+HDG�RU�7DLO

[:H�GHQRWH�E\�θ�WKH��XQNQRZQ��SUREDELOLW\�3�+��

(VWLPDWLRQ�WDVN�[

*LYHQ�D�VHTXHQFH�RI�WRVV�VDPSOHV�[>�@��[>�@��«��[>0@��ZH

ZDQW�WR�HVWLPDWH�WKH�SUREDELOLWLHV�3�+� �θ�DQG�3�7�� �����θ

+HDG 7DLO

Page 16: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

')( * * +-, .0/ 132 1 / 4 5�687�9�, :<; =>; ?34�1 @ 4�A 4 B�, 7�9 5�CED / F 4 FHG8D A 5 F I 6�/ 5 J , KML<N N 9 J 4 1 9 7 J / D�9 7�A ; O<A A 1 / P�Q J FH1 4 F 4 1 R 4 5 S TVU W�X0Y W

�6WDWLVWLFDO�SDUDPHWHU�ILWWLQJ

[&RQVLGHU�LQVWDQFHV�[>�@��[>�@��«��[>0@�VXFK�WKDW

� 7KH�VHW�RI�YDOXHV�WKDW�[�FDQ�WDNH�LV�NQRZQ� (DFK�LV�VDPSOHG�IURP�WKH�VDPH�GLVWULEXWLRQ� (DFK�VDPSOHG�LQGHSHQGHQWO\�RI�WKH�UHVW

[7KH�WDVN�LV�WR�ILQG�D�SDUDPHWHU��VR�WKDW�WKH�GDWD�FDQ�EHVXPPDUL]HG�E\�D�SUREDELOLW\�3�[>M@_����

� 7KH�SDUDPHWHUV�GHSHQG�RQ�WKH�JLYHQ�IDPLO\�RI�SUREDELOLW\

GLVWULEXWLRQV��PXOWLQRPLDO��*DXVVLDQ��3RLVVRQ��HWF�� :H�ZLOO�IRFXV�RQ�PXOWLQRPLDO�GLVWULEXWLRQV� 7KH�PDLQ�LGHDV�JHQHUDOL]H�WR�RWKHU�GLVWULEXWLRQ�IDPLOLHV

L�L�G�

VDPSOHV

')( * * +-, .0/ 132 1 / 4 5�687�9�, :<; =>; ?34�1 @ 4�A 4 B�, 7�9 5�CED / F 4 FHG8D A 5 F I 6�/ 5 J , KML<N N 9 J 4 1 9 7 J / D�9 7�A ; O<A A 1 / P�Q J FH1 4 F 4 1 R 4 5 S TVU W�X0Y��

7KH�/LNHOLKRRG�)XQFWLRQ

[�+RZ�JRRG�LV�D�SDUWLFXODU�θ",W�GHSHQGV�RQ�KRZ�OLNHO\�LW�LV�WR�JHQHUDWH�WKH�REVHUYHG

GDWD

[7KXV��WKH�OLNHOLKRRG�IRU�WKH�VHTXHQFH�+�7��7��+��+�LV

∏ θ=θ=θP

P[3'3'/ �_@>��_����

θ⋅θ⋅θ−⋅θ−⋅θ=θ ��������� '/

� ��� � ��� ��� ��� � �

θ

/��'

Page 17: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

')( * * +-, .0/ 132 1 / 4 5�687�9�, :<; =>; ?34�1 @ 4�A 4 B�, 7�9 5�CED / F 4 FHG8D A 5 F I 6�/ 5 J , KML<N N 9 J 4 1 9 7 J / D�9 7�A ; O<A A 1 / P�Q J FH1 4 F 4 1 R 4 5 S TVU W�X0Y�Y

6XIILFLHQW�6WDWLVWLFV

[7R�FRPSXWH�WKH�OLNHOLKRRG�LQ�WKH�WKXPEWDFN�H[DPSOH�ZHRQO\�UHTXLUH�1 � �DQG�1 �

��WKH�QXPEHU�RI�KHDGV�DQG�WKH�QXPEHU�RI�WDLOV�

1 � ��DQG�1 � �DUH�VXIILFLHQW�VWDWLVWLFV�IRU�WKH�ELQRPLDO

GLVWULEXWLRQ

[$�VXIILFLHQW�VWDWLVWLF�LV�D�IXQFWLRQ�WKDW�VXPPDUL]HV��IURP

WKH�GDWD��WKH�UHOHYDQW�LQIRUPDWLRQ�IRU�WKH�OLNHOLKRRG� ,I�V�'�� �V�'·����WKHQ�/�θ�_'�� �/�θ�_'·�

�� 11'/ ������ θ−⋅θ=θ

')( * * +-, .0/ 132 1 / 4 5�687�9�, :<; =>; ?34�1 @ 4�A 4 B�, 7�9 5�CED / F 4 FHG8D A 5 F I 6�/ 5 J , KML<N N 9 J 4 1 9 7 J / D�9 7�A ; O<A A 1 / P�Q J FH1 4 F 4 1 R 4 5 S TVU W�X0Y��

0D[LPXP�/LNHOLKRRG�(VWLPDWLRQ

0/(�3ULQFLSOH�

/HDUQ�SDUDPHWHUV�WKDW�PD[LPL]H�WKH�OLNHOLKRRG

IXQFWLRQ

7KLV�LV�RQH�RI�WKH�PRVW�FRPPRQO\�XVHG�HVWLPDWRUV�LQ�VWDWLVWLFV

,QWXLWLYHO\�DSSHDOLQJ

Page 18: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

')( * * +-, .0/ 132 1 / 4 5�687�9�, :<; =>; ?34�1 @ 4�A 4 B�, 7�9 5�CED / F 4 FHG8D A 5 F I 6�/ 5 J , KML<N N 9 J 4 1 9 7 J / D�9 7�A ; O<A A 1 / P�Q J FH1 4 F 4 1 R 4 5 S TVU W�X0Y��

0D[LPXP�/LNHOLKRRG�(VWLPDWLRQ��&RQW��

[&RQVLVWHQW

� (VWLPDWH�FRQYHUJHV�WR�EHVW�SRVVLEOH�YDOXH�DV�WKH

QXPEHU�RI�H[DPSOHV�JURZ

[$V\PSWRWLF�HIILFLHQF\

� (VWLPDWH�LV�DV�FORVH�WR�WKH�WUXH�YDOXH�DV�SRVVLEOH

JLYHQ�D�SDUWLFXODU�WUDLQLQJ�VHW

[5HSUHVHQWDWLRQ�LQYDULDQW

� $�WUDQVIRUPDWLRQ�LQ�WKH�SDUDPHWHU�UHSUHVHQWDWLRQ

GRHV�QRW�FKDQJH�WKH�HVWLPDWHG�SUREDELOLW\�GLVWULEXWLRQ

')( * * +-, .0/ 132 1 / 4 5�687�9�, :<; =>; ?34�1 @ 4�A 4 B�, 7�9 5�CED / F 4 FHG8D A 5 F I 6�/ 5 J , KML<N N 9 J 4 1 9 7 J / D�9 7�A ; O<A A 1 / P�Q J FH1 4 F 4 1 R 4 5 S TVU W�X0Y��

([DPSOH��0/(�LQ�%LQRPLDO�'DWD

[$SSO\LQJ�WKH�0/(�SULQFLSOH�ZH�JHW

��:KLFK�FRLQFLGHV�ZLWK�ZKDW�RQH�ZRXOG�H[SHFW�

� ��� � ��� ��� ��� � �

/��'

([DPSOH��1 � �1 � ��� ������

0/(�HVWLPDWH�LV����� ����

7+

+

11

1

+=θ×

Page 19: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

')( * * +-, .0/ 132 1 / 4 5�687�9�, :<; =>; ?34�1 @ 4�A 4 B�, 7�9 5�CED / F 4 FHG8D A 5 F I 6�/ 5 J , KML<N N 9 J 4 1 9 7 J / D�9 7�A ; O<A A 1 / P�Q J FH1 4 F 4 1 R 4 5 S TVU W�X0Y��

/HDUQLQJ�3DUDPHWHUV�IRU�WKH�%XUJODU\

6WRU\

� �

⋅⋅⋅⋅⋅⋅⋅⋅

=

@>@>@>@>

@�>@�>@�>@�>

0&0$0%0(

&$%(

'

∏∏ ∏∏

Θ⋅ΘΘΘ=

Θ⋅Θ⋅Θ⋅Θ=

Θ=Θ

��� �

��

��� ��

P(3P%30(P%P$3P$P&3

P(3P%30(P%P$3P$P&3

P&P$P%P(3'/

��@>���@>���@>@�>_@>���@>_@>�

��@>���@>���@>@�>_@>���@>_@>�

��@>@�>@�>@�>����

� �

L�L�G��VDPSOHV

1HWZRUN�IDFWRUL]DWLRQ

:H�KDYH���LQGHSHQGHQW�HVWLPDWLRQ�SUREOHPV

��� � � ��� ��� ��� � � � ��� ��!�� "$# %&# '���� ( ��) � *�� ��! �,+.- � / � /10 - ) � / 2 �,� � 3 � 465$7 7 ! 3 � � ! � 3 � -�! ��) # 8$) ) � � 9�: 3 /1� � / � � ; � � < =?>A@CB�D�E

*HQHUDO�%D\HVLDQ�1HWZRUNV

:H�FDQ�GHILQH�WKH�OLNHOLKRRG�IRU�D�%D\HVLDQ�QHWZRUN�

7KH�OLNHOLKRRG�GHFRPSRVHV�DFFRUGLQJ�WR�WKH�VWUXFWXUH�RI�WKH

QHWZRUN�

∏∏

∏∏

Θ=

Θ=

Θ=

Θ=Θ

F FFFHG FFFGIF FFF

G J

'/

P3DP[3

P3DP[3

P[P[3'/

���

��@>_@>�

��@>_@>�

��@>�@�>���� K K

L�L�G��VDPSOHV

1HWZRUN�IDFWRUL]DWLRQ

Page 20: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� ��� ��� � � � ��� ��!�� "$# %&# '���� ( ��) � *�� ��! �,+.- � / � /10 - ) � / 2 �,� � 3 � 465$7 7 ! 3 � � ! � 3 � -�! ��) # 8$) ) � � 9�: 3 /1� � / � � ; � � < =?>A@CB�D��

*HQHUDO�%D\HVLDQ�1HWZRUNV��&RQW��

'HFRPSRVLWLRQ��,QGHSHQGHQW�(VWLPDWLRQ�3UREOHPV

,I�WKH�SDUDPHWHUV�IRU�HDFK�IDPLO\�DUH�QRW�UHODWHG��WKHQ�WKH\�FDQ

EH�HVWLPDWHG�LQGHSHQGHQWO\�RI�HDFK�RWKHU�

��� � � ��� ��� ��� � � � ��� ��!�� "$# %&# '���� ( ��) � *�� ��! �,+.- � / � /10 - ) � / 2 �,� � 3 � 465$7 7 ! 3 � � ! � 3 � -�! ��) # 8$) ) � � 9�: 3 /1� � / � � ; � � < =?>A@CB����

)URP�%LQRPLDO�WR�0XOWLQRPLDO

�)RU�H[DPSOH��VXSSRVH�;�FDQ�KDYH�WKH�YDOXHV�����«�.

�:H�ZDQW�WR�OHDUQ�WKH�SDUDPHWHUV�θ� � ��θ� � ��«��θ�

6XIILFLHQW�VWDWLVWLFV��

1 � ��1 � ��«��1 ���WKH�QXPEHU�RI�WLPHV�HDFK�RXWFRPH�LV

REVHUYHG

/LNHOLKRRG�IXQFWLRQ�

0/(�

∏=

θ=θ

��

�� '/ ����

∑=θ

��1

1 ��×

Page 21: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254�672

/LNHOLKRRG�IRU�0XOWLQRPLDO�1HWZRUNV

8:KHQ�ZH�DVVXPH�WKDW�3�; 9 �_�3D 9 ���LV�PXOWLQRPLDO��ZH�JHW

IXUWKHU�GHFRPSRVLWLRQ�

8)RU�HDFK�YDOXH�SD 9 �RI�WKH�SDUHQWV�RI�; 9 �ZH�JHW�DQ

LQGHSHQGHQW�PXOWLQRPLDO�SUREOHP8

7KH�0/(�LV

∏∏∏∏

∏ ∏

θ=Θ=

Θ=

Θ=Θ

=

: :::::: :

::: ::

;=<?>;=<>@

;=<>;=<A>;B<>@CCC

;=< ;=<DE <DCCC

DCCCCC

SD[3

SDP[3

P3DP[3'/

FGHIFGHJKG

��_�

��_@>�

��@>_@>����

��

���× L MMM

NPOQSD1

SD[1RR =θ

S�T U U V�W XY Z�[ Z Y \ ]�^�_�`�W a�b c�b d�\�Z e \�f \ g�W _�` ]�h i Y j \ j#k�i f ] j l ^�Y ] m W n(o�p p ` m \ Z ` _ m Y i�` _�f b q�f f Z Y r�s m j#Z \ j \ Z t \ ] u v1w3x5y�z|{

,V�0/(�DOO�ZH�QHHG"

}6XSSRVH�WKDW�DIWHU����REVHUYDWLRQV�

������������������0/�HVWLPDWHV�3�+�� �����IRU�WKH�WKXPEWDFN~ :RXOG�\RX�EHW�RQ�KHDGV�IRU�WKH�QH[W�WRVV"

}6XSSRVH�QRZ�WKDW�DIWHU����REVHUYDWLRQV�

�������������������0/�HVWLPDWHV�3�+�� �����IRU�D�FRLQ~ :RXOG�\RX�SODFH�WKH�VDPH�EHW"

Page 22: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

S�T U U V�W XY Z�[ Z Y \ ]�^�_�`�W a�b c�b d�\�Z e \�f \ g�W _�` ]�h i Y j \ j#k�i f ] j l ^�Y ] m W n(o�p p ` m \ Z ` _ m Y i�` _�f b q�f f Z Y r�s m j#Z \ j \ Z t \ ] u v1w3x5y�z��

}0/(�FRPPLWV�WR�D�VSHFLILF�YDOXH�RI�WKH�XQNQRZQ

SDUDPHWHU�V�

}0/(�LV�WKH�VDPH�LQ�ERWK�FDVHV

}&RQILGHQFH�LQ�SUHGLFWLRQ�LV�FOHDUO\�GLIIHUHQW

%D\HVLDQ�,QIHUHQFH

� � � � � � � � � � � � � � � � � � � � � � � � � � � �

YV�

&RLQ 7KXPEWDFN

S�T U U V�W XY Z�[ Z Y \ ]�^�_�`�W a�b c�b d�\�Z e \�f \ g�W _�` ]�h i Y j \ j#k�i f ] j l ^�Y ] m W n(o�p p ` m \ Z ` _ m Y i�` _�f b q�f f Z Y r�s m j#Z \ j \ Z t \ ] u v1w3x5y�z z

%D\HVLDQ�,QIHUHQFH��FRQW��

)UHTXHQWLVW�$SSURDFK�}$VVXPHV�WKHUH�LV�DQ�XQNQRZQ�EXW�IL[HG�SDUDPHWHU�θ

}(VWLPDWHV�θ�ZLWK�VRPH�FRQILGHQFH

}3UHGLFWLRQ�E\�XVLQJ�WKH�HVWLPDWHG�SDUDPHWHU�YDOXH

%D\HVLDQ�$SSURDFK�}5HSUHVHQWV�XQFHUWDLQW\�DERXW�WKH�XQNQRZQ�SDUDPHWHU

}8VHV�SUREDELOLW\�WR�TXDQWLI\�WKLV�XQFHUWDLQW\�

~ 8QNQRZQ�SDUDPHWHUV�DV�UDQGRP�YDULDEOHV}3UHGLFWLRQ�IROORZV�IURP�WKH�UXOHV�RI�SUREDELOLW\�

~ ([SHFWDWLRQ�RYHU�WKH�XQNQRZQ�SDUDPHWHUV

Page 23: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

S�T U U V�W XY Z�[ Z Y \ ]�^�_�`�W a�b c�b d�\�Z e \�f \ g�W _�` ]�h i Y j \ j#k�i f ] j l ^�Y ] m W n(o�p p ` m \ Z ` _ m Y i�` _�f b q�f f Z Y r�s m j#Z \ j \ Z t \ ] u v1w3x5y�z��

%D\HVLDQ�,QIHUHQFH��FRQW��

}:H�FDQ�UHSUHVHQW�RXU�XQFHUWDLQW\�DERXW�WKH�VDPSOLQJ

SURFHVV�XVLQJ�D�%D\HVLDQ�QHWZRUN

~ 7KH�REVHUYHG�YDOXHV�RI�;�DUH�LQGHSHQGHQW�JLYHQ�θ~ 7KH�FRQGLWLRQDO�SUREDELOLWLHV��3�[>P@�_�θ����DUH�WKHSDUDPHWHUV�LQ�WKH�PRGHO

~ 3UHGLFWLRQ�LV�QRZ�LQIHUHQFH�LQ�WKLV�QHWZRUN

θ

������� ����� ���� �� ��� ������

������������������ "! #�$%�&� '

S�T U U V�W XY Z�[ Z Y \ ]�^�_�`�W a�b c�b d�\�Z e \�f \ g�W _�` ]�h i Y j \ j#k�i f ] j l ^�Y ] m W n(o�p p ` m \ Z ` _ m Y i�` _�f b q�f f Z Y r�s m j#Z \ j \ Z t \ ] u v1w3x5y�z�(

%D\HVLDQ�,QIHUHQFH��FRQW��

}3UHGLFWLRQ�DV�LQIHUHQFH�LQ�WKLV�QHWZRUN

ZKHUH

θθθ+=

θθθ+=

+

∫∫

G0[[30[3

G0[[30[[0[3

0[[0[3

@�>�@��>_��_@�>�

@�>�@��>_�@�>�@��>�_@�>�

@�>�@��>_@�>�

K

KK

K

@�>@��>�

���_@>@��>�@�>@��>_�

0[[3

30[[30[[3

K

K

K

θθ=θ

3RVWHULRU

/LNHOLKRRG 3ULRU

3UREDELOLW\�RI�GDWD

θ

)�* +�, )�* - , )�* .�, )/* .102+�,

Page 24: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

S�T U U V�W XY Z�[ Z Y \ ]�^�_�`�W a�b c�b d�\�Z e \�f \ g�W _�` ]�h i Y j \ j#k�i f ] j l ^�Y ] m W n(o�p p ` m \ Z ` _ m Y i�` _�f b q�f f Z Y r�s m j#Z \ j \ Z t \ ] u v1w3x5y�z��

([DPSOH��%LQRPLDO�'DWD�5HYLVLWHG

}6XSSRVH�WKDW�ZH�FKRRVH�D�XQLIRUP�SULRU�3�θ��� ���IRU�θ�LQ�>���@

}7KHQ�3�θ�_'��LV�SURSRUWLRQDO�WR�WKH�OLNHOLKRRG�/�θ��'�

� �1 � �1 � ��� ������

� 0/(�IRU�3�;� �+���LV����� ����� %D\HVLDQ�SUHGLFWLRQ�LV

K�������

��_��_@�>� ==θθ⋅θ==+ ∫ G'3'+0[3

� �� �� � �� � � � � �

���_@>@��>�@�>@��>_� θ⋅θ∝θ 30[[30[[3 KK

��� � � ��� ��� ��� � � � � �"! # � $&% '(% )�� � * � + � , � ! # �.-0/ � 1 � 132"/ + � 1 4 �.� � 5 � 687&9 9 # 5 � � # ! 5 � / # ! + % :&+ + � � ; < 5 13� � 1 � � = � � > ?A@CBED�F�G

%D\HVLDQ�,QIHUHQFH�DQG�0/(

H,Q�RXU�H[DPSOH��0/(�DQG�%D\HVLDQ�SUHGLFWLRQ�GLIIHU

%XW«

,I�SULRU�LV�ZHOO�EHKDYHGH

'RHV�QRW�DVVLJQ���GHQVLW\�WR�DQ\�³IHDVLEOH´�SDUDPHWHU

YDOXH

��7KHQ��ERWK�0/(�DQG�%D\HVLDQ�SUHGLFWLRQ�FRQYHUJH�WR

���WKH�VDPH�YDOXHH

%RWK�FRQYHUJH�WR�WKH�³WUXH´�XQGHUO\LQJ�GLVWULEXWLRQ

�DOPRVW�VXUHO\�

Page 25: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� ��� ��� � � � � �"! # � $&% '(% )�� � * � + � , � ! # �.-0/ � 1 � 132"/ + � 1 4 �.� � 5 � 687&9 9 # 5 � � # ! 5 � / # ! + % :&+ + � � ; < 5 13� � 1 � � = � � > ?A@CBED�F��

'LULFKOHW�3ULRUV

H5HFDOO�WKDW�WKH�OLNHOLKRRG�IXQFWLRQ�LV

H$�'LULFKOHW�SULRU�ZLWK�K\SHUSDUDPHWHUV��α � �«�α � LV�GHILQHG�DV

�������������������������������������IRU�OHJDO�θ� � �«��θ� �

7KHQ�WKH�SRVWHULRU�KDV�WKH�VDPH�IRUP��ZLWK�K\SHUSDUDPHWHUV

α � �1� � �«�α � �1� �

∏=

θ=Θ�

��

���'/ ���

∏=

−αθ∝Θ

�� �3 �

���

∏∏∏=

−+α

==

−α θ=θθ∝

ΘΘ∝Θ�

��

��

��

��

�� ����

'33'3

��

���

�_����_�

��� � � ��� ��� � � � � ! "$#&%$'$� (*) +,) - !$� . !$/ ! 0$� %$' "2143 � 5 ! 576&3 / " 5 8 #2� " 9 � :<;*= = ' 9 ! � ' % 9 � 3$' %$/ ) >*/ / � � ?$@ 9 57� ! 5 ! � A ! " B CEDGFIH�JLK

'LULFKOHW�3ULRUV��FRQW��

M:H�FDQ�FRPSXWH�WKH�SUHGLFWLRQ�RQ�D�QHZ�HYHQW�LQ�FORVHG

IRUP�N ,I�3�Θ��LV�'LULFKOHW�ZLWK�K\SHUSDUDPHWHUV�α O �«�α P WKHQ

�������6LQFH�WKH�SRVWHULRU�LV�DOVR�'LULFKOHW��ZH�JHW

∑∫ αα

=ΘΘ⋅θ==Q

QRR G3N;3 ���@�>�

∑∫ +α+α

=ΘΘ⋅θ==+S

SS ���_��_@�>�

1

1G'3'N0;3

TTT

Page 26: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254672

3ULRUV�,QWXLWLRQ

87KH�K\SHUSDUDPHWHUV�α 9 �«�α : �FDQ�EH�WKRXJKW�RI�DV³LPDJLQDU\´�FRXQWV�IURP�RXU�SULRU�H[SHULHQFH

8(TXLYDOHQW�VDPSOH�VL]H� �α 9 �«�α :

87KH�ODUJHU�WKH�HTXLYDOHQW�VDPSOH�VL]H�WKH�PRUH

FRQILGHQW�ZH�DUH�LQ�RXU�SULRU

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /1032546<;

(IIHFW�RI�3ULRUV

3UHGLFWLRQ�RI�3�; +���DIWHU�VHHLQJ�GDWD�ZLWK�1 = � ������1 >IRU�GLIIHUHQW�VDPSOH�VL]HV

?A@ BDC?A@ E?A@ EAC?A@ F?A@ FGC?A@ H?A@ HIC?A@ C?A@ CGC

? EJ? HA? KJ? L5? BM?J? ?

?A@ B

?A@ E

?A@ F

?A@ H?A@ C

?A@ K

? EJ? HA? KG? L5? B�?G?

'LIIHUHQW�VWUHQJWK�α N ���α O �)L[HG�UDWLR�α N ���α O �

)L[HG�VWUHQJWK�α N ���α O'LIIHUHQW�UDWLR�α N ���α O �

Page 27: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /1032546��

(IIHFW�RI�3ULRUV��FRQW��8,Q�UHDO�GDWD��%D\HVLDQ�HVWLPDWHV�DUH�OHVV�VHQVLWLYH�WR

QRLVH�LQ�WKH�GDWD

?A@ B

?A@ E

?A@ F

?A@ H

?A@ C

?A@ K

?A@ �

C B�? BMC EJ? EJC FJ? F5C HG? HAC C5?

3�;� ��_'

1�

������ � �� �� ��� � @ C�� @ C���� � �� �� ��� � B�� B���� � �� �� ��� � C�� C��

�� � �� �� ��� � B�?�� BD?��

?

B ������! � ��" � �

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /1032546�#

&RQMXJDWH�)DPLOLHV

87KH�SURSHUW\�WKDW�WKH�SRVWHULRU�GLVWULEXWLRQ�IROORZV�WKH

VDPH�SDUDPHWULF�IRUP�DV�WKH�SULRU�GLVWULEXWLRQ�LV�FDOOHG

FRQMXJDF\$ 'LULFKOHW�SULRU�LV�D�FRQMXJDWH�IDPLO\�IRU�WKH�PXOWLQRPLDO�OLNHOLKRRG

8&RQMXJDWH�IDPLOLHV�DUH�XVHIXO�VLQFH�

$ )RU�PDQ\�GLVWULEXWLRQV�ZH�FDQ�UHSUHVHQW�WKHP�ZLWK

K\SHUSDUDPHWHUV$ 7KH\�DOORZ�IRU�VHTXHQWLDO�XSGDWH�ZLWKLQ�WKH�VDPH�UHSUHVHQWDWLRQ$ ,Q�PDQ\�FDVHV�ZH�KDYH�FORVHG�IRUP�VROXWLRQ�IRU�SUHGLFWLRQ

Page 28: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /1032546 6

%D\HVLDQ�1HWZRUNV�DQG�%D\HVLDQ

3UHGLFWLRQ

83ULRUV�IRU�HDFK�SDUDPHWHU�JURXS�DUH�LQGHSHQGHQW

8'DWD�LQVWDQFHV�DUH�LQGHSHQGHQW�JLYHQ�WKH�XQNQRZQ

SDUDPHWHUV

θ �

������� ������ ���� � �����������

��������� ������� �"!#�$&% '�(*),+"- !.�"!0/ -1+

23����� 23���� 2����4� 23���,�5���

θ 6�7 � θ �

θ 6�7 �P ����89�

23��89�

:<;���� =

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /1032546�>

%D\HVLDQ�1HWZRUNV�DQG�%D\HVLDQ

3UHGLFWLRQ��&RQW��

8:H�FDQ�DOVR�³UHDG´�IURP�WKH�QHWZRUN�

���&RPSOHWH�GDWD�Æ

�������������������������SRVWHULRUV�RQ�SDUDPHWHUV�DUH�LQGHSHQGHQW

θ ?

@�A�B�C @�A�DC @A�E C @�A�E�F�B�C

G�H�I�J�K L�J�M�M N"O#NP&Q R�S*T,U"V O.N"O0W V1U

X3A�B�C X3A�DC X�A�E4C X3A�E,F5B�C

θ Y�Z ? θ ?

θ Y�Z ?P @�A�[9C

X3A�[9C

\<]�J�K ^

Page 29: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254687

%D\HVLDQ�3UHGLFWLRQ�FRQW��

96LQFH�SRVWHULRUV�RQ�SDUDPHWHUV�IRU�HDFK�IDPLO\�DUH

LQGHSHQGHQW��ZH�FDQ�FRPSXWH�WKHP�VHSDUDWHO\9

3RVWHULRUV�IRU�SDUDPHWHUV�ZLWKLQ�IDPLOLHV�DUH�DOVR

LQGHSHQGHQW�

9&RPSOHWH�GDWD�⇒�WKH�SRVWHULRUV�RQ�θ :<;�=?>A@ �DQG�θ� :<;�=B>DC ��DUH

LQGHSHQGHQW

θ ?

θ Y9Z ?P @A�[9C

X�A�[9C

EGFIH J KLFNMPORQNMNFTS

θ ?

θ Y9Z ?8U�VP @�A�[9C

X3A�[9C

θ Y�Z ?WU#X

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /10325468Y

%D\HVLDQ�3UHGLFWLRQ�FRQW��

9*LYHQ�WKHVH�REVHUYDWLRQV��ZH�FDQ�FRPSXWH�WKH�SRVWHULRU

IRU�HDFK�PXOWLQRPLDO�θ� =?Z[;]\G^GZ �LQGHSHQGHQWO\_ 7KH�SRVWHULRU�LV�'LULFKOHW�ZLWK�SDUDPHWHUV

α�; Z �_SD Z ��1��; Z �_SD Z ��«��α�; Z N_SD Z ��1��; Z N_SD Z �

97KH�SUHGLFWLYH�GLVWULEXWLRQ�LV�WKHQ�UHSUHVHQWHG�E\�WKH

SDUDPHWHUV

��������������������������ZKLFK�LV�ZKDW�ZH�H[SHFWHG�

7KH�%D\HVLDQ�DQDO\VLV�MXVW�PDGH�WKH�DVVXPSWLRQV

H[SOLFLW

����

������a `aa

aaaabGcd

SD1SD

SD[1SD[ee +α

+α=θ

Page 30: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254687

$VVHVVLQJ�3ULRUV�IRU�%D\HVLDQ�1HWZRUNV

:H�QHHG�WKHα�[ 9 �SD 9 ���IRU�HDFK�QRGH�[:

;:H�FDQ�XVH�LQLWLDO�SDUDPHWHUV�Θ < DV�SULRU�LQIRUPDWLRQ

= 1HHG�DOVR�DQ�HTXLYDOHQW�VDPSOH�VL]H�SDUDPHWHU�0 <= 7KHQ��ZH�OHW�α�[ 9 �SD 9 �� �0 < •3�[ 9 �SD 9_Θ < �

;7KLV�DOORZV�WR�XSGDWH�D�QHWZRUN�XVLQJ�QHZ�GDWD

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254(>@?

/HDUQLQJ�3DUDPHWHUV��&DVH�6WXG\��FRQW��

;([SHULPHQW�

= 6DPSOH�D�VWUHDP�RI�LQVWDQFHV�IURP�WKH�DODUP�QHWZRUN= /HDUQ�SDUDPHWHUV�XVLQJ

� 0/(�HVWLPDWRU

� %D\HVLDQ�HVWLPDWRU�ZLWK�XQLIRUP�SULRU�ZLWK�GLIIHUHQW

VWUHQJWKV

Page 31: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254(> 2

/HDUQLQJ�3DUDPHWHUV��&DVH�6WXG\��FRQW��

&RPSDULQJ�WZR�GLVWULEXWLRQ�3�[���WUXH�PRGHO��YV��4�[��OHDUQHG�GLVWULEXWLRQ�����0HDVXUH�WKHLU�./�'LYHUJHQFH

= ��./�GLYHUJHQFH��ZKHQ�ORJV�DUH�LQ�EDVH���� � 7KH�SUREDELOLW\�3�DVVLJQV�WR�DQ�LQVWDQFH�ZLOO�EH��RQ�DYHUDJH�

WZLFH�DV�VPDOO�DV�WKH�SUREDELOLW\�4�DVVLJQV�WR�LW� ./�3__4��≥��� ./�3__4�� ���LII�DUH�3��DQG�4��HTXDO

∑=[ [4

[3[343./

����

ORJ���__�

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254(>��

/HDUQLQJ�3DUDPHWHUV��&DVH�6WXG\��FRQW��

??�� �?�� �?�� >?�� �

22�� �2�� �

? 68?@? 2 ?8?8? 2 68?@? �@?@?8? � 68?8? �@?8?@? �868?@? � ?@?8? � 68?@? 68?8?@?

./�'LYHUJHQFH

0

0/(%D\HV�Z��8QLIRUP�3ULRU��0 �%D\HV�Z��8QLIRUP�3ULRU��0 ��%D\HV�Z��8QLIRUP�3ULRU��0 ��%D\HV�Z��8QLIRUP�3ULRU��0 ��

Page 32: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254(> �

/HDUQLQJ�3DUDPHWHUV��6XPPDU\

;(VWLPDWLRQ�UHOLHV�RQ�VXIILFLHQW�VWDWLVWLFV= )RU�PXOWLQRPLDO�WKHVH�DUH�RI�WKH�IRUP�1��[ 9 �SD 9 �= 3DUDPHWHU�HVWLPDWLRQ

;%D\HVLDQ�PHWKRGV�DOVR�UHTXLUH�FKRLFH�RI�SULRUV

;%RWK�0/(�DQG�%D\HVLDQ�DUH�DV\PSWRWLFDOO\�HTXLYDOHQW�DQG

FRQVLVWHQW;%RWK�FDQ�EH�LPSOHPHQWHG�LQ�DQ�RQ�OLQH�PDQQHU�E\

DFFXPXODWLQJ�VXIILFLHQW�VWDWLVWLFV

����

������a � ������

����SD1SD

SD[1SD[�� +α

+α=θ

��

���× �

���SD1

SD[1 =θ

0/( %D\HVLDQ��'LULFKOHW�

��� � � ��� ��� ��� � � � ��� ��!�� "$# %&# '���� ( ��) � *�� ��! �,+.- � / � /10 - ) � / 2 �,� � 3 � 465$7 7 ! 3 � � ! � 3 � -�! ��) # 8$) ) � � 9�: 3 /1� � / � � ; � � < =?>A@CB6DFE

2XWOLQH

G,QWURGXFWLRQ

G%D\HVLDQ�QHWZRUNV��D�UHYLHZ

G3DUDPHWHU�OHDUQLQJ��&RPSOHWH�GDWD

ª3DUDPHWHU�OHDUQLQJ��,QFRPSOHWH�GDWDG

6WUXFWXUH�OHDUQLQJ��&RPSOHWH�GDWDG

$SSOLFDWLRQ��FODVVLILFDWLRQG

/HDUQLQJ�FDXVDO�UHODWLRQVKLSVG

6WUXFWXUH�OHDUQLQJ��,QFRPSOHWH�GDWDG

&RQFOXVLRQ

HJILKNMOIAPRQ S�TLU,Q TVS WYXZIV[\I\K,MOI]PJQ S TCU,Q TCS W^ K`_]aVb WVQ W]cCdVQ d

e I\U\K`_Oafb WLQ WgcCdCQ d

Page 33: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� ��� ��� � � � ��� ��!�� "$# %&# '���� ( ��) � *�� ��! �,+.- � / � /10 - ) � / 2 �,� � 3 � 465$7 7 ! 3 � � ! � 3 � -�! ��) # 8$) ) � � 9�: 3 /1� � / � � ; � � < =?>A@CB6D��

,QFRPSOHWH�'DWD

'DWD�LV�RIWHQ�LQFRPSOHWHG

6RPH�YDULDEOHV�RI�LQWHUHVW�DUH�QRW�DVVLJQHG�YDOXH

7KLV�SKHQRPHQD�KDSSHQ�ZKHQ�ZH�KDYHG

0LVVLQJ�YDOXHVG

+LGGHQ�YDULDEOHV

��� � � ��� ��� ��� � � � ��� ��!�� "$# %&# '���� ( ��) � *�� ��! �,+.- � / � /10 - ) � / 2 �,� � 3 � 465$7 7 ! 3 � � ! � 3 � -�! ��) # 8$) ) � � 9�: 3 /1� � / � � ; � � < =?>A@CB6DJD

0LVVLQJ�9DOXHV

G([DPSOHV�

G6XUYH\�GDWD

G0HGLFDO�UHFRUGV

� 1RW�DOO�SDWLHQWV�XQGHUJR�DOO�SRVVLEOH�WHVWV

Page 34: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� ��� ��� � � � ��� ��!�� "$# %&# '���� ( ��) � *�� ��! �,+.- � / � /10 - ) � / 2 �,� � 3 � 465$7 7 ! 3 � � ! � 3 � -�! ��) # 8$) ) � � 9�: 3 /1� � / � � ; � � < =?>A@CB6D��

0LVVLQJ�9DOXHV��FRQW��

&RPSOLFDWLQJ�LVVXH�G

7KH�IDFW�WKDW�D�YDOXH�LV�PLVVLQJ�PLJKW�EH�LQGLFDWLYH�RI�LWV

YDOXH� 7KH�SDWLHQW�GLG�QRW�XQGHUJR�;�5D\�VLQFH�VKH�FRPSODLQHG

DERXW�IHYHU�DQG�QRW�DERXW�EURNHQ�ERQHV«�

7R�OHDUQ�IURP�LQFRPSOHWH�GDWD�ZH�QHHG�WKH�IROORZLQJ

DVVXPSWLRQ�

0LVVLQJ�DW�5DQGRP��0$5��G

7KH�SUREDELOLW\�WKDW�WKH�YDOXH�RI�; � �LV�PLVVLQJ�LV�LQGHSHQGHQW

RI�LWV�DFWXDO�YDOXH�JLYHQ�RWKHU�REVHUYHG�YDOXHV

��� � � ��� ��� ��� � � � ��� ��!�� "$# %&# '���� ( ��) � *�� ��! �,+.- � / � /10 - ) � / 2 �,� � 3 � 465$7 7 ! 3 � � ! � 3 � -�! ��) # 8$) ) � � 9�: 3 /1� � / � � ; � � < =?>A@CB6D��

0LVVLQJ�9DOXHV��FRQW��G

,I�0$5�DVVXPSWLRQ�GRHV�QRW�KROG��ZH�FDQ�FUHDWH�QHZ

YDULDEOHV�WKDW�HQVXUH�WKDW�LW�GRHVG

:H�QRZ�FDQ�SUHGLFW�QHZ�H[DPSOHV��Z��SDWWHUQ�RI�RPPLVLRQV�G

:H�PLJKW�QRW�EH�DEOH�WR�OHDUQ�DERXW�WKH�XQGHUO\LQJ�SURFHVV

�=

+

7

+

+

7

<

"

"

+

7

7

7

"

"

7

+

'DWD �=

+

7

+

+

7

<

"

"

+

7

7

7

"

"

7

+

$XJPHQWHG�'DWD�������2EV�=

<

<

<

<

<

2EV�<

1

1

<

<

<

<

1

1

<

<

;

<=

; <

=

�������� ��������

��������

Page 35: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� ��� ��� � � � ��� ��!�� "$# %&# '���� ( ��) � *�� ��! �,+.- � / � /10 - ) � / 2 �,� � 3 � 465$7 7 ! 3 � � ! � 3 � -�! ��) # 8$) ) � � 9�: 3 /1� � / � � ; � � < =?>A@CB6D��

+LGGHQ��/DWHQW��9DULDEOHV

G$WWHPSW�WR�OHDUQ�D�PRGHO�ZLWK�YDULDEOHV�ZH�QHYHU�REVHUYH� ,Q�WKLV�FDVH��0$5�DOZD\V�KROGV

G:K\�VKRXOG�ZH�FDUH�DERXW�XQREVHUYHG�YDULDEOHV"

��� ��� ���

�� �� ��

��� ��� ���

�� �� ��

���SDUDPHWHUV ���SDUDPHWHUV

��� � � ��� ��� ��� � � � ��� ��!�� "$# %&# '���� ( ��) � *�� ��! �,+.- � / � /10 - ) � / 2 �,� � 3 � 465$7 7 ! 3 � � ! � 3 � -�! ��) # 8$) ) � � 9�: 3 /1� � / � � ; � � < =?>A@CB ���

+LGGHQ�9DULDEOHV��FRQW��

G+LGGHQ�YDULDEOHV�DOVR�DSSHDU�LQ�FOXVWHULQJ

G$XWRFODVV�PRGHO�

� +LGGHQ�YDULDEOHV�DVVLJQV

FODVV�ODEHOV� 2EVHUYHG�DWWULEXWHV�DUH

LQGHSHQGHQW�JLYHQ�WKH�FODVV

�� ���������

��� ������ ���

+LGGHQ

2EVHUYHG��� ���! "#� �%$& �'�( )+*-,�.�� �'�!�

Page 36: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254672

/HDUQLQJ�3DUDPHWHUV�IURP�,QFRPSOHWH

'DWD

&RPSOHWH�GDWD�8

,QGHSHQGHQW�SRVWHULRUV�IRU�θ � ��θ 9;: �=<?> �DQG�θ 9@: �A<3B,QFRPSOHWH�GDWD�8

3RVWHULRUV�FDQ�EH�LQWHUGHSHQGHQW8

&RQVHTXHQFH�C 0/�SDUDPHWHUV�FDQ�QRW�EH�FRPSXWHG�VHSDUDWHO\�IRU�HDFK

PXOWLQRPLDOC 3RVWHULRU�LV�QRW�D�SURGXFW�RI�LQGHSHQGHQW�SRVWHULRUV

θ D

θ EGF DIH#JP KML�N;O

P L�N;Oθ EGF DIH�Q

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /1032546IR

([DPSOH

��PLVVLQJ�YDOXHV��PLVVLQJ�YDOXHQR�PLVVLQJ�YDOXHV

S6LPSOH�QHWZRUN�

8 3�;��DVVXPHG�WR�EH�NQRZQ

S/LNHOLKRRG�LV�D�IXQFWLRQ�RI���SDUDPHWHUV��3�< +_; +���3�< +_; 7�

S&RQWRXU�SORWV�RI�ORJ�OLNHOLKRRG�IRU�GLIIHUHQW�QXPEHU�RI�PLVVLQJ�YDOXHV�RI�;�0� ����

3�< +

_; +

3�< +_; 7� 3�< +_; 7� 3�< +_; 7�

; <

Page 37: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /1032546��

/HDUQLQJ�3DUDPHWHUV�IURP�,QFRPSOHWH

'DWD��FRQW���

8,Q�WKH�SUHVHQFH�RI�LQFRPSOHWH�GDWD��WKH�OLNHOLKRRG�FDQ�KDYH

PXOWLSOH�JOREDO�PD[LPD

8([DPSOH�

C :H�FDQ�UHQDPH�WKH�YDOXHV�RI�KLGGHQ�YDULDEOH�+C ,I�+�KDV�WZR�YDOXHV��OLNHOLKRRG�KDV�WZR�JOREDO�PD[LPD

86LPLODUO\��ORFDO�PD[LPD�DUH�DOVR�UHSOLFDWHG

80DQ\�KLGGHQ�YDULDEOHV�⇒�D�VHULRXV�SUREOHP

+ <

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /1032546��

/�Θ

_'�

([SHFWDWLRQ�0D[LPL]DWLRQ��(0��S

8VH�³FXUUHQW�SRLQW´�WR�FRQVWUXFW�DOWHUQDWLYH�IXQFWLRQ��ZKLFK�LV�³QLFH´�S

*XDUDQW\��PD[LPXP�RI�QHZ�IXQFWLRQ�LV�EHWWHU�VFRULQJ�WKH�FXUUHQW�SRLQW

0/(�IURP�,QFRPSOHWH�'DWD

8)LQGLQJ�0/(�SDUDPHWHUV��QRQOLQHDU�RSWLPL]DWLRQ�SUREOHP

*UDGLHQW�$VFHQW�S

)ROORZ�JUDGLHQW�RI�OLNHOLKRRG�Z�U�W��WR�SDUDPHWHUVS

$GG�OLQH�VHDUFK�DQG�FRQMXJDWH�JUDGLHQW�PHWKRGV�WR�JHW�IDVW�FRQYHUJHQFH

Θ%RWK�S

)LQG�ORFDO�PD[LPD�S

5HTXLUH�PXOWLSOH�UHVWDUWV�WR�ILQG�DSSUR[��WR�WKH�JOREDO�PD[LPXPS

5HTXLUH�FRPSXWDWLRQV�LQ�HDFK�LWHUDWLRQ

Page 38: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /1032546��

*UDGLHQW�$VFHQW

80DLQ�UHVXOW

85HTXLUHV�FRPSXWDWLRQ��3�[ � �3D � _R>P@����IRU�DOO�L��P

83URV�

� )OH[LEOH� &ORVHO\�UHODWHG�WR�PHWKRGV�LQ�QHXUDO�QHWZRUN�WUDLQLQJ

8&RQV�

� 1HHG�WR�SURMHFW�JUDGLHQW�RQWR�VSDFH�RI�OHJDO�SDUDPHWHUV� 7R�JHW�UHDVRQDEOH�FRQYHUJHQFH�ZH�QHHG�WR�FRPELQH�ZLWK�³VPDUW´

RSWLPL]DWLRQ�WHFKQLTXHV

�@�>_����_�ORJ

��Θ

θ=

θ∂Θ∂ ∑ PRSD[3

'3 ������� ����

�� � � ��� ��� ��� � � � ��������� "! #$! %���� & ��' � (�� ��� �*),+ � - � -/.�+ ' � - 0 �*� � 1 � 243"5 5 � 1 � � � � 1 � +�� ��' ! 6"' ' � � 7�8 1 -/� � - � � 9 � � : ;=<?>A@�BDC

([SHFWDWLRQ�0D[LPL]DWLRQ��(0�

E$�JHQHUDO�SXUSRVH�PHWKRG�IRU�OHDUQLQJ�IURP�LQFRPSOHWH�GDWD

,QWXLWLRQ�E

,I�ZH�KDG�DFFHVV�WR�FRXQWV��WKHQ�ZH�FDQ�HVWLPDWH�SDUDPHWHUVE

+RZHYHU��PLVVLQJ�YDOXHV�GR�QRW�DOORZ�WR�SHUIRUP�FRXQWVE

³&RPSOHWH´�FRXQWV�XVLQJ�FXUUHQW�SDUDPHWHU�DVVLJQPHQW

���

���

���

���

F= 1��;�<��FHGJI+

7++7

<

""+77

7""7+

+7+7

++77KDL�M�NPORQ STNPUWV

Θ X NZY\[ ]

([SHFWHG�&RXQWVKDL�M�NPORQ STNDO^V _`NPUTVΘ X N^Ya[ b

'DWD

&XUUHQW�

PRGHO

Page 39: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

�� � � ��� ��� ��� � � � ��������� "! #$! %���� & ��' � (�� ��� �*),+ � - � -/.�+ ' � - 0 �*� � 1 � 243"5 5 � 1 � � � � 1 � +�� ��' ! 6"' ' � � 7�8 1 -/� � - � � 9 � � : ;=<?>A@�B�B

(0��FRQW��

7UDLQLQJ

'DWD

��� ��� ����

� � � � � �

��� �� ������� �������� L�� V���

+

([SHFWHG�&RXQWV "!�#%$ � "!�#%& � "!�#(' � "!*),+�#-$�+.#%$/+�#-' � "!10�$�+�) � "!10�&�+2) � "!10�'�+2) �

354687�9.: ;2:=< 4?>

@1A�B�C : D?7�EF D72;?GH;6ID2:HD?G1< J�D

K�L K�M K�NO

P?L P?M P?N

Q5R.S�T�U�V�SXW�V�U�Y�Z�[�\ !�](+Θ$ �

@1^IB/C :HD?7.E

5HLWHUDWH

_a` b b ced f�g h=i h g j kHl*mHnHd o/p qrp s=jHh t jHu j vHd mHn kxwzy g { j {}|*y u k { ~ lxg k � d �1�/� � n � j h n m � g yHn mHu p �/u u h g �H� � {}h j { j h � j k � ^�����B����

(0��FRQW��

)RUPDO�*XDUDQWHHV�� /�Θ � �'��≥�/�Θ � �'�

� (DFK�LWHUDWLRQ�LPSURYHV�WKH�OLNHOLKRRG

�,I�Θ �-� Θ � ��WKHQ�Θ � LV�D�VWDWLRQDU\�SRLQW��RI�/�Θ�'�

� 8VXDOO\��WKLV�PHDQV�D�ORFDO�PD[LPXP

0DLQ�FRVW��

&RPSXWDWLRQV�RI�H[SHFWHG�FRXQWV�LQ�(�6WHS�

5HTXLUHV�D�FRPSXWDWLRQ�SDVV�IRU�HDFK�LQVWDQFH�LQ�WUDLQLQJ�VHW� 7KHVH�DUH�H[DFWO\�WKH�VDPH�DV�IRU�JUDGLHQW�DVFHQW�

Page 40: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

_a` b b ced f�g h=i h g j kHl*mHnHd o/p qrp s=jHh t jHu j vHd mHn kxwzy g { j {}|*y u k { ~ lxg k � d �1�/� � n � j h n m � g yHn mHu p �/u u h g �H� � {}h j { j h � j k � ^�����B����

([DPSOH��(0�LQ�FOXVWHULQJ

�&RQVLGHU�FOXVWHULQJ�H[DPSOH

(�6WHS�� &RPSXWH�3�&>P@_; � >P@�«�; � >P@�Θ�� 7KLV�FRUUHVSRQGV�WR�³VRIW´�DVVLJQPHQW�WR�FOXVWHUV� &RPSXWH�H[SHFWHG�VWDWLVWLFV�

0�6WHS� 5H�HVWLPDWH�3�; � _&���3�&�

��� ��� ���

��� ������ ���

∑=

Θ=�� ���� �

� P[P[F3F[1( ��� �@�>@�����>_��@��>

!#" $ $ %'& (*) +-, + ) . /1032141& 576 896 :-.1+ ; .1< . =1& 214 />@? ) A . A B3? < / A C 0) / D & EGF7H H 4 D . + 4 2 D ) ?14 21< 6 I7< < + ) J1K D A + . A . + L . / M NPORQTS*U�V

(0�LQ�3UDFWLFH

,QLWLDO�SDUDPHWHUV�W

5DQGRP�SDUDPHWHUV�VHWWLQJW

³%HVW´�JXHVV�IURP�RWKHU�VRXUFH

6WRSSLQJ�FULWHULD�W

6PDOO�FKDQJH�LQ�OLNHOLKRRG�RI�GDWDW

6PDOO�FKDQJH�LQ�SDUDPHWHU�YDOXHV

$YRLGLQJ�EDG�ORFDO�PD[LPD�W

0XOWLSOH�UHVWDUWVW

(DUO\�³SUXQLQJ´�RI�XQSURPLVLQJ�RQHV

6SHHG�XS�W

YDULRXV�PHWKRGV�WR�VSHHG�FRQYHUJHQFH

Page 41: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

!#" $ $ %'& (*) +-, + ) . /1032141& 576 896 :-.1+ ; .1< . =1& 214 />@? ) A . A B3? < / A C 0) / D & EGF7H H 4 D . + 4 2 D ) ?14 21< 6 I7< < + ) J1K D A + . A . + L . / M NPORQTS*U Q

(UURU�RQ�WUDLQLQJ�VHW��$ODUP�

([SHULPHQW�E\�%DXU��.ROOHU�DQG�6LQJHU�>8$,��@

!#" $ $ %'& (*) +-, + ) . /1032141& 576 896 :-.1+ ; .1< . =1& 214 />@? ) A . A B3? < / A C 0) / D & EGF7H H 4 D . + 4 2 D ) ?14 21< 6 I7< < + ) J1K D A + . A . + L . / M NPORQTS*U �

7HVW�VHW�HUURU��DODUP�

Page 42: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

!#" $ $ %'& (*) +-, + ) . /1032141& 576 896 :-.1+ ; .1< . =1& 214 />@? ) A . A B3? < / A C 0) / D & EGF7H H 4 D . + 4 2 D ) ?14 21< 6 I7< < + ) J1K D A + . A . + L . / M NPORQTS*U��

3DUDPHWHU�YDOXH��$ODUP�

!#" $ $ %'& (*) +-, + ) . /1032141& 576 896 :-.1+ ; .1< . =1& 214 />@? ) A . A B3? < / A C 0) / D & EGF7H H 4 D . + 4 2 D ) ?14 21< 6 I7< < + ) J1K D A + . A . + L . / M NPORQTS*U��

3DUDPHWHU�YDOXH��$ODUP�

Page 43: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

!#" $ $ %'& (*) +-, + ) . /1032141& 576 896 :-.1+ ; .1< . =1& 214 />@? ) A . A B3? < / A C 0) / D & EGF7H H 4 D . + 4 2 D ) ?14 21< 6 I7< < + ) J1K D A + . A . + L . / M NPORQTS*U��

3DUDPHWHU�YDOXH��$ODUP�

!#" $ $ %'& (*) +-, + ) . /1032141& 576 896 :-.1+ ; .1< . =1& 214 />@? ) A . A B3? < / A C 0) / D & EGF7H H 4 D . + 4 2 D ) ?14 21< 6 I7< < + ) J1K D A + . A . + L . / M NPORQTS*U��

%D\HVLDQ�,QIHUHQFH�ZLWK�,QFRPSOHWH�'DWD

5HFDOO��%D\HVLDQ�HVWLPDWLRQ�

&RPSOHWH�GDWD��FORVHG�IRUP�VROXWLRQ�IRU�LQWHJUDO

,QFRPSOHWH�GDWD��

1R�VXIILFLHQW�VWDWLVWLFV��H[FHSW�WKH�GDWD��

3RVWHULRU�GRHV�QRW�GHFRPSRVH�

1R�FORVHG�IRUP�VROXWLRQ

×1HHG�WR�XVH�DSSUR[LPDWLRQV

θθθ+=+ ∫ G'30[3'0[3 �_��_@�>��_@�>�

Page 44: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

!#" $ $ %'& (*) +-, + ) . /1032141& 576 896 :-.1+ ; .1< . =1& 214 />@? ) A . A B3? < / A C 0) / D & EGF7H H 4 D . + 4 2 D ) ?14 21< 6 I7< < + ) J1K D A + . A . + L . / M NPORQTS*U��

0$3�$SSUR[LPDWLRQ

�6LPSOHVW�DSSUR[LPDWLRQ��0$3�SDUDPHWHUV

� 0$3�����0D[LPXP�$�SRVWHULRUL�3UREDELOLW\

ZKHUH

$VVXPSWLRQ��

3RVWHULRU�PDVV�LV�GRPLQDWHG�E\�D�0$3�SDUDPHWHUV

)LQGLQJ�0$3�SDUDPHWHUV��

6DPH�WHFKQLTXHV�DV�ILQGLQJ�0/�SDUDPHWHUV�

0D[LPL]H�3�θ_'��LQVWHDG�RI�/�θ�'�

�a_@�>��_@�>� θ+≈+ 0[3'0[3

�_�PD[DUJa'3 θ=θ θ

!#" $ $ %'& (*) +-, + ) . /1032141& 576 896 :-.1+ ; .1< . =1& 214 />@? ) A . A B3? < / A C 0) / D & EGF7H H 4 D . + 4 2 D ) ?14 21< 6 I7< < + ) J1K D A + . A . + L . / M NPORQTS*U�U

6WRFKDVWLF�$SSUR[LPDWLRQV

6WRFKDVWLF�DSSUR[LPDWLRQ��

6DPSOH�θ � ��«��θ � �IURP�3�θ_'��

$SSUR[LPDWH

∑ θ+≈+ ��

0[3N

'0[3 �_@�>��

�_@�>�

Page 45: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

!#" $ $ %'& (*) +-, + ) . /1032141& 576 896 :-.1+ ; .1< . =1& 214 />@? ) A . A B3? < / A C 0) / D & EGF7H H 4 D . + 4 2 D ) ?14 21< 6 I7< < + ) J1K D A + . A . + L . / M NPORQTS*U��

6WRFKDVWLF�$SSUR[LPDWLRQV��FRQW��

+RZ�GR�ZH�VDPSOH�IURP�3�θ_'�"

0DUNRY�&KDLQ�0RQWH�&DUOR��0&0&��PHWKRGV�W

)LQG�D�0DUNRY�&KDLQ�ZKRVH�VWDWLRQDU\�SUREDELOLW\�,V�3�θ_'�W

6LPXODWH�WKH�FKDLQ�XQWLO�FRQYHUJHQFH�WR�VWDWLRQDU\�EHKDYLRUW

&ROOHFW�VDPSOHV�IRU�WKH�³VWDWLRQDU\´�UHJLRQV

3URV�W

9HU\�IOH[LEOH�PHWKRG��ZKHQ�RWKHU�PHWKRGV�IDLOV��WKLV�RQH�XVXDOO\�ZRUNVW

7KH�PRUH�VDPSOHV�FROOHFWHG��WKH�EHWWHU�WKH�DSSUR[LPDWLRQ

&RQV�W

&DQ�EH�FRPSXWDWLRQDOO\�H[SHQVLYHW

+RZ�GR�ZH�NQRZ�ZKHQ�ZH�DUH�FRQYHUJLQJ�RQ�VWDWLRQDU\�GLVWULEXWLRQ"

!#" $ $ %'& (*) +-, + ) . /1032141& 576 896 :-.1+ ; .1< . =1& 214 />@? ) A . A B3? < / A C 0) / D & EGF7H H 4 D . + 4 2 D ) ?14 21< 6 I7< < + ) J1K D A + . A . + L . / M NPORQTS���V

6WRFKDVWLF�$SSUR[LPDWLRQV�

*LEEV�6DPSOLQJ

*LEEV�6DPSOHU��

$�VLPSOH�PHWKRG�WR�FRQVWUXFW

0&0&�VDPSOLQJ�SURFHVV

6WDUW��

&KRRVH��UDQGRP��YDOXHV�IRU�DOO�XQNQRZQ�YDULDEOHV

,WHUDWLRQ��

&KRRVH�DQ�XQNQRZQ�YDULDEOH� $�PLVVLQJ�GDWD�YDULDEOH�RU�XQNQRZQ�SDUDPHWHU� (LWKHU�D�UDQGRP�FKRLFH�RU�URXQG�URELQ�YLVLWV

�6DPSOH�D�YDOXH�IRU�WKH�YDULDEOH�JLYHQ�WKH�FXUUHQW�YDOXHV�RI�DOO

RWKHU�YDULDEOHV

θ �

θ ��� ���P �� ����

� ����θ ��� ����

Page 46: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

!#" $ $ %'& (*) +-, + ) . /1032141& 576 896 :-.1+ ; .1< . =1& 214 />@? ) A . A B3? < / A C 0) / D & EGF7H H 4 D . + 4 2 D ) ?14 21< 6 I7< < + ) J1K D A + . A . + L . / M NPORQTS�� Q

3DUDPHWHU�/HDUQLQJ�IURP�,QFRPSOHWH

'DWD��6XPPDU\�

1RQ�OLQHDU�RSWLPL]DWLRQ�SUREOHP�

0HWKRGV�IRU�OHDUQLQJ��(0�DQG�*UDGLHQW�$VFHQW� ([SORLW�LQIHUHQFH�IRU�OHDUQLQJ

'LIILFXOWLHV��

([SORUDWLRQ�RI�D�FRPSOH[�OLNHOLKRRG�SRVWHULRU� 0RUH�PLVVLQJ�GDWD�⇒�PDQ\�PRUH�ORFDO�PD[LPD� &DQQRW�UHSUHVHQW�SRVWHULRU�⇒�PXVW�UHVRUW�WR�DSSUR[LPDWLRQV

�,QIHUHQFH

� 0DLQ�FRPSXWDWLRQDO�ERWWOHQHFN�IRU�OHDUQLQJ� /HDUQLQJ�ODUJH�QHWZRUNV

�⇒�H[DFW�LQIHUHQFH�LV�LQIHDVLEOH

�⇒�UHVRUW�WR�VWRFKDVWLF�VLPXODWLRQ�RU�DSSUR[LPDWH�LQIHUHQFH

��H�J���VHH�-RUGDQ¶V�WXWRULDO�

!#" $ $ %'& (*) +-, + ) . /1032141& 576 896 :-.1+ ; .1< . =1& 214 />@? ) A . A B3? < / A C 0) / D & EGF7H H 4 D . + 4 2 D ) ?14 21< 6 I7< < + ) J1K D A + . A . + L . / M NPORQTS�� �

2XWOLQH

�,QWURGXFWLRQ

�%D\HVLDQ�QHWZRUNV��D�UHYLHZ

�3DUDPHWHU�OHDUQLQJ��&RPSOHWH�GDWD

�3DUDPHWHU�OHDUQLQJ��,QFRPSOHWH�GDWD

ª6WUXFWXUH�OHDUQLQJ��&RPSOHWH�GDWD

ª 6FRULQJ�PHWULFV� 0D[LPL]LQJ�WKH�VFRUH� /HDUQLQJ�ORFDO�VWUXFWXUH

�$SSOLFDWLRQ��FODVVLILFDWLRQ

�/HDUQLQJ�FDXVDO�UHODWLRQVKLSV

�6WUXFWXUH�OHDUQLQJ��,QFRPSOHWH�GDWD

�&RQFOXVLRQ

����������� �� ��� �� ������������������ � ��� �� �

� ������� �� ���� �

! ���������"� �� �#�� �

Page 47: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254687

%HQHILWV�RI�/HDUQLQJ�6WUXFWXUH

9(IILFLHQW�OHDUQLQJ����PRUH�DFFXUDWH�PRGHOV�ZLWK�OHVV�GDWD

: &RPSDUH��3�$��DQG�3�%��YV�MRLQW�3�$�%�IRUPHU�UHTXLUHV�OHVV�GDWD�

: 'LVFRYHU�VWUXFWXUDO�SURSHUWLHV�RI�WKH�GRPDLQ: ,GHQWLI\LQJ�LQGHSHQGHQFLHV�LQ�WKH�GRPDLQ�KHOSV�WR

� 2UGHU�HYHQWV�WKDW�RFFXU�VHTXHQWLDOO\

� 6HQVLWLYLW\�DQDO\VLV�DQG�LQIHUHQFH9

3UHGLFW�HIIHFW�RI�DFWLRQV: ,QYROYHV�OHDUQLQJ�FDXVDO�UHODWLRQVKLS�DPRQJ�YDULDEOHV

⇒GHIHU�WR�ODWHU�SDUW�RI�WKH�WXWRULDO

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /1032546<;

:K\�6WUXJJOH�IRU�$FFXUDWH�6WUXFWXUH

=,QFUHDVHV�WKH�QXPEHU�RI

SDUDPHWHUV�WR�EH�ILWWHG=

:URQJ�DVVXPSWLRQV�DERXW

FDXVDOLW\�DQG�GRPDLQ�VWUXFWXUH

=&DQQRW�EH�FRPSHQVDWHG�E\

DFFXUDWH�ILWWLQJ�RI�SDUDPHWHUV=

$OVR�PLVVHV�FDXVDOLW\�DQG

GRPDLQ�VWUXFWXUH

>@?#ACB D�E#F�? G H IKJ ? A�LNM<HOB

M<P�F�Q#R

S5F�A TUJ ?#A�V>@? A�B D�E#F�? G H I8J ? A�LNM<HWB

M<P�F�Q R

S5F�A TOJ ?#AV

>X?#ACB D E F�?#G�H I8J ? A�LYM<HWB

M<P�F�Q R

S5F�A TOJ ?�A�V

$GGLQJ�DQ�DUF 0LVVLQJ�DQ�DUF

Page 48: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /1032546��

$SSURDFKHV�WR�/HDUQLQJ�6WUXFWXUH

9&RQVWUDLQW�EDVHG

: 3HUIRUP�WHVWV�RI�FRQGLWLRQDO�LQGHSHQGHQFH: 6HDUFK�IRU�D�QHWZRUN�WKDW�LV�FRQVLVWHQW�ZLWK�WKH�REVHUYHG

GHSHQGHQFLHV�DQG�LQGHSHQGHQFLHV

96FRUH�EDVHG

: 'HILQH�D�VFRUH�WKDW�HYDOXDWHV�KRZ�ZHOO�WKH

�LQ�GHSHQGHQFLHV�LQ�D�VWUXFWXUH�PDWFK�WKH�REVHUYDWLRQV: 6HDUFK�IRU�D�VWUXFWXUH�WKDW�PD[LPL]HV�WKH�VFRUH

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /1032546��

&RQVWUDLQWV�YHUVXV�6FRUHV9

&RQVWUDLQW�EDVHG: ,QWXLWLYH��IROORZV�FORVHO\�WKH�GHILQLWLRQ�RI�%1V: 6HSDUDWHV�VWUXFWXUH�FRQVWUXFWLRQ�IURP�WKH�IRUP�RI�WKH

LQGHSHQGHQFH��WHVWV: 6HQVLWLYH�WR�HUURUV�LQ�LQGLYLGXDO�WHVWV

96FRUH�EDVHG

: 6WDWLVWLFDOO\�PRWLYDWHG: &DQ�PDNH�FRPSURPLVHV

9%RWK

: &RQVLVWHQW���ZLWK�VXIILFLHQW�DPRXQWV�RI�GDWD�DQG

FRPSXWDWLRQ��WKH\�OHDUQ�WKH�FRUUHFW�VWUXFWXUH

Page 49: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /1032546��

/LNHOLKRRG�6FRUH�IRU�6WUXFWXUHV

)LUVW�FXW�DSSURDFK�: 8VH�OLNHOLKRRG�IXQFWLRQ

95HFDOO��WKH�OLNHOLKRRG�VFRUH�IRU�D�QHWZRUN�VWUXFWXUH�DQG

SDUDPHWHUV�LV

96LQFH�ZH�NQRZ�KRZ�WR�PD[LPL]H�SDUDPHWHUV�IURP�QRZ�ZH

DVVXPH

∏ ∏

Θ=

Θ=Θ

� ���

���

����

*P3DP[3

*P[P[3'*/

���@>_@>�

���@>�@�>�����

�K

����PD[��� '*/'*/ � Θ= Θ

�� ��� ��� ��� � � � ��������� � � !"� #���� $ ��% � &�� ��� �('*) � + � +-,�) % � + . �(� � / � 021 3 3 � / � � � � / � )�� ��% � 4 % % � � 5�6 / +-� � + � � 7 � � 8 9;:=<?>�@�A

/LNHOLKRRG�6FRUH�IRU�6WUXFWXUH��FRQW��

5HDUUDQJLQJ�WHUPV�

ZKHUHB+�;��LV�WKH�HQWURS\�RI�;

B,�;�<��LV�WKH�PXWXDO�LQIRUPDWLRQ�EHWZHHQ�;�DQG�<

C ,�;�<��PHDVXUHV�KRZ�PXFK�³LQIRUPDWLRQ´�HDFK

YDULDEOHV�SURYLGHV�DERXW�WKH�RWKHUD ,�;�<��≥��D ,�;�<�� ���LII�;�DQG�<�DUH�LQGHSHQGHQWD ,�;�<�� �+�;��LII�;�LV�WRWDOO\�SUHGLFWDEOH�JLYHQ�<

( )∑ −=

=

LL

*LL ;+3D;,0

'*/'*O

�����

���ORJ���

Page 50: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

�� ��� ��� ��� � � � ��������� � � !"� #���� $ ��% � &�� ��� �('*) � + � +-,�) % � + . �(� � / � 021 3 3 � / � � � � / � )�� ��% � 4 % % � � 5�6 / +-� � + � � 7 � � 8 9;:=<?>�@�@

/LNHOLKRRG�6FRUH�IRU�6WUXFWXUH��FRQW��

*RRG�QHZV�B

,QWXLWLYH�H[SODQDWLRQ�RI�OLNHOLKRRG�VFRUH�D 7KH�ODUJHU�WKH�GHSHQGHQF\�RI�HDFK�YDULDEOH�RQ�LWV�SDUHQWV��WKH

KLJKHU�WKH�VFRUHD /LNHOLKRRG�DV�D�FRPSURPLVH�DPRQJ�GHSHQGHQFLHV��EDVHG�RQ

WKHLU�VWUHQJWK

%DG�QHZV�B

$GGLQJ�DUFV�DOZD\V�KHOSVD ,�;�<��≤�,�;�<�=�D 0D[LPDO�VFRUH�DWWDLQHG�E\�³FRPSOHWH´�QHWZRUNVD 6XFK�QHWZRUNV�FDQ�RYHUILW�WKH�GDWD�����WKH�SDUDPHWHUV�WKH\�OHDUQ

FDSWXUH�WKH�QRLVH�LQ�WKH�GDWD

( )∑ −=L

L*LL ;+3D;,0'*O ��������

�� ��� ��� ��� � � � ��������� � � !"� #���� $ ��% � &�� ��� �('*) � + � +-,�) % � + . �(� � / � 021 3 3 � / � � � � / � )�� ��% � 4 % % � � 5�6 / +-� � + � � 7 � � 8 9 : < >-< ���

$YRLGLQJ�2YHUILWWLQJ

³&ODVVLF´�LVVXH�LQ�OHDUQLQJ�

6WDQGDUG�DSSURDFKHV�B

5HVWULFWHG�K\SRWKHVHVC /LPLWV�WKH�RYHUILWWLQJ�FDSDELOLW\�RI�WKH�OHDUQHUC ([DPSOH��UHVWULFW���RI�SDUHQWV�RU���RI�SDUDPHWHUV

B0LQLPXP�GHVFULSWLRQ�OHQJWK

C 'HVFULSWLRQ�OHQJWK�PHDVXUHV�FRPSOH[LW\C &KRRVH�PRGHO�WKDW�FRPSDFWO\�GHVFULEHV�WKH�WUDLQLQJ�GDWD

B%D\HVLDQ�PHWKRGV

C $YHUDJH�RYHU�DOO�SRVVLEOH�SDUDPHWHU�YDOXHVC 8VH�SULRU�NQRZOHGJH

Page 51: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

�� ��� ��� ��� � � � ��������� � � !"� #���� $ ��% � &�� ��� �('*) � + � +-,�) % � + . �(� � / � 021 3 3 � / � � � � / � )�� ��% � 4 % % � � 5�6 / +-� � + � � 7 � � 8 9 : < >-< � <

$YRLGLQJ�2YHUILWWLQJ��FRQW���

2WKHU�DSSURDFKHV�LQFOXGH�

B+ROGRXW�&URVV�YDOLGDWLRQ�/HDYH�RQH�RXW

C 9DOLGDWH�JHQHUDOL]DWLRQ�RQ�GDWD�ZLWKKHOG�GXULQJ�WUDLQLQJ

B6WUXFWXUDO�5LVN�0LQLPL]DWLRQ

C 3HQDOL]H�K\SRWKHVHV�VXEFODVVHV�EDVHG�RQ�WKHLU�9&

GLPHQVLRQ

�� ��� ��� ��� � � � ��������� � � !"� #���� $ ��% � &�� ��� �('*) � + � +-,�) % � + . �(� � / � 021 3 3 � / � � � � / � )�� ��% � 4 % % � � 5�6 / +-� � + � � 7 � � 8 9 : < >-< ���

0LQLPXP�'HVFULSWLRQ�/HQJWK

5DWLRQDOH�B

SUHIHU�QHWZRUNV�WKDW�IDFLOLWDWH�FRPSUHVVLRQ�RI�WKH�GDWDB

&RPSUHVVLRQ�⇒�VXPPDUL]DWLRQ�⇒�JHQHUDOL]DWLRQ

'DWD

%DVH

6

/

&

(

';

(QFRGLQJ

6FKHPH&RPSUHVVHG

'DWD

'HFRGLQJ

6FKHPH

'DWD

%DVH

Page 52: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

�� ��� ��� ��� � � � ��������� � � !"� #���� $ ��% � &�� ��� �('*) � + � +-,�) % � + . �(� � / � 021 3 3 � / � � � � / � )�� ��% � 4 % % � � 5�6 / +-� � + � � 7 � � 8 9 : < >-< ���

0LQLPXP�'HVFULSWLRQ�/HQJWK��FRQW��

B&RPSXWLQJ�WKH�GHVFULSWLRQ�OHQJWK�RI�WKH�GDWD��ZH�JHW

B0LQLPL]LQJ�WKLV�WHUP�LV�HTXLYDOHQW�WR�PD[LPL]LQJ

����GLP��

ORJ����� '*O*

0*'/*''/ −+=

���GLP��

ORJ������ *'/*

0'*O'*0'/ −−=

����� ���� ������������������ ���� ��������������� ����� ������������������!

" ��� �$# %&�(')���+*

�� ��� ��� ��� � � � ��������� � � !"� #���� $ ��% � &�� ��� �('*) � + � +-,�) % � + . �(� � / � 021 3 3 � / � � � � / � )�� ��% � 4 % % � � 5�6 / +-� � + � � 7 � � 8 9 : < >-< ��,

0LQLPXP�'HVFULSWLRQ��&RPSOH[LW\

3HQDOL]DWLRQ

B/LNHOLKRRG�LV��URXJKO\��OLQHDU�LQ�0

B3HQDOW\�LV�ORJDULWKPLF�LQ�0

$V�ZH�JHW�PRUH�GDWD��WKH�SHQDOW\�IRU�FRPSOH[�VWUXFWXUH�LV

OHVV�KDUVK

���GLP��

ORJ������ *'/*

0'*O'*0'/ −−=

�@�_�(>ORJ

��_@>�ORJ���

Θ⋅≈

Θ= ∑

*[30

*P[3'*O -

Page 53: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

�� ��� ��� ��� � � � ��������� � � !"� #���� $ ��% � &�� ��� �('*) � + � +-,�) % � + . �(� � / � 021 3 3 � / � � � � / � )�� ��% � 4 % % � � 5�6 / +-� � + � � 7 � � 8 9 : < >-< ���

0LQLPXP�'HVFULSWLRQ��([DPSOH

B,GHDOL]HG�EHKDYLRU�

� %&��� �*������� �������'���� � %&��*��������� %&��� �*������� ���������'���� � % ��*��� ����� %&��� �*����!�� �#"$����'���� � % ��*��%��&"

> ��,

> � �

> � �

>-< A

>-<('

� � ��� < ����� < � ��� � ����� �)� ��� � ��� �

* +,- ./ 0

132 4 4 5�6 798 :<; : 8 = >@?A@B@6 C�D E�D F<=@: G =@H = I@6 A@B >KJML 8 N = NPOL H > N Q ?K8 > R 6 SUT�V V B R = : B A R 8 L@B A@H D W�H H : 8 X@Y R NP: = N = : Z = > [ \^]`_(aP_!bdc

0LQLPXP�'HVFULSWLRQ��([DPSOH��FRQW��

5HDO�GDWD�LOOXVWUDWLRQ�ZLWK�WKUHH�QHWZRUN�e

³7UXH´�DODUP������SDUDP���VLPSOLILHG������SDUDP���WUHH������SDUDP�

aUfdg

aUf�f

aUf)b

aM_(h

aM_�c

b i�b)b _(b�b)b _(i)b�b f)b�b)b f�i�b�b j�b�b)b

* +,- ./ 0

k

l3m(n�oqpro)s(tvu)mxwy{z |~}�� z �dz o)�qp~o)s(t�u)m(wl3m�odo���o)s(t�u)m(w

Page 54: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

132 4 4 5�6 798 :<; : 8 = >@?A@B@6 C�D E�D F<=@: G =@H = I@6 A@B >KJML 8 N = NPOL H > N Q ?K8 > R 6 SUT�V V B R = : B A R 8 L@B A@H D W�H H : 8 X@Y R NP: = N = : Z = > [ \^]`_(aP_!b��

&RQVLVWHQF\�RI�WKH�0'/�6FRUH

0'/�6FRUH�LV�FRQVLVWHQW�

$V�0�→∞�WKH�³WUXH´�VWUXFWXUH�* �PD[LPL]HV�WKH�VFRUH

�DOPRVW�VXUHO\��

)RU�VXIILFLHQWO\�ODUJH�0��WKH�PD[LPDO�VFRULQJ�VWUXFWXUHV�DUH

HTXLYDOHQW�WR�*

3URRI��RXWOLQH��e

6XSSRVH�*�LPSOLHV�DQ�LQGHSHQGHQFH�VWDWHPHQW�QRW�LQ�* ��WKHQ

DV�0�→∞��O�*�'��→�O�* �'����H0����H�GHSHQGV�RQ�*�

VR�0'/�* �'����0'/�*�'��→�H0����GLP�* ��GLP�*�����ORJ�0e

1RZ�VXSSRVH�* �LPSOLHV�DQ�LQGHSHQGHQFH�VWDWHPHQW�QRW�LQ�*��WKHQ

DV�0�→∞���O�*�'��→�O�* �'�

VR�0'/�*�'����0'/�* �'��→��GLP�*��GLP�* �����ORJ�0

132 4 4 5�6 798 :<; : 8 = >@?A@B@6 C�D E�D F<=@: G =@H = I@6 A@B >KJML 8 N = NPOL H > N Q ?K8 > R 6 SUT�V V B R = : B A R 8 L@B A@H D W�H H : 8 X@Y R NP: = N = : Z = > [ \^]`_(aP_!b�h

%D\HVLDQ�,QIHUHQFH

�%D\HVLDQ�5HDVRQLQJ���FRPSXWH�H[SHFWDWLRQ�RYHU�XQNQRZQ�*

ZKHUH

$VVXPSWLRQ��*V�DUH�PXWXDOO\�H[FOXVLYH�DQG�H[KDXVWLYH

∑ +=+ � '*3*'0[3'0[3 �_���_@�>��_@�>�

∫ θθθ=

���_���_�

���_��_�

*3G*3*'3

*3*'3'*3

0DUJLQDO�OLNHOLKRRG3ULRU�RYHU�VWUXFWXUHV

/LNHOLKRRG 3ULRU�RYHU�SDUDPHWHUV�

3RVWHULRU�VFRUH

Page 55: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

132 4 4 5�6 798 :<; : 8 = >@?A@B@6 C�D E�D F<=@: G =@H = I@6 A@B >KJML 8 N = NPOL H > N Q ?K8 > R 6 SUT�V V B R = : B A R 8 L@B A@H D W�H H : 8 X@Y R NP: = N = : Z = > [ \^]`_(aP_!b��

0DUJLQDO�/LNHOLKRRG��%LQRPLDO�FDVH

�$VVXPH�ZH�REVHUYH�D�VHTXHQFH�RI�FRLQ�WRVVHV«�

�%\�WKH�FKDLQ�UXOH�ZH�KDYH�

UHFDOO�WKDW

ZKHUH�1���

�LV�WKH�QXPEHU�RI�KHDGV�LQ�ILUVW�P�H[DPSOHV�

@��>�@��>_@>�@��>_@�>�@��>�

@�>�@��>�

=

0[[0[3[[3[3

0[[3

KL

K

��

���

P

1P[[+P[3

α+α+α+

==+ @�>�@��>_@�>� K

132 4 4 5�6 798 :<; : 8 = >@?A@B@6 C�D E�D F<=@: G =@H = I@6 A@B >KJML 8 N = NPOL H > N Q ?K8 > R 6 SUT�V V B R = : B A R 8 L@B A@H D W�H H : 8 X@Y R NP: = N = : Z = > [ \^]`_(aP_)_(b

0DUJLQDO�/LNHOLKRRG��%LQRPLDOV��FRQW��

:H�VLPSOLI\�WKLV�E\�XVLQJ

7KXV

=@�>�@��>� 0[[3 K

����

��

11

1

1

1

1

α+α+−+α+−

α+α+α

⋅α+α+−

α+−α+α

α

L

L

��

����������

αΓα+Γ=α+−α+α 1

1L

=@�>�@��>� 0[[3 K

��

��

��

��

��

��

��

��

� 11

11 αΓ+αΓ

αΓ+αΓ

++α+αΓα+αΓ

Page 56: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#26272

%LQRPLDO�/LNHOLKRRG��([DPSOH

�,GHDOL]HG�H[SHULPHQW�ZLWK�3�+�� �����

895: ;

895: <

895: 9

89

8�=>: ?

8�=>: @

8�=>: A

8�=>: B

= C 9D= 9�C <E= <5C ;5= ;FC GH= GEC CF=

0

0'/

'LULFKOHW�������

'LULFKOHW�����

'LULFKOHW�����

�/RJ

�3�'

���0

I�J K K L�M NO P�Q P O R S�T�U�V�M W�X Y�X Z�R�P [ R�\ R ]�M U�V S�^ _ O ` R `#a�_ \ S ` b T�O S c M d(e�f f V c R P V U c O _�V U�\ X g�\ \ P O h�i c `#P R ` R P j R S k l1m3n5o#n6nFp

0DUJLQDO�/LNHOLKRRG��([DPSOH��FRQW��

q$FWXDO�H[SHULPHQW�ZLWK�3�+�� �����

8(95: ;

8(95: <

8(95: 9

89

8 =>: ?

8 =>: @

8 =>: A

8 =>: B

= C 9�= 9DC <5= <5C ;5= ;FC GH= GEC C5=

�/RJ

�3�'

���0

0

0'/

'LULFKOHW�������

'LULFKOHW�����

'LULFKOHW�����

Page 57: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

I�J K K L�M NO P�Q P O R S�T�U�V�M W�X Y�X Z�R�P [ R�\ R ]�M U�V S�^ _ O ` R `#a�_ \ S ` b T�O S c M d(e�f f V c R P V U c O _�V U�\ X g�\ \ P O h�i c `#P R ` R P j R S k l1m3n5o#n6n��

0DUJLQDO�/LNHOLKRRG��0XOWLQRPLDOV

7KH�VDPH�DUJXPHQW�JHQHUDOL]HV�WR�PXOWLQRPLDOV�ZLWK�'LULFKOHW

SULRU

q 3�Θ��LV�'LULFKOHW�ZLWK�K\SHUSDUDPHWHUV�α � �«�α �q '�LV�D�GDWDVHW�ZLWK�VXIILFLHQW�VWDWLVWLFV�1 � �«�1 ��7KHQ

∏∑

∑αΓ+αΓ

+αΓ

αΓ

= � ���

���

��

��

��

��

��1

1

'3

I�J K K L�M NO P�Q P O R S�T�U�V�M W�X Y�X Z�R�P [ R�\ R ]�M U�V S�^ _ O ` R `#a�_ \ S ` b T�O S c M d(e�f f V c R P V U c O _�V U�\ X g�\ \ P O h�i c `#P R ` R P j R S k l1m3n5o#n6n��

0DUJLQDO�/LNHOLKRRG��%D\HVLDQ�1HWZRUNV

+ 7 7 + 7 + +

+ 7 + + 7 7 +

;

<

q1HWZRUN�VWUXFWXUH

GHWHUPLQHV�IRUP�RI

PDUJLQDO�OLNHOLKRRG

� � � � � � �

1HWZRUN���q

7ZR�'LULFKOHW�PDUJLQDO�OLNHOLKRRGVq 3�;>�@�«�;>�@�q 3�<>�@�«�<>�@�

; <

1HWZRUN���q

7KUHH�'LULFKOHW�PDUJLQDO�OLNHOLKRRGVq 3�;>�@�«�;>�@�q 3�<>�@�<>�@�<>�@�<>�@�q 3�<>�@�<>�@�<>�@�

; <

Page 58: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

I�J K K L�M NO P�Q P O R S�T�U�V�M W�X Y�X Z�R�P [ R�\ R ]�M U�V S�^ _ O ` R `#a�_ \ S ` b T�O S c M d(e�f f V c R P V U c O _�V U�\ X g�\ \ P O h�i c `#P R ` R P j R S k l1m3n5o#n6n��

0DUJLQDO�/LNHOLKRRG��FRQW��

,Q�JHQHUDO�QHWZRUNV��WKH�PDUJLQDO�OLNHOLKRRG�KDV�WKH�IRUP�

qZKHUH

q1�����DUH�WKH�FRXQWV�IURP�WKH�GDWD

q ����DUH�WKH�K\SHUSDUDPHWHUV�IRU�HDFK�IDPLO\�JLYHQ�*

( )( )∏ ∏ ∏ αΓ

+αΓ

+αΓ

αΓ= � ��� �

��

������

SD[

SD[1SD[

SD1SD

SD*'3

�����

��������

����

���_�

'LULFKOHW�0DUJLQDO�/LNHOLKRRG

)RU�WKH�VHTXHQFH�RI�YDOXHV�RI�; �ZKHQ

; �V�SDUHQWV�KDYH�D�SDUWLFXODU�YDOXH

�� � � ��� ��� ��� � � � ��������� "! #$! %���� & ��' � (�� ��� �*),+ � - � -/.�+ ' � - 0 �*� � 1 � 243"5 5 � 1 � � � � 1 � +�� ��' ! 6"' ' � � 7�8 1 -/� � - � � 9 � � : ;=<?>�@/>A>CB

3ULRUV�DQG�%'H�VFRUH

D:H�QHHG��SULRU�FRXQWV�����IRU�HDFK�QHWZRUN�VWUXFWXUH�*

D7KLV�FDQ�EH�D�IRUPLGDEOH�WDVN

E 7KHUH�DUH�H[SRQHQWLDOO\�PDQ\�VWUXFWXUHV«

3RVVLEOH�VROXWLRQ��7KH�%'H�SULRUF 8VH�SULRU�RI�WKH�IRUP�0 G ��% G �* G ��Θ G �

� &RUUHVSRQGV�WR�0 H �SULRU�H[DPSOHV�GLVWULEXWHG�DFFRUGLQJ�WR�% HF 6HW�α�[ I �SD I

J�� �0 G 3�[ I �SD I

J_�* G ��Θ G �

� 1RWH�WKDW�SD KL�DUH��LQ�JHQHUDO��QRW�WKH�VDPH�DV�WKH�SDUHQWV�RI�; K �LQ

* HNM :H�FDQ�FRPSXWH�WKLV�XVLQJ�VWDQGDUG�%1�WRROV

F 7KLV�FKRLFH�DOVR�KDV�GHVLUDEOH�WKHRUHWLFDO�SURSHUWLHV

� (TXLYDOHQW�QHWZRUNV�DUH�DVVLJQHG�WKH�VDPH�VFRUH

Page 59: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

�� � � ��� ��� ��� � � � ��������� "! #$! %���� & ��' � (�� ��� �*),+ � - � -/.�+ ' � - 0 �*� � 1 � 243"5 5 � 1 � � � � 1 � +�� ��' ! 6"' ' � � 7�8 1 -/� � - � � 9 � � : ;=<?>�@/>A>��

%D\HVLDQ�6FRUH��$V\PSWRWLF�%HKDYLRU�

7KH�%D\HVLDQ�VFRUH�VHHPV�TXLWH�GLIIHUHQW�IURP�WKH�0'/�VFRUH�

+RZHYHU��WKH�WZR�VFRUHV�DUH�DV\PSWRWLFDOO\�HTXLYDOHQW

7KHRUHP��,I�WKH�SULRU�3�Θ�_*��LV�³ZHOO�EHKDYHG´��WKHQ

3URRI�D

�6LPSOH��8VH�6WLUOLQJ¶V�DSSUR[LPDWLRQ�WR�Γ���F $SSOLHV�WR�%D\HVLDQ�QHWZRUNV�ZLWK�'LULFKOHW�SULRUV

D�*HQHUDO��8VH�SURSHUWLHV�RI�H[SRQHQWLDO�PRGHOV�DQG

/DSODFH¶V�PHWKRG�IRU�DSSUR[LPDWLQJ�LQWHJUDOVE $SSOLHV�WR�%D\HVLDQ�QHWZRUNV�ZLWK�RWKHU�SDUDPHWULF�IDPLOLHV

����GLP��

ORJ����_�ORJ 2*

0'*O*'3 +−=

�� � � ��� ��� ��� � � � ��������� "! #$! %���� & ��' � (�� ��� �*),+ � - � -/.�+ ' � - 0 �*� � 1 � 243"5 5 � 1 � � � � 1 � +�� ��' ! 6"' ' � � 7�8 1 -/� � - � � 9 � � : ;=<?>�@/>A>��

%D\HVLDQ�6FRUH��$V\PSWRWLF�%HKDYLRU

&RQVHTXHQFHV�D

%D\HVLDQ�VFRUH�LV�DV\PSWRWLFDOO\�HTXLYDOHQW�WR�0'/�VFRUHF 7KH�WHUPV�ORJ�3�*��DQG�GHVFULSWLRQ�OHQJWK�RI�*�DUH

FRQVWDQW�DQG�WKXV�WKH\�DUH�QHJOLJLEOH�ZKHQ�0�LV�ODUJH�

D%D\HVLDQ�VFRUH�LV�FRQVLVWHQW

F )ROORZV�LPPHGLDWHO\�IURP�FRQVLVWHQF\�RI�0'/�VFRUH

D2EVHUYHG�GDWD�HYHQWXDOO\�RYHUULGHV�SULRU�LQIRUPDWLRQ

F $VVXPLQJ�WKDW�WKH�SULRU�GRHV�QRW�DVVLJQ�SUREDELOLW\���WR

VRPH�SDUDPHWHU�VHWWLQJV

Page 60: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

�� � � ��� ��� ��� � � � ��������� "! #$! %���� & ��' � (�� ��� �*),+ � - � -/.�+ ' � - 0 �*� � 1 � 243"5 5 � 1 � � � � 1 � +�� ��' ! 6"' ' � � 7�8 1 -/� � - � � 9 � � : ;=<?>�@/>A>��

6FRUHV����6XPPDU\

D/LNHOLKRRG��0'/�DQG��ORJ��%'H�KDYH�WKH�IRUP

D%'H�UHTXLUHV�DVVHVVLQJ�SULRU�QHWZRUN��,W�FDQ�QDWXUDOO\

LQFRUSRUDWH�SULRU�NQRZOHGJH�DQG�SUHYLRXV�H[SHULHQFHD

%RWK�0'/�DQG�%'H�DUH�FRQVLVWHQW�DQG�DV\PSWRWLFDOO\

HTXLYDOHQW��XS�WR�D�FRQVWDQW�D

$OO�WKUHH�DUH�VFRUH�HTXLYDOHQW���WKH\�DVVLJQ�WKH�VDPH�VFRUH

WR�HTXLYDOHQW�QHWZRUNV

����_���� ���

�� 3D;13D;6FRUH'*6FRUH �∑=

�� � � ��� ��� ��� � � � ��������� "! #$! %���� & ��' � (�� ��� �*),+ � - � -/.�+ ' � - 0 �*� � 1 � 243"5 5 � 1 � � � � 1 � +�� ��' ! 6"' ' � � 7�8 1 -/� � - � � 9 � � : ;=<?>�@/>����

2XWOLQH

D,QWURGXFWLRQ

D%D\HVLDQ�QHWZRUNV��D�UHYLHZ

D3DUDPHWHU�OHDUQLQJ��&RPSOHWH�GDWD

D3DUDPHWHU�OHDUQLQJ��,QFRPSOHWH�GDWD

ª6WUXFWXUH�OHDUQLQJ��&RPSOHWH�GDWDE 6FRULQJ�PHWULFV

ª 0D[LPL]LQJ�WKH�VFRUHE /HDUQLQJ�ORFDO�VWUXFWXUH

D$SSOLFDWLRQ��FODVVLILFDWLRQ

D/HDUQLQJ�FDXVDO�UHODWLRQVKLSV

D6WUXFWXUH�OHDUQLQJ��,QFRPSOHWH�GDWD

D&RQFOXVLRQ

���� ����� ������� ��� ���������� ���� � ����� ��� �

�"!�#�$ ��� ��%�&�� &

' ����"!�#($ ��� �)%�&�� &

Page 61: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#25672

2SWLPL]DWLRQ�3UREOHP

,QSXW�8 7UDLQLQJ�GDWD8 6FRULQJ�IXQFWLRQ��LQFOXGLQJ�SULRUV��LI�QHHGHG�8 6HW�RI�SRVVLEOH�VWUXFWXUHV

� ,QFOXGLQJ�SULRU�NQRZOHGJH�DERXW�VWUXFWXUH

2XWSXW�8 $�QHWZRUN��RU�QHWZRUNV��WKDW�PD[LPL]H�WKH�VFRUH

.H\�3URSHUW\�8 'HFRPSRVDELOLW\��WKH�VFRUH�RI�D�QHWZRUN�LV�D�VXP�RI

WHUPV�

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#25696

/HDUQLQJ�7UHHV

:7UHHV�

8 $W�PRVW�RQH�SDUHQW�SHU�YDULDEOH

::K\�WUHHV"

8 (OHJDQW�PDWK

⇒ZH�FDQ�VROYH�WKH�RSWLPL]DWLRQ�SUREOHP8 6SDUVH�SDUDPHWHUL]DWLRQ

⇒DYRLG�RYHUILWWLQJ

Page 62: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#256��

/HDUQLQJ�7UHHV��FRQW��

:/HW�S�L��GHQRWH�WKH�SDUHQW�RI�; � ��RU���LI�; � �KDV�QR�SDUHQWV

::H�FDQ�ZULWH�WKH�VFRUH�DV

:6FRUH� �VXP�RI�HGJH�VFRUHV���FRQVWDQW

( ) ∑∑

∑∑

+−=

+=

=

>

=>

��

�������

����

������

���

;6FRUH;6FRUH;;6FRUH

;6FRUH;;6FRUH

3D;6FRUH'*6FRUH

�������

�����

������

������

����������

6FRUH�RI�´HPSW\µQHWZRUN

,PSURYHPHQW�RYHU�´HPSW\µ�QHWZRUN

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#256

/HDUQLQJ�7UHHV��FRQW�

$OJRULWKP�:&RQVWUXFW�JUDSK�ZLWK�YHUWLFHV��������«

:6HW�Z�L→M��EH�6FRUH��;� �_�; � �����6FRUH�;� �

:)LQG�WUHH��RU�IRUHVW��ZLWK�PD[LPDO�ZHLJKW

8 7KLV�FDQ�EH�GRQH�XVLQJ�VWDQGDUG�DOJRULWKPV�LQ�ORZ�RUGHU

SRO\QRPLDO�WLPH�E\�EXLOGLQJ�D�WUHH�LQ�D�JUHHG\�IDVKLRQ

�.UXVNDO¶V�PD[LPXP�VSDQQLQJ�WUHH�DOJRULWKP�

7KHRUHP��7KLV�SURFHGXUH�ILQGV�WKH�WUHH�ZLWK�PD[LPDO�VFRUH

:KHQ�VFRUH�LV�OLNHOLKRRG��WKHQ�Z�L→M���LV�SURSRUWLRQDO�WR,�; � ��;� ��WKLV�LV�NQRZQ�DV�WKH�&KRZ��/LX�PHWKRG

Page 63: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#256��

/HDUQLQJ�7UHHV��([DPSOH

7UHH�OHDUQHG�IURP

DODUP�GDWD

:*UHHQ����FRUUHFW

DUFV:5HG����VSXULRXV

DUFV

� ��� � ���� '� �

� '����" � '&�*�

�(''�(*�����('� '� ) &���#'�

��*����� � ���

&�*���� � � � �����

*�'�������

� �(�� *�� �

� �(���� ���" � ��) ������

� ) &��� �������

� ) � � ��� &���

� �(����#*� ��) ���� � (�����) ��(��� *�) � �� ��� ����������� �(&

� * � & � ����

*���* ��� �� *���) &

��) ��� � �

��� &�*���) ���

� '��(&�&

) �&����� *�����&� � � '

� � � *�) � ��'��

�('�'��� �����#�� � ��&�('���� � ��� ����� � � ��� ��� ����

� � � � ��� ����) *

� ���

� �:1RW�HYHU\�HGJH�LQ�WUHH�LV�LQ�WKH�WKH�RULJLQDO�QHWZRUN

:7UHH�GLUHFWLRQ�LV�DUELWUDU\�����ZH�FDQ¶W�OHDUQ�DERXW�DUF

GLUHFWLRQ

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#256��

%H\RQG�7UHHV

:KHQ�ZH�FRQVLGHU�PRUH�FRPSOH[�QHWZRUN��WKH�SUREOHP�LV�QRW

DV�HDV\

:6XSSRVH�ZH�DOORZ�WZR�SDUHQWV

:$�JUHHG\�DOJRULWKP�LV�QR�ORQJHU�JXDUDQWHHG�WR�ILQG�WKH

RSWLPDO�QHWZRUN

:,Q�IDFW��QR�HIILFLHQW�DOJRULWKP�H[LVWV

7KHRUHP��)LQGLQJ�PD[LPDO�VFRULQJ�QHWZRUN�VWUXFWXUH�ZLWK�DWPRVW�N�SDUHQWV�IRU�HDFK�YDULDEOHV�LV�13�KDUG�IRU�N�!��

Page 64: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#256��

+HXULVWLF�6HDUFK

::H�DGGUHVV�WKH�SUREOHP�E\�XVLQJ�KHXULVWLF�VHDUFK

:'HILQH�D�VHDUFK�VSDFH�

8 QRGHV�DUH�SRVVLEOH�VWUXFWXUHV8 HGJHV�GHQRWH�DGMDFHQF\�RI�VWUXFWXUHV

:7UDYHUVH�WKLV�VSDFH�ORRNLQJ�IRU�KLJK�VFRULQJ�VWUXFWXUHV

6HDUFK�WHFKQLTXHV�8 *UHHG\�KLOO�FOLPELQJ8 %HVW�ILUVW�VHDUFK8 6LPXODWHG�$QQHDOLQJ8 ���

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#256��

+HXULVWLF�6HDUFK��FRQW��

:7\SLFDO�RSHUDWLRQV�

6 &

(

'

6 &

(

'

6 &

(

'

6 &

(

'

$GG�&�→'

5HYHUVH�&�→(5HPRYH�&�→(

Page 65: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#256��

([SORLWLQJ�'HFRPSRVDELOLW\�LQ�/RFDO

6HDUFK

:&DFKLQJ��7R�XSGDWH�WKH�VFRUH�RI�DIWHU�D�ORFDO�FKDQJH��ZH

RQO\�QHHG�WR�UH�VFRUH�WKH�IDPLOLHV�WKDW�ZHUH�FKDQJHG�LQ

WKH�ODVW�PRYH

6 &

(

'

6 &

(

'

6 &

(

'

6 &

(

'

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#2 ���

*UHHG\�+LOO�&OLPELQJ

6LPSOHVW�KHXULVWLF�ORFDO�VHDUFK8 6WDUW�ZLWK�D�JLYHQ�QHWZRUN

� HPSW\�QHWZRUN

� EHVW�WUHH

� D�UDQGRP�QHWZRUN

8 $W�HDFK�LWHUDWLRQ

� (YDOXDWH�DOO�SRVVLEOH�FKDQJHV

� $SSO\�FKDQJH�WKDW�OHDGV�WR�EHVW�LPSURYHPHQW�LQ�VFRUH

� 5HLWHUDWH

8 6WRS�ZKHQ�QR�PRGLILFDWLRQ�LPSURYHV�VFRUH

:(DFK�VWHS�UHTXLUHV�HYDOXDWLQJ�DSSUR[LPDWHO\�Q�QHZ�FKDQJHV

Page 66: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#2 � 2

*UHHG\�+LOO�&OLPELQJ��FRQW��

:*UHHG\�+LOO�&OLPELQJ�FDQ�JHW�VWUXFN�LQ�

8 /RFDO�0D[LPD�

� $OO�RQH�HGJH�FKDQJHV�UHGXFH�WKH�VFRUH8 3ODWHDXV�

� 6RPH�RQH�HGJH�FKDQJHV�OHDYH�WKH�VFRUH�XQFKDQJHG

:%RWK�DUH�RFFXU�LQ�WKH�VHDUFK�VSDFH

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#2 � 6

*UHHG\�+LOO�&OLPELQJ��FRQW��

7R�DYRLG�WKHVH�SUREOHPV��ZH�FDQ�XVH�:

7$%8�VHDUFK� .HHS�OLVW�RI�.��PRVW�UHFHQWO\�YLVLWHG�VWUXFWXUHV� $SSO\�EHVW�PRYH�WKDW�GRHV�QRW�OHDG�WR�D�VWUXFWXUH�LQ�WKH�OLVW� 7KLV�HVFDSHV�SODWHDXV�DQG�ORFDO�PD[LPD�DQG�ZLWK�³EDVLQ´�VPDOOHUWKDQ��.�VWUXFWXUHV

:5DQGRP�5HVWDUWV

� 2QFH�VWXFN��DSSO\�VRPH�IL[HG�QXPEHU�RI�UDQGRP�HGJH�FKDQJHV�DQG

UHVWDUW�VHDUFK� 7KLV�FDQ�HVFDSH�IURP�WKH�EDVLQ�RI�RQH�PD[LPD�WR�DQRWKHU

Page 67: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#2 � �

*UHHG\�+LOO�&OLPELQJ

:*UHHG\�+LOO�&OLPELQJ�ZLWK�7$%8�OLVW�DQG�UDQGRP�UHVWDUWV�RQ

DODUP6FRUH�0

VWHS��

4#2 �

4#2 ��

4#2 ��

4#2 ��

4#2 ��

6

4#2 �

4#2���

� 2 ��� 6���� ����� ��� ����� � ��� �����

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#2 �

2WKHU�/RFDO�6HDUFK��+HXULVWLFV

:6WRFKDVWLF�)LUVW�$VFHQW�+LOO�&OLPELQJ

� (YDOXDWH�SRVVLEOH�FKDQJHV�DW�UDQGRP� $SSO\�WKH�ILUVW�RQH�WKDW�OHDGV�³XSKLOO´� 6WRS�ZKHQ�D�IL[�DPRXQW�RI�³XQVXFFHVVIXO´�DWWHPSWV�WR�FKDQJH�WKH

FXUUHQW�FDQGLGDWH

:6LPXODWHG�$QQHDOLQJ

� 6LPLODU�LGHD��EXW�DOVR�DSSO\�³GRZQKLOO´�FKDQJHV�ZLWK�D�SUREDELOLW\�WKDW

LV�SURSRUWLRQDO�WR�WKH�FKDQJH�LQ�VFRUH� 8VH�D�WHPSHUDWXUH�WR�FRQWURO�DPRXQW�RI�UDQGRP�GRZQKLOO�VWHSV

� 6ORZO\�³FRRO´�WHPSHUDWXUH�WR�UHDFK�D�UHJLPH�ZKHUH�SHUIRUPLQJ�VWULFWXSKLOO�PRYHV

Page 68: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#2 ���

,�(TXLYDOHQFH�&ODVV�6HDUFK

6R�IDU��ZH�VHHQ�JHQHULF�VHDUFK�PHWKRGV«:&DQ�H[SORLW�WKH�VWUXFWXUH�RI�RXU�GRPDLQ"

,GHD�:6HDUFK�WKH�VSDFH�RI�,�HTXLYDOHQFH�FODVVHV

:(DFK�,�HTXLYDOHQFH�FODVV�LV�UHSUHVHQWHG�E\�D�3'$*��SDUWLDOO\

RUGHUHG�JUDSK�����VNHOHWRQ���Y�VWUXFWXUHV

%HQHILWV�:7KH�VSDFH�RI�3'$*V�KDV�IHZHU�ORFDO�PD[LPD�DQG�SODWHDXV

:7KHUH�DUH�IHZHU�3'$*V�WKDQ�'$*V

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#2 � �

,�(TXLYDOHQFH�&ODVV�6HDUFK��FRQW��

(YDOXDWLQJ�FKDQJHV�LV�PRUH�H[SHQVLYH

:7KHVH�DOJRULWKPV�DUH�PRUH�FRPSOH[�WR�LPSOHPHQW

; =<

; =<

; =<

$GG����������

2ULJLQDO�3'$*

1HZ�3'$*

&RQVLVWHQW�'$*

6FRUH

Page 69: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#2 � �

6HDUFK�DQG�6WDWLVWLFV

:(YDOXDWLQJ�WKH�VFRUH�RI�D�VWUXFWXUH�UHTXLUHV�WKH

FRUUHVSRQGLQJ�FRXQWV��VXIILFLHQW�VWDWLVWLFV�:6LJQLILFDQW�FRPSXWDWLRQ�LV�VSHQW�LQ�FROOHFWLQJ�WKHVH�FRXQWV

� 5HTXLUHV�D�SDVV�RYHU�WKH�WUDLQLQJ�GDWD:5HGXFH�RYHUKHDG�E\�FDFKLQJ�SUHYLRXVO\�FRPSXWHG�FRXQWV

� $YRLG�GXSOLFDWHG�HIIRUWV� 0DUJLQDOL]H�FRXQWV��1�;�<��→1�;�

7UDLQLQJ

'DWD

6WDWLVWLFV

&DFKH

6HDUFK

6FRUH

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#2 � �

/HDUQLQJ�LQ�3UDFWLFH��7LPH��6WDWLVWLFV

:8VLQJ�JUHHG\�+LOO�&OLPELQJ�RQ�������LQVWDQFHV�IURP�DODUP

�����

�������

�������

�������

�������

�������

�������

�� �� ��� ����� ����� ����� ���� ����� �����

&D

FK

H�6

L]H

6HFRQGV

���

���

��

���

���

���

���

��

���

�� �� ��� ����� ����� ����� ���� ����� �����

6F

RUH

6HFRQGV

Page 70: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#2 ���

/HDUQLQJ�LQ�3UDFWLFH��$ODUP�GRPDLQ

��� �

��� �

� ����� ������� ������� ������ ����� ��� ��� ����� ������� �������

./�'LYHUJHQFH

0

7UXH�6WUXFWXUH�%'H�0� ���8QNQRZQ�6WUXFWXUH�%'H�0� ���

7UXH�6WUXFWXUH�0'/8QNQRZQ�6WUXFWXUH�0'/

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#2 �

0RGHO�$YHUDJLQJ

:5HFDOO��%D\HVLDQ�DQDO\VLV�VWDUWHG�ZLWK

8 7KLV�UHTXLUHV�XV�WR�DYHUDJH�RYHU�DOO�SRVVLEOH�PRGHOV

∑ +=+

'*3*'0[3'0[3 �_���_@�>��_@�>�

Page 71: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#27682

0RGHO�$YHUDJLQJ��FRQW��

96R�IDU��ZH�IRFXVHG�RQ�VLQJOH�PRGHO

: )LQG�EHVW�VFRULQJ�PRGHO: 8VH�LW�WR�SUHGLFW�QH[W�H[DPSOH

9,PSOLFLW�DVVXPSWLRQ�

: %HVW�VFRULQJ�PRGHO�GRPLQDWHV�WKH�ZHLJKWHG�VXP

93URV�

; :H�JHW�D�VLQJOH�VWUXFWXUH; $OORZV�IRU�HIILFLHQW�XVH�LQ�RXU�WDVNV

9&RQV�

; :H�DUH�FRPPLWWLQJ�WR�WKH�LQGHSHQGHQFLHV�RI�D�SDUWLFXODU�VWUXFWXUH; 2WKHU�VWUXFWXUHV�PLJKW�EH�DV�SUREDEOH�JLYHQ�WKH�GDWD

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#276=<

0RGHO�$YHUDJLQJ��FRQW��

&DQ�ZH�GR�EHWWHU"9

)XOO�$YHUDJLQJ: 6XP�RYHU�DOO�VWUXFWXUHV: 8VXDOO\�LQWUDFWDEOH���WKHUH�DUH�H[SRQHQWLDOO\�PDQ\�VWUXFWXUHV

9$SSUR[LPDWH�$YHUDJLQJ

; )LQG�.�ODUJHVW�VFRULQJ�VWUXFWXUHV; $SSUR[LPDWH�WKH�VXP�E\�DYHUDJLQJ�RYHU�WKHLU�SUHGLFWLRQ; :HLJKW�RI�HDFK�VWUXFWXUH�GHWHUPLQHG�E\�WKH�%D\HV�)DFWRU

����

�_����_���

�_��_�

'3

'3

*'3*3

*'3*3

'*3

'*3 ⋅=

7KH�DFWXDO�VFRUH�ZH�FRPSXWH

Page 72: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#276��

6HDUFK��6XPPDU\9

'LVFUHWH�RSWLPL]DWLRQ�SUREOHP

9,Q�JHQHUDO��13�+DUG

: 1HHG�WR�UHVRUW�WR�KHXULVWLF�VHDUFK: ,Q�SUDFWLFH��VHDUFK�LV�UHODWLYHO\�IDVW��a����YDUV�LQ�a��

PLQ��

� 'HFRPSRVDELOLW\

� 6XIILFLHQW�VWDWLVWLFV

9,Q�VRPH�FDVHV��ZH�FDQ�UHGXFH�WKH�VHDUFK�SUREOHP�WR�DQ

HDV\�RSWLPL]DWLRQ�SUREOHP: ([DPSOH��OHDUQLQJ�WUHHV

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#276 6

2XWOLQH

9,QWURGXFWLRQ

9%D\HVLDQ�QHWZRUNV��D�UHYLHZ

93DUDPHWHU�OHDUQLQJ��&RPSOHWH�GDWD

93DUDPHWHU�OHDUQLQJ��,QFRPSOHWH�GDWD

ª6WUXFWXUH�OHDUQLQJ��&RPSOHWH�GDWD; 6FRULQJ�PHWULFV; 0D[LPL]LQJ�WKH�VFRUH

ª /HDUQLQJ�ORFDO�VWUXFWXUH9

$SSOLFDWLRQ��FODVVLILFDWLRQ9

/HDUQLQJ�FDXVDO�UHODWLRQVKLSV9

6WUXFWXUH�OHDUQLQJ��,QFRPSOHWH�GDWD9

&RQFOXVLRQ

��������� � ������� ��� ����������������� � ����� ��� �

� ������ ��� ��!�"�� "

# ��������$ ��� �%!�"�� "

Page 73: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#276��

/RFDO�DQG�*OREDO�6WUXFWXUH

���������� ������ ��� �����������

���� ����

! "�#�� ���%$&('������������)����*

+-,/.1032 4�536)7 +�8 ,�8 .%9:;:/: < =?>:;:A@ < =?>:B@C: < DE@:B@/@ < @?>@F:/: < @?@G:@F:A@ < @?@G:@;@C: < @?@G:@;@/@ < @?@G:

H

I

J

KL'����������M�����*

N O

N O

N�P Q"RN O

N�P N"R N�P S�N

N�P N"N�O

*OREDO�VWUXFWXUH

([SOLFLWO\�UHSUHVHQWV

3�(_$�� �3�(�

/RFDO�VWUXFWXUH

([SOLFLWO\�UHSUHVHQWV

3�6_$� ���� �3�6_%�(�$ ��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#276!T

/RFDO�VWUXFWXUH��'HFLVLRQ�WUHHV

9&DSWXUH�SURSHUWLHV�RI�FRQWH[W�VSHFLILF�LQGHSHQGHQFH

: %�DQG�6�DUH�LQGHSHQGHQW�JLYHQ�$� �IDOVH

9,QWHUQDO�QRGHV��$�WHVWV�RQ�;·V�SDUHQWV�YDOXHV

9/HDIV��'LVWULEXWLRQ�RQ�;

H

I

J

KL'������������)����*

N O

N O

N�P Q"RN O

N�P N"R N�P S%N

N�P N"N�O

�U�������� ������ �V� �����A�����

���� ����

! "�#M� ���%$

Page 74: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#276��

/HDUQLQJ�GHFLVLRQ�WUHHV

93DUDPHWHU�OHDUQLQJ�

: $V�ZLWK�WDEXODU�UHSUHVHQWDWLRQV: 0XOWLQRPLDO�GLVWULEXWLRQ�DW�HDFK�OHDI: &RXQWV�DUH�DW�WKH�OHYHO�RI�OHDYHV

96WUXFWXUH�OHDUQLQJ

: 'HILQH�WKH�0'/�RU�PDUJLQDO�OLNHOLKRRG: *HQHUDO�VWUXFWXUH�VLPLODU�WR�VFRUHV�RI�%D\HVLDQ�QHWZRUNV

H

I

J

KL'!���������M�����*

N O

N O

N�P Q%RN O

N�P N%R N�P S%N

N�P N%N�O

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#276��

(IIHFWV�RQ�OHDUQLQJ

9*OREDO�VWUXFWXUH�

: (QDEOHV�GHFRPSRVDELOLW\�RI�WKH�VFRUH

� 6HDUFK�LV�IHDVLEOH9

/RFDO�VWUXFWXUH�: 5HGXFHV�WKH�QXPEHU�RI�SDUDPHWHUV�WR�EH�ILWWHG

� %HWWHU�HVWLPDWHV

� 0RUH�DFFXUDWH�JOREDO�VWUXFWXUH�

Page 75: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#276��

/RFDO�6WUXFWXUH��0RUH�$FFXUDWH�*OREDO

6WUXFWXUH

������� ��� ����� ��� �����������

���������

��� � � � ����� ����� � ��� ����� �!� �����"���#�

��� � ���

��� � � � �����

YHUVXV

ILWQHVV

PRGHO�FRPSOH[LW\

7KH�VFRUH�EDODQFLQJ�DFW

$GGLQJ�DQ�DUF�PD\�LPSO\�DQ�H[SRQHQWLDO�LQFUHDVH�RQ

������WKH�QXPEHU�RI�SDUDPHWHUV�WR�ILW�

�LQGHSHQGHQWO\�RI�WKH�UHODWLRQ�EHWZHHQ�WKH�YDULDEOHV

:LWKRXW�ORFDO�VWUXFWXUH���

SUHIHUUHG�PRGHO

$&% ' ' (*) +-, .0/ . , 1 243 5464) 7�8 9�8 :014. ; 14< 1 =4) 546 2�>�? , @ 1 @�A ? < 2 @ B 3�, 2 C ) DFE�G G 6 C 1 . 6 5 C , ?46 54< 8 H�< < . , I4J C @�. 1 @ 1 . K 1 2 L MONQP#R�PTS!U

/RFDO�VWUXFWXUH��1RLV\�2U

V,QWXLWLRQ��&DXVDO�,QGHSHQGHQFH

W 0DQ\�SRVVLEOH�FDXVHV�WKDW�GR�QRW�LQWHUDFW

� 6HYHUDO�GLVHDVHV�FDQ�FDXVH�IHYHU�

,I�RQH�³VXFFHHGV´��WKH�SDWLHQW�KDV�WKH�V\PSWRP

XZY [�\^]�[�\`_ XaY [�\T]b[�\dc

e�\�fF\�g

XaY [�\T]b[�\`_!h���

Page 76: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

$&% ' ' (*) +-, .0/ . , 1 243 5464) 7�8 9�8 :014. ; 14< 1 =4) 546 2�>�? , @ 1 @�A ? < 2 @ B 3�, 2 C ) DFE�G G 6 C 1 . 6 5 C , ?46 54< 8 H�< < . , I4J C @�. 1 @ 1 . K 1 2 L MONQP#R�PTS P

/RFDO�VWUXFWXUH��1RLVH�2U�GHFRPSRVLWLRQ

V%HQHILWV�

W /LQHDU�QXPEHU�RI�SDUDPHWHUVW *RRG�DSSUR[LPDWLRQ�IRU�PDQ\�GRPDLQV

V7UDLQLQJ�

W 8VLQJ�PLVVLQJ�GDWD�PHWKRGVW 2U�JDWH�SDUDPHWHUV�DUH�IL[HG�DQG�QRW�UHWUDLQHG

XZY [ \^]�[�\ _ XZY [�\^]�[�\ c

e�\ f0\#g

XaY [�\T]b[�\��

XaY [�\T]b[ \ _e�\ f0\#g

XZY [�\^]�[�\ ce�\ fF\�g

e�\ f0\#g

XZY [�\^]�[�\��e�\ fF\�g

XZY [�\^]�[�\`_ XaY [�\T]b[�\dc XZY [ \^]�[�\��

/RFDO�RU�JDWH

IL[HG�LQ�DGYDQFH

$&% ' ' (*) +-, .0/ . , 1 243 5464) 7�8 9�8 :014. ; 14< 1 =4) 546 2�>�? , @ 1 @�A ? < 2 @ B 3�, 2 C ) DFE�G G 6 C 1 . 6 5 C , ?46 54< 8 H�< < . , I4J C @�. 1 @ 1 . K 1 2 L MONQP#R�PTS��

2WKHU�7\SHV�RI�/RFDO�6WUXFWXUH

V([WHQVLRQV�RI�WUHHV��*UDSKV

V([WHQVLRQV�RI�1RLV\�RU��1RLV\�PD[��&DXVDO�LQGHSHQGHQFH

V5HJUHVVLRQ

V1HXUDO�QHWV

V&RQWLQXRXV�UHSUHVHQWDWLRQV��VXFK�DV�*DXVVLDQV

V7R�³SOXJ�LQ´�D�GLIIHUHQW�UHSUHVHQWDWLRQ��ZH�QHHG�WKH�IROORZLQJ

W 6XIILFLHQW�6WDWLVWLFVW (VWLPDWLRQ�RI�SDUDPHWHUVW 0DUJLQDO�OLNHOLKRRG

$Q\�W\SH�RI�UHSUHVHQWDWLRQ�WKDW�UHGXFHV�WKH�QXPEHU

RI�SDUDPHWHUV�WR�ILW

Page 77: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

$&% ' ' (*) +-, .0/ . , 1 243 5464) 7�8 9�8 :014. ; 14< 1 =4) 546 2�>�? , @ 1 @�A ? < 2 @ B 3�, 2 C ) DFE�G G 6 C 1 . 6 5 C , ?46 54< 8 H�< < . , I4J C @�. 1 @ 1 . K 1 2 L MONQP#R�PTS��

2XWOLQH

V,QWURGXFWLRQ

V%D\HVLDQ�QHWZRUNV��D�UHYLHZ

V3DUDPHWHU�OHDUQLQJ��&RPSOHWH�GDWD

V3DUDPHWHU�OHDUQLQJ��,QFRPSOHWH�GDWD

V6WUXFWXUH�OHDUQLQJ��&RPSOHWH�GDWD

ª$SSOLFDWLRQ��FODVVLILFDWLRQV

/HDUQLQJ�FDXVDO�UHODWLRQVKLSVV

6WUXFWXUH�OHDUQLQJ��,QFRPSOHWH�GDWDV

&RQFOXVLRQ

$&% ' ' (*) +-, .0/ . , 1 243 5464) 7�8 9�8 :014. ; 14< 1 =4) 546 2�>�? , @ 1 @�A ? < 2 @ B 3�, 2 C ) DFE�G G 6 C 1 . 6 5 C , ?46 54< 8 H�< < . , I4J C @�. 1 @ 1 . K 1 2 L MONQP#R�PTS��

7KH�&ODVVLILFDWLRQ�3UREOHP

V)URP�D�GDWD�VHW�GHVFULELQJ�REMHFWV�E\�YHFWRUV�RI�IHDWXUHV

DQG�D�FODVV

V)LQG�D�IXQFWLRQ�)��IHDWXUHV�→�FODVV�WR�FODVVLI\�D�QHZ�REMHFW

9HFWRU � �������������������������������������������������!��3UHVHQFH

9HFWRU � �������������������������������������������������!��3UHVHQFH

9HFWRU � ���������������������������������������������������!�3UHVHQFH

9HFWRU � ��������������������������������������������������!�$EVHQFH

9HFWRU � �������������������������������������������������!�$EVHQFH

9HFWRU ��������������������������������������������������!�3UHVHQFH

9HFWRU �������������������������������������������������!�$EVHQFH

Page 78: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

$&% ' ' (*) +-, .0/ . , 1 243 5464) 7�8 9�8 :014. ; 14< 1 =4) 546 2�>�? , @ 1 @�A ? < 2 @ B 3�, 2 C ) DFE�G G 6 C 1 . 6 5 C , ?46 54< 8 H�< < . , I4J C @�. 1 @ 1 . K 1 2 L MONQP#R�PTS!S

([DPSOHV

V3UHGLFWLQJ�KHDUW�GLVHDVH

� )HDWXUHV��FKROHVWHURO��FKHVW�SDLQ��DQJLQD��DJH��HWF�� &ODVV��^SUHVHQW��DEVHQW`

V)LQGLQJ�OHPRQV�LQ�FDUV

� )HDWXUHV��PDNH��EUDQG��PLOHV�SHU�JDOORQ��DFFHOHUDWLRQ�HWF�� &ODVV��^QRUPDO��OHPRQ`

V'LJLW�UHFRJQLWLRQ

� )HDWXUHV��PDWUL[�RI�SL[HO�GHVFULSWRUV� &ODVV��^����������������������������`

V6SHHFK�UHFRJQLWLRQ

� )HDWXUHV��6LJQDO�FKDUDFWHULVWLFV��ODQJXDJH�PRGHO� &ODVV��^SDXVH�KHVLWDWLRQ��UHWUDFWLRQ`

$&% ' ' (*) +-, .0/ . , 1 243 5464) 7�8 9�8 :014. ; 14< 1 =4) 546 2�>�? , @ 1 @�A ? < 2 @ B 3�, 2 C ) DFE�G G 6 C 1 . 6 5 C , ?46 54< 8 H�< < . , I4J C @�. 1 @ 1 . K 1 2 L MONQP#R�PTS��

$SSURDFKHV

V0HPRU\�EDVHG

W 'HILQH�D�GLVWDQFH�EHWZHHQ�VDPSOHVW 1HDUHVW�QHLJKERU��VXSSRUW�YHFWRU�PDFKLQHV

V'HFLVLRQ�VXUIDFH

W )LQG�EHVW�SDUWLWLRQ�RI�WKH�VSDFHW &$57��GHFLVLRQ�WUHHV

V*HQHUDWLYH�PRGHOV

W ,QGXFH�D�PRGHO�DQG�LPSRVH�D�GHFLVLRQ�UXOHW %D\HVLDQ�QHWZRUNV

Page 79: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

$&% ' ' (*) +-, .0/ . , 1 243 5464) 7�8 9�8 :014. ; 14< 1 =4) 546 2�>�? , @ 1 @�A ? < 2 @ B 3�, 2 C ) DFE�G G 6 C 1 . 6 5 C , ?46 54< 8 H�< < . , I4J C @�. 1 @ 1 . K 1 2 L MONQP#R�PTS��

*HQHUDWLYH�0RGHOV

V%D\HVLDQ�FODVVLILHUV

W ,QGXFH�D�SUREDELOLW\�GHVFULELQJ�WKH�GDWD

3�) � �«�) � �&�W ,PSRVH�D�GHFLVLRQ�UXOH���*LYHQ�D�QHZ�REMHFW���I P �«�I � �!

F� �DUJPD[ � �3�&� �F�_�I � �«�I � �

V:H�KDYH�VKLIWHG�WKH�SUREOHP�WR�OHDUQLQJ�3�) � �«�) � �&�

�/HDUQ�D�%D\HVLDQ�QHWZRUN�UHSUHVHQWDWLRQ�IRU�3�) � �«�) � �&�

� � � �� ��� ��� � � � ��������� ��� !� "���� # ��$ � %�� ��� �'&)( � * � *,+�( $ � * - �'� � . � /10�2 2 � . � � � � . � (�� ��$ � 3�$ $ � � 4�5 . *,� � * � � 6 � � 7 8:9<;>=,;@?�A

2SWLPDOLW\�RI�WKH�GHFLVLRQ�UXOH

0LQLPL]LQJ�WKH�HUURU�UDWH����

/HW�F B �EH�WKH�WUXH�FODVV��DQG�OHW�OC �EH�WKH�FODVV�UHWXUQHG�E\

WKH�FODVVLILHU�

$�GHFLVLRQ�E\�WKH�FODVVLILHU�LV�FRUUHFW�LI�F B OC ��DQG�LQ�HUURU

LI�F B≠�OC ��

7KH�HUURU�LQFXUUHG�E\�FKRRVH�ODEHO�OC �LV

�7KXV��KDG�ZH�KDG�DFFHVV�WR�3��ZH�PLQLPL]H�HUURU�UDWH�E\

FKRRVLQJ�O B �ZKHQ

ZKLFK�LV�WKH�GHFLVLRQ�UXOH�IRU�WKH�%D\HVLDQ�FODVVLILHU

�_���_��_��_� D IO3IO3OF/F( EF

GGEEE

rr

−=λ= ∑=

LMIO3IO3 HI ≠∀> �_��_�rr

Page 80: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

� � � �� ��� ��� � � � ��������� ��� !� "���� # ��$ � %�� ��� �'&)( � * � *,+�( $ � * - �'� � . � /10�2 2 � . � � � � . � (�� ��$ � 3�$ $ � � 4�5 . *,� � * � � 6 � � 7 8:9<;>=,;@?��

$GYDQWDJHV�RI�WKH�*HQHUDWLYH�0RGHO

$SSURDFK

�2XWSXW��5DQN�RYHU�WKH�RXWFRPHV���OLNHOLKRRG�RI�SUHVHQW

YV��DEVHQW�

([SODQDWLRQ��:KDW�LV�WKH�SURILOH�RI�D�³W\SLFDO´�SHUVRQ�ZLWK

D�KHDUW�GLVHDVH�

0LVVLQJ�YDOXHV��ERWK�LQ�WUDLQLQJ�DQG�WHVWLQJ�

9DOXH�RI�LQIRUPDWLRQ��,I�WKH�SHUVRQ�KDV�KLJK�FKROHVWHURO

DQG�EORRG�VXJDU��ZKLFK�RWKHU�WHVW�VKRXOG�EH�FRQGXFWHG"�

9DOLGDWLRQ��FRQILGHQFH�PHDVXUHV�RYHU�WKH�PRGHO�DQG�LWV

SDUDPHWHUV�

%DFNJURXQG�NQRZOHGJH��SULRUV�DQG�VWUXFWXUH

� � � �� ��� ��� � � � ��������� ��� !� "���� # ��$ � %�� ��� �'&)( � * � *,+�( $ � * - �'� � . � /10�2 2 � . � � � � . � (�� ��$ � 3�$ $ � � 4�5 . *,� � * � � 6 � � 7 8:9<;>=,;����

���

���

�������������� ����������

������������������

������������ ������ ! �"

#$�� �%������������

���������

&����$���'

��(�������)�

*����������

(������

& ���+��-,$+HDUW�GLVHDVH$FFXUDF\� ����'DWD�VRXUFH8&,�UHSRVLWRU\

$GYDQWDJHV�RI�8VLQJ�D�%D\HVLDQ�1HWZRUN

.(IILFLHQF\�LQ�OHDUQLQJ�DQG�TXHU\�DQVZHULQJ

/ &RPELQH�NQRZOHGJH�HQJLQHHULQJ�DQG�VWDWLVWLFDO

LQGXFWLRQ/ $OJRULWKPV�IRU�GHFLVLRQ�PDNLQJ��YDOXH�RI�LQIRUPDWLRQ�

GLDJQRVLV�DQG�UHSDLU

Page 81: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#25672

&

) 8 ) 9 ) :) ; ) < ) =>@? A5B5C@D5CFE D@B@A G CIHFJLK G C M >@N O D5H�H BPK JFQIRLH�A

'LDEHWHV�LQ

3LPD�,QGLDQV

�IURP�8&,�UHSRVLWRU\�

7KH�1DwYH�%D\HVLDQ�&ODVVLILHU

.)L[HG�VWUXFWXUH�HQFRGLQJ�WKH�DVVXPSWLRQ�WKDW�IHDWXUHV�DUH

LQGHSHQGHQW�RI�HDFK�RWKHU�JLYHQ�WKH�FODVV�

./HDUQLQJ�DPRXQWV�WR�HVWLPDWLQJ�WKH�SDUDPHWHUV�IRU�HDFK

3�) S _&��IRU�HDFK�) S �

���_��_��_����_� TUVTV &3&)3&)3&)3))&3 ••••∝ LK

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#256XW

7KH�1DwYH�%D\HVLDQ�&ODVVLILHU��FRQW��

.&RPPRQ�SUDFWLFH�LV�WR�HVWLPDWH

.7KHVH�HVWLPDWH�DUH�LGHQWLFDO�WR�0/(�IRU�PXOWLQRPLDOV

.(VWLPDWHV�DUH�UREXVW�FRQVLVWLQJ�RI�ORZ�RUGHU�VWDWLVWLFV

UHTXLULQJ�IHZ�LQVWDQFHV.

+DV�SURYHQ�WR�EH�D�SRZHUIXO�FODVVLILHU

��

���× YF1

FD1 Z[\P] =θ

&

) ^ ) _ ) `) a ) b ) c

Page 82: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#25687

,PSURYLQJ�1DwYH�%D\HV

91DwYH�%D\HV�HQFRGHV�DVVXPSWLRQV�RI�LQGHSHQGHQFH�WKDW

PD\�EH�XQUHDVRQDEOH�

������$UH�SUHJQDQF\�DQG�DJH�LQGHSHQGHQW�JLYHQ�GLDEHWHV"

��3UREOHP��VDPH�HYLGHQFH�PD\�EH�LQFRUSRUDWHG�PXOWLSOH

WLPHV9

7KH�VXFFHVV�RI�QDwYH�%D\HV�LV�DWWULEXWHG�WR: 5REXVW�HVWLPDWLRQ: 'HFLVLRQ�PD\�EH�FRUUHFW�HYHQ�LI�SUREDELOLWLHV�DUH�LQDFFXUDWH

9,GHD��LPSURYH�RQ�QDwYH�%D\HV�E\�ZHDNHQLQJ�WKH

LQGHSHQGHQFH�DVVXPSWLRQV

%D\HVLDQ�QHWZRUNV�SURYLGH�WKH�DSSURSULDWH�PDWKHPDWLFDO

ODQJXDJH�IRU�WKLV�WDVN

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#256<;

7UHH�$XJPHQWHG�1DwYH�%D\HV��7$1�

9$SSUR[LPDWH�WKH�GHSHQGHQFH�DPRQJ�IHDWXUHV�ZLWK�D�WUHH

%D\HV�QHW9

7UHH�LQGXFWLRQ�DOJRULWKP= 2SWLPDOLW\��PD[LPXP�OLNHOLKRRG�WUHH= (IILFLHQF\��SRO\QRPLDO�DOJRULWKP

95REXVW�SDUDPHWHU�HVWLPDWLRQ

&

>@?#ACB5DFECDHG E@B@AI DFJHKML I D

N >MO P E5J�J

BCL KHQFRMJ�A) ^ ) _ ) `) a

) b ) c

����_���_��_����_� STUVUTU &3&))3&))3&)3))&3 ••••∝ LK

Page 83: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#25687

(YDOXDWLQJ�WKH�SHUIRUPDQFH�RI�D

FODVVLILHU��Q�IROG�FURVV�YDOLGDWLRQ

'� '� '� 'Q

5XQ��

5XQ��

5XQ��

5XQ�Q

93DUWLWLRQ�WKH�GDWD�VHW�LQ�Q

VHJPHQWV9'R�Q�WLPHV

: 7UDLQ�WKH�FODVVLILHU�ZLWK�WKH

JUHHQ�VHJPHQWV: 7HVW�DFFXUDF\�RQ�WKH�UHG

VHJPHQWV9&RPSXWH�VWDWLVWLFV�RQ�WKH�Q�UXQV

� 9DULDQFH

� 0HDQ�DFFXUDF\9$FFXUDF\��RQ�WHVW�GDWD�RI�VL]H�P

: $FF� 2ULJLQDO�GDWD�VHW

P

OF

;

<=><∑

=λ? �_�

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#256@6

3HUIRUPDQFH��7$1�YV��1DwYH�%D\HV

��

��

��

��

��

��

��

���

�� �� �� �� �� �� �� ���7$1

1D

wYH

�%D

\H

V

9���'DWD�VHWV

IURP�8&,

UHSRVLWRU\

0HGLFDO

6LJQDO

SURFHVVLQJ

)LQDQFLDO

*DPHV

9$FFXUDF\�EDVHG

RQ���IROG�FURVV�

YDOLGDWLRQ91R�SDUDPHWHU

WXQLQJ

Page 84: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#256��

3HUIRUPDQFH��7$1�YV�&���

��

��

��

��

��

��

��

���

�� �� �� �� �� �� �� ���

&�

��

7$1

9���'DWD�VHWV

IURP�8&,

UHSRVLWRU\

0HGLFDO

6LJQDO

SURFHVVLQJ

)LQDQFLDO

*DPHV

9$FFXUDF\�EDVHG

RQ���IROG�FURVV�

YDOLGDWLRQ91R�SDUDPHWHU

WXQLQJ

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#256��

%H\RQG�7$1

�&DQ�ZH�GR�EHWWHU�E\�OHDUQLQJ�D�PRUH�IOH[LEOH�VWUXFWXUH"

�([SHULPHQW��OHDUQ�D�%D\HVLDQ�QHWZRUN�ZLWKRXW�UHVWULFWLRQV�RQ

WKH�VWUXFWXUH

Page 85: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#256��

3HUIRUPDQFH��7$1�YV��%D\HVLDQ

1HWZRUNV

��

��

��

��

��

��

��

���

�� �� �� �� �� �� �� ���

7$1

%D

\H

VLD

Q�1

HWZ

RUN

V9���'DWD�VHWV

IURP�8&,

UHSRVLWRU\

0HGLFDO

6LJQDO

SURFHVVLQJ

)LQDQFLDO

*DPHV

9$FFXUDF\�EDVHG

RQ���IROG�FURVV�

YDOLGDWLRQ91R�SDUDPHWHU

WXQLQJ

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#2 ���

:KDW�LV�WKH�SUREOHP"

�2EMHFWLYH�IXQFWLRQ

: /HDUQLQJ�RI�DUELWUDU\�%D\HVLDQ�QHWZRUNV�RSWLPL]HV�3�&��) � �����) � �: ,W�PD\�OHDUQ�D�QHWZRUN�WKDW�GRHV�D�JUHDW�MRE�RQ�3�) � �����)� ��EXW�D

SRRU�MRE�RQ�3�&�_�) � �����) � �������������������������������������������*LYHQ�HQRXJK�GDWD«�1R�SUREOHP«�

: :H�ZDQW�WR�RSWLPL]H�FODVVLILFDWLRQ�DFFXUDF\�RU�DW�OHDVW

WKH�FRQGLWLRQDO�OLNHOLKRRG�3�&�_�) � �����) � �

� 6FRUHV�EDVHG�RQ�WKLV�OLNHOLKRRG�GR�QRW�GHFRPSRVH

��������������⇒��OHDUQLQJ�LV�FRPSXWDWLRQDOO\�H[SHQVLYH�

� &RQWURYHUV\�DV�WR�WKH�FRUUHFW�IRUP�IRU�WKHVH�VFRUHV

�1DLYH�%D\HV��7DQ��HWF�FLUFXPYHQW�WKH�SUREOHP�E\�IRUFLQJ

D�VWUXFWXUH�ZKHUH�DOO�IHDWXUHV�DUH�FRQQHFWHG�WR�WKH�FODVV

Page 86: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#2 � 2

&ODVVLILFDWLRQ��6XPPDU\

�%D\HVLDQ�QHWZRUNV�SURYLGH�D�XVHIXO�ODQJXDJH�WR

LPSURYH�%D\HVLDQ�FODVVLILHUV: /HVVRQ��ZH�QHHG�WR�EH�DZDUH�RI�WKH�WDVN�DW�KDQG��WKH�DPRXQW

RI�WUDLQLQJ�GDWD�YV�GLPHQVLRQDOLW\�RI�WKH�SUREOHP��HWF�$GGLWLRQDO�EHQHILWV

: 0LVVLQJ�YDOXHV: &RPSXWH�WKH�WUDGHRIIV�LQYROYHG�LQ�ILQGLQJ�RXW�IHDWXUH�YDOXHV: &RPSXWH�PLVFODVVLILFDWLRQ�FRVWV

�5HFHQW�SURJUHVV�

: &RPELQH�JHQHUDWLYH�SUREDELOLVWLF�PRGHOV��VXFK�DV�%D\HVLDQ

QHWZRUNV��ZLWK�GHFLVLRQ�VXUIDFH�DSSURDFKHV�VXFK�DV�6XSSRUW

9HFWRU�0DFKLQHV

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254#2 ���

2XWOLQH

�,QWURGXFWLRQ

�%D\HVLDQ�QHWZRUNV��D�UHYLHZ

�3DUDPHWHU�OHDUQLQJ��&RPSOHWH�GDWD

�3DUDPHWHU�OHDUQLQJ��,QFRPSOHWH�GDWD

�6WUXFWXUH�OHDUQLQJ��&RPSOHWH�GDWD

�$SSOLFDWLRQ��FODVVLILFDWLRQ

ª/HDUQLQJ�FDXVDO�UHODWLRQVKLSV: &DXVDOLW\�DQG�%D\HVLDQ�QHWZRUNV: &RQVWUDLQW�EDVHG�DSSURDFK: %D\HVLDQ�DSSURDFK

�6WUXFWXUH�OHDUQLQJ��,QFRPSOHWH�GDWD

�&RQFOXVLRQ

Page 87: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

/103254#2 ���

/HDUQLQJ�&DXVDO�5HODWLRQV�7KDQNV�WR�'DYLG�+HFNHUPDQ�DQG�3HWHU�6SLUWHV�IRU�WKH�VOLGHV�

�'RHV�VPRNLQJ�FDXVH�FDQFHU"

�'RHV�LQJHVWLRQ�RI�OHDG�SDLQW�GHFUHDVH�,4"

�'R�VFKRRO�YRXFKHUV�LPSURYH�HGXFDWLRQ"

�'R�0LFURVRIW�EXVLQHVV�SUDFWLFHV�KDUP�FXVWRPHUV"

/103254#2 ���

&DXVDO�'LVFRYHU\�E\�([SHULPHQW

UDQGRPL]H

PHDVXUH�UDWHOXQJ�FDQFHU

PHDVXUH�UDWH

OXQJ�FDQFHU

VPRNHGRQ¶WVPRNH

&DQ�ZH�GLVFRYHU�FDXVDOLW\�IURP�REVHUYDWLRQDO�GDWD�DORQH"�

Page 88: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

/103254#2 �@7

:KDW�LV�³&DXVH´�$Q\ZD\"

3UREDELOLVWLF�TXHVWLRQ�

:KDW�LV�S��OXQJ�FDQFHU�_�\HOORZ�ILQJHUV���"

&DXVDO�TXHVWLRQ�

:KDW�LV�S��OXQJ�FDQFHU�_�VHW�\HOORZ�ILQJHUV����"

/103254#2 � 6

3UREDELOLVWLF�YV��&DXVDO�0RGHOV

3UREDELOLVWLF�TXHVWLRQ�

:KDW�LV�S��OXQJ�FDQFHU�_�\HOORZ�ILQJHUV���"

&DXVDO�TXHVWLRQ�

:KDW�LV�S��OXQJ�FDQFHU�_�VHW�\HOORZ�ILQJHUV����"

6PRNLQJ

<HOORZ

)LQJHUV

/XQJ

&DQFHU

6PRNLQJ

<HOORZ

)LQJHUV

/XQJ

&DQFHU

Page 89: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

/103254#2 � �

7R�3UHGLFW�WKH�(IIHFWV�RI�$FWLRQV�

0RGLI\�WKH�&DXVDO�*UDSK

6PRNLQJ

<HOORZ

)LQJHUV

/XQJ

&DQFHU

6PRNLQJ

<HOORZ

)LQJHUV

/XQJ

&DQFHU

VHW\HOORZ

ILQJHUV

S��OXQJ�FDQFHU�_�VHW�\HOORZ�ILQJHUV���� �S�OXQJ�FDQFHU�

�����������

&DXVDO�0RGHO

; �

3D �

; �

3D �VHW�;L�

$�GHILQLWLRQ�RI�FDXVDO�PRGHO�«�"

Page 90: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

������������

,GHDO�,QWHUYHQWLRQV

6PRNLQJ

<HOORZ

)LQJHUV

/XQJ

&DQFHU

6PRNLQJ

<HOORZ

)LQJHUV

/XQJ

&DQFHU

VHW

\HOORZ

ILQJHUV

�3HDUO��LGHDO�LQWHUYHQWLRQ�LV�SULPLWLYH��GHILQHV�FDXVH

�6SLUWHV�HW�DO���FDXVH�LV�SULPLWLYH��GHILQHV�LGHDO�LQWHUYHQWLRQ

�+HFNHUPDQ�DQG�6KDFKWHU��IURP�GHFLVLRQ�WKHRU\�RQH�FRXOG

GHILQH�ERWK

�����������

+RZ�&DQ�:H�/HDUQ�&DXVH�DQG�(IIHFW

IURP�2EVHUYDWLRQDO�'DWD"

¬Ι�$,%� ⇒

$�����%

$�����%

$�����%

+

$������������%

+ +

RU

RU

RU

HWF�

Page 91: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��������� �

/HDUQLQJ�&DXVH�IURP�2EVHUYDWLRQV�

&RQVWUDLQW�%DVHG�$SSURDFK

%ULGJLQJ�DVVXPSWLRQV��

&DXVDO�0DUNRY�DVVXPSWLRQ�

)DLWKIXOQHVV

6PRNLQJ

<HOORZ

)LQJHUV

/XQJ

&DQFHU

6PRNLQJ

<HOORZ

)LQJHUV

/XQJ

&DQFHU

�FRQG�LQGHSHQGHQFLHV

IURP�GDWD��FDXVDO�DVVHUWLRQV�

"

&DXVDO�0DUNRY�DVVXPSWLRQ

6PRNLQJ

<HOORZ

)LQJHUV

/XQJ

&DQFHU

6PRNLQJ

<HOORZ

)LQJHUV

/XQJ

&DQFHU

:H�FDQ�LQWHUSUHW�WKH�FDXVDO�JUDSK�DV�D�SUREDELOLVWLF�RQH

L�H���DEVHQFH�RI�FDXVH�⇒ FRQGLWLRQDO�LQGHSHQGHQFH

Page 92: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

)DLWKIXOQHVV

6PRNLQJ/XQJ

&DQFHUDQG

7KHUH�DUH�QR�DFFLGHQWDO�LQGHSHQGHQFLHV

L�H���FRQGLWLRQDO�LQGHSHQGHQFH�⇒ DEVHQFH�RI�FDXVH�

(�J���FDQQRW�KDYH�

,�VPRNLQJ�OXQJ�FDQFHU�

�����������

2WKHU�DVVXPSWLRQV

�$OO�PRGHOV�XQGHU�FRQVLGHUDWLRQ�DUH�FDXVDO

�$OO�PRGHOV�DUH�DF\FOLF

Page 93: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

�����������

�$OO�PRGHOV�XQGHU�FRQVLGHUDWLRQ�DUH�FDXVDO

FRIIHH

FRQVXPSWLRQ

OXQJ

FDQFHU

FRIIHH

FRQVXPSWLRQ

OXQJ

FDQFHU

VPRNLQJ

1R�XQH[SODLQHG�FRUUHODWLRQV

/HDUQLQJ�&DXVH�IURP�2EVHUYDWLRQV�

&RQVWUDLQW�EDVHG�PHWKRG

;�����<�����=

;�����<�����=

;�����<�����=

;�����<�����=

;�����<�����=

;�����<�����=

;�����<�����=

;�����<�����=

$VVXPSWLRQ��7KHVH�DUH�DOO�WKH�SRVVLEOH�PRGHOV

Page 94: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

/HDUQLQJ�&DXVH�IURP�2EVHUYDWLRQV�

&RQVWUDLQW�EDVHG�PHWKRG

;�����<�����=

;�����<�����=

;�����<�����=

;�����<�����=

;�����<�����=

;�����<�����=

;�����<�����=

;�����<�����=

'DWD��7KH�RQO\�LQGHSHQGHQFH�LV�,�;�<�

/HDUQLQJ�&DXVH�IURP�2EVHUYDWLRQV�

&RQVWUDLQW�EDVHG�PHWKRG

;�����<�����=

;�����<�����=

;�����<�����=

;�����<�����=

;�����<�����=

;�����<�����=

;�����<�����=

;�����<�����=

&0$��$EVHQFH�RI�FDXVH�⇒�FRQGLWLRQDO�LQGHSHQGHQFH

RQO\

,�;�<�

Page 95: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

/HDUQLQJ�&DXVH�IURP�2EVHUYDWLRQV�

&RQVWUDLQW�EDVHG�PHWKRG

;�����<�����=

;�����<�����=

;�����<�����=

;�����<�����=

;�����<�����=

;�����<�����=

;�����<�����=

;�����<�����=

)DLWKIXOQHVV��&RQGLWLRQDO�LQGHSHQGHQFH�⇒ DEVHQFH�RI�FDXVH

,�;�<�

/HDUQLQJ�&DXVH�IURP�2EVHUYDWLRQV�

&RQVWUDLQW�%DVHG�0HWKRG

;�����<�����=

;�����<�����=

;�����<�����=

;�����<�����=

;�����<�����=

;�����<�����=

;�����<�����=

;�����<�����=

&RQFOXVLRQ��;�DQG�<�DUH�FDXVHV�RI�=

RQO\

,�;�=�

Page 96: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��������� � �

&DQQRW�$OZD\V�/HDUQ�&DXVH

¬Ι�$,%� ⇒

$�����%

$�����%

$�����%

+

$������������%

+ +

$�����%

2

RU

RU

RU

RU

HWF�

KLGGHQ

FRPPRQ

FDXVHV

VHOHFWLRQ

ELDV

��������� ���

%XW�ZLWK�IRXU��RU�PRUH��YDULDEOHV«

6XSSRVH�ZH�REVHUYH�WKH�LQGHSHQGHQFLHV��GHSHQGHQFLHV

FRQVLVWHQW�ZLWK

7KHQ��LQ�HYHU\�DF\FOLF�FDXVDO�VWUXFWXUH�QRW�H[FOXGHG�E\�

&0$�DQG�IDLWKIXOQHVV��WKHUH�LV�D�GLUHFWHG�SDWK�IURP�=�WR�:�

=�FDXVHV�:

;�����<

=

:

,�;,<�

,�;�<,:_=�

¬Ι�;,<_=�HWF.

WKDW�LV���

Page 97: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . ��������� �0/

&RQVWUDLQW�%DVHG�$SSURDFK

�$OJRULWKP�EDVHG�RQ�WKH�V\VWHPDWLF�DSSOLFDWLRQ�RI

1 ,QGHSHQGHQFH�WHVWV1 'LVFRYHU\�RI�³<´�DQG�³9´�VWUXFWXUHV

�'LIILFXOWLHV�

1 1HHG�LQILQLWH�GDWD�WR�OHDUQ�LQGHSHQGHQFH�ZLWK�FHUWDLQW\

� �:KDW�VLJQLILFDQFH�OHYHO�IRU�LQGHSHQGHQFH�WHVWV�VKRXOG

ZH�XVH"

� /HDUQHG�VWUXFWXUHV�DUH�VXVFHSWLEOH�WR�HUURUV�LQ

LQGHSHQGHQFH�WHVWV

7KH�%D\HVLDQ�$SSURDFK

;�����<�����=

;�����<�����=

;�����<�����=

;�����<�����=

������ 2 =*S

������ 3 =*S

������ 4 =*S

������ 5 =*S

'DWD�G

�����_� 6 =G*S

����_� 7 =G*S

����_� 8 =G*S

�����_� 9 =G*S

2QH�FRQFOXVLRQ��S�;�DQG�<�FDXVH�=_G� �������� ����

Page 98: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

7KH�%D\HVLDQ�DSSURDFK

;�����<�����=

;�����<�����=

;�����<�����=

;�����<�����=

�������VHW_�����VHW_� GGG _*S*<=S<=S �∑=

������ � =*S

������ � =*S

������ � =*S

������ � =*S

'DWD�G

�����_� � =G*S

����_� =G*S

����_� � =G*S

�����_� � =G*S

����� ����

$VVXPSWLRQV

�&DXVDO�0DUNRY�DVVXPSWLRQ

�)DLWKIXOQHVV

�$OO�PRGHOV�XQGHU�FRQVLGHUDWLRQ�DUH�FDXVDO

�HWF�

6PRNLQJ

<HOORZ

)LQJHUV

/XQJ

&DQFHU

'DWD

�FDXVDO�DVVHUWLRQV�

"

QHZ

DVVXPSWLRQ

Page 99: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

��

����� ����

'HILQLWLRQ�RI�0RGHO�+\SRWKHVLV�*

7KH�K\SRWKHVLV�FRUUHVSRQGLQJ�WR�'$*�PRGHO�*��P�LV�D�FDXVDO�PRGHO

���&0$��WKH�WUXH�GLVWULEXWLRQ�KDV�WKH�LQGHSHQGHQFLHV

LPSOLHG�E\�P

�������� ��

���� � ���� �� �����

��� �������� ���

�������� ��

���� � ���� �� �����

��� �������� ���

DQG

�������� ��

���� � ���� �� �����

��� �������� ���

"!$# �%��& ����#(' )"*�+ �-,/. �0� �1# '

����� ���2

)DLWKIXOQHVV

�_� *S θ LV�D�SUREDELOLW\�GHQVLW\�IXQFWLRQ�IRU�HYHU\�*

⇓WKH�SUREDELOLW\�WKDW�IDLWKIXOQHVV�LV�YLRODWHG� ��

([DPSOH��������������'$*�PRGHO�*��;�!<

��_� =⊥ *<;S

Page 100: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

���

����� �� �

&DXVHV�RI�SXEOLVKLQJ�SURGXFWLYLW\5RGJHUV�DQG�0DUDQWR�����

*34

$%,/,7<

4)-

35(352'

6(;

0HDVXUH�RI�DELOLW\��XQGHUJUDGXDWH�

*UDGXDWH�SURJUDP�TXDOLW\

0HDVXUH�RI�SURGXFWLYLW\

4XDOLW\�RI�ILUVW�MRE

6H[

&LWDWLRQ�UDWH

3XEOLFDWLRQ�UDWH

'DWD�����PHQ�����ZRPHQ

&,7(6

38%6

�����������

&DXVHV�RI�SXEOLVKLQJ�SURGXFWLYLW\

$VVXPSWLRQV��1R�KLGGHQ�YDULDEOHV

�7LPH�RUGHULQJ�

�2WKHUZLVH�XQLIRUP�GLVWULEXWLRQ�RQ�VWUXFWXUH

�1RGH�OLNHOLKRRG��OLQHDU�UHJUHVVLRQ�RQ�SDUHQWV

6(; $%,/,7<

*34

35(352'

4)-

38%6 &,7(6

Page 101: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

���

���������

5HVXOWV�RI�*UHHG\�6HDUFK���

*34

$%,/,7<

4)-

35(352'

6(;

&,7(6

38%6

��������� �

2WKHU�0RGHOV

*34

$%,/,7<

4)-

35(352'

6(;

&,7(6

38%6

%D\HVLDQ

*34

$%,/,7<

4)-

35(352'

6(;

&,7(6

38%6

3&������

*34

$%,/,7<

4)-

35(352'

6(;

&,7(6

38%6

5RGJHUV��0DUDQWR

*34

$%,/,7<

4)-

35(352'

6(;

&,7(6

38%6

3&�������

Page 102: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

���

�����������

%D\HVLDQ�0RGHO�$YHUDJLQJ

*34

$%,/,7<

4)-

35(352'

6(;

&,7(6

38%6

S ����

SDUHQWV�RI�38%6 SURE

6(;��4)-��$%,/,7<��35(352' ����

6(;��4)-��$%,/,7< ����

6(;��4)-��35(352' ����

6(;��4)- ����

�����������

&KDOOHQJHV�IRU�WKH�%D\HVLDQ�$SSURDFK

�(IILFLHQW�VHOHFWLYH�PRGHO�DYHUDJLQJ���PRGHO�VHOHFWLRQ

�+LGGHQ�YDULDEOHV�DQG�VHOHFWLRQ�ELDV

� 3ULRU�DVVHVVPHQW� &RPSXWDWLRQ�RI�WKH�VFRUH��SRVWHULRU�RI�PRGHO�� 6WUXFWXUH�VHDUFK

�([WHQG�VLPSOH�GLVFUHWH�DQG�OLQHDU�PRGHOV

Page 103: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

���

�����������

%HQHILWV�RI�WKH�7ZR�$SSURDFKHV

%D\HVLDQ�DSSURDFK��(QFRGH�XQFHUWDLQW\�LQ�DQVZHUV

�1RW�VXVFHSWLEOH�WR�HUURUV�LQ�LQGHSHQGHQFH�WHVWV

&RQVWUDLQW�EDVHG�DSSURDFK��0RUH�HIILFLHQW�VHDUFK

�,GHQWLI\�SRVVLEOH�KLGGHQ�YDULDEOHV

�����������

6XPPDU\

�7KH�FRQFHSWV�RI

� ,GHDO�PDQLSXODWLRQ� &DXVDO�0DUNRY�DQG�IDLWKIXOQHVV�DVVXPSWLRQV

HQDEOH�XV�WR�XVH�%D\HVLDQ�QHWZRUNV�DV�FDXVDO�JUDSKV

IRU�FDXVDO�UHDVRQLQJ�DQG�FDXVDO�GLVFRYHU\

�8QGHU�FHUWDLQ�FRQGLWLRQV�DQG�DVVXPSWLRQV��ZH�FDQ

GLVFRYHU�FDXVDO�UHODWLRQVKLSV�IURP�REVHUYDWLRQDO�GDWD

�7KH�FRQVWUDLQW�EDVHG�DQG�%D\HVLDQ�DSSURDFKHV�KDYH

GLIIHUHQW�VWUHQJWKV�DQG�ZHDNQHVVHV

Page 104: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

���

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254�687:9

2XWOLQH

;,QWURGXFWLRQ

;%D\HVLDQ�QHWZRUNV��D�UHYLHZ

;3DUDPHWHU�OHDUQLQJ��&RPSOHWH�GDWD

;3DUDPHWHU�OHDUQLQJ��,QFRPSOHWH�GDWD

;6WUXFWXUH�OHDUQLQJ��&RPSOHWH�GDWD

;$SSOLFDWLRQ��FODVVLILFDWLRQ

;/HDUQLQJ�FDXVDO�UHODWLRQVKLSV

ª6WUXFWXUH�OHDUQLQJ��,QFRPSOHWH�GDWD;

&RQFOXVLRQ

<:=?>A@B=DCFE G�H?I�E H5G JLKM=5NO=O>�@B=PC:E G HQI�E HQG J

R >TSPU5V J5E JPWQX5E X

Y =OIO>TSBUZV J?E J[WQXQE X

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254�687:\

/HDUQLQJ�6WUXFWXUH�IRU�,QFRPSOHWH�'DWD

'LVWLQJXLVK�;

/HDUQLQJ�VWUXFWXUH�IRU�JLYHQ�VHW�RI�UDQGRP�YDULDEOHV] +DUG�VHDUFK�SUREOHP

;,QWURGXFLQJ�QHZ�KLGGHQ�YDULDEOHV

] +RZ�WR�UHFRJQL]H�WKH�QHHG�IRU�D�QHZ�KLGGHQ�YDULDEOH"] :KHUH�WR�LQWURGXFH�WKH�KLGGHQ�YDULDEOH�LQ�FXUUHQW

VWUXFWXUH"] 2SHQ�HQGHG���

Page 105: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

���

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254�687��

,QFRPSOHWH�'DWD���6WUXFWXUH�6FRUHV

0'/

;8VH�VDPH�0'/�IRUPXOD�ZLWK�SUREDELOLW\�RI�WKH�GDWD

;5HTXLUHV�ILQGLQJ�PD[LPXP�OLNHOLKRRG�SDUDPHWHUV

] 8VLQJ�PHWKRGV�IRU�SDUDPHWHU�OHDUQLQJ��H�J���(0�

;7KHRUHWLFDO�UHVXOWV�VKRZ�WKDW�SHQDOW\�VKRXOG�EH�DGMXVWHG

���GLP��

ORJ������ *'/*

0'*O'*0'/ −−=

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254�6 257

,QFRPSOHWH�'DWD���6WUXFWXUH�6FRUHV��FRQW��

%D\HVLDQ�

;:H�FDQQRW�HYDOXDWH�WKH�PDUJLQDO�OLNHOLKRRG

;:H�KDYH�WR�UHVRUW�WR�DSSUR[LPDWLRQV�

] $V\PSWRWLF�DSSUR[LPDWLRQV

� (YDOXDWH�VFRUH�DURXQG�0$3�SDUDPHWHUV

� 1HHG�WR�ILQG�0$3�SDUDPHWHUV��H�J���(0�] 6WRFKDVWLF�DSSUR[LPDWLRQV

� $SSO\�VWRFKDVWLF�LQWHJUDWLRQ�PHWKRGV

� 0XFK�VORZHU

∫ θΘΘ=

G*3*'3*3

*'3*3'*3

�_���_���

�_����_�

Page 106: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

���

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254�6 2:2

6 &

(

'

0�������������� ��� �� ��� ������ ���0���������������� � �� ��

6 &

(

'

6 &

(

'

6 &

(

'

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254�6 2Q6

3UREOHP

6XFK�SURFHGXUHV�DUH�FRPSXWDWLRQDOO\�H[SHQVLYH�;

&RPSXWDWLRQ�RI�RSWLPDO�SDUDPHWHUV��SHU�FDQGLGDWH��UHTXLUHV

QRQ�WULYLDO�RSWLPL]DWLRQ�VWHS;

6SHQG�QRQ�QHJOLJLEOH�FRPSXWDWLRQ�RQ�D�FDQGLGDWH��HYHQ�LI�LW

LV�D�ORZ�VFRULQJ�RQH

,Q�SUDFWLFH��VXFK�OHDUQLQJ�SURFHGXUHV�DUH�IHDVLEOH�RQO\�ZKHQ

ZH�FRQVLGHU�VPDOO�VHWV�RI�FDQGLGDWH�VWUXFWXUHV

Page 107: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

���

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254�6 2��

6WUXFWXUDO�(0

;,GHD��8VH�SDUDPHWHUV�IRXQG�IRU�SUHYLRXV�VWUXFWXUHV�WR�KHOS

HYDOXDWH�QHZ�VWUXFWXUHV�;

6FRSH��VHDUFKLQJ�RYHU�VWUXFWXUHV�RYHU�WKH�VDPH�VHW�RI

UDQGRP�YDULDEOHV�

2XWOLQH�;

3HUIRUP�VHDUFK�LQ��6WUXFWXUH��3DUDPHWHUV��VSDFH�;

8VH�(0�OLNH�LWHUDWLRQV��XVLQJ�SUHYLRXVO\�EHVW�IRXQG�VROXWLRQ

DV�D�EDVLV�IRU�ILQGLQJ�HLWKHU�] %HWWHU�VFRULQJ�SDUDPHWHUV�����³SDUDPHWULF´�(0�VWHS

RU] %HWWHU�VFRULQJ�VWUXFWXUH�����³VWUXFWXUDO´�(0�VWHS

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254�6 2��

6WUXFWXUDO�(0

;5HFDOO��LQ�FRPSOHWH�GDWD�ZH�KDG] 'HFRPSRVLWLRQ�⇒�HIILFLHQW�VHDUFK

,GHD�;

,QVWHDG�RI�RSWLPL]LQJ�WKH�UHDO�VFRUH«;

)LQG�DQ�DOWHUQDWLYH�VFRUH�WKDW�LV�DPHQDEOH�WR�VHDUFK;

6XFK�WKDW] :H�UHFRYHU�GHFRPSRVDELOLW\�DQG�VXIILFLHQW�VWDWLVWLFV] 0D[LPL]LQJ�QHZ�VFRUH�¼�LPSURYHPHQW�LQ�UHDO�VFRUH

Page 108: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

���

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254�6 2��

([SHFWHG�VFRUHV

;/HW�2�GHQRWH�WKH�REVHUYHG�GDWD

;/HW�+�GHQRWH�WKH�KLGGHQ�YDULDEOHV

;,I�ZH�KDYH�D�GLVWULEXWLRQ�4�+���WKHQ�³FRPSOHWH´�GDWD

���������������������������� �����

+

2

'DWD�

�������@���> +206FRUH+4+206FRUH( �� ∑=

;6LQFH�2��+�GHVFULEH�FRPSOHWH�GDWD

u7KH�H[SHFWHG�VFRUH�LV�GHFRPSRVDEOH�

∑=

=

� ���������� ����������

������

������

16FRUH(

16FRUH(+206FRUH(

�@�>

@��>�@�_�>

����

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254�6 2��

+RZ�GR�ZH�FKRRVH�4�+�"

7KHRUHP��,I�4�+�� �3�+_2�0 � ��WKHQ

&RQVHTXHQFHV�

u0�LV�EHWWHU�WKDQ�0��DFFRUGLQJ�WR�H[SHFWHG�VFRUH�

������⇒�0�LV�DOVR�EHWWHU�DFFRUGLQJ�WR�WUXH�VFRUH

�@�_�>�@�_�>

�_��_��

�2+06FRUH(2+06FRUH(

206FRUH206FRUH

�� −

≥−

Page 109: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

���

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254�67258

9)RU�WKH�0'/�VFRUH��ZH�JHW�WKDW

&RQVHTXHQFH�9

:H�FDQ�XVH�FRPSOHWH�GDWD�PHWKRGV��ZHUH�ZH�XVH�H[SHFWHG

FRXQWV��LQVWHDG�RI�DFWXDO�FRXQWV

�3HQDOW\��_�ORJ@�_���>

�3HQDOW\�@�_�_�ORJ���>

3HQDOW\�%����@��_�_(>ORJ3��

�@��_�(>0'/�

:

::

:

%3D;3'�%3D;1(

%'�%3D;33D;1(

'�%%'

'�%%���'

; ;;;;; ;;;;

−=

−=

=

+

+

6WUXFWXUDO�(0�IRU�0'/

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254�6725<

7UDLQLQJ'DWD

([SHFWHG�&RXQWV=?>@BA�C=?>@BD C=?>@FE�C=?>�GIHJ@KAHL@BA�HJ@KE�C=?>(MNAHOGBC=?>(MNDPHQGBC=?>(MRESHQGBC=?>@BD�T @KA�C=?>�GIHJ@KAHL@FE�C=?>(M A HL@ D C=?>(M D H5M A HQGKC

&RPSXWDWLRQ

U7V U7W U7X

Y

Z[V Z[W Z\X

U7V U7W U7X

Y

Z[V Z[W Z\X+

6FRUH�

�3DUDPHWHUL]H

U V U W U X

Y

Z V Z W Z X

Page 110: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

���

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254�67258

6WUXFWXUDO�(0�LQ�3UDFWLFH

,Q�WKHRU\�9

(�6WHS��FRPSXWH�H[SHFWHG�FRXQWV�IRU�DOO�FDQGLGDWH�VWUXFWXUHV9

0�6WHS��FKRRVH�VWUXFWXUH�WKDW�PD[LPL]HV�H[SHFWHG�VFRUH

3UREOHP��WKHUH�DUH��H[SRQHQWLDOO\��PDQ\�VWUXFWXUHV::H�FDQQRW�FRPSXWHG�H[SHFWHG�FRXQWV�IRU�DOO�RI�WKHP�LQ�DGYDQFH

6ROXWLRQ�9

0�6WHS��VHDUFK�RYHU�QHWZRUN�VWUXFWXUHV��H�J���KLOO�FOLPELQJ�9

(�6WHS��RQ�GHPDQG��IRU�HDFK�VWUXFWXUH�*�H[DPLQHG�E\�0�

6WHS��FRPSXWH�H[SHFWHG�FRXQWV98VH�VPDUW�FDFKLQJ�VFKHPHV�WR�PLQLPL]H�RYHUDOO

FRPSXWDWLRQV

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254�6;6;<

7KH�6WUXFWXUDO�(0�3URFHGXUH

,QSXW��% = � ��* = �Θ = �ORRS�IRU�Q�= 0,1,«�XQWLO�FRQYHUJHQFH

,PSURYH�SDUDPHWHUV�

ΘC > = 3DUDPHWULF�(0��* > �Θ > �OHW�%C > ��* >@? �ΘC > �

,PSURYH�VWUXFWXUH�

6HDUFK�IRU�D�QHWZRUN�% >BADC � ��* >FEHG �Θ >FEDG ��V�W�(>6FRUH�% >IADC �'��_�%C > @�!�(>6FRUH�%C > �'��_�%C > @

93DUDPHWULF�(0���FDQ�EH�UHSODFHG�E\�*UDGLHQW�$VFHQW�

1HZWRQ�5DSKVRQ�PHWKRGV��RU�DFFHOHUDWHG�(0�9(DUO\�VWRSSLQJ�SDUDPHWHU�RSWLPL]DWLRQ�VWDJH�DYRLGV

³HQWUHQFKPHQW´�LQ�FXUUHQW�VWUXFWXUH�

Page 111: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

���

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254�6;6 2

6WUXFWXUDO�(0��&RQYHUJHQFH�3URSHUWLHV

7KHRUHP��7KH�6(0�SURFHGXUH�FRQYHUJHV�LQ�VFRUH�

�����7KH�OLPLW�����������������������������������������H[LVWV�

7KHRUHP��&RQYHUJHQFH�SRLQW�LV�D�ORFDO�PD[LPD�

�����,I�* � � �*�LQILQLWHO\�RIWHQ��WKHQ��Θ � ��WKH�OLPLW�RI�WKH

SDUDPHWHUV�LQ�WKH�VXEVHTXHQFH�ZLWK�VWUXFWXUH�*��LV�D

VWDWLRQDU\�SRLQW�LQ�WKH�SDUDPHWHU�VSDFH�RI�*�

��6FRUH�OLP '% �� ∞→

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254�6;6 6

/HDUQLQJ�6WUXFWXUH�IURP�,QFRPSOHWH

'DWD��6XPPDU\9+DUG�SUREOHP�

9,QLWLDO�SURJUHVV�

� (0�OLNH�VHDUFK�WHFKQLTXHV

� ��&38�\HDUV�⇒���&38�KRXUV

93UREOHPV�

� (VFDSLQJ�ORFDO�PD[LPD� ,QGXFLQJ�QHZ�YDULDEOHV

Page 112: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

���

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254�6;6��

2XWOLQH

9,QWURGXFWLRQ

9%D\HVLDQ�QHWZRUNV��D�UHYLHZ

93DUDPHWHU�OHDUQLQJ��&RPSOHWH�GDWD

93DUDPHWHU�OHDUQLQJ��,QFRPSOHWH�GDWD

96WUXFWXUH�OHDUQLQJ��&RPSOHWH�GDWD

9$SSOLFDWLRQ��FODVVLILFDWLRQ

9/HDUQLQJ�FDXVDO�UHODWLRQVKLSV

96WUXFWXUH�OHDUQLQJ��,QFRPSOHWH�GDWD

ª&RQFOXVLRQ

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254�6;6��

6XPPDU\��/HDUQLQJ�%D\HVLDQ�1HWZRUNV

93DUDPHWHUV��6WDWLVWLFDO�HVWLPDWLRQ

96WUXFWXUH��6HDUFK�RSWLPL]DWLRQ

9,QFRPSOHWH�GDWD��(0

6

/

&

(

';

'DWD���

3ULRU�LQIRUPDWLRQ

6WUXFWXUH

6HDUFK

6XIILFLHQW

6WDWLVWLFV

*HQHUDWRU

3DUDPHWHU

)LWWLQJ

,QGXFHU

Page 113: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

���

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254�67678

8QWRXFKHG�LVVXHV

9)HDWXUH�HQJLQHHULQJ

: )URP�PHDVXUHPHQWV�WR�IHDWXUHV

9)HDWXUH�VHOHFWLRQ

: 'LVFRYHULQJ�WKH�UHOHYDQW�IHDWXUHV

96PRRWKLQJ�DQG�SULRUV

: 1RW�HQRXJK�GDWD�WR�FRPSXWH�UREXVW�VWDWLVWLFV

9�5HSUHVHQWLQJ�VHOHFWLRQ�ELDV

: $OO�WKH�VXEMHFWV�IURP�WKH�VDPH�SRSXODWLRQ

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254�676<;

8QWRXFKHG�,VVXHV��&RQW��

98QVXSHUYLVHG�OHDUQLQJ

: &OXVWHULQJ�DQG�H[SORUDWRU\�GDWD�DQDO\VLV

9,QFRUSRUDWLQJ�WLPH

: /HDUQLQJ�'%1V

96DPSOLQJ��DSSUR[LPDWH�LQIHUHQFH�DQG�OHDUQLQJ

: 1RQ�FRQMXJDWH�IDPLOLHV��DQG�WLPH�FRQVWUDLQWV

92Q�OLQH�OHDUQLQJ��UHODWLRQ�WR�RWKHU�JUDSKLFDO�PRGHOV

Page 114: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

���

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254�676��

6RPH�$SSOLFDWLRQV

9%LRVWDWLVWLFV����0HGLFDO�5HVHDUFK�&RXQFLO��%XJV�

9'DWD�$QDO\VLV����1$6$��$XWR&ODVV�

9&ROODERUDWLYH�ILOWHULQJ����0LFURVRIW��06%1�

9)UDXG�GHWHFWLRQ����$77

9&ODVVLILFDWLRQ����65,��7$1�%/7�

96SHHFK�UHFRJQLWLRQ����8&�%HUNHOH\

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254�676��

6\VWHPV

9%8*6���%D\HVLDQ�LQIHUHQFH�8VLQJ�*LEEV�6DPSOLQJ

� $VVXPHV�IL[HG�VWUXFWXUH� 1R�UHVWULFWLRQV�RQ�WKH�GLVWULEXWLRQ�IDPLOLHV� 5HOLHV�RQ�0DUNRY�&KDLQ�0RQWHFDUOR�0HWKRGV�IRU�LQIHUHQFH� ZZZ�PUF�EVX�FRP�DF�XN�EXJV

9$XWR&ODVV���8QVXSHUYLVHG�%D\HVLDQ�FODVVLILFDWLRQ

� $VVXPHV�D�QDwYH�%D\HV�VWUXFWXUH�ZLWK�KLGGHQ�YDULDEOH�DW�WKH�URRW

UHSUHVHQWLQJ�WKH�FODVVHV� ([WHQVLYH�OLEUDU\�RI�GLVWULEXWLRQ�IDPLOLHV�� DFN�DUF�QDVD�JRY�LF�SURMHFWV�ED\HV�JURXS�JURXS�DXWRFODVV�

Page 115: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

���

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254�676��

6\VWHPV��&RQW��

906%1���0LFURVRIW�%HOLHI�1HWZRNV

� /HDUQV�ERWK�SDUDPHWHUV�DQG�VWUXFWXUH��YDULRXV�VHDUFK�PHWKRGV� 5HVWULFWLRQV�RQ�WKH�IDPLO\�RI�GLVWULEXWLRQV� ZZZ�UHVHDUFK�PLFURVRIW�FRP�GWDV�

97$1�%/7���7UHH�$XJPHQWHG�1DwYH�%D\HV�IRU�VXSHUYLVHG

FODVVLILFDWLRQ� &RUUHODWLRQV�DPRQJ�IHDWXUHV�UHVWULFWHG�WR�IRUHVWV�RI�WUHHV� 0XOWLQRPLDO��*DXVVLDQV��PL[WXUHV�RI�*DXVVLDQV��DQG�OLQHDU

*DXVVLDQV� ZZZ�HUJ�VUL�FRP�SURMHFWV�/$6

90DQ\�PRUH���ORRN�LQWR�$8$,��:HE�HWF�

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254�6����

&XUUHQW�7RSLFV

97LPH

: %H\RQG�GLVFUHWH�WLPH�DQG�EH\RQG�IL[HG�UDWH

9&DXVDOLW\

: 5HPRYLQJ�WKH�DVVXPSWLRQV

9+LGGHQ�YDULDEOHV

: :KHUH�WR�SODFH�WKHP�DQG�KRZ�PDQ\"

90RGHO�HYDOXDWLRQ�DQG�DFWLYH�OHDUQLQJ

: :KDW�SDUWV�RI�WKH�PRGHO�DUH�VXVSHFW�DQG�ZKDW�DQG

KRZ�PXFK�GDWD�LV�QHHGHG"

Page 116: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

���

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254�6�� 2

3HUVSHFWLYH��:KDW¶V�2OG�DQG�:KDW¶V�1HZ

92OG��6WDWLVWLFV�DQG�SUREDELOLW\�WKHRU\

� 3URYLGH�WKH�HQDEOLQJ�FRQFHSWV�DQG�WRROV�IRU�SDUDPHWHU

HVWLPDWLRQ�DQG�ILWWLQJ��DQG�IRU�WHVWLQJ�WKH�UHVXOWV

91HZ��5HSUHVHQWDWLRQ�DQG�H[SORLWDWLRQ�RI�GRPDLQ

VWUXFWXUH: 'HFRPSRVDELOLW\

(QDEOLQJ�VFDODELOLW\�DQG�FRPSXWDWLRQ�RULHQWHG�PHWKRGV

: 'LVFRYHU\�RI�FDXVDO�UHODWLRQV�DQG�VWDWLVWLFDO

LQGHSHQGHQFH

(QDEOLQJ�H[SODQDWLRQ�JHQHUDWLRQ�DQG�LQWHUSUHWDELOLW\

: 3ULRU�NQRZOHGJH

(QDEOLQJ�D�PL[WXUH�RI�NQRZOHGJH�HQJLQHHULQJ�DQG�LQGXFWLRQ

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254�6�� 6

7KH�)XWXUH���9

3URJUHVV�ZLOO�SDUDOOHO�DQG�OHYHUDJH�RQ�H[WHQVLRQV�WR

PRGHOLQJ� 0RUH�H[SUHVVLYH�UHSUHVHQWDWLRQ�ODQJXDJHV� %HWWHU�FRQWLQXRXV�GLVFUHWH�PRGHOV� ,QFUHDVH�FURVV�IHUWLOL]DWLRQ�ZLWK�QHXUDO�QHWZRUNV

95DQJH�RI�DSSOLFDWLRQV

� %LRORJ\���'1$��FRQWURO��ILQDQFLDO��SHUFHSWLRQ���9

%H\RQG�FXUUHQW�OHDUQLQJ�PRGHO� )HDWXUH�GLVFRYHU\� 0RGHO�GHFLVLRQV�DERXW�WKH�SURFHVV��GLVWULEXWLRQV��IHDWXUH

VHOHFWLRQ� 8WLOLWLHV

9+\EULG�PHWKRGV����%D\HVLDQ�QHWZRUNV�DV�³JOXH´"

Page 117: Iowa State Universityweb.cs.iastate.edu/~jtian/cs673/cs673_spring05/...!!#" $ % &(' )) * + ,!#-. /103254 /HDUQLQJ LQ WKLV FRQWH[W 7 3URFH V ,QSXW˛ GDWDVHW DQG SULRU LQIRUPDWLRQ 2XWSXW˛

���

��� � � ��� �� �� � ��������� ��� ��� �� � � �� ��� ��� ��� � � ! !#"�� � � ! $ ��� � % � &('�) ) � % � � % � ��� ��� � *�� � � +�, % !# ! - � . /103254�6����

0DQ\�WKDQNV�WR���

9*LO�%HMHUDQR

9/LVH�*HWRRU

9'DYLG�+HFNHUPDQ

9'DSKQH�.ROOHU

98UL�/HUQHU

95RQ�3DUU

93HWHU�6SLUWHV

9%LNDVK�6DEDWD

«��$QG�UHPHPEHU

)RU�FXUUHQW�VOLGHV��DGGLWLRQDO�PDWHULDO��DQG�UHDGLQJ�OLVW�VHH

KWWS���ZZZ�FV�EHUNHOH\�HGX�aQLU�7XWRULDO