Ni

c

( )

(

, )

* v a~ x b

* x v b

- .

( ) (

i c j

g

•• a

•• d a i

a a jaa a

• d a g a• g d9F T a NIL

••

•

u L•• TV u d a

a• s• wd a• s wd7 VI a

•• C D d g• s w d a

• g• 9F T a NIL

• i j•• NIL• 9F T

•

•

•−2.5

0.0

2.5

5.0

−3 −2 −1 0 1 2 3X

Y

a i• ! " #$, #&, … , #( )

x * = 1,… , )

!$ = -. + -$#$$ + ⋯+ -(#$( + 1$!& = -. + -$#&$ + ⋯+ -(#&( + 1&

⋮!3 = -. + -$#3$ + ⋯+ -(#3( + 13

⋮!4 = -. + -$#4$ + ⋯+ -(#4( + 14

a i• x w a

5 =

!$!&⋮!3⋮!4

, 6 =

1 #$$ ⋯1⋮1

#&$⋮#3$

⋯⋱⋯

⋮1

⋮#4$

⋱⋯

#$(#&(⋮#3(⋮#4(

, 8 =

-.-$-&⋮-(

, 9 =

1$1&⋮13⋮14

• x5 = 68 + 9v w

!3 = 63:8 + 13

a i• 9 = 5 − 68 < a

< = 5 − 68 : 5 − 68 = 5:5 − 28:6:65 + 8:6:68• a

><>8

= −26:5 + 26:68 = 0

•

6:68 = 6:5

• 6:6 @$ a A8

A8 = 6:6 @$6:5

i d s lt• ax ~ w

• 6 Ba Σ x6~E B, Σ a68 + 9~E 68 + 9, 8Σ8:

• 9 0a F& x9~E 0, F&G~ w b

• 5 68a F&G x

5 = 68 + 9~E 68, F&G

a• 8c 9c 5 aA8c H5c I x

•A8 = 6:6 @$6:5~E 8, F& 6:6 @$

•H5 = 6A8 = 6 6:6 @$6:5 = J5~E 68, F&J

J = 6 6:6 @$6:

•I = 5 − H5 = G − J 5~E 0, F& G − J

J

a• x

A8~E 8, F& 6:6 @$

• a x

A8 − 8

F& 6:6 @$~E 0, 1

Rg• g 6a g 8a F& y

a 5 wx• * #3 = 1, #3$, … , #3( a !3

a x

K !3|#3; 8, F& =1

2NF&O#K −

!3 − #38 &

2F&

Rg• z

•P !3|#3; 8, F& = #38

•Q !3|#3; 8, F& = F&

E yi | xi ;β,σ2( ) = xiβV yi | xi ;β,σ

2( ) =σ 2

Rg• * w

K 5|6; 8, F& =R3S$

4

K !3|#3; 8, F&

=R3S$

41

2NF&O#K −

!3 − #38 &

2F&

=1

2NF&

4

O#K −∑3S$4 !3 − #38 &

2F&

og Rg•

ln K 5|6; 8, F& = ln R3S$

4

K !3|#3; 8, F&

= ln1

2NF&

4

O#K −∑3S$4 !3 − #38 &

2F&

= −)2ln2N −

)2lnF& −

12F&

W3S$

4!3 − #38 &

e•

g

• a

W3S$

4!3 − #38 &

yi − xiβ( )2

i=1

n∑

-4 -2 0 2 4

-12

-10

-8-6

-4-2

Values of parameter

Log

likel

ihoo

d

山登り

対数尤度関数が最大となる点を点推定

e N• g

xW

3S$

4!3 − #38 &

•A8 =

∑3S$4 #3!3∑3S$4 #3

&

XF& =1

) − " + 1W3S$

4

!3 − #3A8&

/ ) − (" + 1)

Rg E• g 6a g 8a F& y a

! a ax

K 5|6; 8, F& =1

2NF&O#K −

5 − 68 &

2F&

Rg E• z

•P 5|6; 8, F& = 68

•Q 5|6; 8, F& = F&

E y | X ;β,σ 2( ) = Xβ

Rg E• g w

K 5|6; 8, F& =1

2NF&O#K −

5 − 68 &

2F&

=1

2NF&O#K −

5 − 68 : 5 − 682F&

og Rg E• g w

ln K 5|6; 8, F& = ln1

2NF&O#K −

5 − 68 : 5 − 682F&

= −)2ln2N −

)2lnF& −

5 − 68 : 5 − 682F&

e N E• r

A8 = 6:6 @$6:5

XF& =1[5 − 6A8

:5 − 6A8

[ = ) − " + 1 hp y

e N z• A8 6:6 @$ ~

wA8 = 6:6 @$6:5

• 6 w a 6:6 @$ ~ w• ) < " i w j• w v• w v a

E• k " y ln g " w

g

• 183 a 2( NP a

• a g w i jw

• _ ~ x wx

vy

( s

• a ~ w

r• _

wx• y x

_ =

11

1

g• _• ak w l

wwa rank _ v

o

• w a g~ a

• w w• w• w• 6:6 @$ x ok l wx

nL ) ( (• 6:6 @$ x a x 6:6

~ w• y a g c G(d$ x w

a8e = 6:6 + cG(d$ @$6:5

• a fe 8 x8 w

fe 8 = ! − 68 : ! − 68 + c8:8

n• a )×) _wx

_h = G4 = h_

• aG4 )×) ah )×)

ij nL• y

5 = 68 + 9

• x a ij ~ wx

fe 8 = 5 − 68 & + c 8 k = 5 − 68 : 5 − 68 + cWlS$

(

-lk

• c 8 k = c∑lS$( -l

k wxim ≥ 0, c ≥ 0j• c HTR NX] FVFR X Vb w i

j a w• fe 8 a 8

ij nL• ij ~ fe 8 8

a w

arg min8∈ℝtuv

fe 8 = arg min8∈ℝtuv

5 − 68 & + c 8 k

= arg min8∈ℝtuv

5 − 68 : 5 − 68 + cWlS$

(

-lk

= arg min8∈ℝtuv

W3S$

4

!3 −-. −WlS$

(

-l#3l

&

+ cWlS$

(

-lk

arg min

• max x # d x #• arg max x # d x # #

• min x # d x #• arg min x # d x # #

• d arg min

ik nL• ik ~ fe 8 a

arg min8∈ℝtuv

fe 8 = arg min8∈ℝtuv

! − 68 & + c 8 k

• m = 2 NIL• m = 1 9F T• 0 < y < 1 y w c 8 kx y a5 F XNH S X 5 y

cWlS$

(

2y -l + 1 − y -l&

(• ik ~ fe 8

arg min8∈ℝtuv

fe 8 = arg min8∈ℝtuv

! − 68 & + c 8 k

• NIL m = 2 • m

• 9F T m = 1 • m

• 5 F XNH S X 0 < y < 1 • m m

i& nL (• m = 2 fe 8 x

A8z{|}~ = arg min8∈ℝtuv

fe 8

= arg min8∈ℝtuv

! − 68 & + c 8 &

= arg min8∈ℝtuv

! − 68 : ! − 68 + cWlS$

(

-l&

= arg min8∈ℝtuv

W3S$

4

!3 −-. −WlS$

(

-l#3l

&

+ cWlS$

(

-l&

i& nL (• NIL A8z{|}~ a

v

A8z{|}~ = arg min8∈ℝtuv

W3S$

4

!3 −-. −WlS$

(

-l#3l

&

�. Å.WlS$

(

-l& ≤ Å

9

i& nL (• x ~

A8z{|}~ = arg min8∈ℝtuv

fe 8 = arg min8∈ℝtuv

5 − 68 : 5 − 68 + c8:8

• a NIL g

A8z{|}~ = 6:6 + cG(d$ @$6:5

i& nL (•

fe 8 = 5 − 68 & + c 8 &= 5 − 68 : ! − 68 + c8:8= 5:5 − 28:É:5 + 8:É:É8 + c8:8

• 8fe 8>8

= −2É:5 + 2É:É8 + 2c8

• ÑÖ 8Ü8

= 0A8z{|}~ = 6:6 + cG(d$ @$6:5

i& nL (• A8z{|}~

A8z{|}~ = arg min8∈ℝtuv

W3S$

4

!3 −-. −WlS$

(

-l#3l

&

�. Å.WlS$

(

-l& ≤ Å

• Å -l x

• i& w c w ag

0 500 1000 1500 2000

−200

020

040

0

L1 Norm

Coe

ffici

ents

10 10 10 10 10

1

2

3

4

5

6

7

8

9

10

Å = WlS$

(

-l&

-l

0 500 1000 1500 2000

−200

020

040

0

L1 Norm

Coe

ffici

ents

10 10 10 10 10

1

2

3

4

5

6

7

8

9

10

i& nL (• i& w c w a g

2 4 6 8 10

−200

020

040

0

Log Lambda

Coe

ffici

ents

10 10 10 10 10

1

2

3

4

5

6

7

8

9

10-l

i& nL (• fe 8

arg min8∈ℝtuv

! − 68 & + c 8 &

• c g 3A : 5i jx w

2 4 6 8 10

3000

3500

4000

4500

5000

5500

6000

log(Lambda)

Mea

n−Sq

uare

d Er

ror

10 10 10 10 10 10 10 10 10 10 10 10 10

2 4 6 8 10

−200

020

040

0

Log Lambda

Coe

ffici

ents

10 10 10 10 10

1

2

3

4

5

6

7

8

9

10

i& nL (• g 3A : 5i j

c -l ”

i$ nL• m = 1 fe 8 x

A8áàââä = arg min8∈ℝtuv

fe 8

= arg min8∈ℝtuv

! − 68 & + c 8 $

= arg min8∈ℝtuv

! − 68 : ! − 68 + cWlS$

(

-l

= arg min8∈ℝtuv

W3S$

4

!3 −-. −WlS$

(

-l#3l

&

+ cWlS$

(

-l

i$ nL• 9F T A8áàââä a

v

A8áàââä = arg min8∈ℝtuv

W3S$

4

!3 −-. −WlS$

(

-l#3l

&

�. Å. WlS$

(

-l ≤ Å

i$ nL• 9F T A8áàââä

A8áàââä = arg min8∈ℝtuv

W3S$

4

!3 −-. −WlS$

(

-l#3l

&

�. Å. WlS$

(

-l ≤ Å

• Å ≥

• ã- ~a Å. =∑lS$( ã-l a Å < Å.

(• NIL 9F T a 8 k

ww

• w g -$ -& y a 8 kx

• 9F T d -$ + -& ≤ Åi j• NIL d -$

& + -&& ≤ Åi j

(• ã- = ã-$, ã-&• ã-z -$ -&

a x

9F T NIL

8 $ = -$ + -& ≤ Å 8 & = -$& + -&

& ≤ Å

7F XN X F ,

! − 68 & ! − 68 &

fe 8

8 k

• 8 k = ∑lS$( -l

k

x

• 5 F XNH S X c ∑lS$( 1 − y -l& + 2 -l x

7F XN X F ,

i$ nL• i$ w e

• i$ x 8 s go w ~

• f w ay

i$ nL• 9F T A8áàââä

min8∈ℝtuv

12W3S$

4

!3 −-. −WlS$

(

-l#3l

&

�. Å. WlS$

(

-l ≤ Å

• a #3l a w

i$ nL• a 9F T A8áàââä a

x -

x - =12W3S$

4

!3 −-. −WlS$

(

-l#3l

&

+ åWlS$

(

-l

~a å å ≥ 0

i$ nL• ã-

• a x - A8áàââä å ã- åw x

A8áàââä å = sin ã- ã- − åd

i$ nL• c 8 $ 8 w a x

w 8 w

• 3TTVINSFX I 4 H SX 1 LTVNXMR• MXX / [[[ O XFX T X TVL FVXNH N [ ((N

• 9 F X 1SL LV NTS 91 • MXX / VTO HX H NI TVL H NI FT -( ,-.(

• 8X VFXN MVNSPFL MV MT INSL 1 LTVN R 8 1 • MXX / T V SS NSVNF V 3 IVNH 7 V X 3 IVNH 7 V X FV E RNSF

V 5SXV E1E6F XE8X VFXN E MVNSPFLMV MT INSLE1 LTVNXMR TVE9NS FVE8S V E=VT R E 1 E2 HP E: E T E N 2V HPE . I

• 1 X VSFXNSL 4NV HXNTS : XMTI T : XN N V 14:: • MXX / [ XFS TVI I _ T]I F V I FIRREIN XVE XFX I

i$ nL• 9F T A8áàââä

min8∈ℝtuv

12W3S$

4

!3 −-. −WlS$

(

-l#3l

&

�. Å. WlS$

(

-l ≤ Å

• Å -lx

• i$ w c w ag

0 500 1000 1500 2000 2500 3000

−600

−400

−200

020

040

060

0

L1 Norm

Coe

ffici

ents

0 2 4 6 8 10 10

1

2

3

4

5

6

7

8

9

10

-l

Å = WlS$

(

-l

−4 −2 0 2 4

−600

−400

−200

020

040

060

0

Log Lambda

Coe

ffici

ents

10 10 7 4 0

1

2

3

4

5

6

7

8

9

10

0 500 1000 1500 2000 2500 3000

−600

−400

−200

020

040

060

0

L1 Norm

Coe

ffici

ents

0 2 4 6 8 10 10

1

2

3

4

5

6

7

8

9

10-l

i$ nL• i$ w c w a g

i$ nL• fe 8

A8áàââä = arg min8∈ℝtuv

! − 68 & + c 8 $

• c g 3A : 5i j

x w−4 −2 0 2 4

3000

3500

4000

4500

5000

5500

6000

log(Lambda)

Mea

n−Sq

uare

d Er

ror

10 10 10 9 10 8 8 8 7 7 7 5 4 4 3 2 0

i$ nL• g 3A : 5i j

c -l ”

−4 −2 0 2 4

−600

−400

−200

020

040

060

0

Log Lambda

Coe

ffici

ents

10 10 7 4 0

1

2

3

4

5

6

7

8

9

10

i• FV g v INF X g

• MXX / [ XFS TVI I _MF XN XFX9 FVS FV NX]E N 41 1 INF X MXR• )) g• i jdFL 2:8 2TI] RF NSI RF

• d

• XH e g • I 949 g d 0 g • MI 749 g d 0 g • XHM g • XL d • L d g

• g ] dt• Bd g