BIT

SF4

64:M

achi

neLe

arni

ng

05Tow

ardsB

ayesian

Machin

eLearn

ingD

r.K

amle

shTi

war

iA

ssis

tant

Pro

fess

or,D

epar

tmen

tofC

SIS

,B

ITS

Pila

ni,P

ilani

Cam

pus,

Raj

asth

an-3

3303

1IN

DIA

Jan

27,2

021

ON

LIN

E(C

ampu

s@

BIT

S-P

ilani

Jan-

May

2021

)

http://ktiwari.in/ml

Let’s

Pre

dict

Con

side

rtab

lebe

low

:C

anyo

ude

term

ine

the

valu

eof

???

Todo

this

One

have

tode

term

ine

how

Yde

pend

sup

onat

trib

utes

x1,

x2,x

3,x4

,x5

y=

f(x1

,x2,

x3,x

4,x5

)

Wha

tabo

ut

f=

mod

(�w

i×

x i,2

)

with

(w1,

w2,

w3,

w4,

w5)

=(0

,1,0

,1,0

)

Mac

hine

Lear

ning

(BIT

SF4

64)

MW

F(1

0-11

AM

)onl

ine@

BIT

S-P

ilani

Lect

ure-

05(J

an27

,202

1)2

/7

Let’s

chan

geou

rfoc

usa

bit

Inpu

t:x

Out

put:

yTr

aini

ngda

ta:(x

(1) ,

y(1)),(x

(2) ,

y(2)),

...,(

x(m) ,

y(m) )

Targ

etfu

nctio

nf:x→

yH

ypot

hesi

sh:

x→

y

Inst

ead

ofre

port

ing

yle

t’sre

port

P(y)

Fora

give

nin

putx

,out

puti

sno

tTru

e/Fa

lse

But

,

P(T

rue)

=0.

3

P(F

alse

)=

0.7

Mac

hine

Lear

ning

(BIT

SF4

64)

MW

F(1

0-11

AM

)onl

ine@

BIT

S-P

ilani

Lect

ure-

05(J

an27

,202

1)3

/7

Bia

s:H

owto

com

eup

with

thes

eva

lues

Obs

erve

the

data

Ass

ume

data

for1

0co

into

sses

be-H

-H-T

-H-H

-T-T

-H-H

-H-

Now

agai

nth

eco

inis

tobe

flipp

ed,w

hati

sex

pect

edou

tput

?P(H

)=

0.7

and

P(T

)=

0.3

You

can

inco

rpor

ate

your

bias

Ass

ume

you

know

the

coin

was

unbi

ased

(and

you

have

ahi

ghco

nfide

nce

onth

is)

You

hypo

thet

ical

lyco

nsid

er10

0m

ore

flips

1 .B

eing

afa

irco

init

wou

ldgi

ve50

Han

d50

TS

oyo

ures

timat

efo

rpro

babi

lity

ofhe

adis

asbe

low

P(H

)=

(7+

50)/(1

10)=

0.51

8an

dP(T

)=

0.48

2W

hy10

0?w

hyno

t100

00?

OK

;ify

ouha

vem

ore

confi

denc

eon

bias

take

1000

0.1

Als

oca

lled

lapl

acia

nsm

ooth

ing

Mac

hine

Lear

ning

(BIT

SF4

64)

MW

F(1

0-11

AM

)onl

ine@

BIT

S-P

ilani

Lect

ure-

05(J

an27

,202

1)4

/7

Bay

esTh

eore

mE

xam

ple:

Thre

eco

mpa

nies

A,B

and

Cm

akes

35%

,35%

and

30%

ofal

lthe

lam

psin

mar

ket.

Pro

babi

lity

ofth

eirl

amp

bein

gde

fect

ive

is1.

5%,1

%an

d2%

resp

ectiv

ely.

Wha

tis

the

prob

abili

tyth

ata

rand

omly

sele

cted

defe

ctiv

ela

mp

was

man

ufac

ture

din

fact

ory

C?

Bay

esTh

eore

m

P(A

|B)=

P(B

|A)P

(A)

P(B

)

Prio

r,po

ster

ior,

likel

ihoo

dan

dev

iden

ce(P

(A),

P(A

|B),

P(B

|A),

P(B

))

Wha

twe

wan

tis

P(C

|D)

P(D

)=

P(A

)P(D

|A)+

P(B

)P(D

|B)+

P(C

)P(D

|C)=

.35∗0

.015

+.3

5∗0

.01+

.3∗0

.02=

0.01

475

P(C

|D)=

P(D

|C)P

(C)

P(D

)=

0.02

∗0.3

0.01

475

=0.

407

Mac

hine

Lear

ning

(BIT

SF4

64)

MW

F(1

0-11

AM

)onl

ine@

BIT

S-P

ilani

Lect

ure-

05(J

an27

,202

1)5

/7

Pro

babi

lity

ofob

serv

ing

ada

tase

t

Ass

ume

you

are

flipp

ing

abi

ased

coin

whe

rep(

H)=

0.4.

Wha

tis

the

prob

abili

tyth

atyo

use

eth

isda

tase

tD=<

H,H

,T,T

,H,H

>

p(H)=

0.4

p(T)=

1−

p(H)=

1−

0.4=

0.6

Ifal

lthe

trai

lsar

ein

depe

nden

tthe

np(

D|θ)

=p(

H)×

p(H)×

p(T)×

p(T)×

p(H)×

p(H)

=0.

44×

0.62

=0.

0092

16

Not

e:O

rder

ofel

emen

tsin

the

data

setd

ono

tmat

teri

nth

etr

ial.

So

p(<

H,H

,H,H

,T,T

>)

issa

me

(infa

ctan

yot

herp

erm

utat

ion)

Wha

tisθ

Itis

the

para

met

er.

Foro

urca

seit

repr

esen

tsp(

H)=

0.4

Mac

hine

Lear

ning

(BIT

SF4

64)

MW

F(1

0-11

AM

)onl

ine@

BIT

S-P

ilani

Lect

ure-

05(J

an27

,202

1)6

/7

Than

kYo

u!

Than

kyo

uve

rym

uch

for

your

atte

ntio

n!

Que

ries

?

Mac

hine

Lear

ning

(BIT

SF4

64)

MW

F(1

0-11

AM

)onl

ine@

BIT

S-P

ilani

Lect

ure-

05(J

an27

,202

1)7

/7