aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

36
w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w w ¢sQJaV„alZ JpNQ ·;gealZ £fissp¡“ iaJ^;Qg ~;£ealL ¡;£eal@jJJjQw¡fi MQs“w pV I¢L '› ifilaJ^ :s¡ag qL ‹…qm iw ~;£eal t'›iu ¢sQJaV„alZ JpNQ ·;gealZ £fissp¡“ :s¡ag qL ‹…qm q§ƒ

Transcript of aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

Page 1: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

XXX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

X

aT2+B7vBM; +Q/2 r�HFBM; bmTTQ`i

JB+?�2H _�bFBM- `�bFBM!K++K2X`m

.2TiX Q7 *a- hl JmMB+?

�T`BH R- kyRN

JX _�bFBM UhlJV aT2+B7vBM; +Q/2 r�HFBM; bmTTQ`i �T`BH R- kyRN R f e

Page 2: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

XXX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

X

1ti2M/BM; *QKKQM GBbT, h?2Q`v

*QKKQM GBbT Bb � T`Q;`�KK�#H2 T`Q;`�KKBM; H�M;m�;2

J�DQ` H�M;m�;2 72�im`2b +�M #2 �//2/ pB� K�+`Qb

LQ HBKBib ě +Q/2 Bb /�i�5

*QM7Q`KBM; +Q/2 Bb TQ`i�#H2 #2ir22M BKTH2K2Mi�iBQMb

JX _�bFBM UhlJV aT2+B7vBM; +Q/2 r�HFBM; bmTTQ`i �T`BH R- kyRN k f e

Page 3: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

XXX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

X

1ti2M/BM; *QKKQM GBbT, S`�+iB+2

*GPa r�b � HB#`�`v*2HHb- *QMi2tiG

A7 M2+2bb�`v- QM2 +�M +Q/2 r�HF5Ai2`�i2

AKTH2K2Mi�iBQM@bT2+B}+ +Q/2 r�HFBM; +Q/2 #Bi`Qib

�;MQbiB+@GBx�`/, kyRd- TQ`i�#H2 ě biBHH MQi iQQ H�i2 iQ i?2 ;�K2\h2bi bmBi2 iQ #`2�F i?2 Qi?2` TQ`i�#H2 +Q/2 r�HF2`bĘ

JX _�bFBM UhlJV aT2+B7vBM; +Q/2 r�HFBM; bmTTQ`i �T`BH R- kyRN j f e

Page 4: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

XXX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

X

1ti2M/BM; *QKKQM GBbT, S`�+iB+2

*GPa r�b � HB#`�`v*2HHb- *QMi2tiG

A7 M2+2bb�`v- QM2 +�M +Q/2 r�HF5Ai2`�i2

AKTH2K2Mi�iBQM@bT2+B}+ +Q/2 r�HFBM; +Q/2 #Bi`Qib

�;MQbiB+@GBx�`/, kyRd- TQ`i�#H2 ě biBHH MQi iQQ H�i2 iQ i?2 ;�K2\h2bi bmBi2 iQ #`2�F i?2 Qi?2` TQ`i�#H2 +Q/2 r�HF2`bĘ

JX _�bFBM UhlJV aT2+B7vBM; +Q/2 r�HFBM; bmTTQ`i �T`BH R- kyRN j f e

Page 5: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

XXX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

X

*Q/2 r�HFBM; bmTTQ`i

ai�M/�`/ Bb MQi 2MQm;?QT�[m2 H2tB+�H 2MpB`QMK2Mib

*QKKQM GBbT, i?2 G�M;m�;2 Uk 2/XV Bb 2MQm;?

AKTH2K2Mi�iBQMb ?�p2 2MQm;? 7mM+iBQMbĘ rBi? mMB[m2 M�K2bAMi2`2biBM; 2tT�MbBQMb Q7 bi�M/�`/ K�+`Qb

JX _�bFBM UhlJV aT2+B7vBM; +Q/2 r�HFBM; bmTTQ`i �T`BH R- kyRN 9 f e

Page 6: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

XXX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

X

ai�M/�`/ K�+`Qb

U/27mM 7 UtV URY tVVBb +m``2MiHv �HHQr2/ iQ 2tT�M/ iQ �Mv Q7 i?2 7QHHQrBM;,

c *GAaSUBKTH,,/27BM2@7mM+iBQM

^7 U7mM+iBQM 7 ^UH�K#/� UtV U#HQ+F 7 URY tVVVVVc a"*GUBKTH,,/27BM2@7mM+iBQM

^7 UBKTH,,M�K2/@H�K#/� 7 UtV U#HQ+F 7 URY tVVVVc SH2�b2 MQc "QMmb ĺMQĻ B7 /27BM2@7mM+iBQM Bb bT2+B�H QT2`�iQ`UBKTH,,/27BM2@7mM+iBQM ^7 ]t @= URY tV]Vc SH2�b2\UBKTH,,/27BM2@7mM+iBQM

^7 UH�#2Hb UU7 UtV URY tVVV U7mM+iBQM 7VVV

JX _�bFBM UhlJV aT2+B7vBM; +Q/2 r�HFBM; bmTTQ`i �T`BH R- kyRN 8 f e

Page 7: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

XXX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

XX

X

*�M r2 �;`22 QM r?�i BKTH2K2Mi�iBQMb +�M bB;M�H\

*._\

*._@LL T�+F�;2 7Q` M2+2bb�`v 7mM+iBQM�HBiv\1Bi?2` rBi?@�m;K2Mi2/@2MpB`QMK2Mi- Q` �m;K2Mi@2MpB`QMK2Mi- Q`2MpB`QMK2Mi@2Mi`v@M�K2b- Q` K�+`Q2tT�M/@�HHX

,*._@LL@1sS�LaAPLa 72�im`2 7Q` bi�M/�`/@+QM7Q`KBM; +Q/2 BM i?22tT�MbBQMb\

:Q�Hb,lMB}2/ M�KBM; 7Q` 7mM+iBQMfK�+`Q �HB�b2b ě +?2�T 7Q` BKTH2K2Mi�iBQMb6mHH +Q/2@r�HFBM; 7`B2M/HBM2bb ě +?2�T 2MQm;? iQ ?�p2 � +?�M+2Ę

*m``2Mi /`�7i, ?iiTb,ff;BiH�#X+QKKQM@HBbTXM2ifK`�bFBMf+/`@r�HF�#BHBiv

JX _�bFBM UhlJV aT2+B7vBM; +Q/2 r�HFBM; bmTTQ`i �T`BH R- kyRN e f e

Page 8: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

A SHORT NOTE ON

TRIES AND COMPRESSED

DATA Structures

- ANDREWLAWSON

(P&A#EoPA¥

Page 9: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

A - B - C- > Resort

A - D - C → Result ②

A - B - D - ( → Result③

WE HAVE SEQUENCES OF Tokens

THAT INDICATE A PARTICULAR RESULT .

Page 10: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

,C

'

/B

- D- C - ③A

\D

\C- ②

WE BUILD A TM2 AND CAN

EASILY WRITE A Function THAT

CHECKS For MATCHES

Page 11: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

C is REQUIRED

,①

trc

/B

- D- C - ③A

\D

\C- ②

CAN WE Phone THE TREE?

Page 12: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

AT ANY NODE WE CAN

ENCODE THE SYMBOLS NEEDED

IN THE FOLLOWING NODES IN

A BIT AMAT .WITH 100,000

SYMBOLS WE BUCKET BY FIRST

LEITER,

STICKING TO 4

LETTERS WE GET→

Page 13: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

,

0010 /(

• oio

/B 0010

- D- C - ③A

,c. ②

WORK OUT THE BIT ARRAY For

THE CANDCDATE SEQUENCE

A- B - C ⇒ # ( I 110 )

Page 14: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

But WE DON'T WANT TO

ONLY

BUCKET Byv First LEITER ,

.

IN THE REAL WORLD WE

ARE LIMITED BY MEMORY FOR

B. IT VECTOUS,

OR THE INTEGER

SIZE LIMITS

Page 15: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

→ Bloom FILTERS

( New Discovery FOR ME,

I'm NOT A COMPUTER

SCIENTIST )

Page 16: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

A Bloom filter is a space-efficient probabilistic data structure, conceived by Burton Howard Bloom in 1970, that is used to test whether an element is a member of a set. False positive matches are possible, but false negatives are not

WIKIPEDIA

L

WE REPRESENT our Bloom FILTER

As A BIT ARRAY AND A SET OF

HASH FUNCTIONS

Page 17: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

ADDIOC AN ELEMENT :

WE APPLY OUR HASH Functions

And SET THE INDICATED Bets

ITEM

A ten .

✓ v s

000 I 000 I 0000I 000

Page 18: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

① UERYIOG AN ELEMENT

Run THE HASH FUNCTIONS AGAIN

NOT ALL BITS SET → DEFINITELY

NOT N THE SET

ALL Bits SET →PROBABLY

#

InTHE

SET

* VARIES DEPENDING ON TOO . Bits,

No . HASH FUNCTIONS

Page 19: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

So :

COMPACT REPRESENTATION Using-

SAME COMPARISON LOGIC Etc

- FALSE Positives ARE FINE,

JUST

MEANS THAT WE PASS To THE NEXT

NODE

- CAN NAVE A VERY GRANULAR FILTER

- Now Working ON TUNING of

MEASURING

Page 20: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

But :

SURELY THERE ARE BETTER WAYS

To Do ALL OF This CTREE PRUNING ).

Page 21: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

REMINDER :

###U Is HIRING

MULTIPLE Lisp PROGRAMMERS

RIGHT Now

→ COME TALK To

( OR ANDREW WICK KYLE )

Page 22: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

C O M M O N L I S P F O U N D AT I O N U P D AT E 2 0 1 9

D AV E C O O P E R < D AV E @ G E N . W O R K S >

Page 23: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

B A C K G R O U N D

• Ten years of advocating Common Lisp (founded 2009)

• Volunteers working in “copious” spare time to promote Common Lisp

• Provides resources for development common-lisp.net including GitLab, mailing lists, project pages, and continuous integration

Page 24: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

C O M M O N - L I S P. N E T

• Complete site revamp (late 2018)

• Engaged a team of volunteers spearheaded by Marinano Montone (over 72 discrete contributions)

• Site built automatically from every commit to the repository

• Eating our own dogfood: transitioned from Ruby tooling to Common Lisp site generator

• Test server is using portable Allegro Serve

Page 25: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

F U N D R A I S I N G

• ASDF Appreciation Fundraiser

• Sponsors contributed $5000 matching funds

• Currently achieved over half this goal

• Fundraiser has been back online for ELS2019

• https://payments.common-lisp.net/asdf

Page 26: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

F U T U R E D I R E C T I O N S

• Fundraising

• We would like to allow each project hosted with common-lisp.net to activate their own fundraiser

• Services

• Would like to provide convenient ways for projects to utilize continuous integration services across CL implementations

• Hosting of source code escrow with particular support for Common Lisp based projects and products

Page 27: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

Vsevolod Dyomkin

European Lisp Symposium ’2019

Page 28: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

Inspiration

Page 29: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

But only* text-based* Emacs-based

Not a poor-man’s REPL, butA robust history recorder

Page 30: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

slime-history.eld- custom sexp-based format – not human-readable;; -*- coding: utf-8-unix -*-;; History for SLIME REPL. Automatically written.;; Edit only if you know what you're doing("(push \"~/prj/lisp/cl-nlp/\" asdf:*central-registry*)(push \"~/prj/lisp/wiki-lang-detect/\" asdf:*central-registry*)(push \"~/prj/lisp/crawlik/\" asdf:*central-registry*)(ql:quickload :cl-nlp)(ql:quickload :crawlik)(in-package :nlp-user)"…

Page 31: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

slime-history.eld- custom sexp-based format – not human-readable- not robust- hard to control- etc.

Page 32: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

Robustness Requirements

* recording history from concurrently running sessions* keeping unlimited history* identifying the time of the record and its context

Page 33: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

frlog* client-server application: CL server, EL/CL/… client* logs to human-readable *.md

Page 34: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

### nlp-user (14) 2018-10-23_19:09:21

(drakma:http-request "http://schema.org/" :accept "application/ld+json")

;;; 2018-10-23_19:09:27

((:ACCESS-CONTROL-ALLOW-ORIGIN . "*") (:CONTENT-TYPE . "application/ld+json") (:VARY . "Accept, Accept-Encoding") (:ETAG . "d61d9aceb39f0342f4dc4c54cb30baf4") (:LAST-MODIFIED . "Thu, 27 Sep 2018 13:34:07 GMT") (:X-CLOUD-TRACE-CONTEXT . "f1823f5633d337de3df2369fe1e86d1c") (:DATE . "Tue, 23 Oct 2018 16:08:59 GMT") (:SERVER . "Google Frontend") (:CONTENT-LENGTH . "136698") (:CACHE-CONTROL . "public, max-age=600") ...)

;;; 2018-10-23_19:09:27

200

;;; 2018-10-23_19:09:27

#<FLEXI-STREAMS:FLEXI-IO-STREAM {101D8B3AA3}>

;;; 2018-10-23_19:09:27

T

;;; 2018-10-23_19:09:27

#<PURI:URI https://schema.org/docs/jsonldcontext.json>

;;; 2018-10-23_19:09:27

"OK"

Page 35: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

.EL problemIt uses Emacs native indenting function:

(with-temp-buffer (lisp-mode) (insert text) (let ((inhibit-message t)) (indent-region 0 (point))) (string-trim (buffer-string))))))

But after some time we get:

error in process sentinel: url-http-idle-sentinel: Lisp nesting exceeds ‘max-lisp-eval-depth’error in process sentinel: Lisp nesting exceeds max-lisp-eval-depth

Page 36: aT2+B7vBM;+Q/2r HFBM;bmTTQ`i

Thanks! Read more

http://lisp-univ-etc.blogspot.com/2018/09/ann-flight-recorder-robust-repl-logging.html