Clojurians Log v2

I'm not an expert, but I was playing with packaging all of this work up, and tools.build skipped my sha-based sci because it expects me to have a version ... any timeframe on some of these recent changes being included in a release version of sci, @borkdude ?

define "packaging up"?

Sorry - I was trying to package this stuff in a jar (using tools.build) (having the following dependency in my deps.edn: org.babashka/sci {:git/url "" :git/sha "fcf7e6067f091fd2cf2843660349e820c4449dd8"}) and I saw a message I'd never seen before:

Skipping coordinate: {:git/url , :git/sha fcf7e6067f091fd2cf2843660349e820c4449dd8, :deps/manifest :deps, :deps/root /Users/pmooser/.gitlibs/libs/org.babashka/sci/fcf7e6067f091fd2cf2843660349e820c4449dd8, :parents #{[]}, :paths [/Users/pmooser/.gitlibs/libs/org.babashka/sci/fcf7e6067f091fd2cf2843660349e820c4449dd8/resources /Users/pmooser/.gitlibs/libs/org.babashka/sci/fcf7e6067f091fd2cf2843660349e820c4449dd8/src]}

And reading the source code of tools.build it appears that it does that if an artifact doesn't have a mvn/version ... so that meant it wasn't clear to me how I was supposed to include (read: depend upon) such an artifact in a build.

You mean build an uberjar right? I think tools.build should be able to do that? Ask in #tools-build first maybe

I could just release a version, but I think it's better to fix things at their root cause and being able to use SCI as a git dep will be better for quicker iteration

Sure thing. Happy to give that a try. Thanks.

Started some work on async eval for CLJS (based on promises). Here are some tests: https://github.com/babashka/sci/blob/promised-eval/test/sci/async_test.cljs

How safe is SCI’s default eval-string (with no additions) for the purposes of running completely untrusted code?

@devn Pretty safe, there is no file system access, no Threads. The only unsafe thing is that it might never terminate since it's turing complete

(sci/eval-string "#=(clojure.core/list clojure.core/- #=(clojure.core/+ 1 5) 2 3)") ran, which I guess I was just a bit surprised at

interesting, I'm surprised too :)

out of curiosity, does sci expose something like a “safe” read-string?

edamame does: https://github.com/borkdude/edamame - this is used in SCI. I think the read-eval thing should probably be turned off by default

Hmm:

:all true
:read-eval false

even with read-eval here, it still shouldn’t really matter much with no fs access, etc. — i think?

are the default options used. I'll see what will happen. Yes, that's true

user=> (e/parse-string "#=(+ 1 2 3)" {:all true})
(read-eval (+ 1 2 3))
user=> (e/parse-string "#=(+ 1 2 3)" {:all true :read-eval false})
Execution error (ExceptionInfo) at edamame.impl.parser/throw-reader (parser.cljc:44).
Read-eval not allowed. Use the `:read-eval` option

since im running like 100k untrusted expressions, i am just dotting my i’s. frankly sci seems safer than when i ran these in clojail. i had little java swing windows popping up on my VM and such. 😄

haha

Oh I see what happened. Support for *read-eval* got added later and this enabled that option:

user=> (sci/eval-string "(binding [*read-eval* false] (read-string \"#=( 1 2 3)\"))")
Execution error (ExceptionInfo) at sci.impl.parser/throw-eval-read (parser.cljc:87).
EvalReader not allowed when *read-eval* is false.

but I can expose sci/read-eval so you can bind that to false

@devn fixed on SCI master: *read-eval* defaults to false now and you can do:

(sci/binding [sci/read-eval true] ...)

very nice. thank you!

fwiw, i just got it back into a deployed state at http://getclojure.org search is not as good as it once was when i was using elasticsearch with my own custom tokenizer/analyzer combo. certainly a lot of junk sexps in there, but occasionally there’s something interesting

awesome! and this is eval-ed using SCI right?

lol, yes I can see that here:

(fn [n]
  (let [zeros (vec (repeat n 0))]
    (map #(assoc zeros % 1 (inc %) 1) (range (dec n)))))
#object[sci.impl.fns$fun$arity_1__2260 0x2a5b0042 "sci.impl.fns$fun$arity_1__2260@2a5b0042"]

@devn Maybe examples that return functions aren't so interesting to show?

yeah, all of this was eval’d in SCI yeah i tend to agree on examples returning functions. there’s also some pretty clear cases where clojurebot macroexpanded an expression for someone, so you wind up with a giant block of (clojure.core/list (clojure.core/+ (clojure.core/list …)))

the behavior in elasticsearch was “better” insofar as the complexity of the input seemed to correspond to it’s ranking, so page one for a search of comp would yield boring things like (comp comp comp comp comp) , but the last page would have longer examples. im not sure how the ranking is happening here but it’s a little odd

like, i have no idea why this query http://getclojure.org/search?q=%2B%2B%2B%2B%2B&num=0 returns the example (list 1 2)

why not use elasticsearch then?

i mean, i don’t know if elasticsearch would be better here on a more recent version. the last time i used it was like 6 major versions ago. i started working on setting it up, but found the docker setup for it to be annoying enough that i figured i’d happy path it on algolia first

so in short: laziness 😄

sure :)

and well, i’ve had good experiences with algolia for business-type applications, but this is a weird one because it’s like “no really, i want users to be able to search for .”

so a lot of the defaults probably just need to be removed

https://grep.app/ does allow you to search for anything, basically like grep

oooo!

haven't found out if that's open source - seems not

the original idea was to try and come up with some kind of hoogle-type search for clojure. this is WELL short of that. do you recall crossclj DOT info by any chance? it was around years ago and it was indexing lots of clojure repos and let you click on a function from another lib and hop around like a graph

it’s been dead for awhile, but it was a cool project

I've made a hoogle like thing here: https://borkdude.github.io/re-find.web/?args=2%20%5B%3Aa%20%3Ab%20%3Aa%20%3Ab%5D&ret=%5B%3Aa%20%3Aa%5D I remember crossclj - I actually wanted to remake that using clj-kondo analysis output, it contains all the info necessary for that

oh hey, that’s really cool!

I wouldn't mind if someone stole my idea of using clj-kondo analysis to re-build cross clj

or collaborate on that with someone else

https://github.com/clj-kondo/clj-kondo/blob/master/analysis/README.md

this is less interesting than the spec thing, but https://github.com/Raynes/findfn/blob/master/src/findfn/core.clj wasn’t actually all that bad

yeah, that was one of the inspirations as well. I explained in my talk why re-find was better since it excluded non-sensical suggestions like using merge on a number

yeah, that was definitely the brute force attempt

but still a very cool project. Raynes made a lot of original things

but it ran pretty fast despite that

yeah, sad he’s not around. hard to believe he was 16 back when he started with clojure

he was a badass and also a very nice person

yeah, 100%

it seems he passed away so suddently, don't really know any details

i know some folks in the community reached out to his family, but i don’t know any details, and in the end, the result is the same anyway

re-find.web is really neat

continuing the off-topic theme briefly, was pleased to see SCI managed to run this one. wasn’t sure if things in walk would work without explicit inclusion

(clojure.walk/postwalk
  #(if (coll? %) % (str %))
  '((E (C) (F I) F)
     (A A)
     H
     ((G) B)
     ((E G (I (I I A)) H I) (B) (A) C (D (H)))
     (F C C)))

=> (("E" ("C") ("F" "I") "F") ("A" "A") "H" (("G") "B") (("E" "G" ("I" ("I" "I" "A")) "H" "I") ("B") ("A") "C" ("D" ("H"))) ("F" "C" "C"))

there are a lot of examples which would probably run if they were transformed to include the likely ns. plenty of examples of people writing out (postwalk-replace …) unprefixed

you can make that work by evaluating a :refer [..] in the same context

@devn Can I share http://getclojure.org on Twitter or is it too early?

of course! there is some work to do yet, but contributions, feedback, ideas, etc. all welcome

(sci/eval-string "(print \"#=(spit \\\"foo.txt\\\" \\\"hello\\\")\")") Any idea why this fails to evaluate? (Note: I’m still on the version that allows read-eval)