Fork me on GitHub
#clojure
<
2019-01-02
>
gfredericks01:01:30

@rutledgepaulv if you're taking code as input at runtime, I think that's basically the definition of what eval is for

gfredericks01:01:26

if the code has unbound locals, which I think is what you're describing, I've used a trick to wrap it in a function that takes the named arguments you want to supply e.g.,

(eval `(fn [{:keys [~@some-arg-names]}] ~the-expression-with-unbound-locals))

gfredericks01:01:42

then you can call that function with a map

rutledgepaulv01:01:52

I think that would do the trick. thanks for the idea!

rutledgepaulv01:01:33

should’ve known I was trying to do too much in one macro + eval instead of just using a function lol

rutledgepaulv02:01:32

thanks @gfredericks that allowed me to do what I was trying to.

rutledgepaulv02:01:07

I’m not actually trying to rebind merge to inc.. just an example. lol

rutledgepaulv02:01:37

though a “guess what function I replaced your original function with” game sounds fun

vemv03:01:18

Is there something like https://github.com/lightyear/sql-logging#options but for Clojure? tldr it aides you see if you have costly SQL being emitted, and actually points out the culprit LOC

vemv03:01:28

On second thoughts, probably this library wouldn't make that much sense in Clojure, given "magical" ORMs aren't the norm here... so emitted SQL should have no surprises ...but still I can see myself using some simple metrics, e.g. query x took 30ms

seancorfield03:01:51

@vemv I believe there are Java libraries that can act as a proxy for the actual JDBC driver and provide that sort of statistical logging.

vemv03:01:30

thanks! will look into it

vemv03:01:53

Fantastic. TIL 🙂

seancorfield03:01:45

If you have any problems integrating it with clojure.java.jdbc, ping me in the #sql channel and I'll see if I can help.

rutledgepaulv03:01:24

@vemv you can always just hook the function yourself too if you can find the place you want to instrument

vemv03:01:34

Handy, thanks! 🍻

rutledgepaulv04:01:11

np! and if you want to see what code led to it you could create an exception and parse the stack trace (or I think later versions of java have mechanisms to reflect on the stack without making exceptions). Obviously this solution has potential issues with laziness or the arguments not containing the info you want, but it’s often a quick easy option for local dev and can be applied to almost any clojure library.

vemv04:01:05

Got it, good call!

kah0ona09:01:39

why does (read-string "07") return 7, but (read-string "08") throws an exception?

Lennart Buit09:01:03

a leading zero often means that the following number is octal

Lennart Buit09:01:12

8 however, is not a valid literal in octal

Lennart Buit09:01:44

(read-string "010") is also 8 (base 10) instead of the 10 (base 10) you would have expected

Lennart Buit09:01:55

WHY they decided to have 0 be the prefix for octal instead of 0o is beyond me. As the hex/binary prefix is 0x and 0b respectively…

seancorfield16:01:43

Because everything dating back to C did it. Technically back to B, C's predecessor. B's predecessor BCPL used # for octal but B used that as an operator. So leading 0 means octal in almost every language out there today.

Lennart Buit16:01:38

Yeah, but, “because language X did it” is not a technically sound argument IMO 😉

seancorfield16:01:52

Octal was much more common back in the day. I worked with octal a lot. Then later "graduated" to hex. So those early conventions were very strong for a generation of programmers, maybe even two generations. And those were the folks that designed and built all the languages we use today.

Lennart Buit16:01:48

Right, it makes sense. But endlessly sticking with the convensions of your predecessors ends up with situations where newer developers wonder why a leading zero changes read-string to a base that is hardly used anymore.

Lennart Buit16:01:21

At some point, someone needs to decide that that is crazy, and throw it out of (newer) languages

danm16:01:11

Was there ever an A?

seancorfield16:01:56

BCPL -> B -> C. There could have been a P but instead we got C++ and D. Although we did also get PL/1. Programmers are such wits 🙃

seancorfield16:01:44

Since BCPL was influenced by CPL which in turn was influenced by ALGOL 60, I guess you could say A was ALGOL? 😀

seancorfield16:01:11

I wonder if I'm the only one here who has programmed in BCPL and ALGOL 60? 😕

danm16:01:03

I did C and Java at Uni. Nothing older than that

jaide16:01:43

I played with ASM a bit, I think that’s the oldest language I’ve worked with.

Lennart Buit17:01:54

(Fun fact, Rust, which is a relatively young language, chose to adopt 0o for its octal syntax)

andy.fingerhut18:01:12

So obviously people are not "endlessly" sticking with conventions of predecessors?

seancorfield18:01:03

Using 0o, 0b, 0x as prefixes is certainly more consistent.

Lennart Buit20:01:45

Haha, you are right andy. I actually made my statement before looking up whether newer gen languages would have adapted. As often, Rust serves as a positive surprise

Lennart Buit20:01:28

Swift also goes for 0o whereas Go goes for the traditional 0. Interesting

danm09:01:16

Is leading 0o used for anything else? It would be nice if we could see older languages adopting that pattern too. At least then if 0o and 0 both represented octal people could start writing code where it was clearer that a number was intended to be octal

Lennart Buit09:01:48

I would say that alternatives in syntax are actually worse than having inconsistent, but only one, syntax

danm09:01:33

Yay Java. I remember that one hitting me way back when I was doing Java for my degree...

seancorfield16:01:43

Because everything dating back to C did it. Technically back to B, C's predecessor. B's predecessor BCPL used # for octal but B used that as an operator. So leading 0 means octal in almost every language out there today.

valerauko17:01:16

if i have an input array of strings, and i want to pick those that are also in a predefined set of valid strings, is it better to use (filter #{"valid1" "valid",,,} input) or some other way?

👍 10
Lennart Buit17:01:47

I think with the set is quite elegant

Lennart Buit17:01:07

def the set of accepted strings somewhere, or make a predicate is-allowed-string? I guess

valerauko17:01:44

yeah that's the plan. i was wondering if for example set/intersection would be a better choice?

valerauko17:01:51

the result would need to be all unique too, so i'd need to fit a distinct in there

Lennart Buit17:01:53

intersection would work too if your input is a set

Lennart Buit17:01:34

(remember, all set operations are explicitly undefined for non-sets)

valerauko17:01:09

i'd need to cast the input to set, making me again wonder if that's faster or running distinct on it

jaihindhreddy-duplicate17:01:47

Benchmark it with criterium

valerauko17:01:59

timing it now

👍 5
jaihindhreddy-duplicate17:01:38

Unless performance is a bottleneck, pick the appropriate semantics.

jaihindhreddy-duplicate17:01:58

i.e, if order doesn't matter and you don't care about duplicates

jaihindhreddy-duplicate17:01:06

use clojure.set/intersection

valerauko17:01:25

in the normal case it isn't (as only 3-4 elements should be in the input), but i'd rather not accidentally create an attack vector

jaihindhreddy-duplicate17:01:35

else (filter #{"input1" "input2" ...} input)

Lennart Buit17:01:54

distinct has an … interesting implementation

valerauko17:01:53

this is interesting

valerauko17:01:26

user=> (time (->> input distinct (filter valid)))
"Elapsed time: 0.229339 msecs"

user=> (time (->> input distinct (filter valid) sort))
"Elapsed time: 0.581802 msecs"

user=> (time (set/intersection valid (set input)))
"Elapsed time: 0.219935 msecs"

user=>   (time (sort (set/intersection valid (set input))))
"Elapsed time: 0.199401 msecs"

valerauko17:01:11

why is sort so slow? the resulting filtered collection in the second case is only 4 elements, why is it taking such a toll?

Alex Miller (Clojure team)17:01:20

you have to be careful on some of those - filter is lazy and you aren’t forcing realization

Alex Miller (Clojure team)17:01:29

so the first timing isn’t doing the filter at all

Alex Miller (Clojure team)17:01:13

all of the other ops here are eager so will force realization

Alex Miller (Clojure team)17:01:53

timing a single instance of any of these is also likely meaningless

valerauko17:01:55

will into serve that purpose?

Alex Miller (Clojure team)17:01:39

but really you should time 1000s of instances of these ops to compare

Alex Miller (Clojure team)17:01:02

like (dotimes [i 5] (time (dotimes [i 1000] (->> input distinct (filter valid) sort))))

Alex Miller (Clojure team)17:01:28

the outer loop is 5 trials, the inner loop is 1000 reps per trial

Alex Miller (Clojure team)17:01:38

you’ll usually notice it gets faster as the jit warms up

Alex Miller (Clojure team)17:01:56

critierium is a fancier version of this

valerauko17:01:49

thanks for the advice!

valerauko17:01:43

using sets seems to be significantly faster

valerauko17:01:51

like 3-4 times

Lennart Buit17:01:34

hah, I did not expect that. I expected converting to a set to be a pricey operation

valerauko17:01:08

i tried generating the input randomly so the jvm would sweat a little

valerauko17:01:58

user=> (time (dotimes [m 10000] (->> (input) distinct (filter valid) sort)))
"Elapsed time: 693.849881 msecs"
user=> (time (dotimes [m 10000] (sort (set/intersection valid (set (input))))))
"Elapsed time: 279.379693 msecs"
i didn't expect this big a difference

valerauko17:01:07

moving distinct after the filter makes it a little faster but still twice as slow

Noah Bogart17:01:04

input being a list?

Noah Bogart17:01:16

try making it a set in the first one, i wonder if that'll speed it up at all

valerauko18:01:15

using set instead of distinct made it 2-3 times faster

Noah Bogart18:01:30

i suspect using the filter set predicate is using some sort of set-based stuff under the hood that makes comparisons to other sets much faster than comparing against a list

Noah Bogart18:01:38

the only issue is if your input has duplicates

Noah Bogart18:01:45

and you need them for some reason

valerauko18:01:01

no, i need it to be distinct so it's perfect

valerauko18:01:43

user=> (time (dotimes [m 10000] (->> (input) set (filter valid) sort)))
"Elapsed time: 272.51252 msecs"
user=> (time (dotimes [m 10000] (->> (input) set (set/intersection valid) sort)))
"Elapsed time: 271.146103 msecs"
basically no difference

seancorfield18:01:05

@vale Just FYI, (time (dotimes ..)) doesn't always give you accurate benchmarks -- check out criterium for a better approach.

seancorfield18:01:36

(uh-oh, is Alex going to correct me?)

Alex Miller (Clojure team)18:01:01

with a big enough N and enough sample points, dotimes is sufficient, imo

👍 5
phill23:01:57

And criterium is the agreed definition of "a big enough N and enough sample points" !

noisesmith23:01:49

the fact that it loudly warns you if JIT is turned off is also a bonus

Alex Miller (Clojure team)23:01:35

I find the benefit of NOT using criterium is that you can see when the jit is sufficiently warm when the sample stabilize

Alex Miller (Clojure team)23:01:53

generally this is much faster than using criterium

Alex Miller (Clojure team)23:01:17

I also don’t find that usually the gc stuff in criterium is worth much

Alex Miller (Clojure team)23:01:31

and if you don’t use leiningen you never run into the jit thing :)