Fork me on GitHub
#beginners
<
2020-01-31
>
emmanueloga09:01:41

https://lambdaisland.com/guides/clojure-repls ... super useful to untangle the repl mess πŸ™‚ I don't even know where I got that link from!

teodorlu10:01:54

That's a great resource!

feikas10:01:02

what does this mean |#<[email protected] Unbound: #'clj-journal.timbre/data_> . I just evaluated a var and this other vars commes properly.

andy.fingerhut10:01:48

One way to get a similar result is to do (def a)in a REPL, then try to evaluate a

andy.fingerhut10:01:06

I do not know if there are other ways to cause a var to be unbound.

bronsa11:01:22

most likely declare but yeah that's pretty much the same

feikas11:01:47

weird. I declare this var with def in source, but still not bound...

lucio12:01:09

Every time I get unbound I realize that the specific code with the var doesn’t run .. so the def is never set .. but yours might be a different scenario

feikas13:01:39

this is the case - I checked

feikas13:01:12

is there a String size limit in Clojure?

alexmiller14:01:01

well they're int indexed...

alexmiller14:01:33

user=> Integer/MAX_VALUE
2147483647

feikas14:01:51

same as in Java πŸ€”

alexmiller14:01:58

they are Java strings

alexmiller14:01:10

so, exactly the same :)

jaihindhreddy18:01:10

Unlike in other language ecosystems, I've noticed that in the Clojure ecosystem, fns don't tend to check for wrong args and throw InvalidArgumentExceptions. I've seen systems external to the fns' source code itself to do that ex: spec, schema, malli etc. The fns themselves are usually GIGO. You give 'em garbage, you get garbage back. Not saying Clojurists are indisciplined or something like that. Just that most fns don't do this, but validation is done "at the gate", or at the edges of the program. Is this a fair thing to say?

seancorfield18:01:31

@jaihindhreddy I don't know that you can make much of a generalization there. I've seen plenty of Clojure code that either assert stuff about arguments or explicitly checks them and throws IllegalArgumentException.

seancorfield18:01:14

I just searched the projects I have open in my editor right now and all of these do that to some degree: HoneySQL, Expectations, next.jdbc, clojure.java.data. I know that clojure.java.jdbc does it too. Our own code at work uses a mixture of assert and conditional IllegalArgumentException throwing in some places.

jaihindhreddy19:01:33

Thank you for doing that! I guess one thing that in no-small-part made me think way is nil-punning. Python for example, throws KeyError instead of just returning None.

seancorfield21:01:47

True, nil-punning means that a lot of times you don't need to check for/exclude nil arguments (which is a win, in my opinion).

seancorfield18:01:28

But that's going to produce a runtime failure either way so if an illegal argument value is going to cause your code to blow up anyway, there's not much point in explicitly checking that (and throwing) -- unless you really want to provide a "better" error message for incorrect usage of your functions πŸ™‚

ikitommi18:01:40

If there was an easy way to add functions schemas/specs in the language core, or in a de-facto library, I think we would see a lot more of those.

hindol.adhya18:01:18

I think that's true for all languages. Only that if there's a type system, it enforces some rules of its own.

ikitommi18:01:39

adding a dependency to 3rd party schema lib, which adds 1sec to repl startup time, is not something that lib authors want to do.

borkdude18:01:05

specs can be opt-in by providing them in a different namespace

borkdude18:01:10

and leave the loading/instrumentation to the lib user

borkdude18:01:57

but even then, it's not enforced so not something you will see in most cases

seancorfield18:01:43

(and instrumented specs can add quite an overhead at runtime so it's something you might see in dev/test but not in production)

borkdude18:01:02

(totally agree! even more so with core specs)

borkdude18:01:35

I had an Advent of Code problem with some hot loops which took 15 minutes instead of 2 seconds with instrumentation enabled for speculative specs πŸ™‚

andy.fingerhut18:01:25

There are significantly faster ways to do GI checking than spec, at least for some kinds of GI checking: https://github.com/jafingerhut/funjible#wait-isnt-this-what-clojurespec-is-for

andy.fingerhut18:01:37

The Garbage In part of GIGO

borkdude18:01:21

@andy.fingerhut btw, clj-kondo has some type checking around this too now:

andy.fingerhut18:01:23

Out of curiosity, have you ever tried clj-kondo on the Clojure code in its implementation?

feikas18:01:31

Hello, how would you escape %n in strings so when I call C code via JNA nothing crashes ?

andy.fingerhut18:01:54

Do you mean you want to pass a string to C code that contains the two character % followed by n ?

andy.fingerhut18:01:10

Neither of those should require special escaping that I can think of.

feikas18:01:11

yes exactly

feikas18:01:41

but it will go to printf and it interprets things some way

andy.fingerhut18:01:55

C code libraries often expect strings to be 8-bit, not 16-bit as they are in Clojure/Java, but I don't know what JNA might have in the way of translation of string contents there.

andy.fingerhut18:01:29

Well, 16-bit as they are in Clojure/Java, except when they are restricted to the ASCII subset, then JDK 9 and later will actually store them as 8-bit.

andy.fingerhut18:01:20

Oh, you want the C printf to get a format string such that it will output % followed by n ?

feikas18:01:28

like here https://github.com/runejuhl/clj-journal/commit/15b78e7f4c691bb61df18ce8814a2db087462ac4#commitcomment-37062267 it crashes, but the author escapes things with adding _ betwean those symbols

andy.fingerhut18:01:33

Then typically you escape the % using %%

feikas18:01:04

yes - I fust want to use printf and print the %n

andy.fingerhut18:01:57

If you want C printf to print out %n you should be able to do printf("%%n")

feikas18:01:26

thank you!

borkdude18:01:03

@andy.fingerhut Do you mean lint Clojure itself? or clj-kondo itself?

andy.fingerhut18:01:11

lint Clojure itself.

borkdude18:01:49

That happens on every CI build as a way of linting a corpus for checking regressions. Why you ask?

andy.fingerhut18:01:44

Just curious if you had done it. At least tools.analyzer.jvm that Eastwood uses, that was a big step making that not give any errors running on most/all clojure.core code.

borkdude18:01:28

yeah. I used a lot of false positives from clojure itself to improve clj-kondo in the beginning

andy.fingerhut19:01:32

In particular, one line of code in the Clojure.data namespace might cause that warning on the use of union you showed to fire.

borkdude19:01:54

Yeah, I think that also came up when speculative got specs for clojure.set: https://github.com/borkdude/speculative/issues/70#issuecomment-435185067

andy.fingerhut19:01:02

and now that I remember some discussions on speculative, I probably sound like a broken record there πŸ™‚

borkdude19:01:02

it seems clj-kondo doesn't catch that error, maybe because it has no type annotation for keys yet.. I'll check

borkdude19:01:16

I don't mind some repetition of interesting facts πŸ˜‰

borkdude19:01:15

ah, it has, but the return type is :seqable . clj-kondo has a somewhat relaxed type system, so when something can take the form of some valid input it doesn't complain. and keys can return nil which would be a valid input for set/union

borkdude19:01:33

I could change the return type to :seq and then it would work

borkdude19:01:45

although it wouldn't be entirely accurate...

andy.fingerhut19:01:24

I guess it depends on whether you avoid a warning when something might be given valid input, vs. when it must be given valid input

borkdude19:01:59

if you take the first route then you end up changing your code just to avoid type warnings

borkdude19:01:12

sorry, the second route

andy.fingerhut19:01:13

I know small changes to conditions for issuing warnings can change false positive rates in unexpected ways, until you try it on many code bases, sometimes.

borkdude19:01:42

it's kind of what you get with typescript where you must convince the compiler that some object has a certain key

borkdude19:01:03

you end up writing more code to make the system happy

borkdude19:01:35

but in practice, changing the return type of keys to :seq might not be that bad.

borkdude19:01:53

for the purpose of linting that probably just works fine

borkdude19:01:50

I'll try it and make a diff

borkdude19:01:06

works and no other regressions: https://github.com/borkdude/clj-kondo/runs/419883444 expand the diff thing

borkdude19:01:31

< clojure/data.cljs:52:39: warning: Expected: set or nil, received: seq.
< clojure/data.cljs:52:48: warning: Expected: set or nil, received: seq.

borkdude19:01:14

hmm, only .cljs though, that's a bit funny...

borkdude19:01:02

oh I see, clj-kondo skips over extend-protocol

didibus19:01:10

But keys can return nil or seq

didibus19:01:27

So isn't a return type of seq wrong?

borkdude19:01:51

yeah, it's wrong, but for the purpose of linting it's probably fine

borkdude19:01:26

the nil is the bit that breaks the type checking

didibus19:01:38

It might be better to warn if nil is passed to keys, then if it returns nil ?

didibus19:01:06

I feel the latter might give an error somewhere else and be more confusing no ?

didibus19:01:42

But I don't know, keys on an empty map seems common to me

didibus19:01:20

And I use keys often with map for example. So the nil punning seems really useful, so that if the map is empty then I don't need to loop over anything

borkdude19:01:42

an empty map is not the same as nil

didibus19:01:01

(keys {}) returns nil

didibus19:01:32

That's why I'm saying if the linting was changed to check that nil isn't passed as input to keys, but the return type still allowed nil

borkdude19:01:31

@ Can you come up with a practical example where a return type of keys being spec-ed as seq instead of seqable results in a linter warning?

borkdude19:01:52

I mean real code, not words

borkdude19:01:00

Relying on keys returning nil might even be relying on an implementation detail, since it's not even in the docstring

didibus19:01:31

No, I'm just thinking through this, no real example

didibus19:01:12

And, now that I think some more, I wasn't really thinking type checking but return type validation. In a type check scenario, I think you're probably right

borkdude19:01:31

yeah, it's statically checked where this probably works out

didibus19:01:52

A fucntion that takes seqable, if passed a seq, that's valid correct?

didibus19:01:31

Ya, so I guess all the scenarios I'm thinking of anyways would still work

borkdude19:01:25

The only example where this would not work is if you had a function that insists it be called with nil and nil only. Then (keys {}) would be valid, but not if you spec it as seq. But that seems very unlikely in Clojure.

didibus19:01:36

Ya, totally agree now. I really don't see why someone would ever use keys to a function that takes only nil.

didibus19:01:59

I was thinking it would fail when passed to a fn on seqable input, that was my only concern

borkdude19:01:23

I think in general it works out if functions are typed with correct inputs (broad enough) but somewhat too narrow outputs.

borkdude19:01:42

In static linting

borkdude19:01:00

Thanks for the discussion

didibus19:01:52

You probably have the most experience there, so I trust you in that observation

didibus19:01:33

It does make sense, especially in linting, since you can just ignore a few false positive in the rare edge case

borkdude19:01:09

if you spec a function with a too narrow output (a instead of a or b) and you feed the output into a function which expected a or b, it still works fine.

didibus19:01:45

right, makes sense

didibus19:01:02

If I had say a function, and it could return nil or a seq based on the implementation, and I said it returns a seq, that would warn ?

didibus19:01:25

Like say the last thing I return is the result of next

andy.fingerhut19:01:35

I am tossing in two cents based upon no real knowledge of clj-kondo internals, but I was under the impression that for certain core functions, borkdude simply tells it in some data structure "I claim that this returns type foo", and clj-kondo uses that, but does not check it.

borkdude19:01:53

"I claim that keys always returns a seq" that's how it works yes. And no code breaks on that assumption. If there would be code broken by this assumption, it's probably too specific

borkdude19:01:08

breaks as in false positive warnings

didibus19:01:31

Okay, I see for core. What about for my code being linted? Do you just check inputs as well? So if I return the result of next and I claimed my fn returns seq only, it would be valid?

borkdude19:01:27

return type checking isn't implemented yet

andy.fingerhut19:01:30

Is clj-kondo's mechanism for declaring function return types open to user extension?

borkdude19:01:44

it is, via a configuration format

didibus19:01:45

Just asking out of curiosity by the way, I don't have any grudge either way

borkdude19:01:47

that's how it works. not a lot of people are using this format already and I want to give it some time before committing to it, but it is open to configuration

borkdude19:01:49

There is one schema library (malli) which is going to try to generate this format from information that users provide via their defn-like macro

didibus19:01:50

Excited to see where this all goes, if more library authors start providing clj-kondo type configurations for their libs, linting would get really great

matthew.pettis20:01:53

Are there any recommendations on doing reinforcement learning on Clojure? My shallow search shows cortex, which hasn't been updated in 2 years, and the interop for Clojure in deeplearning4j. Are there others that one should be aware of?

andy.fingerhut20:01:22

I have not read this book, but the author has worked on libraries for using Clojure to generate code for GPUs, and I believe reinforcement learning is one of the use cases he covers: https://aiprobook.com/deep-learning-for-programmers/

matthew.pettis21:01:40

Thanks. Yep, I threw in to support the project, and I can now read his book. He's got general neural networks addressed, but upon first skim, I don't really see anything for reinforcement learning. Still looks promising for other areas I care about though...

andy.fingerhut21:01:27

I do not know how open the author is to suggestions for additional chapters, but you could try contacting him to see his plans for covering reinforcement learning, if any