Fork me on GitHub
#clojure
<
2017-07-13
>
cmal06:07:51

Hi, does task defined using mount (https://github.com/tolitius/mount) ‘s defstate and started using mount/start running in a new thread or in the main thread? Can I defstate a function that blocks the thread ? Thanks.

grav07:07:39

@hiredman and others: Thanks for the great hints regarding integration-testing and databases! I think docker sounds like an interesting approach.

schmee08:07:04

what do you have to do to make reader tags work?

schmee08:07:25

I have a data_readers.clj in my source dir which looks like {kh/test kleinheit.pg.impl/maybe-map}

schmee08:07:00

but when I try it out in the REPL I just get

dev=> #kh/test {:a 1 :b 2}

        java.lang.IllegalStateException: Attempting to call unbound fn: #'kleinheit.pg.impl/maybe-map
clojure.lang.LispReader$ReaderException: java.lang.IllegalStateException: Attempting to call unbound fn: #'kleinheit.pg.impl/maybe-map

cpmcdaniel12:07:09

so, I need a function that takes a nested data structure (only concerned with maps right now) like {:a {:b {:c 1} :d 2}} and returns all the paths to leaves: [[:a ::b :c] [:a :d]]… not sure where to start

schmee13:07:27

if you ask your question in #specter I’m sure someone will help you out

schmee13:07:48

I’d love to take a look but that will have to be later, I’m at work at the moment

raspasov14:07:43

I’ve been using Clojure for a long time now and one question randomly popped into my head:

raspasov14:07:19

is there ANY case where map destructuring of keys only can throw an exception?

raspasov14:07:40

I know that vector destructuring can throw all sorts of fun stuff 🙂

bfabry14:07:53

if you pass it something that's not a map

raspasov14:07:19

hm … 🙂 really?

raspasov14:07:23

=> (let [{:keys [x y]} 1]) nil

bfabry14:07:39

blows up on my version of clojure

moxaj14:07:42

(let [{:keys [a b]} 0]) throws Can't type hint a primitive local lol

raspasov14:07:11

woah… different in ClojureScript!

bfabry14:07:16

I (think) it's probably much more likely to blow up in 1.9 with spec too

bfabry14:07:31

huh, nope. interesting

raspasov14:07:34

latest ClojureScript

raspasov14:07:45

yes it blows up on Clojure JVM

raspasov14:07:55

ok that’s very interesting… good to know

bfabry14:07:57

yeah, blowing up is an implementation detail by the looks

bfabry14:07:05

(macroexpand '(let [{:keys [foo]} 1]))
=>
(let*
 [map__43828
  1
  map__43828
  (if (clojure.core/seq? map__43828) (clojure.lang.PersistentHashMap/create (clojure.core/seq map__43828)) map__43828)
  foo
  (clojure.core/get map__43828 :foo)])

bfabry14:07:47

sooooo maybe if clojure.core/get gets a spec it will start blowing up

raspasov14:07:26

that is so very interesting… it’s important to think about if your inputs are unpredictable/dynamic

bfabry14:07:57

if your inputs are unpredictable you're screwed, add a layer that makes them predictable 😛

raspasov14:07:23

well yea… but if {:keys []} was NEVER throwing

raspasov14:07:42

that layer could just be a (if (nil? x) …) check after

raspasov14:07:10

now it’s clear that you need a more “proper” check ala-spec, etc

bfabry14:07:23

yeah definitely, if you have uncertain input then use tools that are made for validating it as data (spec, schema, clojure data functions etc)

raspasov14:07:39

yea… I was just calling some JavaScript APIs from ClojureScript, so I assume everything there is unpredictable 😝

bfabry14:07:44

safe assumption

raspasov14:07:13

yea…. ok this is actually more interesting than I thought …

raspasov14:07:19

((fn [x] (let [{:keys [a b]} x] a)) 1)

raspasov14:07:29

so in this more… ahem realistic example

raspasov14:07:34

since we’re no longer destructuring a Primitive Local!

raspasov14:07:21

((fn [^long x] (let [{:keys [a b]} x] a)) 1)

raspasov14:07:37

throws… which is logical given the previous things we just tried

raspasov15:07:15

so actually… as long as: 1. the things you’re passing are not literals (which I’ve never done in real program) 2. and you don’t type hint an unpredictable dynamic thing (highly unlikely but possible to do) … it seems that {:keys […]} is actually “safe” from throwing exceptions

raspasov15:07:32

not saying that you should rely on it lol… I tried looking and I couldn’t find any place where it’s stated that it’s safe… so I assume it can be considered an implementation detail? someone pls correct me if you know better 🙂

bfabry15:07:41

if it's not documented otherwise then you should consider the behaviour of something when you pass it an unexpected value to be "undefined"

bfabry15:07:57

ie, will do any random ole shit, and will change between versions and builds and days

bfabry15:07:24

nature of the beast with a dynamic language

dpsutton15:07:07

i'm not sure its the keys part that is throwing. I think you're just doing something Clojure doesn't allow in typehinting :

user> (let [^long x 1]
        x)
CompilerException java.lang.UnsupportedOperationException: Can't type hint a local with a primitive initializer, compiling:(*cider-repl employee-resizer*:67:7) 

dpsutton15:07:15

it's not the keys, that's just not a valid place for a type hint

raspasov15:07:24

@dpsutton yes that’s correct

raspasov15:07:33

see my follow up

dpsutton15:07:35

ah ok. i thought you were still attributing it to the :keys destructuring

cpmcdaniel15:07:54

@schmee - I ended up using postwalk with core.match

schmee15:07:14

@cpmcdaniel nice! mind sharing the solution?

cpmcdaniel15:07:13

I suppose the map should be mapv

cpmcdaniel15:07:25

hmm, not sure how this is actually working without that 😉

cpmcdaniel15:07:33

oh, and the outer seq needs to be there because our tree may have multiple paths, duh

cpmcdaniel15:07:23

ok, I have a bug when adding another path in the tree. I’m flattening at the wrong place

schmee16:07:43

here’s my attempt with Specter:

(def TreeValues
  (s/recursive-path [] p
    (s/if-path map?
      [(s/collect-one s/FIRST s/FIRST) s/MAP-VALS p]
      s/STAY)))

schmee16:07:05

user => (s/select TreeValues m)
[[:a :b :c 1] [:a :b 2]]
user => (s/select TreeValues {:a {:fields {:b {:options [{:label "foo" :value "bar"}]}}}})
[[:a :fields :b :options [{:label "foo" :value "bar"}]]]

schmee16:07:39

I haven’t figured out yet how to not include the leaf itself

cpmcdaniel16:07:26

map over the results with butlast

schmee16:07:34

just noticed that mine is not correct either, I get [:a :b 2] instead of [:a :d 2]

madstap16:07:05

@cpmcdaniel Here's one implementation.

(defn all-paths [m]
  (letfn [(step [acc current-path x]
            (reduce-kv (fn [acc k v]
                         (let [path (conj current-path k)]
                           (if (map? v)
                             (step acc path v)
                             (conj acc path))))
                       acc, x))]
    (step [] [] m)))

(all-paths {:a {:b {:c 1} :d 2}}) ;=> [[:a :b :c] [:a :d]]

cpmcdaniel16:07:02

thanks, I’ll try that on my data set

schmee16:07:56

sweet, there you have a working version of what I was trying to do:

(def TreeValues
  (s/recursive-path [] p
    (s/if-path map?
       [s/ALL (s/collect-one s/FIRST) s/LAST p]
       s/STAY)))

user=> (map butlast (s/select TreeValues {:a {:b {:c 1} :d 2}}))
((:a :b :c) (:a :d))

qqq17:07:13

what is the best data format for exchanging data between clojure and python?

Alex Miller (Clojure team)17:07:15

depends on what your needs are, but you should look at https://github.com/cognitect/transit-format

Janet A. Carr18:07:18

hey, has anybody ever had trouble requiring clj-time.types in a ns? I have the latest version from clojars but I get an exception when I compile complaining the ns clj-time.types doesn't exist or can't be found on the class path.

Janet A. Carr18:07:46

the namespace clearly exists tho in the github repo

hiredman18:07:45

yeah, but the latest in git doesn't have to match the latest jar on clojars

hiredman18:07:03

there are also a few clj-time artifacts on clojars, are you sure you are using the one that maps to the github repo you are looking at?

hiredman18:07:51

you should check lein deps :tree

hiredman18:07:42

if any of your dependencies require one of these other clj-time artifacts (which unfortunately likely have the same namespace names) you will get all kinds of weird behavior

Janet A. Carr18:07:58

I followed the clojars link on github, maybe they just didn't push a newer artifact to maven?

Janet A. Carr18:07:27

that's exactly

hiredman18:07:28

could be, you also haven't shared the exception you are getting, clj-time.types could be failing to load because something is

Janet A. Carr18:07:01

Nah, the last push to clojars was in Dec. 2016, the change I want was added in march 2017

Janet A. Carr18:07:21

crisis averted, I guess.

seancorfield18:07:58

@janetacarr We'll be releasing a new clj-time very soon. I've been meaning to get to that for a week or two!

Janet A. Carr18:07:47

@seancorfield , oh cool, I didn't even realize you we're in this channel! 😛

seancorfield18:07:07

Here's all the fun new stuff https://github.com/clj-time/clj-time/compare/86acab8a122e3229a2e5337075f52b6f75ae4584...master -- I think I'll make it 0.14.0 when I do it (hopefully in the next few days).

kurt-o-sys19:07:28

clojure and jdbc... there seem to be 2 libs: clojure.java.jdbc and clojure.jdbc. Anyone any experience reports? Reasons why using one over the other? ...?

ghadi19:07:00

the first one is the standard one. the second one's name is unfortunate

seancorfield19:07:26

@kurt-o-sys I'm the maintainer of the standard (contrib) one: clojure.java.jdbc and we use it extremely heavily at work (with MySQL) for large volumes of data/queries.

seancorfield19:07:57

It's also tested against MS SQL Server (both MS driver and jTDS), PostgreSQL (both Postgres and Impossibl drivers), and H2, SQLite, Derby etc. There are Oracle and Terabase(?) users of it too. It's what most other JDBC wrapper libraries use under the hood.

seancorfield19:07:13

If you run into any issues with it, feel free to ask in the #sql channel.

kurt-o-sys19:07:21

Right... Thx.

seancorfield20:07:04

You can also see from this initial commit that the author basically copied clojure.java.jdbc (without permission and without attribution -- I took him to task on the mailing list for that!): https://github.com/funcool/clojure.jdbc/commit/dfb16894734c2a95838cd5f5b3fef3e63ac1c4bf

seancorfield20:07:53

In later commits he added some acknowledge of code he'd taken from clojure.java.jdbc and he did, over time, substantially rewrite the code base.

seancorfield20:07:14

(but, yeah, years later I'm still pretty annoyed about it)

lwhorton20:07:46

a list isn’t IAssociative, but I want to update a list with deeply nested values somehow like:

(def L '( '( '( :a :b '( :c :d '( 'needs-updating )))))
(update-in L [0 0 1 1] fn)
now it’s easy to do if the lists are vectors… but given that most-everything defaults to a list after running some process (map filter reduce etc.), I wonder whats the better way to do this that works for a more general seq?

noisesmith20:07:33

if you need associative updates, don’t use lazy operations for modification - map, filter, etc all provide transducers that can make vectors

lwhorton20:07:38

so pour things into [] or use vec-* for map/filter/etc.?

noisesmith20:07:32

@lwhorton right - but especially when chaining operations into with composed transducers is a better option

noisesmith20:07:49

and I don’t think it’s vec-*, it’s *v

noisesmith20:07:02

(and not all of the foov exist - but filterv and mapv do at least)

lwhorton20:07:22

ah, it’s *v yea … but why is into a better option? does it preserve laziness?

noisesmith20:07:34

no - vectors cannot be lazy

noisesmith20:07:55

by composing transducers you avoid creating intermediate data (which does nothing but create gc churn)

lwhorton20:07:33

hm, not totally following you but that’s okay. i know (comp a b c) will produce a transducer. are you saying there’s a way to do into [] some-fn where it doesnt use a transducer?

noisesmith20:07:56

no - and comp doesn’t generate a transducer unless its args are transducers

noisesmith20:07:38

if you call (filterv pred? (mapv f coll)) that generates a vector you don’t need

noisesmith20:07:10

if you use (into [] (comp (map f) (filter pred?)) coll) you get the same result without creating a data structure nobody uses

lwhorton20:07:48

ooof. so much to learn I do have. thanks for the tips yoda

noisesmith20:07:23

you can still get the right answer without composing the transducers, and it’s easy to change once you know how they work

noisesmith20:07:08

but back to the original question, if you need to do updates by index, keep things in associative collections, there are good options for making them but you don’t need to do it the optimal way on the first pass through

lxsameer20:07:38

what's your favorite logger library and why ?

hiredman20:07:58

I tend to use tools.logging on top of log4j2. it works great, just about every other routing framework can be rerouted to log4j2

hiredman20:07:03

the dependencies for that are something like

[org.apache.logging.log4j/log4j-core "2.8"]                                                                                                                                                                                   
                 [org.apache.logging.log4j/log4j-slf4j-impl "2.8"]                                                                                                                                                                             
                 [org.apache.logging.log4j/log4j-jul "2.8"]                                                                                                                                                                                    
                 [org.clojure/tools.logging "0.3.1"] 

hiredman20:07:19

since you were asking about pedestal logging on irc, last I checked(years ago), it was a simple wrapper on log4j

hiredman20:07:05

my mistake, it is a wrapper on slf4j

roklenarcic20:07:31

when accessing nested maps, do you guys prefer (-> a-map :a :d :c) or (get-in a-map [:a :d :c]) ?

hiredman20:07:01

I think most people prefer get-in

hiredman20:07:10

I like the uniformity of ->

hiredman20:07:28

you can stick some functions in there

hiredman20:07:01

-> may also be faster if you are dealing with nested records

hiredman20:07:06

-> is a syntax short cut, it is gone at runtime, get-in is actually a function and will run and exist at runtime

hiredman20:07:30

and you have this key vector that has to exist at runtime

ghadi20:07:14

I'll add one consideration: keyword accesses add quite a bit of bytecode (-> :a :b :c :d)

ghadi20:07:42

probably won't matter tho -- but if you care about jvm inlining i'd look into it

hiredman20:07:01

sure, but at that point do you want to be dealing with nested maps?

ghadi21:07:24

true that

nathanmarz20:07:57

@lwhorton you can do that with specter very easily:

(def L '(((:a :b (:c :d (10))))))
(transform (nthpath 0 0 2 2 0) inc L)
;; => (((:a :b (:c :d (11)))))

eriktjacobsen20:07:21

I prefer get-in because it makes the intention clear, we know that the first argument is a map, and that the second argument is a path. That information is lost with ->, also, that means you can't pull the path, [:b :c :d] from a data source or generate it without your own macro, because you can't apply to ->

hiredman20:07:00

sure, that is the flip side to what I said about the key vector

hiredman20:07:41

if it is constant and known at compile time (which it seems like it almost always is) then keeping it around at runtime is (admittedly an insignificant) waste

lwhorton21:07:48

specter keeps popping up here and there with some nice solutions.. i’ll have to look more closely around its api

qqq21:07:53

https://aws.amazon.com/lambda/faqs/ <-- java is listed as supporrted language; does this mean one can do "clojure -> java" for lambda instead of "cljs -> js" ?

bfabry21:07:55

@qqq my guess is that startup times might be problematic for many lambda use cases

bfabry21:07:15

but yes, you definitely could

ballpointcarrot21:07:19

Both clj and cljs can be used, but yeah, startup times become a thing.

bfabry21:07:45

just a lot of the use cases I can think of for lambda, a very fast starting VM such as the js vm is more appropriate than jvm+clj

hiredman21:07:26

to be clear, by java they mean jvm, clojure doesn't compile to java like clojurescript compiles to javascript

hiredman21:07:38

clojure generates jvm byte code

bfabry21:07:20

and with the incredible support cljs has for the node ecosystem these days... I would choose whichever vm makes the most sense for the scenario. which is a pretty amazing thing to be able to choose while keeping the language constant 🙂

zylox21:07:47

relevant ^ though im sure most have seen it

qqq21:07:52

zylox: thanks for link, first time I'm seeing it

eriktjacobsen21:07:30

@qqq @bfabry the data from this talk suggest it might not be as big of a deal as you'd assume (if you are running lambdas frequently enough to limit cold starts): https://www.youtube.com/watch?v=GINI0T8FPD4

bfabry21:07:33

@eriktjacobsen yes! true. my assumption is that most people using lambda are using it for reasonably infrequent operations (to take advantage of paying for only what you use). but if that's not the case then clj could be perfectly fine

hiredman21:07:41

somethings I know are different between clojure and clojurescript because I follow the jira issues aren't even on the list yet

qqq21:07:30

actually I'm using it to not think about scaling

qqq21:07:46

i.e. if my problem can be decomposed into an 'obviously parallel' setup, then AWS just scales everything for me

qqq21:07:53

and I no longer think at the grainualarity of machines

bcbradley22:07:20

if you create a language binding to a library written in another language

bcbradley22:07:23

how do you license it?

bcbradley22:07:40

do you keep the license of the source library?

eriktjacobsen22:07:37

depends on the license of the source library. If you aren't distributing their library, you might be able to license it however you want. Not something that can be answered in the abstract