Fork me on GitHub
#clojure
<
2015-12-09
>
dvcrn02:12:27

In clojure, how can I merge 2 n deep maps? I have multiple functions that return maps (that can consist of maps of maps of maps of maps of maps) and want to retrieve one big combined one

dvcrn03:12:07

gonna try that. Thanks @dmitrig01 !

onetom03:12:49

is anyone aware of a mock Google Cloud Storage implementation?

onetom03:12:23

or is #C03RZGPG1 a better channel to ask such questions?

sveri07:12:01

@rantingbob: You can define a shortcut for the REPL, I have mine set to CTRL+ALT+R. It will switch to the repl from wherever you are and open it, if it was minimized before. You get out of it by pressing ESC. This is a default behavior. ESC will switch to the editor window from everywhere else. SHIFT+ESC will switch to the editor and close the active window

jaen10:12:44

Anyone knows how namespaced keywords play into :keys destructuring option?

tcrayford10:12:18

@jaen: as far as I know you can't used namespaced keywords in :keys

tcrayford10:12:43

(because then the symbols they bind to would have to be namespaced, which could play real havoc and would require a change of clojure's scoping rules)

jaen10:12:15

Yeah, I thought so, thanks for confirmation

maio11:12:52

@jaen: (let [{:keys [:xxx/yyy]} {:xxx/yyy 10}] yyy)

johannjohann11:12:58

you can namespace keys

johannjohann11:12:12

uh destructure namespace keys

jaen11:12:50

Oh, so it just looses the namespace part?

maio11:12:52

@jaen: it works (but it strips ns)

jaen11:12:18

So I guess if you have :aggregate/id and :event/id then you have to do it the non-shortcut way, but otherwise it's ok?

robert-stuttaford11:12:55

this is particularly awesome when working with datomic data simple_smile

jaen11:12:06

Anyone can spot something obviously wrong with this transit handler?

(def ^:private clj-time-writer
  (transit/write-handler
   (constantly "m")
   (fn [v] v)
   (fn [v] (time-utils/to-str v))))

(def ^:private clj-time-reader
  (transit/read-handler
   (fn [v] (time-utils/from-str v))))
then I specify it as {DateTime clj-time-writer. I've had no problems with JSON repr, but msgpack repr blows up.

jaen11:12:40

(where the object in question is JodaTime's DateTime)

robert-stuttaford11:12:30

never had any luck with msgpack

robert-stuttaford11:12:57

@dnolen might be able to share information on that

jaen11:12:03

I suspect it might be that JSON repr uses the string repr (the second function), while msgpack repr uses the repr (first function), but I couldn't really find any docs on how they should differ.

jaen11:12:37

I guess I'll just go with JSON in the mean time then.

jaen11:12:29

It's especially confusing how write handlers have two varieties - string and non-string, but read handlers only one.

jaen11:12:15

Yeah, giving it a more sensible repr than identity function helped. But I'm unsure how the read handler should look now - how will it know if it reads the repr or the string repr?

jaen12:12:11

Hm, is there any way to use multimethods without def'ing them as top level symbols?

tcrayford12:12:38

@jaen not from what I know

roelof12:12:00

someone here who use the brave book to learn clojure. I have a problem with the last challenge of the do-things chapter

jaen13:12:24

@tcrayford: I've come up with something like this - https://gist.github.com/jaen/027660fbf55bfff6241f - and it works, but I'm unsure if it's relying on implementation details too much or not.

crankyadmin13:12:30

Hi I'm using Flambo (a Clojure DSL for spark) and I'm getting a strange error when running on the cluster: 15/12/09 07:39:32 ERROR yarn.ApplicationMaster: User class threw exception: java.lang.ClassCastException: org.apache.solr.client.solrj.impl.HttpSolrServer cannot be cast to org.apache.solr.client.solrj.SolrServer The code that I'm running runs fine locally. Could anyone translate what the exception means in this case?

jaen13:12:10

Doesn't it look like HttpSolrServer not implementing SolrServer interface?

jaen13:12:20

But that doesn't seem right, does it?

crankyadmin13:12:45

Agreed. But it *works* locally. It only throws this error on a yarn cluster

crankyadmin13:12:44

This is exactly what my confusion is...

jaen13:12:53

Are you sure you have the same version of dependencies locally and on cluster?

jaen13:12:17

Maybe somehow you get different jars and that's what causes it

crankyadmin13:12:21

yeah. Build from the same lein project.clj

jaen13:12:13

Hmpfh. I didn't use Spark, but maybe it provides it's own solr jar that's incompatible with the version you use

jaen13:12:24

And it's used instead of the one you provide in your uberjar

crankyadmin13:12:47

True enough thats possible... let me go dig...

rcanepa13:12:00

For the people who has developed a backend server that deals with relation databases, which sql abstraction library do you recommend?. I am comparing two approaches, the one from Yesql, in which you have to write queries directly in SQL files, and the one from Korma, which works with a new DSL. I think the later facilitates making changes on the queries (good for the development process), and the former seems much more capable (you have all the power of SQL). Any recommendation on this?. I have to say that my backend will work with just one db engine.

robert-stuttaford13:12:29

i’ve seen a lot more praise for yesql than i have for korma

robert-stuttaford13:12:34

korma is also quite old now

jaen13:12:22

Korma is also complicated if you would need to modify it somehow.

jaen13:12:12

There's also a middleground in form of HoneySQL - you have a data structure DSL for building up queries, but it's not a full-blown complex ORM like Korma is.

jaen13:12:30

If you expect your queries to be somehow dynamic it might be a better solution than yesql

jaen13:12:40

Since yesql doesn't have any templating options so far.

roberto13:12:46

+1 for HoneySQL

roberto13:12:24

how can I tell prismatic schema that a field in a map must of a certain java type?

jaen13:12:41

Just use the type, I think

roberto13:12:24

E.g. something like :

(def MySchema {:name s/Str
                                     :date (is org.jodatime.Instant)})

jaen13:12:50

No, just

jaen13:12:56

{:date Instant}

jaen13:12:01

(provided you imported it)

rcanepa13:12:07

Ok. What I don’t like about Yesql (maybe because I don’t know how to properly use it) is that when I want to insert something I need to fill all the defaults/blanks in my code, and I can't rely on the database default values. Inserting seems very unpleasant if sometimes I want to insert a record with 5 columns and the next time with 7 (in the same model/table).

rcanepa13:12:38

Should I make a constructor to deal with this?, to fill the blanks before inserting them?

jaen13:12:00

@roberto:

;; On the JVM, you can use classes for instance? checks
(s/validate java.lang.String "schema")

;; On JS, you can use prototype functions
(s/validate Element (js/document.getElementById "some-div-id"))
straight from the readme at https://github.com/Prismatic/schema so you can be sure I'm not just making things up.

roberto13:12:28

hehehe, I wasn’t doubting you

jaen13:12:30

Not saying you were, but it's always nice to have an official confirmation, I might have been relying on something undocumented by doing that.

jaen13:12:03

@rcanepa: I think that's just how it is with yesql - if you want to omit some fields, you have to write another query with those fields omitted.

rcanepa13:12:20

Ok. I will give Yesql more time to experiment with. Thank @jaen and @roberto !

jaen13:12:26

In general if your queries are not too dynamic yesql will probably work out okay in the long run. If you start noticing you would want to use some form of templating more and more, I suggest trying HoneySQL then.

rcanepa14:12:06

Great, good to know that!

bhauman15:12:27

@jaen: just letting you know I'm doing the advent of code as well simple_smile https://github.com/bhauman/advent-of-clojure

naomarik15:12:33

how do you guys have time for so many things 😛

jaen15:12:08

I don't, really, I'm behind two days right now D :

jaen15:12:35

@bhauman: nice, that day one is pretty succinct.

base69817:12:14

I like the idea of the sql libs being a dsl which creates lazy lists. I was pleasently surprised I could do (take 10 (map :username (select ...)) with cassaforte, not quite relational sql but close enough.

jaredly18:12:01

Is there a way to clear a core.async buffered channel?

geekyhybrid19:12:22

(Just peeking into the Slack room...)

geekyhybrid19:12:59

Haven't been using Clojure much of late, but want to revisit it as a nice side-learning-project.

roelof19:12:30

someone who can help a beginner in the beginners channel with a problem of the brave book ?

roelof19:12:55

@geekyhybrid: which language do you use now then ?

geekyhybrid19:12:28

I have worked with C#, Java, Python, HTML/CSS/JS, some Clojure.

geekyhybrid19:12:14

I want to do more Clojure - I like that it forces me to think about things differently (different from OOP).

roelof19:12:35

oke, that can be

roelof19:12:02

Im trying to learn clojure coming from the ruby / erlang world

geekyhybrid19:12:38

I actually used to be a chemist - and moved over into informatics and lab automation...and picked up programming and dev along the way.

geekyhybrid19:12:27

I actually have some well-earned time away from work coming up - so I think I want to use Clojure learning as my unstructured learning project.

cab19:12:28

i like your avatar @geekyhybrid

cab19:12:38

fitting simple_smile

shriphani19:12:11

Hi everyone, there’s a JNI-heavy library I’ve been using with clojure 1.6 with no issues - and upgrade to 1.7 produced a whole host of linkage errors. Are there any known issues / pitfalls about using 1.7 with JNI-heavy code ?

rcanepa19:12:29

Which unit testing library is similar to tape from node.js? Has the Clojure community a preferred one?

eraserhd19:12:24

@rcanepa: What’s interesting about tape? The test syntax, the output format?

eraserhd19:12:03

The support for multiple browsers?

rcanepa19:12:35

Oh, the simplicity. No global vars/functions… easy to use and understand. The output format was on TAP so you could you others libraries to consume it and print a “prettier” version.

rcanepa19:12:00

I saw that clojure.test seems very similar, however, I am not sure if there are better options around.

rcanepa19:12:23

Also, I read a recommendation about one named expectations.

rcanepa19:12:32

Actually, I am reading its github page right now.

eraserhd19:12:06

I’m partial to midje; however, this seems opposed to what you are looking for. Midje is a does-everything library.

eraserhd19:12:04

The node pattern of avoiding global vars is not so common in Clojure land. What is common is avoiding global mutation. Namespacing prevents collisions.

darwin20:12:39

in default configuration figwheel uses cljs.repl/repl* to create and run repl, I need to hook into that repl and execute arbitrary commands as if they were typed on the repl command prompt (commands are incoming over network socket from chrome devtools), it looks like :read option is the right way to go, but I need somehow spin default repl-read on a separate thread and implement my own :read which either gets something from original repl-read from stdin, or receives a message from the socket, any pointers how to do this? @bhauman?

darwin20:12:17

in other words, I need a side channel, how to enter my own commands into that running repl, while not breaking existing functionality

trinity20:12:17

hello again! wondering if anyone has experience with the clj-kafka lib (https://github.com/pingles/clj-kafka)? having a bit of trouble understanding processing the lazy seq that it returns for consuming msgs and need a quick sanity check simple_smile

dmitrig0120:12:49

@trinity: i have played with it - works best in a doseq

trinity20:12:24

@dmitrig01 thanks for the response! yeah that's how i have it now

trinity20:12:26

(defn consume-msgs
  [conf topic msg-handler]
  (info "** Consuming msgs, fn: consume-msgs, topic:" topic)
  (kf-core/with-resource [c (zk-consumer/consumer conf)]
                            zk-consumer/shutdown
                               (doseq [m (zk-consumer/messages c topic)]
                                (msg-handler m))))

ghadi20:12:44

you can't abort a doseq though

trinity20:12:34

yeah, outside of shutting down the entire program, but based on the kafka docs ""Next, your logic should expect to get an iterator from Kafka that may block if there are no new messages available.", so it should run until

ghadi20:12:34

Also, make sure you handle the timeout option when you reach the tip of the stream (I forget what clj-kafka) does about it

dmitrig0120:12:42

you could try converting to a core.async channel

ghadi20:12:59

so it will block if you set the option...

trinity20:12:48

i read that as saying that it will block, unless there is a message to be processed

trinity20:12:52

"The interesting part here is the while (it.hasNext()) section. Basically this code reads from Kafka until you stop it." - kafka docs

dmitrig0120:12:01

tbh it did not work so well for me, i ended up writing my own consumer

dmitrig0120:12:24

actually, i have some code that still uses it i think

trinity20:12:25

this is the part that i was confused about, bc the doseq does what i need, but i wasn't sure if that was a poor design choice that i basically can't run anything after the doseq

ghadi20:12:34

dmitrig01: if you have auto-commit set, and you use a core.async channel, it is easy to commit the message without it being processed.

dmitrig0120:12:40

yea i do use doseq

dmitrig0120:12:09

ghadi yes, i wrote my own low-level consumer to go to core.async

dmitrig0120:12:48

gotta publish it at some point

trinity21:12:50

okay great, thank you @dmitrig01 + @ghadi i will look into converting it to a core.async channel simple_smile

bhauman21:12:38

@darwin look into the :special-fns arg for cljs.repl/repl

darwin21:12:45

@bhauman: that does not seem to be what I need, special-fns seem to be caller-defined fns to be available inside repl, no?

darwin21:12:03

I’m not familiar with this repl stuff, maybe I’m missing something here, that massive repl* function starts loop, where it is continuously calling read-eval-print, which does blocking reading from in stream

darwin21:12:22

I need to hook into that, and provide my own inputs in the loop (to emulate user typing)

darwin21:12:56

the problem is that blocking call, if it blocks in waiting for user input, I cannot hook into it, I have to use multiple threads and do some magic

darwin21:12:57

btw, read-eval-print calls that :read (eval-read by default) function to do the actual blocking call, so I thought I could reimplement it to use core async channel and that channel would be pushed to by multiple producers

darwin21:12:27

one blocking reading from in, second one would be that network thread

bhauman21:12:00

@darwin: you could also probably hijack the in input stream

bhauman21:12:15

by binding it to your own

darwin21:12:54

@bhauman: I thought about it a bit, I googled, but didn’t find any good pointers how to do that, this is what I got so far: https://gist.github.com/darwin/e4879f8442f9a60d4f89

darwin21:12:11

seems to work, need to test it more

darwin21:12:56

I’m just unsure if this reading from stdio can safely run from a worker thread

darwin21:12:24

I assume that is what is happening when running it from go block with >!!

darwin21:12:20

ah, forgot to deref the volatile

darwin21:12:43

gist updated^

blueberry21:12:08

@shriphrani I used JNI with both clojure 1.7 and 1.8 with no issues. Maybe your problem lies in JVM version?

blueberry21:12:45

@shriphani: I used JNI with both clojure 1.7 and 1.8 with no issues. Maybe your problem lies in JVM version?

shriphani21:12:14

hm my issue was with 1.7

shriphani21:12:22

it worked fine with 1.6 (the clojure version)

shriphani21:12:47

I tested it with java7 and java8

shriphani21:12:34

that’s the lib

shriphani21:12:27

switching to clojure 1.7.0 causes it to break.

base69821:12:39

Does anyone know how to change the color of magit-blame to not black/grey?

larhat21:12:31

@base698 M-x customize-face and than search for magit-blame-* face names, probably

base69823:12:19

@larhat: That's life changing thanks