Fork me on GitHub
#clojure
<
2019-10-25
>
deas04:10:17

New to travis and looking for nice pipelines to borrow from. Would like to have it cover publishing to clojars and dockerhub. Preferably even multiple jdk versions and images. Any suggestion is appreciated.

valerauko05:10:48

@seancorfield is there a simple way to make next.jdbc return column names (keys in the map) without namespacing them with the table name?

valerauko05:10:33

as-unqualified-maps will work. thanks!

seancorfield16:10:35

Also, I might miss SQL questions in popular channels like this one -- #sql is the best place to ask about c.j.j. and next.jdbc -- and there's a #honeysql channel too, if you're using that @vale

Akiz06:10:06

Good Morning everybody. I am stuck like for two days and maybe somebody can help me? If this message is too long or I should ask elsewhere tell me 🙂 Thanks. Function below takes list of maps like

'({:country_code "CZ" :direction_id "1" :direction_name "1"
  :zone_id "x" :zone_name "x" :hub_id "x" :hub_name "x" 
:dealer_number "1025" :dealer_name "x"}
{:country_code "CZ" :direction_id "2" :direction_name "2"
:zone_id "x" :zone_name "x" :hub_id "x" :hub_name "x" 
:dealer_number "1025" :dealer_name "y"}
...)
Groups them by key and calculates KPIS (aggregates selected keys from these maps) wich are associated to the group key and then returned.
(defn group-and-transform
  [datamart group-key]
  (for [m (group-by group-key datamart)]
    (let [info (extract-info 
                (apply merge-with (comp distinct flatten (into list)) (val m)) 
                group-key)
          kpis (calculate-kpis (val m))]
      (assoc
      (merge info kpis)
       group-key (key m)))))
Then there is info (function extract-info) that groups and list other values, which I am trying to assoc together with KPIs. So in the end I want one map with dimension values (possibly grouped in list) and aggregated KPIs for every group-key.
(defn extract-info [data group-key]
  (let [country (if (list? (:country_code data)) 
                  (first (:country_code data))
                  (:country_code data))
        info (try
               (merge
                (case country
                  "CZ" {:country_code "" :direction_id "" :direction_name ""
                        :zone_id "" :zone_name "" :hub_id "" :hub_name "" 
                        :dealer_number "" :dealer_name "" :type group-key}
                  :default {})
                  (select-keys data
                               [:country_code :country_name :direction_id :direction_name
                                :zone_id :zone_name :hub_id :hub_name :dealer_number :dealer_name]))
               (catch Exception e (println (str "Error!" e))))]
    info))
But it fails with the error (when it should produce map with values in list - made from 2nd and 3rd value in dm) which I cant understand and identify.
(group-and-transform  (take 3 dm) :zone_id)
Error! java.lang.IllegalArgumentException: No matching clause: clojure.lang.LazySeq@896
({:kpi773 -100, :kpi755 -61.60700M, :kpi760 -61.60700M, :kpi768 1, :kpi807 0, :kpi1326 1, :kpi756 1, :kpi771 0, :zone_id "51", :kpi770 0, :kpi758 129.00M} {:kpi773 0, :hub_id "", :kpi755 0, :kpi760 0, :zone_name "Z≤na 52", :direction_name "", :kpi768 1, :type :zone_id, :kpi807 0, :kpi1326 1, :country_code "CZ", :country_name "Czech republic", :hub_name "", :kpi756 1, :dealer_number "20318039", :dealer_name "K N car spol. s r.o.", :kpi771 0, :zone_id "52", :direction_id "", :kpi770 0, :kpi758 57.00M})

andy.fingerhut07:10:06

In your function group-and-transform you have the expression (into list). list has no local value defined, so it refers to the function list in Clojure

jaide07:10:29

If you were to write a custom CLI to better combine your git & ticketing system's API, would you reach for Clojure or ClojureScript? I'm having trouble deciding between the two.

jaide07:10:39

After more thought, I've reached a possible direction: Write it in Clojure first because it will be the easiset\fastest development experience. Write as much of the business logic as cljc. Then once the project is done, I can choose to migrate it into a terminal user interface on ClojureScript with that template I made a few months back.

andy.fingerhut07:10:33

Try evaluating small subexpressions and see what results you get for those smaller things, then build up from there.

andy.fingerhut07:10:15

For example, consider the difference betwen these:

user=> (into list [5 3])
Execution error (ClassCastException) at user/eval4356 (REPL:111).
class clojure.lang.PersistentList$Primordial cannot be cast to class clojure.lang.IPersistentCollection (clojure.lang.PersistentList$Primordial and clojure.lang.IPersistentCollection are in unnamed module of loader 'app')
user=> (into (list) [5 3])
(3 5)

andy.fingerhut07:10:17

flatten is often a bad idea, because it flattens arbitrary depths of data structures. It isn't clear to me what your intended behavior is that you want to get from this expression? (apply merge-with (comp distinct flatten (into list)) (val m))

Akiz07:10:17

Hello Andy, thank you for your interest! About the second function -> I need to merge some values accross multiple maps into one value. When I did only into list it ended like (1 1(1(1(1(1))) so I use flatten to get (1 1 1 1 1 1) and then distinct to get (1). I feel that this is not the best way how to do it but I did not find better yet. Edit: When I use (into (list))) and call (group-and-transform (take 2 dm) :zone_id) it leads to

Error printing return value (ClassCastException) at clojure.core/comp$fn (core.clj:2570).
clojure.lang.PersistentList$EmptyList
When i use (into list) and call (group-and-transform (take 2 dm) :zone_id) it leads to Error! java.lang.IllegalArgumentException: No matching clause: clojure.lang.LazySeq@896 And when I call (group-and-transform dm :zone_id) the error is different:
Error printing return value (StackOverflowError) at clojure.core/distinct$step$fn$fn (core.clj:5048).
null
class clojure.lang.ExceptionInfo
PART OF SOLUTION: Ha! That function was really the problem that I did not realized before... It seems that I am not using comp correctly as (apply merge-with list (val m)) works better. But it fails because country can end like (((("CZ" "CZ") "CZ") "CZ") which cant be extracted by
country (if (list? (:country_code data)) 
                  (first (:country_code data))
                  (:country_code data)) 
so it fails with java.lang.IllegalArgumentException: No matching clause: (((("CZ" "CZ") "CZ") "CZ") this is why I went with flatten ..... but then it produces that failing lazy sequence... 😞

Akiz08:10:27

So my question ends like this: How to tranform this: '(((("CZ" "CZ") "CZ") "CZ"))) (but ideally also this): "CZ" into ("CZ") not producing lazy sequence. Thank you

vlaaad08:10:36

something like that?

(defn- flat-conj [to from]
  (if (sequential? from)
    (reduce flat-conj to from)
    (conj to from)))

(flat-conj #{} ["cz" ["cz" [[["cz"]]]]]) ;=> #{"cs"}
(flat-conj #{} "cz") ;=> #{"cs"}

Akiz08:10:13

Thank you, I will use this if I dont find out how to not produce that sequence in a first place 🙂.

andy.fingerhut08:10:50

Instead of answering how to transform this '(((("CZ" "CZ") "CZ") "CZ")))into what you want, I would ask you: what expression are you using that returns that result? Because I suspect that changing that expression to return ("CZ" "CZ" "CZ" "CZ") instead is very likely possible, and will avoid any need to add a step to flatten it.

Akiz08:10:42

(for [m dm] (apply merge-with list (val m))

andy.fingerhut08:10:23

So using merge-with with the function list is causing the first two maps to produce a list of two elements, then if you merge in a 3rd map, you get the list of two elements, plus the 3rd thing, in a list. and so on.

Akiz08:10:56

You are right, i have been confused by example (merge-with +) which returns one value. Can you think about better function to be used for merging values into one sequence? Best regards

andy.fingerhut08:10:22

Sorry, got into a separate conversation for a while there. Looking ahead to the next step after this, I think you said your end goal is not this map with values that are lists of the input map values, but to then find sets of distinct values, and then maybe do something further after that?

andy.fingerhut08:10:46

I am asking that because even this intermediate step might be unnecessary, depending upon the desired end result you are looking for.

andy.fingerhut08:10:53

It might be easier to help if you linked to a document with an example of a sample input with a sequence of say 3 maps, and the return value you hope that group-and-transform would return, with a little English explaining how the return value is related to the input

💯 4
Akiz11:10:46

No problem Andy, thanks. Here is the file, I kept only relevant things..

Akiz11:10:06

I updated it as there were bad leftovers 🙂

andy.fingerhut08:10:34

So if you had a different expression instead, that took a sequence of maps as input, you want it to return one map that contains keys that are in any of those individual input maps, and the values associated with the keys to be lists of the values from each input map?

👍 4
Akiz08:10:16

This is exactly what I need But there is possibility that there will be sequence with only one map so it would be nice if it returns list with the only one value also (as nothing can be merged ) if possible - but this is not so important 🙂. Thank you

vemv09:10:32

Anyone familiar with clojure.test.check.clojure-test/defspec opaquely failing builds (e.g. a CircleCI one)? I can successfully run a defspec-based test in Emacs, or over a vanilla repl. In CI, the build just fails with Tests failed. and no other useful output

vemv11:10:48

Solved: there wasn't an explicit test.check dep in the project. An explicit and up-to-date one fixed builds.

patrkris15:10:29

Does anyone here know of a macro or something else for retrying a failing serializable PostgreSQL transaction a specified max number of times?

bfabry15:10:33

I’ve always created that macro/fn myself for projects so I don’t think there’s anything in core

patrkris15:10:52

Thanks 🙂

souenzzo15:10:48

There is some (two?) talks about it on youtube

patrkris15:10:46

@U050MP39D Do you happen to know a good way to simulate a serializable error?

bfabry15:10:46

sure, just try and serialize a function or an atom or whatever

patrkris15:10:35

@U050MP39D No, I mean in regard to PostgreSQL serializable transactions 🙂

bfabry15:10:16

oh, nope. I’d probably just with-redef and throw the exception you’re expecting

patrkris15:10:58

Ah okay 🙂 I thought I would specifically check for the right error code from PostgreSQL

souenzzo15:10:08

It will throw just when you try to (ser)/(rel)ialize

(sequence (map (fn [x]
                       (/ 1 (- x 33))))
                ;; map will do 32 items chunk
                (range 33))

Roman Tsopin18:10:19

I’m sure someone has already thought about Clojure over llvm (like Kotlin native). As I understand llvm could be very dynamic (as Swift for Tensorflow shows), so good REPL experience is possible. What do you think? Are there any progress in this area? Or what are the main obstacles?

gklijs11:10:01

I wonder want the use case would be. Running clojure on embedded devices, or for iOS apps?

hiredman18:10:17

llvm basically has none of the things called out as being provided by platforms there. there is nothing stopping you from implementing something like clojure on the llvm, but you will need to implement everything from scratch, and won't even be able to share your facilities with other languages that use llvm

gklijs11:10:49

It could be built with rust, using .cljr extensions. But it won't be easy implementing the whole clojure core in a rust interpreter and also create a nice rust interop. But it's not impossible.

hiredman18:10:24

the fact that llvm has vm in the name and so does the jvm causes people to think they are comparable, they are not, at all

hiredman18:10:33

basically a native code version of clojure throws away a lot of great things about clojure (good interop because everything on the jvm has the same memory model, layout, and calling conventions, a number of great gcs to choose from, etc)

hiredman18:10:39

llvm doesn't even provide a standard multithreading abstraction

Roman Tsopin18:10:22

Thank you. I mean yes, jvm and llvm are quite different, but the way I look at this is more similar to ClojureScript view. Something that’s useful in a context, where Clojure (or ClojureScript) is not applicable. Where you need fast startup time, low footprint, easy C, C++ interop or platforms like iOS. I know many of the things above are possible with graalvm or js, but the idea to have native binaries produced by mature established infrastructure like llvm seems very attractive to me

hiredman18:10:54

there are a number of projects that translate (some subset of) clojure code in to c++

hiredman18:10:38

https://ferret-lang.org/ is maybe the most maintained

Roman Tsopin18:10:44

Thank you, I’ll look at this

hiredman19:10:42

if given some kind of project requiring c++, I would likely write clojure to generate c++ instead of writing a clojure to c++ compiler

borkdude20:10:39

is my understand of reader conditionals correct when I'm thinking that these are processed at read-time by the language and the language picks only the parts that it's interested in and then processes the the forms?

borkdude20:10:25

e.g. you cannot write a macro that emits reader conditional forms which are then again processed by the reader?

ghadi20:10:32

the reader hands things out already filtered -- eval doesn't see them

borkdude20:10:43

that's what I mean yeah

ghadi20:10:04

that's right, you cannot write such a macro. conditionals are at read-time

borkdude20:10:11

phew, ok. thanks 🙂

Alex Miller (Clojure team)20:10:35

you can ask the reader to preserve them

borkdude20:10:04

there is the reader-conditional function to create data representations of reader conditional forms. when is that useful? I have used it once when I generated code to disk which was then executed by some other process

Alex Miller (Clojure team)20:10:09

that's what you get if you ask the reader to preserve

borkdude20:10:37

ah kewl, ok. thanks a lot