Fork me on GitHub
#clojure
<
2018-05-14
>
Bravi15:05:45

I have a vector of vectors and I’m trying to only display unique elements

[ [{:name "hello"}] [{:name "world} {:name "hello"}] ]
I tried to chuck it into a set like so
(into #{} [ [{:name "hello"}] [{:name "world} {:name "hello"}] ])
but I need to unwrap those inner vectors somehow

sundarj15:05:04

note that you can also do this:

(into #{} cat [[{:name "hello"}] [{:name "world"} {:name "hello"}]])
#{{:name "world"} {:name "hello"}}

sundarj15:05:33

=> (doc cat)
-------------------------
clojure.core/cat
([rf])
  A transducer which concatenates the contents of each input, which must be a
  collection, into the reduction.
nil

Bravi15:05:34

nice!

😄 4
dpsutton15:05:26

(let [vectors [ [{:name "hello"}] [{:name "world"} {:name "hello"}] ]]
  (->> vectors
       (apply concat)
       distinct))
-> ({:name "hello"} {:name "world"})

Bravi15:05:43

ah nice, thank you! 🙂

👍 4
Bravi15:05:21

I was trying to create a function that shows the difference in terms of order within 2 lists. so for example running these two vectors

(def a-list
  [{:id "29jd1001"
    :name "A project"}
   {:id "2013uj2"
    :name "B project"}
   {:id "1112398"
    :name "C project"}])

(def b-list
  [{:id "2013uj2"
    :name "B project"}
   {:id "29jd1001"
    :name "A project"}
   {:id "1112398"
    :name "C project"}])
through that function, should give me A project and B project, because their orders changed. and there will only ever be 2 elements swapping positions at one time. so I wrote it like this:
(defn find-diffs
  [a b diff-fn]
  (remove nil? (map diff-fn a b)))

(->> (find-diffs a-list b-list #(when (not= (:id %1) (:id %2)) [%1 %2]))
    (apply concat)
    distinct)

Bravi15:05:15

I wanted to use keep but looks like it only takes 1 collection at a time

joelsanchez15:05:39

sorry but doesn't clojure.data/diff work for you?

joelsanchez15:05:43

cljs.user=> (clojure.data/diff a-list b-list)
[[{:name "A project", :id "29jd1001"}
  {:name "B project", :id "2013uj2"}]
 [{:name "B project", :id "2013uj2"}
  {:name "A project", :id "29jd1001"}]
 [nil nil {:id "1112398", :name "C project"}]]

Bravi15:05:20

but I want it the other way around. so it should give me the first 2, rather than the 3rd one

Bravi15:05:31

because their positions changed

joelsanchez15:05:52

right, that's what it's returning, take a closer look

joelsanchez15:05:03

it returns diff from the left side, diff from the right side, and things in common

joelsanchez15:05:25

"things-only-in-a things-only-in-b things-in-both"

Bravi15:05:59

oh so the whole thing is a return value? damnit 😄

joelsanchez15:05:07

well it's a tuple, but yes

Bravi15:05:17

of course there’s a built - in thing in clojure..

😂 4
noisesmith15:05:48

the thing is, data/diff doesn't look for re-orderings, it just checks differences naiively element per element

noisesmith15:05:52

but maybe that's good enough for you

Bravi15:05:30

yeah, that’s exactly what I was trying to achieve

Bravi15:05:38

by comparing ids in my case

Bravi15:05:52

ah this is very cool!

Bravi15:05:15

another reason to completely switch to clojure

sobel17:05:24

i find it cute how modern languages are finally getting around to having relations as a standard 1st class part of the language

sobel17:05:04

data nerds (SQL) have been onto relations for a while though and it's painful using client languages that have to build a bridge to deal with them

sobel17:05:15

it's almost not painful in clojure

justinlee17:05:32

what does it mean to have first-class relations? foreign-key enforcement?

tanzoniteblack17:05:18

With lein 2.8.1, I'm getting this error:

Tried to use insecure HTTP repository without TLS.
None of the repositories specifically listed in my project.clj are http, so I presume it's a transitive repository ? How can I figure out which repository is being connected over HTTP without TLS so I can see if I can override the repository in my project.clj?

Alex Miller (Clojure team)17:05:59

I’m not sure if this will work, but you could try lein pom to generate a pom.xml, then mvn -X dependency:tree

Alex Miller (Clojure team)17:05:27

I think you’ll see some logs like Using connector WagonRepositoryConnector with priority 0.0 for http://...

Appo718:05:05

also having issues with security on ubuntu 18.04. Lein will complain about java certificates which i was only able to solve by completely purging all versions of java, reinstalling certificates and reinstalling java

ghadi18:05:03

What version of java? JDK9 lacks certificates at all in some packages

ghadi18:05:24

(jdk 8 and 10 are fine)

tanzoniteblack18:05:14

[email protected]:/tmp» java --version
openjdk 10.0.1 2018-04-17
OpenJDK Runtime Environment (build 10.0.1+10-Ubuntu-3ubuntu1)
OpenJDK 64-Bit Server VM (build 10.0.1+10-Ubuntu-3ubuntu1, mixed mode)

tanzoniteblack18:05:49

I'm trying the mvn dependency:tree idea, but I had to add some plugins since I'm using an S3 wagon via https://github.com/s3-wagon-private/s3-wagon-private

tanzoniteblack18:05:00

(which could be the culprit, we'll see)

tanzoniteblack18:05:39

[DEBUG] Using transporter WagonTransporter with priority -1.0 for found it, thanks @alexmiller

tanzoniteblack18:05:03

it would be a very nice feature improvement if lein included which repo was the problem in it's error message 🙂

tanzoniteblack18:05:14

maybe I'll see if I can figure out how to add that and make a PR later

Alex Miller (Clojure team)18:05:14

I have a PhD in obscure dependency debugging

😈 4
Alex Miller (Clojure team)18:05:29

yeah, that would be helpful

andy.fingerhut18:05:46

I have no idea what the root cause is, but I got errors early on with 'mvn install' on Clojure's source code itself with OpenJDK 8 and OpenJDK 10 on an Ubuntu 18.04 VM. Oracle JDKs worked better on the same OS in that regard.

andy.fingerhut19:05:58

This was starting from an empty $HOME/.m2 directory, in case that makes a difference.

Alex Miller (Clojure team)18:05:03

nah, this is new stuff in lein that disallows http (not https) maven repos

tanzoniteblack18:05:13

which is nice, and I appreciate the feature

tanzoniteblack18:05:17

it's just really difficult to debug

Alex Miller (Clojure team)18:05:02

that particular one is actually also available at https

tanzoniteblack18:05:12

@alexmiller yeah, I just had to add

["couchbase" {:url       ""
              :snapshots false}]
to my :repositories in my project.clj

tanzoniteblack18:05:34

because whichever dependency was pulling in that repo had coded in the http version, so I have to hardcode an override in my project.clj

Appo718:05:47

@ghadi I'm pretty sure 10. (openjdk from the ubuntu repos). Had to go back to 8, reinstall certificates and then go back to 10. Not sure what exactly caused the hiccup.

ghadi18:05:27

weird, 10 should be fine

nickmbailey20:05:02

anyone know if later jvms let you tune the method size limit. running into an issue where core.match expands into too large a method for the jvm to handle

hiredman20:05:24

I suspect those limits are actually backed in to the classfile format

noisesmith20:05:26

@nickmbailey - one cause of this is mixing core.match with eg. core.async, and by using function calls you can pull it out so no one method contains code from both libs

nickmbailey20:05:48

i am using core.async but not within my match macro

noisesmith20:05:52

other culprits include doseq or for

nickmbailey20:05:27

afaict its just a big match, if i comment out one or two clauses it compiles again

nickmbailey20:05:32

but not any specific clause

noisesmith20:05:58

oh - worth mentioning the big macros in case that's the easy fix anyway :D

nickmbailey20:05:23

yeah i hunted around for some culprit like that 🙂

nickmbailey20:05:36

well i don’t think it’ll be too bad to split up into multiple functions

nickmbailey20:05:48

just figured i’d check if there was an ‘easy’ solution

noisesmith20:05:08

depending how deep you want to go into this, there's also no.disassemble which will show you all the byte code emitted by a function - you might find a surprising thing you can make smaller

nickmbailey20:05:52

i am doing the match within a ‘loop’ block

nickmbailey20:05:00

maybe i can just pull that out to another function

hiredman20:05:35

that is an interesting little tidbit in the jvm spec

nickmbailey20:05:03

yeah moving the match block outside of the loop block fixed it for now. that match might grow so we’ll see how long it lasts haha

noisesmith20:05:33

replacing destructuring with bindings to lookups can also help sometimes

nickmbailey20:05:53

ah thats good to know

nickmbailey20:05:05

doing a bit of destructuring in my match clauses

Bravi22:05:45

is there a function that I could use instead of conj here and that would give me the results you see in comments?

(conj [1 2 3] 4) ;; -> [1 2 3 4]
(conj [1 2 3] [4]) ;; -> [1 2 3 4]
so basically I have a function that will either get a vector of items or just 1 item in argument and no matter what type, I want it to append to the returned list.
(defn my-merge [item]
  (conj [1 2 3] item))
I know I can do a check using vector? but I was wondering if there’s a trick to always concat whatever is in the second argument

dpsutton22:05:05

most of the time this means you could tighten up code elsewhere. ensure that code that uses this is aware that you prefer collections of things rather than an individual

Bravi22:05:30

so you’d recommend to stick to one type, right?

dpsutton22:05:11

yes. you are dealing with individual things or collections of things. and make sure all producers/whatevers behave in that type

Bravi22:05:28

👍 thank you 🙂

Bravi22:05:21

sometimes I try to create a lot of unnecessary abstraction to save a few keystrokes but then I end up with these dilemmas. I should stop haha

dpsutton22:05:51

i would make sure that you always know whether values are collections or individual values. otherwise there's lurking Don't know how to create ISeq from: clojure.lang.Keyword

👍 4
sobel22:05:44

i see a lot of that 😛

sparkofreason22:05:16

I have two projects using the same library, both using Clojure 1.9.0. One builds the library fine, the other throws "CompilerException clojure.lang.ExceptionInfo: Call to clojure.core/ns did not conform to spec:" How is that possible?

the2bears22:05:29

Likely a dependency trying to use (require ...) instead of (:require ...) in the ns expression.

the2bears22:05:56

But which library...

Bravi22:05:06

one more question, if I may..

(def my-obj {:name {:first {:one "John"}}})

(defn get-obj
  [& args]
  (get-in my-obj (concat [:name] args)))

(get-obj [:first :one]) ;; nil
(get-obj [:name :first :one]) ;; John

Bravi22:05:17

can’t figure out why is that

Bravi22:05:59

even if I wrap that concat in vec, still doesn’t work

dpsutton22:05:07

you're putting & args and then passing a single collection

dpsutton22:05:26

choose if you want to (get-obj [:path :components]) or (get-obj :path :components)

dpsutton22:05:02

if you put a (prn "args:" args) in your defn of get-obj you'll see what you are calling it with

sparkofreason22:05:11

@the2bears The require in the lib is definitely wrong, can't figure out why it doesn't fail in both cases.

the2bears22:05:38

Is the failing lib a dependency (with a newer, fixed version) of something else in the successfully compiling project? I'd check the deps with lein on each project. The failing lib is probably being brought in, up-to-date, by something else.

seancorfield22:05:02

@dave.dixon Is it possible one of the projects is actually pulling in an earlier version of Clojure than you think?

Bravi22:05:09

@dpsutton sorry I mistyped there. I’m actually doing the latter (get-obj :path :components)

dpsutton22:05:57

(get-obj :first :one)
"args:" (:first :one)
"John"

dpsutton22:05:01

it seems to work for me

dpsutton22:05:08

(defn get-obj
  [& args]
  (prn "args:" args)
  (get-in my-obj (concat [:name] args)))

Bravi22:05:24

hmm then it must be something to do with re-frame. 😞 damn it. I’m basically doing this

(re-frame/reg-sub
 ::is-loading?
 (fn [db [_ & path]]
   (let [full-path (into [:loading] path)]
     (get-in db full-path))))
and then using it like this
(re-frame/subscribe [::subs/is-loading? :sortable :projects])

Bravi22:05:51

I’ll see if I can get help in re-frame channel. thank you so much for your help

seancorfield22:05:40

@dave.dixon Another possibility is that one project never causes the bad namespace to be required/loaded.

sparkofreason22:05:38

@seancorfield It definitely refers to 1.8. And both projects are loading the lib in question. Running both projects in the REPL, which reports Clojure 1.9.0.

souenzzo22:05:17

Why my reversible collectin isn't faster then a non-reversible collection?

(let [p @sorted-set-with-1000-items]
  (prn (reversible? p))
  (time (dotimes [i 10000]
          (reverse p))))
true
"Elapsed time: 14603.556531 msecs"
=> nil
(let [p (seq @sorted-set-with-1000-items)]
  (prn (reversible? p))
  (time (dotimes [i 10000]
          (reverse p))))
false
"Elapsed time: 15057.406117 msecs"

dpsutton22:05:04

can you throw a little context out?

andy.fingerhut22:05:43

@souenzzo I am pretty sure that if reversible? is true for a collection, it only means that it is likely to be able to do (rseq coll) faster than (seq (reverse coll)). reverse takes (at least) linear time and space for a collection regardless (see implementation of clojure.core/reverse function)

souenzzo23:05:13

My mistake . Will use rseq

andy.fingerhut23:05:30

Yeah, doc string for 'rseq' promises constant time implementation for any collection where (reversible? coll) is true. reverse makes no such promises.

👍 4
sparkofreason23:05:51

@the2bears The failing lib is the most recent version, and the only version referenced according to lein deps :tree

joaohgomes23:05:32

It depends on the underlying collection implementation righ? Got the following results comparing vecs vs lists:

(let [coll (into () (range 1e5))]
    (time (dotimes [_ 100] (reverse coll))))
"Elapsed time: 1329.904501 msecs"
=> nil
(let [coll (into [] (range 1e5))]
    (time (dotimes [_ 100] (reverse coll))))
"Elapsed time: 187.19594 msecs"
=> nil

andy.fingerhut23:05:42

I suspect that difference is not because one is O(n) time and one is O(n^2) time, but that reduce (used in implementation of reverse) is a significant constant times faster for vectors than for lists, perhaps because lists must follow an arbitrary 'next' pointer for each element, whereas the vector implementation is more cache-hit-friendly.

👍 8
the2bears23:05:17

@dave.dixon weird, but having read @seancorfield's comments I think he's on the right path - something to do with 1.8 even though both REPLs show 1.9.

andy.fingerhut23:05:56

It would be awesomely nice if there were an easy way at a Clojure REPL to invoke 'time' or another macro like it, and get not only the time, but also the number of cache misses experienced during the execution of the expression given, and/or the IPC (Instruction Per Cycle, not Inter Process Communication)

joaohgomes23:05:16

Makes sense. And for reversing a reversible?, the rseq as constant time seems more appropriate performance-wise.

scriptor23:05:51

it might be because (into () (range 1e5)) doesn't return a chunked seq

scriptor23:05:06

so it has to realize all 10,000 elements one by one

scriptor23:05:23

if you removed the into call the performance is similar to that of using a vector

👍 4
joaohgomes23:05:04

I heard of this library criterium that improves on time approach. (http://clojure-goes-fast.com/blog/benchmarking-tool-criterium/)

scriptor23:05:28

yeah, criterium's awesome for real-world profiling

sparkofreason23:05:50

@the2bears Agreed. Not sure how that's possible, but it seems like the only explanation.

seancorfield23:05:31

@dave.dixon If you have a :repl profile that is bringing in libraries, it could "correct" the 1.8 dependency.