Fork me on GitHub
#clojure
<
2020-04-22
>
joefromct02:04:36

Hi, i just came across this on clojure docs and was wondering why; > ;; it is tempting to try an index on a list > (get '(a b c) 1) > ;;=> nil > > ;; but you should use nth > (nth '(a b c) 1) > ;;=> b Is there some performance implication regarding nth on a list? I'm a bit confused why get is off limits here. Thanks,

i03:04:48

get on a list always returns a nil.

joefromct03:04:50

i guess i'm trying to decide if i should use nth and potentially incur a performance penalty, or coerce into a vector and then have get close at hand.

joefromct03:04:06

this is in the context of a list comprehension that i can fit into memory without too much issue

dpsutton03:04:11

how many times do you need random access?

joefromct03:04:30

yeah, i guess thats the right question, i'm walking the entire sequence so i think i should shove it into a vector ?

dpsutton03:04:29

i was getting at the opposite side of that. creating a vector walks the entire sequence. if you just need to find the nth thing once, just linear scan and don't pay the penalty for the datastructure creation on top of the O(n) behavior

dpsutton03:04:51

if you need random access often, use a datastructure that provides the API and performance that you need

joefromct03:04:20

ok, thank you for the tip. I think i get it.

i04:04:21

is finger-tree (https://github.com/clojure/data.finger-tree) . really that slow ?

andy.fingerhut05:04:38

You can run your own benchmarks to verify, but I have a recollection of doing or seeing some benchmark results that showed its constant factors were quite a bit slower for the operations that is has in common with vectors. vectors cannot efficiency insert new elements at the front, whereas finger trees can, so depending upon how often you want to do that, finger trees could be faster overall.

namenu13:04:53

i have used it once for deque. since it has different purpose, comparing it's performance with normal vector is not fair. in my case, it was 2-3 times slower than Java's ArrayDeque. https://github.com/namenu/advent-of-code/blob/c87a9228a5b5bb3e3f70ba65c2a75a12d7fb43e3/src/aoc/year2018/day09.clj#L64

namenu13:04:31

but it's still O(1) and immutable so i think it's worthy

ctamayo05:04:44

I'm trying to containerize my dev environment, which involves connecting to a database at a separate host. Connection is refused because apparently JDBC is trying to connect to localhost, even though the correct DATABASE_URL env var is detected. Here is me debugging from inside the container:

[email protected]:/app$ clojure -A:dev -m 
Starting nREPL server at localhost:7000
Connecting to database at URL: jdbc:
log4j:WARN No appenders could be found for logger (com.zaxxer.hikari.HikariConfig).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See  for more info.
Apr 22, 2020 4:59:04 AM org.postgresql.core.v3.ConnectionFactoryImpl log
WARNING: ConnectException occurred while connecting to localhost:5432
.ConnectException: Connection refused (Connection refused)
...stack trace...
Here is the fn responsible for connecting:
(defn connect! []
  (if-let [database-url (System/getenv "DATABASE_URL")]
    (do
      (println "Connecting to database at URL:" database-url)
      (conman/connect! {:jdbc-url database-url}))
    (throw
     (ex-info "Database exception!"
              {:causes #{:no-database-url}})))
Any ideas for what to poke at?

ctamayo05:04:44

I'm trying to containerize my dev environment, which involves connecting to a database at a separate host. Connection is refused because apparently JDBC is trying to connect to localhost, even though the correct DATABASE_URL env var is detected. Here is me debugging from inside the container:

[email protected]:/app$ clojure -A:dev -m 
Starting nREPL server at localhost:7000
Connecting to database at URL: jdbc:
log4j:WARN No appenders could be found for logger (com.zaxxer.hikari.HikariConfig).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See  for more info.
Apr 22, 2020 4:59:04 AM org.postgresql.core.v3.ConnectionFactoryImpl log
WARNING: ConnectException occurred while connecting to localhost:5432
.ConnectException: Connection refused (Connection refused)
...stack trace...
Here is the fn responsible for connecting:
(defn connect! []
  (if-let [database-url (System/getenv "DATABASE_URL")]
    (do
      (println "Connecting to database at URL:" database-url)
      (conman/connect! {:jdbc-url database-url}))
    (throw
     (ex-info "Database exception!"
              {:causes #{:no-database-url}})))
Any ideas for what to poke at?

hindol.adhya12:04:33

How do you specify what dbhost is? By putting a mapping in etc/hosts?

ctamayo16:04:57

However Docker Compose does it. I've tested it by running psql --host=dbhost --username=postgres rtc inside the container and that works, so I don't think dbhost is the issue?

hiredman05:04:00

See what dbhost resolves to in your container

ctamayo16:04:27

ok, I installed dnsutils inside the container, and both of these work:

psql --hostname=dbhost --username=postgres rtc
psql --hostname=$(dig +short dbhost) --username=postgres rtc

ctamayo17:04:04

So that indicates to me that this is still an issue inside Java/JDBC somewhere...

hiredman17:04:11

Nah, you are still 2 steps away from jdbc

hiredman17:04:46

You need to look at conman, and then at the connection pool library conman actually uses

hiredman17:04:46

Conman pass the uri you give to https://github.com/pupeno/to-jdbc-uri and my guess is that is what is mangling things

hiredman17:04:39

Oh, you know what it is, you have a newline in your environment variable

ctamayo19:04:14

amazing, lol

hiredman19:04:49

A good tip is to use prn, which can make this kind of thing very apparent

ctamayo19:04:21

actually that doesn't appear to be the case

ctamayo19:04:48

unless I'm missing something? What makes you say there's a newline?

ctamayo19:04:34

user=> (prn (System/getenv "DATABASE_URL"))
"jdbc:"
nil

ctamayo19:04:07

also to-jdbc-uri appears to be doing what's expected...

user=> (j/to-jdbc-uri (System/getenv "DATABASE_URL"))
"jdbc:"

hiredman19:04:28

It may just have been how slack formatted things

ctamayo19:04:38

Aha! This is coming from the migrations part of the code...later on in my db ns. The conman/connect! call is actually working!

ctamayo19:04:26

So migratus is defaulting to localhost, which works outside my container but not in, which makes sense. So I just need to dig in to the right :db params to pass to it. Thanks for the help!

myguidingstar11:04:33

is there any tool to find all unused vars (mostly functions) in a big clojure project?

thomas.ormezzano11:04:20

clj-kondo detects unused private vars and unused bindings

thomas.ormezzano11:04:39

and a bunch of other things

sogaiu11:04:58

by the same author, there is this: https://github.com/borkdude/carve

myguidingstar14:04:17

carve looks like what I'm looking for. Thanks

myguidingstar11:04:18

i want to get rid of unused legacy stuff

eudisd13:04:54

Hey guys, is there any way to spread a vector/list into the argument arguments of function call without using apply?

aisamu13:04:17

You can spread it with [email protected]

eudisd13:04:46

nice! Ok let me try this, brb

aisamu13:04:48

(If you're inside a syntax-quoted form, that is, which it looks like you are from the other comment)

eudisd13:04:28

Blah, that didn't work

eudisd13:04:57

I think I'm doing something wrong, (i'm a clojure beginner). I'll read a bit more and try to figure it out, thanks for the help

hindol.adhya14:04:05

Can you not use the routes function directly? Apply should work then. https://github.com/weavejester/compojure/wiki/Routes-In-Detail

hindol.adhya14:04:54

defroutes is just for convenience it seems.

eudisd14:04:50

ah interesting, thanks for the tip! Trying it out now

eudisd13:04:05

There is a third party macro that I want to spread into, since its a macro I cannot use apply. Specifically, in the compojure library, there is a defroutes macro that I would like to call, with a list of pre-compiled routes from various modules. Instead of manually entering each and every module into the call, I'd like to do something like (apply defroutes app-routes list-of-all-my-routes)

alexmiller13:04:27

you can write your own macro that wraps it and use [email protected]

hiredman14:04:37

Don't use defroutes

i14:04:08

I found this behavior quite weird

i14:04:16

(defn test-chan [] (let [c (a/chan 10)] (a/onto-chan c [1 2]) (a/close! c) (println (a/<!! c)) ; prints nil (println (a/<!! c)) ; prints nil (println (a/<!! c)))) ; prints nil

i14:04:07

If I omit this sexp (a/close! c), then everything is fine. wh closing on a channel twice will fail the takes ?

delaguardo15:04:28

takes a val from port. Will return nil if closed. Will block
if nothing is available.
from documentation for <!!

ghadi15:04:47

"Puts the contents of coll into the supplied channel.
  Returns a channel which will close after the items are copied."

ghadi15:04:01

@i onto-chan is async

ghadi15:04:28

you need to (a/<!! (a/onto-chan ...)) before you try to read from c

i15:04:30

aha. because the goblock of onto-chan hasn’t been dispatched.

ghadi15:04:43

it's been dispatched, but not guaranteed to be done

i15:04:43

does using go blocks causing much latency? I previously heard on this channel someone saying go block under jvm (clojure) is quite tricky regarding the performance.

noisesmith15:04:10

by itself, adding a go block will slow down your code (if that's the only change). If you are doing something parallel in order to have better throughput then need to do complex coordinataion, a go block can facilitate that speedup if that makes sense

noisesmith15:04:56

async is a debt / detriment you take on (in terms of both perf and debugability) that should be balanced by some other greater benefit which happens to need async

slawek09817:04:49

Hey! Do you have any general suggestions how to transform XML (possibly parsed via data.xml) efficiently (both in terms of performance and code readability and manageability) into nested map, extracting specific keys and values from tags and content at specific levels?

simongray09:04:00

I recently needed to do similar stuff and ended up using a zipper.

slawek09817:04:11

I thought about Specter, but I'm not sure if it's actually possible to collect stuff from multiple places into map with it.

hiredman18:04:45

I dunno about efficiency but I always start with for

dmillett18:04:45

Do you want to do anything special with each tag? Do you want to flatten it?

josh60418:04:26

I worked with XMPP in Clojure and ended up using transducers inspired by this blog post: https://juxt.pro/blog/posts/xpath-in-transducers.html

slawek09818:04:10

In general I need to build map from various nested properties of this XML. I've got multiple "objects" in this XML which I need to extract.

dmillett18:04:18

attributes for specific nodes? Or counting frequencies for nodes?

dmillett18:04:48

Probably a transducer to import the XML with (data.xml) and then you can easily create/extract nested maps with (get-in m [kp1 kp2 kp3])

dmillett18:04:06

Xpath tools may get this.

dmillett18:04:18

This might be helpful for you to look at the nested values/maps https://github.com/dmillett/clash#data-shape

slawek09818:04:09

So it's something like: first go to tag x > tag y > tag x of id y and then take every children of tag z and build map out of it's tag a and b and c.

isak18:04:19

Sounds like you could just use xpath type tools

hiredman18:04:28

(for [maybe-x (tree-seq map? :content xml) :when (= (:tag maybe-x) :x) maybe-y (:content maybe-x) :when (= (:tag maybe-y) :y) :when (= (:id (:attr maybe-y)) (:id (:attr maybe-x)))] whatever) or something

slawek09818:04:44

Yeah, something like that probably.

hiredman18:04:17

for is a real swiss army knife for querying nested data

hiredman18:04:25

you can even do recursive queries with it by just wrapping it in a recursive function

hiredman18:04:09

(fn f [whatever] (for [x whatever i (f x)] i))

jtth18:04:40

For server-side access to Firebase, should I just use the java admin library or is there a preferred pre-existing wrapper, or should I use something like taika, a wrapper around the REST API?

markaddleman18:04:56

Oh, sorry, you said Firebase 🙂 I've never really understood how *base and *store are related

jtth18:04:25

It’s a cloud firestore. I mess them up constantly.

jtth18:04:57

and it seems like this is precisely what i was looking for, thanks!

yosevuk19:04:17

Hey, does Clojure have an official/unofficial motto or philosophy? For example: Java: "Write once, run anywhere" Erlang: "Write once, run forever" Haskell: "Avoid success at all costs"

andy.fingerhut19:04:58

I think this was discussed some time in the past month or three. I recall there being many proposed answers, but none "official". Lots of proposed unofficial mottos, of course.

kwladyka19:04:50

always parens ;)

kwladyka19:04:59

in the past I saw something like: from practical programmers to programmers

kwladyka19:04:08

it was different but something in this meaning

yosevuk19:04:11

Oh no, I can't search back that far

yosevuk19:04:27

Thanks Alex!

mike145219:04:06

@yosevuk Simple made Easy

pshar1020:04:17

I've created a macro:

(defmacro defelem [name element]
  `(ws/defcard ~name
    (ct.react/react-card
     (r/as-element
      ~element
      )
     )
    )
  )
But when I run it in the repl, I get the error TypeError: Cannot set property 'defelem' of undefined. Why is this and what's the fix?

seancorfield20:04:14

Macros have to be executed in Clojure, even for ClojureScript, since they rely on the Clojure compiler for expansion and then the resulting code can be compiled by ClojureScript. I think.

seancorfield20:04:49

@pshar10 Depending on which ClojureScript REPL you're using, asking in #clojurescript #lumo #planck #devcards ... may yield better answers about how to deal with macros in those environments.

pshar1022:04:04

Say I have a list '(1 3 4 5), and I want to append to it some number n only if it already doesn't exist in it. How to do that in clojure elegantly?

nikolavojicic22:04:54

(let [coll '(1 3 4 5)]
  (if ((set coll) 4)
    coll
    (conj coll 4)))
Btw use vectors instead of lists. Better version:
(let [coll '(1 3 4 5)]
  (if (some #{4} coll)
    coll
    (conj coll 4)))

pshar1022:04:10

@nikolavojicic why vectors and not lists?

nikolavojicic22:04:47

Vector data structure is more performant. List is a linked list, its purpose in Clojure is code writing mostly...

nikolavojicic22:04:47

Vector data structure is more performant. List is a linked list, its purpose in Clojure is code writing mostly...

bfabry22:04:23

there are usage patterns for which a list is better performance wise than a vector

pshar1022:04:40

would you give an example?

nikolavojicic22:04:16

Yes, for adding elements at the beginning.

nikolavojicic22:04:42

For search by index and for adding to the end, vector is faster.

bfabry22:04:56

correct, use as a stack

bfabry22:04:32

they're possibly more memory efficient than a vector too, but I'm unsure

pshar1022:04:50

and for iterating through the whole once, which is better?

pshar1022:04:11

or when used with map?

pshar1022:04:39

and appending if an element doesn't already exist?

nikolavojicic22:04:02

Also, there are vector specific functions such as subvec.

bfabry22:04:21

for iterating all the way through then my guess is a vector because of arrays getting pulled into cache lines, but that's the kind of thing that needs measuring

nikolavojicic22:04:57

Use vector always. Btw don't confuse lazy seq and list!

pshar1022:04:42

@nikolavojicic your example doesn't return what I want though

pshar1022:04:05

(let [coll '(1 3 4 5)]
  (if (#{4} coll)
    coll
    (conj coll 4)))
returns (4 1 3 4 5)

pshar1022:04:09

There are two 4s

pshar1022:04:19

there should only be one

nikolavojicic23:04:14

Fixed it, sorry, should have run it :)

andy.fingerhut23:04:42

And of course, if what you want is a collection of things with no duplicates, and order does not matter, then a set is the most appropriate 🙂

nikolavojicic23:04:44

Added better version that uses some.

pshar1023:04:37

@ how to create an empty set? I'm using #{} but doing a conj on it is returning a vector

andy.fingerhut23:04:08

user=> (conj #{} 5)
#{5}

andy.fingerhut23:04:17

You do not get that result?

nikolavojicic23:04:38

And if order of insertion matters, add all the elements and use distinct at the end.

pshar1023:04:46

@ Yes this works, but I'm using re-frame and initializing a db value to #{}

andy.fingerhut23:04:17

I do not now that db's limitations that it may have on the types of values it allows you to store.

seancorfield23:04:45

I have some code that gives a reflection warning when run on OpenJDK 8 but does not give a reflection warning on OpenJDK 11 or OpenJDK 14. I found that surprising so I'd be interested to hear suggestions/explanations of what might have changed between JDK versions for that to happen. It's with this image metadata extractor library com.drewnoakes/metadata-extractor {:mvn/version "2.13.0"}

seancorfield23:04:34

(I haven't checked whether it's a multi-release JAR which I suppose could account for it -- actually being different JVM bytecode on 8 vs 11/14)

seancorfield23:04:25

(nope, that's not it -- no MR stuff in that lib)

hiredman23:04:46

clojure.lang.Reflector (which the compiler also uses to resolve methods) actually has a check to see if it is running on 1.8 or not

hiredman23:04:31

it may be a module thing

hiredman23:04:45

9+ respects module accessibility, so something that may be making things ambiguous in 8 might be ruled out because of module something or other in 11