Fork me on GitHub
#clojure
<
2017-01-27
>
qqq00:01:55

does [ring/ring-core "1.5.2"] not provide ring.util.servlet ?

qqq00:01:03

if not, what does provide ring.util.servlet ?

qqq00:01:13

am I not understnading clojure apcakges -- or does ring, unlike other github/clojure projects, have "multiple porjects" inside a single "repo" ?

qqq00:01:24

oh, i"m the idiot, I should have done [ring "1.5.1"] which would included all of the packages

plins00:01:43

hi everybody, is anyone aware of a lib that allows me to manage env variables and if they are not provided it would throw an error? im currently using Envinron and doing it by hand 😕

qqq02:01:52

I'm AOT-ing clojure code. I need to generate a class which im[lements a certain Java interface.

qqq02:01:10

I know how to AOT code in Boot. I know I need to use something like :gen-class ? I don't know anything else. WHat should I read up on?

bfabry02:01:18

@qqq the docs for gen-class are here https://clojuredocs.org/clojure.core/gen-class . just saying, my experience with it is that it's a bad idea. a small shim java class that calls your clojure code using Clojure.Var has always worked out much better for me

qqq02:01:11

I need something which satisfies javax.servlet.Servlet -- I know that using ring.servlet/servlet will convert a ring-handler to a java-servlet

bfabry02:01:16

assuming you actually need a compiled class that is. if all you need is something that implements an interface proxy is a pretty good winner

qqq02:01:21

but I need to output the corect *.class file

qqq02:01:36

yeah, I need to output a *.class file implementing a Servlet. I'm also using boot, not lein ring

qqq02:01:05

how do I get src/foo/bar.clj to generate a foo.bar.HelloServlet.class file ?

qqq02:01:22

I tried definin a "HelloServlet" inside of foo/bar.clj , but it does not output a foo/bar/HelloServlet.class

bfabry02:01:43

(gen-class :name foo.bar.HelloServlet.class :implements javax.servlet.Servlet ...)

qqq02:01:56

is that diagram for real or a joke? 🙂

bfabry02:01:13

that diagram is real

bfabry02:01:17

clojure interop is very very good, especially at consuming java code. it gets complicated when you want java code to consume clojure code

qqq02:01:52

yeah, I've had no problem using java from clojure

bfabry02:01:06

(and I, personally, would replace the "use gen-class" box with "write a tiny bit of java")

bfabry03:01:01

although maybe my aot+gen-class+complex-build fu is just lacking

qqq03:01:18

@bfabry: the good news is that I now have a g/m/HelloServlet.class and am getting a different error 🙂

bradford03:01:40

Hi! Quick q. why does

(let [port (cycle (range 1 200))] (dotimes [_ 10] (print (take 1 port)))  )
return
(1)(1)(1)(1)(1)(1)(1)(1)(1)(1)
instead of
1 2 3 4 5
etc. I guess it's lazy, but I'm not sure how to work around that laziness

bfabry03:01:46

take takes 1 value from port and returns it but does not modify the contents of port

7h3kk1d03:01:02

You can use a map and doall.

7h3kk1d03:01:30

or doseq I think

qqq03:01:36

(take n) really means "create a new seq consisting of the first n elements, but don't modify the origal seq)

tolitius03:01:45

@bradford what is the goal? just '(1 2 3 4 5) out of a cycled range?

(->> (cycle (range 1 200)) (take 5))

bradford03:01:53

Yes, I just want to grab a new port every iteration inside dotimes.

7h3kk1d03:01:29

(let [port (cycle (range 1 200))] (doseq [x (take 10 port)] (print x)) )

7h3kk1d03:01:48

You're probably thinking imperatively you don't actually modify the port binding so you need to iterate over it somehow.

bradford03:01:30

The whole context is creating a set number of proxies for webcrawlers, and consuming URLs for them to consume. Each proxy needs a port.

(defn start-async-consumers
  [num-consumers  ^core.CrawlService crawlservice]
  (let [ports (cycle (range 1 200))]
    (dotimes [_ num-consumers]
      (async/thread
        (while true
          (let [url (async/<!! in-chan)]
            (.start crawlservice (Integer. (take 1 ports)) url)
            ))))))

tolitius03:01:57

@bradford

boot.user=> (defn create-crawler [port service] (println "creating a crawler" {:port port}))
#'boot.user/create-crawler
boot.user=> (map create-crawler (cycle (range 4)) (range 10))
creating a crawler {:port 0}
creating a crawler {:port 1}
creating a crawler {:port 2}
creating a crawler {:port 3}
creating a crawler {:port 0}
creating a crawler {:port 1}
creating a crawler {:port 2}
creating a crawler {:port 3}
creating a crawler {:port 0}
creating a crawler {:port 1}

tolitius03:01:31

4 is number of ports 10 is number of consumers

qqq03:01:33

can Clojure extend an existing java class?

qqq03:01:41

which funtion's doc should I read up on?

bradford03:01:07

@tolitius ohhhhhhhhhhhhhh. thanks!

bfabry03:01:09

it would depend on what functionality you want from extension, I would go back to the flowchart. gen-class probably though

bfabry03:01:50

not extend, extend is related to protocols

qqq03:01:13

@bfabry : lol, according to the diagram, yes, gen-calss

tolitius03:01:19

@bradford sure, good luck crawling 🙂

bfabry03:01:27

well if that's what the diagram says, I'm sure it's correct

tolitius03:01:47

> not extend, extend is related to protocols why is it bad?

bfabry03:01:25

it's not bad, it's just not what people mean when they say "extend an existing java class"

tolitius03:01:07

it might be me with a wider meaning of "extend an existing java class" 🙂 but I would say it extends java.util.Date with a method to-ms:

(defprotocol Dateable
  (to-ms [t]))

(extend java.util.Date
  Dateable
  {:to-ms #(.getTime %)})

bfabry03:01:39

yeah, it allows you to extend a java type with a protocol. but that's not what qqq meant

tolitius03:01:16

@qqq what did you mean by: > can Clojure extend an existing java class?

qqq03:01:34

@tolitius: @bfabry 's interpreattion -- I wanted to write clojure code that extends javax.servlet.http.HttpServlet

qqq03:01:08

never have I benn so frtrasted in clojure until I have to do java interop

qqq03:01:50

does proxy generate a HelloServlet.class ?

qqq03:01:05

because the dumbass server I'm using wants a HelloServlet.class 😞

bfabry03:01:42

lol, I think it's voodoo magic that I can have my code called by code written in another language at all

bfabry03:01:58

if your implementation needs to be referred to statically from java then proxy won't work

tolitius03:01:20

@qqq: what's the server? why does it want to greet everybody so fearlessly?

qqq03:01:37

google app engine

bfabry03:01:53

which is one of the points on chas's flowchart. you can probably all tell by now, I'm a big fan of the flowchart

qqq03:01:55

if it was anything besides gae, i'd tell it to go f- itself, but since I want to use gae and datastore, I tolerate this bs

qqq03:01:05

yeah, that flowchart is amazing

qqq03:01:21

i can simplify taht flow chart: if what you're doing trivial? yes => do it no => use gen-calss

tolitius03:01:38

yes, proxy and reify create anonymous classes, so if you need a custom named class, you'd go with gen-class

qqq03:01:40

wtf does this error mean: google/so have been useless

qqq03:01:49

java.lang.NoClassDefFoundError: clojure/lang/Var

bfabry04:01:52

did you import clojure.lang.Var;? (Cursive is very helpful here as you just get automagic java file fixing in intellij)

tolitius04:01:33

usually means you do not have clojure-version.jar on your classpath

qqq04:01:37

why would I have to import clojure.lang.Var ?

qqq04:01:47

is it not provided by oh well, let me try it

qqq04:01:49

as all this is balck magic

bfabry04:01:15

no such thing as provided automatically in java, or clojure really

qqq04:01:52

clojure gen-class javax.servlet.Servlet <-- I kid you not, I think the one result that would be useful for this is cgrand from 2008, and apparently that page hangs

qqq04:01:00

it's like the universe is telling me to give up and just write a java stub

cgrand21:01:56

qqq: write a stub, forget gen-class (I only use gen-class for main classes nowadays)

qqq22:01:05

@U3E46Q1DG : ha! I got gen-class to work, the problem was not gen-class, the problem was I forgot to include clojure (via the (uber) task) for boot -- so the servlet container was loading my class, not finding the clojure support, then bitching about my calss (leading me to think I mis-used genclass), when I used it perfectly and the only problem was I did not include clojure as a dependency

artur06:01:26

Is there a neat way to update all hashmaps in a vector so that if a hashmap is missing a key then add it with certain value?

jr06:01:40

here's 1 way:

(map (partial merge {:missing-key certain-value}) vec-of-maps)

artur06:01:43

Nice! Thanks jr

narendraj907:01:31

(def testing (repeat 1000000 {:name "NJ" :available? true})) Will the map in the sequence testing be interned or there would be copies of the map once testing is realized?

rauh07:01:56

@narendraj9 Yes: (let [[a b] (repeat {:a :b})] (identical? a b))

narendraj907:01:44

@rauh According to this article -- http://nyeggen.com/post/2012-04-09-clojure/ , the values of a hashmap aren't interned. Why is that the case?

rauh07:01:38

@narendraj9 Strings and Keywords are both interned

danielgrosse07:01:23

Hello, I use schema to define my records, which works great. Now I have a question to understand the usage. Is it useful to validate the record in production or only within unittest? Actually I test it when I call (map->Record), but this throws an exception, if the input is wrong.

narendraj907:01:54

@rauh repeat keeps the reference to the object around and hence all of them would be indentical. https://github.com/clojure/clojure/blob/master/src/jvm/clojure/lang/Repeat.java Okay. Hashmaps in Clojure aren't interned it seems. Structural sharing happens because we start with a reference to the original map but two separate maps with the same contents would be two independent objects.

pbaille09:01:45

Hi, what is the most straightforward way to extend a PersistentVector instance to some protocols? or to create a type that behaves and looks exactly like PersistentVector but has extra capabilities (via implementing some extra protocols). Am I suppose to write a 200loc deftype to do that?!

mpenet09:01:51

former is nothing too exotic, later is a bit more involved yeah ...

pbaille09:01:53

it will extend the whole type no?

pbaille09:01:16

I’m seeking for extending particular instance or create a new type

cddr09:01:48

What kind of problems does one avoid by heeding the warnings in lein deps :tree about "possibly confusing dependencies"?

borkdude09:01:26

What is the idiomatic way of getting the namespace part of a keyword as a keyword, (keyword (name kw))?

pbaille09:01:47

(keyword (namespace x)) ?

pbaille09:01:47

how can I use clojurebot? as @mpenet just did

pbaille09:01:33

./clj (keyword (namespace :yop/yop))

pbaille09:01:55

lol i’m missing something 🙂

mpenet09:01:03

remove the dot

mpenet09:01:17

slack reserves the / for commands

mpenet09:01:44

@pbaille for a particular instance you can't do it with extend-proto/type, no easy way to creating a new type from another (potemkin does this for maps for instance)

mpenet09:01:04

you can hack something on top of metadata, but this will be brittle

mpenet09:01:20

(with-meta [] {:stype "myfoo"}) then use a multimethod for your fns

pbaille09:01:58

yes i’ve almost do the metadata stuff but is there a particular reason for this to not being in core?

mpenet09:01:46

well it sounds a lot like class inheritance 🙂

pbaille10:01:00

Is there a lib that do what potemkin do for maps for vecs and sets?

mpenet10:01:03

and if you want to get there you can with what's available

pbaille10:01:18

inheritance is a bad thing?

mpenet10:01:08

not necessarily, I am personally not against this kind of things. and could have used what you described a few times in the past, but that's how it is atm

pbaille10:01:09

I’m not so afraid of it, but i’m a beginner

pbaille10:01:26

okok I see

pbaille10:01:31

I will stick with my ugly defvectype macro

borkdude10:01:30

hmm actually I want a function that returns foo for both :foo/bar and :foo

borkdude10:01:46

I guess string splitting is the only option here

pbaille10:01:59

ho but in the second case foo is not the namespace

mpenet10:01:16

#(or (namespace %) (name %))

mpenet10:01:21

yep, kinda the same

borkdude10:01:27

good enough, thanks!

sveri10:01:32

Hey, I am evaluating bouncer for validation of forms and when using it, bouncer returns a vec like this above after validation. You can see that the errors are hidden in the vec at the second map inside :bouncer.core/errors. Is that how its intended to be? That looks pretty unstructred, almost randomly. Any ideas if I am doing something wrong?

baptiste-from-paris10:01:57

Hello guys, another core.async question for me 🙂 I am running this code to see with YourKit which Threads are live or not

(let [ch (chan 1)]
  (go (>! ch 42)
      (println (<! ch))
      (a/close! ch)))
when I capture thread usage with YourKit, this is what happens =>

baptiste-from-paris10:01:50

the async-dispatch Thread stays in waiting mode

baptiste-from-paris10:01:12

I thought it should be GC’d or killed in some way

mpenet10:01:17

it's a fix sized threadpool, it wont resize I think

baptiste-from-paris10:01:31

so it is supposed to stay in waiting mode until the end of the world ?

mpenet10:01:37

the chan get gc'ed at some point, the threadpool used by go is still here.

mpenet10:01:43

no clue about that

baptiste-from-paris10:01:10

I thought go point to ch

mpenet10:01:29

go returns a chan itself but it doesn't point to ch

baptiste-from-paris10:01:04

I am pretty sure it has a reference to chwhile it’s alive

baptiste-from-paris10:01:32

and nothing has a reference to go so it should be GC’d - I mean, that’s what I thought 🙂

baptiste-from-paris10:01:42

from go doc =>

...
Returns a channel which will receive the result of the body when
completed
in my case close! returns nil ...

baptiste-from-paris10:01:44

US members are still sleeping ^^

mpenet10:01:51

et oui 🙂

rauh11:01:11

@baptiste-from-paris IIRC it's 60s idle until the thread is then cleaned up

baptiste-from-paris11:01:13

@rauh ; after 60s thread is still here

rauh11:01:47

@baptiste-from-paris NVM, that's for the Cached Threadpools, for the async dispatch pool (newFixedThreadPool) they'll never get shut down unless done specifically

rauh11:01:33

See clojure.core.async.impl.exec.threadpool -> java.util.concurrent.Executors

baptiste-from-paris11:01:38

and if I do the test 100x times, it all sounds good =>

(dotimes [_ 100]
  (let [ch (chan 1)]
    (go (>! ch 42)
        (println (<! ch))
        (a/close! ch))))

baptiste-from-paris11:01:20

10 threads which is my availableProcessors +2

rauh11:01:48

No, it's fixed to 8 unless specified otherwise

rauh11:01:30

See the namespace I pasted above, then check the doc of the Executor services

baptiste-from-paris11:01:34

and from the javadoc =>

newFixedThreadPool
...
The threads in the pool will exist until it is explicitly shutdown.

baptiste-from-paris11:01:25

no I am wondering how it impacts me when I am using core.async

rauh11:01:56

num_threads * stack_size_per_thread memory usage... otherwise not at all

rauh11:01:26

Which is usually 8*2MB depending on JVM/Platform (etc?)

baptiste-from-paris11:01:43

by the way, why default to 8 ?

rauh11:01:07

Well it used to be 2*processor + 42 but then people complained that this uses a lot of RAM by default, so it got lowered to 8. But I think people are now expected to choose their pool size (or even their own executor).

tbaldridge12:01:49

@baptiste-from-paris Good morning from the US 🙂. I did a (free) video awhile back about GC'd go blocks you may find interesting: https://www.youtube.com/watch?v=VrwVc-saWLw&amp;index=7&amp;list=PLhi8pL3xn1OTDGCyXnkZStox6yFjn2583

tbaldridge12:01:32

The gist: half the time channels hold a reference to go blocks (not the other way around), and yes that seems strange but there's a reason.

baptiste-from-paris12:01:42

ahah good morning !, @tbaldridge that’s a great video, I think I really need to watch it another time

baptiste-from-paris12:01:23

@mpenet I was wrong about go -> ch ; it’s ch -> go video explain it greatly

baptiste-from-paris13:01:08

and @tbaldridge , do you have any idea why we never shutdown threads ?

tbaldridge13:01:58

I think that's the default for the threadpool. And we expect users to override the threadpool if they want something different.

tbaldridge13:01:10

I'm not aware of any design decisions in that area

mpenet13:01:56

it's not overridable if I recall

mpenet13:01:03

not for go blocks at least

mpenet13:01:14

you can just specify its size

matan13:01:23

coming back to clojure after a while, I am getting this error which I'm not sure about: >java.lang.ClassNotFoundException: clojure.java.shell looks like http://clojure.java.io is found, but not shell. anyone up for quick advice? 🙂

matan13:01:23

Really my code is entirely trivial where this comes up: (clojure.java.shell/sh "git")

matan13:01:21

Environment: >Leiningen 2.7.1 on Java 1.8.0_101 Java HotSpot(TM) 64-Bit Server VM

pesterhazy14:01:28

@matan did you require the namespace? 🙂

mpenet14:01:59

(clojure.java.shell/sh "sh" "-c" "git")

pesterhazy14:01:28

just "git" should work

mpenet14:01:34

alternatively you can pass the full path of "git"

tianshu14:01:06

Is there a tree-map in clojure?

bja14:01:42

@doglooksgood what do you want tree-map to do differently than PersistentHashMap?

rauh14:01:56

@doglooksgood A sorted map is one IIRC

rauh14:01:14

(class (sorted-map))
=> clojure.lang.PersistentTreeMap

matan14:01:45

@pesterhazy I have not noticed in the docs that the namespace needs to be required... isn't it included automatically just like http://clojure.java.io?

bronsa14:01:13

@matan you need to require all namespaces before use

bronsa14:01:25

no namespace is ever guaranteed to be automatically included

bronsa14:01:29

other than clojure.core

pesterhazy14:01:36

what bronsa said

matan14:01:44

I guess that's by the spec, and I should do that then

matan14:01:54

Wondering why http://clojure.java.io simply works without it

pesterhazy14:01:04

probably because it's used bei lein

matan14:01:20

thanks a lot guys!

qqq14:01:59

is there a 'clojurelight' somewhere which is like clojure, but less features, and also less code / classes generated ?

pesterhazy14:01:27

that's a strange question, qqq!

tianshu14:01:00

sorry for my stupid question, it seems like array-map is what I want

bronsa14:01:33

why do you want array-map?

qqq14:01:53

@pesterhazy : yeah, on second thought, it's not clear to me what I would be wiling to "drop" to make clj 'lighter'

bja14:01:07

@qqq theoretically clojurescript with dead code elimination is that

bja14:01:19

it has fewer features (no jvm libraries, you get node libraries instead), but the output size can be significantly smaller due to advanced mode from the google closure compiler

tianshu14:01:21

@bronsa I want some stuff that has an order, and can find value by key.

bronsa14:01:42

@doglooksgood array-map preserves insertion order, but if you conj an element on an array-map, it will be transformed into an hash-map and the ordering will be lost

bja14:01:12

also, you almost never want clojure.core/array-map due to it transparently promoting to a hash-map past a certain size

bronsa14:01:48

if you create an array-map using clojure.core/array-map it will not be promoted regardless of size, but it will as soon as you conj onto it

bja14:01:17

user=> (type (conj (array-map) [:foo :bar]))
clojure.lang.PersistentArrayMap

bronsa14:01:11

.. after you get past the threshold size :)

tianshu14:01:41

Yes, it is

tianshu14:01:08

how should I do if I want to search for an item in a vector? just use (first (fliter ...)) ?

bja14:01:05

(first (filter (comp #{"my-key"} first) my-vector)

mpenet14:01:46

(some #{:foo} v)

bja14:01:54

assuming you have [[:k1 :v1] [:k2 :v2]] situation

bronsa14:01:36

@mpenet often ffilter and some will not be the same

bronsa14:01:50

some returns (f x)

bronsa14:01:57

filter returns x

mpenet14:01:17

depends on the values yes

bronsa14:01:53

unless the predicate is identity, they will return different values

vs4vijay15:01:08

Hey, I am getting Caused by: java.lang.IllegalArgumentException: No value supplied for key: {:handler project1.core/handler, :init project1.core/on-init, :destroy project1.core/on-destroy}

vs4vijay15:01:21

I just bootstrapped project using leiningen

vs4vijay15:01:38

and in my project1.clj file, my configuration is:

vs4vijay15:01:50

`:dependencies [[org.clojure/clojure "1.8.0"]] 6 [[ring "1.5.1"]] 7 :plugins [[lein-ring "0.10.0"]] 8 :ring {:handler project1.core/handler 9 :init project1.core/on-init 10 :destroy project1.core/on-destroy})`

vs4vijay15:01:36

How do I resolve this?

moxaj15:01:37

@vs4vijay you might have messed up some parens, can you post your whole project.clj?

moxaj15:01:39

oh I see, your dependencies is 2 separate vectors

vs4vijay15:01:23

@moxaj, I think parens are at right position... How do I install depedencies? doing lein deps just gives the same error

moxaj15:01:02

just merge those 2 vectors

moxaj15:01:19

[[org.clojure/clojure "1.8.0"] [ring "1.5.1"]]

vs4vijay15:01:28

Got it, Thanks matw

vs4vijay15:01:57

Any good resources for Web app dev. with Clojure?

vs4vijay15:01:24

I I am currently following Packt Publishing course

bja15:01:20

@vs4vijay I usually recommend people start with http://www.luminusweb.net/docs until they form a strong opinion of their own on various libraries

vs4vijay15:01:03

Cool, Let me check

bja15:01:19

I personally made almost none of the same choices as luminus for my latest project, but until you understand the choices, luminus will give you some sane defaults

vs4vijay15:01:43

Okay, and what about any book recommandations? I am reading Joy of Clojure... and have lined up SICP and Clojure for Brave... Is there any other resources worth checking out?

dm316:01:29

what's the proper way to type-hint multi-arity fn return types? Is it

(defn go (^Type []) (^Type [a]))
?

vinai16:01:32

@vs4vijay Clojure for the Brave and True is a good first book. I recommend to read it before Joy of Clojure, which is arguably a bit more advanced.

vinai16:01:54

The other way round you probably can skip Clojure for the brave and true imo.

arnaud_bos16:01:41

Kyle Kingsbury's "Clojure from the ground up" is a very nice series https://aphyr.com/tags/Clojure-from-the-ground-up (the cards are sorted by publication date desc)

arnaud_bos16:01:36

beginner oriented though, sorry I didn't read carefully what you were searching for 😇

borkdude16:01:27

Didn’t know that cljs-ajax supports Clojure nowadays: https://github.com/JulianBirch/cljs-ajax

borkdude16:01:31

Maybe it has done so for a long time without me knowing 🙂

octahedrion16:01:28

when using generators created by s/gen, is there a way to parameterize the generator for a test - for example to fix the size of a collection ?

octahedrion16:01:08

like, e.g. say i have a spec :qwe/coll (s/coll-of double?) which I don't want to specify the size of the collection, but then later when I exercise it I do want to specify a size

iku00088817:01:31

Is it possible to add multiple type hints for example when a thing could be a either a file or an input stream?

pupeno17:01:18

Any idea how to set up the working directory of a lein shell call?

arkh17:01:07

iku00088: no. type hints set meta-data for the :tag key. The purpose of a type hint is to avoid reflection but if you had multiple options within a type hint, reflection would still be needed to determine type

mike_ananev17:01:26

@borkdude realy cool lib for client and server

borkdude17:01:50

@mike1452 I know it, but thanks for reminding 🙂

iku00088817:01:26

@arkh Thanks for nailing the question 🙂

souenzzo17:01:40

#newbie: how to (clojure.instant/now)

souenzzo17:01:07

(-> (clj-time.core/now)
    (clj-time.coerce/to-string)
    (clojure.instant/read-instant-timestamp))
Is is the "easiest" to do this? (I will compare with another timestamp)

sashton17:01:41

did with-ns end up anywhere after the contribs were deprecated: https://clojure.github.io/clojure-contrib/with-ns-api.html

sashton17:01:31

i didn’t find it in any newer libs, maybe i missed it

iku00088817:01:22

@souenzzo perhaps : > The functions equal?, after?, and before? determine the relative position of two DateTime instances:

iku00088817:01:06

in clj-time if you want to tell which time is earlier or something like that

souenzzo17:01:11

equal?/after? allow timestamp? (Other time comes from datomic)

iku00088817:01:46

Sorry, not familiar with datomic... (Really nice that you are using it...)

iku00088817:01:47

Could you tell the result of (type thing-from-datomic)?

souenzzo17:01:57

(-> (clj-time.core/now)
    (clj-time.coerce/to-timestamp))

pesterhazy17:01:57

@souenzzo what's your question?

pesterhazy17:01:08

an instant is simply a java.util.Date

pesterhazy17:01:53

you can use clj-time.coerce/to-date and from-date to translate back and forth to instants

iku00088817:01:31

If I understand correctly, he wants to compare a timestamp from datomic against current time

pesterhazy17:01:53

a datestamp from datomic is also just java.util.Date 🙂

pesterhazy17:01:22

it's the lingua franca of java time management

iku00088817:01:55

Enlightenment 💡

iku00088817:01:04

Good to know when I am living with Postgres atm. Thanks.

souenzzo17:01:27

😮 I can't use >/`<` on #inst

souenzzo17:01:58

Ok, all time stuff will be with clj-time

pesterhazy18:01:13

there's .before

pesterhazy18:01:46

Java's got your back

akjetma19:01:15

Not sure if this is the right channel for this question, please let me know if there's a better place! We just had a checksum mismatch error with a dependency from maven, and I'm wondering how this can happen? It's consistently the same incorrect shasum, so the artifact seems to have changed, just wondering how this can come about so I know who to talk to/where to look.

akjetma19:01:26

(btw, the artifact was hosted on maven central)

moxaj19:01:26

@akjetma is the artifact javassist by any chance?

moxaj19:01:20

@akjetma yeah, it's a transient transit dependency. I manually added the latest version and excluded the old one

akjetma19:01:25

ahh, gotcha

moxaj19:01:42

[org.javassist/javassist "3.21.0-GA"] works

rickhall200019:01:00

anyone know what caused it to break?

akjetma19:01:11

^ that's what i'm wondering

akjetma19:01:32

whether this is an issue with the maven central repo or whether library authors are allowed to modify artifacts at the same version number

akjetma19:01:55

ty though @moxaj, we'll use that for a workaround

tanzoniteblack19:01:40

library authors are not allowed to modify artifacts at the same version, I believe. But I've seen artifact checksum mismatches happen in the past with maven central when sonatype's pushed updates to the servers that hasn't reached all of the mirrors yet (or there was a bug in their updates)

pesterhazy19:01:22

there's some discussion in #datomic

spieden19:01:36

hah, i hit the javassist thing too

spieden19:01:43

just deployed the one from my cache to our artifactory

spieden20:01:18

can anyone help me figure out how to create a rettag hinted fn without defn? trying to do it from a macro, but stripped down problem is:

(set! *warn-on-reflection* true)
(defn bam ^TestBean [] (TestBean.))
(.getFooField (bam))
=> nil

(.getFooField ((fn ^TestBean [] (TestBean.))))
Reflection warning, /private/var/folders/3y/6pws2t7168l1c228pv16scvw0000gn/T/form-init3595576558932812728.clj:1:1 - reference to field getFooField can't be resolved.
=> nil

; trying to replicate what defn seems to do internally doesn't work:
(.getFooField ((with-meta (fn [] (TestBean.)) {:rettag TestBean})))
Reflection warning, /private/var/folders/3y/6pws2t7168l1c228pv16scvw0000gn/T/form-init3595576558932812728.clj:1:1 - reference to field getFooField can't be resolved.
=> nil

spieden20:01:19

i thought maybe it was on the var somehow, but that doesn’t seem to be the case:

(meta #'bam)
=>
{:arglists ([]),
 :line 1,
 :column 1,
 :file "/private/var/folders/3y/6pws2t7168l1c228pv16scvw0000gn/T/form-init3595576558932812728.clj",
 :name bam,
 :ns #object[clojure.lang.Namespace 0x49661d5d "user”]}

moxaj20:01:17

@spieden the meta is actually on the var, but nested: (-> #'bam meta :arglists first meta)

spieden20:01:23

guess i’ll just need to have my macro expand do defn then

spieden20:01:33

it had another side effect anyway, so probably more clear

sophiago20:01:40

can anyone explain the finer points of when to use volatiles vs. atoms? a typical pattern for me is having a core.async function consumed into a hash-map. values are being stored concurrently from multiple threads, but not updated concurrently under the same keys. i've always used atoms for this, but am wondering if that level of safety is necessary and i could speed it up by wrapping the hash-map in a volatile instead. thoughts?

wei20:01:01

can you write multimethods with :pre and :post?

spieden20:01:41

@sophiago first question as to whether it will speed things up is knowing it’s the bottleneck i suppose

sophiago20:01:43

based on the pattern i described i'd sort of doubt it. unless it just adds overhead for each call to swap!?

spieden20:01:47

i’m actually not clear on where the atom comes in if you’re building up a map within a single async/go-loop style form(?)

spieden20:01:21

i.e. multi threaded producing, single threaded consuming that’s building up a map inside a go block

sophiago20:01:58

the hash-map is a global i'm mutating with multiple threads that finish at different times

sophiago20:01:19

similar to how Om.Next works

sophiago20:01:47

except in this case i'm not actually mutating each key, hence why i'm wondering whether atomicity is really necessary

spieden20:01:33

i think thread safety is typically by object not by key within it

Gabriel20:01:58

is there any interaction between the values of the map? why not have, say, a hash-map of atoms?

spieden20:01:00

i’d maybe serialize everything with a channel and consume it that way

sophiago20:01:48

@spieden that route really defeats the purpose of how i'm using core.async and would lead to a very significant speed decrease

sophiago20:01:04

@lewis i didn't consider that?

spieden20:01:11

@sophiago "That’s all there is to using transients, but they have another important constraint: Transients require thread isolation.” from https://clojure.org/reference/transients

bfabry20:01:12

rather than go too much down the design path I'll just say "volatiles won't work for your use case"

spieden20:01:36

i’m getting all confused, volatiles not transients

spieden20:01:50

well, i guess it matters what you’re holding inside the volatile =)

sophiago20:01:10

@bfabry ah ok, thanks. i was having trouble understanding exactly how they differ from atoms in terms of thread safety

spieden20:01:36

@sophiago depends whether the “reduce” phase is the expensive one or the “map” phase. i.e. collecting all the values from expensive operations serially shouldn’t be a bottleneck i wouldn’t think. also you can buffer with channels

Gabriel20:01:05

> volatiles - there are a new set of functions (volatile!, vswap!, vreset!, volatile?) to create and use volatile "boxes" to hold state in stateful transducers. Volatiles are faster than atoms but give up atomicity guarantees so should only be used with thread isolation.

Gabriel20:01:18

that should answer it for you, no?

sophiago20:01:31

i'm not actually reducing anything at this point though. which also means there is thread isolation

sophiago20:01:57

i've used transients in other cases, but definitely don't think they make sense here

spieden20:01:17

yeah sorry, i was muddying the waters with transients

sophiago20:01:45

actually, since you brought it up and we're discussing mutation...i have an unrelated and more minor question about transients. i noticed you can't use nthnext with them. is there something similar to that for when i need to access the part of a sequence past a certain index?

bfabry20:01:54

I think you'd have to write your own, my guess is it's just not implemented because it wasn't needed, and they want to keep that api relatively small

sophiago20:01:58

and @lewis that's the thing...the description makes them seem like they would be suitable for my case since the threads are isolated and i'm trying to optimize speed. but i've always seen atoms used for this pattern so am a bit confused

sophiago20:01:30

@bfabry ah ok, thanks. i'll just take a look at the source if i can't think of a way around it

ghadi20:01:01

nthnext is a sequence operation. as @bfabry says the transient API is intentionally small

ghadi20:01:30

(there are no transient lists)

ghadi20:01:40

(only vectors maps and sets)

bfabry20:01:20

very vague thought: if I wanted fast thread coordination I wouldn't use an atom, I'd have all the threads pushing changes to a non-blocking queue of some kind and then a consumer thread consuming them. buuuut that's probably very specific to the "type" of concurrency problem I'm used to

sophiago20:01:44

@ghadi, but nth works with transients so i would think nthnext is ommitted for no reason other than simplicity

ghadi20:01:13

nth is an indexed operation which falls under vectors

Gabriel20:01:14

it seems that you want thread safety for the values, but not for the map itself. is that the case? if so, I really do think hash-map of atoms would be your best bet here

Gabriel20:01:27

your top-level map would still be immutable etc

bfabry20:01:42

the consuming thread could even use a transient 😱

ghadi20:01:43

nthnext is more next than nth

sophiago20:01:44

@bfabry do you say that because swap! is inherently slow? my issue with that approach would be the complexity involved in labeling the output

bfabry20:01:12

swap! is not slow, but it is atomic, so if you have many threads trying to do it at once they will block each other

sophiago20:01:23

well, i'm using blocking threads at this point anyway because i think i was overloading the go macro somehow... it was taking a ridiculous amount of memory and seemed almost as if the gc wasn't keeping up. although i'd have to do more tests to know whether that's actually a general issue with jdk on debian 😕

sophiago20:01:47

i think more likely the latter since my repl was slowing to a crawl today with just some simple sequential code and i kept having to call System/gc directly, which is...bizarre

sophiago20:01:27

but otoh, my core.async code is a bit abusive in terms of the number of channels it generates with more complex input

sophiago20:01:05

i'm using more channels than actual threads in almost all cases, so i think having them block/park just makes sense regardless

spieden20:01:37

@sophiago channels to threads shouldn’t matter as you’ll do any blocking stuff inside an async/thread i presume

sophiago20:01:53

i've actually never explicitly called async/thread. from looking at the threadcall code it seems that might be the solution to my memory management issue? since it calls clojure.lang.Var/resetThreadBindingFrame?

spieden21:01:55

that just brings thread local var bindings into the new thread

spieden21:01:48

basically go forms aren’t meant for blocking IO or heavy compute, just coordination

spieden21:01:05

.. but async/thread will give you back a channel for such things

sophiago21:01:31

i learned that about go-blocks the hard way 🙂

sophiago21:01:43

but i wasn't aware async/thread did anything other than just using put! and <!! on channels. i guess i'll give it a try

spieden21:01:17

it runs blocking code in a separate thread is all

spieden21:01:21

hmm, if partial-diff is the blocking thing you probably want to spin off a bunch of threads, collect the channels they return and do something like async/merge with them(?)

spieden21:01:38

given m isn’t used in the computation, just used to collect results

spieden21:01:01

also channels aren’t IFns so not sure what 8 is trying to do

sophiago21:01:22

oh, i may have misread the docs

sophiago21:01:31

i guess it doesn't make sense to call it inside a channel since it returns a channel...i'd likely have to rewrite this code to take advantage of it then.

bradford21:01:46

Once again, something that took me 3+ days and failed in Java took me 30 minutes in Clojure. whyyy do I never learn

aaelony21:01:00

=> (bigint "1e8")
NumberFormatException For input string: "1e8"  java.lang.NumberFormatException.forInputString (NumberFormatException.java:65)
=> (bigint 1e8)
100000000N
Is there a parse method for something like “1e8” ?

laujensen21:01:27

Whats the idiomatic way to tell clojure.java.shel/sh to send the process to the background ?

sophiago21:01:36

@aaelony what do you mean exactly?

7h3kk1d21:01:42

@aaelony (Double/valueOf "1e8")

aaelony21:01:51

@sophiago, at the cli the value comes in as “1e8” rather than 1e8. 1e8 is fine, but “1e8” is not. For an integer, a “123” can become 123. But what about a bigint?

aaelony21:01:38

yeah, @7h3kk1d but not a Double

7h3kk1d21:01:55

(.toBigInteger (new BigDecimal “1e8”))

7h3kk1d21:01:07

There may be a better way

bfabry21:01:36

if it's just some random script this is the point where I'd use the edn/read-string cannon

aaelony21:01:57

thanks @7h3kk1d, that may work...

7h3kk1d21:01:16

@aaelony I’m just stumbling blind so YMMV

sophiago21:01:29

@aaelony sorry, got interrupted. yeah, i'd do: (bigint (read-string "1e8"))

7h3kk1d21:01:18

That uses a double though

7h3kk1d21:01:23

so you may lose precision I think

aaelony21:01:56

yes. Good enough for now. Wish there was a bigint specific thing

sophiago21:01:03

true. it's not ideal

7h3kk1d21:01:00

Trying to find the java source for it.

stuartsierra22:01:40

As far as I know, nothing will parse 1e8 as an integer. The closest you can get without losing precision is (bigint (BigDecimal. "1e8"))

7h3kk1d22:01:20

http://developer.classpath.org/doc/java/math/BigDecimal-source.html Line number 377 is what does it I think for BigDecimal

tanzoniteblack22:01:26

or if you want it to throw an error if the string comes in with something that shouldn't be an integer: (.toBigIntegerExact (BigDecimal. "1e8"))

tanzoniteblack22:01:44

(i.e. (.toBigIntegerExact (BigDecimal. "18.5")) throws an exception, .toBigInterger will just ignore the error)

stuartsierra22:01:07

Note: java.math.BigInteger is not the same as Clojure's bigint.

sophiago22:01:15

^ that really confused me at first. it looks odd in the writer, but it's a good thing 🙂

spieden23:01:39

@moxaj regarding my earlier, seems there may be more going on — this is when i set the metadata after the var’s initialized

(.getFooField (map->TestBean {:foo-field 1}))
Reflection warning, /private/var/folders/3y/6pws2t7168l1c228pv16scvw0000gn/T/form-init3595576558932812728.clj:1:1 - reference to field getFooField can't be resolved.
=> 1
(-> #'map->TestBean meta :arglists first meta)
=> {:tag bean_dip.TestBean}

(.getFooField (bam))
=> nil
(-> #'bam meta :arglists first meta)
=> {:tag bean_dip.TestBean}

spieden23:01:43

wonder if the compiler does more than just set that metadata..

spieden23:01:51

guess the metadata has to be on the symbol when the compiler encounters it(?)

moxaj23:01:00

@spieden afaik, for vars, it has to be on the symbol. For functions, it has to be on the args vector. Also, my map->X isn't type hinted for some reason

spieden23:01:41

@moxaj ah yeah, this isn’t a defrecord generated fn, it’s for a bean translation lib i’m doing

spieden23:01:53

@moxaj will push soon if you want to peek =)