Fork me on GitHub
#clojure
<
2019-12-16
>
brandon05:12:27

I’ve encountered a weird issue - when serializing booleans with normal .Serializable means, false comes back as… truthy

brandon05:12:21

it seems to be because Clojure is essentially asking, “are you the same object as Boolean/FALSE or are you null? if not, you’re truthy my friend!”

brandon05:12:33

and when deserializing false, it comes out as not the exact same object as Boolean/FALSE but rather a new instance of Boolean - therefore, truthy

brandon05:12:42

but it still prints as “false” confusedparrot

andy.fingerhut05:12:24

@Brandon that is what is happening. Well known issue for Java, too. Don't use the Boolean constructor to create Boolean values: https://clojuredocs.org/clojure.core/if

brandon06:12:59

@andy.fingerhut this seems like a subtle bug in clojure itself, no? Though it is bad style to construct new Booleans, sometimes it’s out of one’s control. I would expect Clojure to at least support evaluating Booleans’ truthiness by value in this case

andy.fingerhut06:12:48

If it is a bug in Clojure, then it is a bug in Java, too. I believe Java devs are advised not to do this, too.

brandon06:12:25

hm, does the java if work the same way?

andy.fingerhut06:12:28

You can use (boolean x) to convert a new'd Boolean to Boolean/TRUE or Boolean/FALSE I believe.

andy.fingerhut06:12:31

(at least, I have half a recollection Java if works the same way. I believe I did that experiment when writing the page of docs I linked to above)

brandon06:12:43

the (boolean x) workaround is what I’m going with for now 🙂

andy.fingerhut06:12:29

user=> (if (Boolean. false) :truthy :falsey)
:truthy
user=> (if (boolean (Boolean. false)) :truthy :falsey)
:falsey

andy.fingerhut06:12:52

It seems to me that one could consider this a bug in the serialize/deserialize code.

valerauko07:12:10

I tried on http://repl.it and it seems to work as expected

public static void main() {
    if (new Boolean(false)) {
      System.out.println("Hello, world!");
    } else {
      System.out.println("Goodbye, world!");
    }
  }
outputs "Goodbye, world!"

andy.fingerhut07:12:23

Strange to me: @alexmiller mentioned something recently that "default serialization" should do the right thing with Java booleans, and only "clever" things should have trouble. I do not know whether .Serializable is something he would consider default or "clever"

andy.fingerhut07:12:08

@vale I tried compiling a similar one-class Java source file using AdoptOpenJDK 11, and got this message:

$ javac -Xlint:deprecation MyA.java 
MyA.java:3: warning: [deprecation] Boolean(boolean) in Boolean has been deprecated
    if (new Boolean(false)) {
        ^
1 warning

andy.fingerhut07:12:07

No such warning with AdoptOpenJDK 8, so must have been deprecated some time between there.

valerauko07:12:12

Boolean(boolean x) was deprecated (apparently) in 9 and http://repl.it runs on 10 so it's still working

valerauko07:12:19

makes sense if it's removed in 11

andy.fingerhut07:12:32

Deprecated could continue working indefinitely in Java, given desire for backwards compatibility.

andy.fingerhut07:12:56

Not removed in Java 11. Only gives a warning.

andy.fingerhut07:12:35

Still in Java 13, still deprecated

mping09:12:42

How does dependency resolution works in clj when supplying aliases? I though aliases were additive:

clj -A:test -e "(require 'next.jdbc)" ;;OK
clj -Atest:benchmark -e "(require 'next.jdbc)" ;; Syntax error (ClassNotFoundException) compiling at (clj_lx/benchmark.clj:2:3).

mping09:12:19

nvmind, extra parenthesis at the require… troll

borkdude16:12:11

is memoize thread-safe?

potetm01:12:03

Don’t use it for caching objects (like connections).

potetm01:12:29

Don’t use it when you want identical? equality.

seancorfield02:12:30

@U07S8JGF7 I assume you mean "don't use it for caching functions that accept objects (like connections)"?

potetm02:12:03

or return them

potetm02:12:37

e.g. “take a URL string, return a connection” -> oops memory leak

seancorfield02:12:40

memoize only cares about the arguments, not the return value, but I see what you mean.

seancorfield02:12:05

Well, not a memory leak, but you get the same connection object back, over and over again 🙂

potetm02:12:46

no you get different objects

potetm02:12:50

that’s the problem

potetm02:12:59

it’s a race condition on intialization

seancorfield02:12:12

user=> (defn f [url] (.openConnection (.URL. url)))
#'user/f
user=> (def ff (memoize f))
#'user/ff
user=> (def a (ff ""))
#'user/a
user=> (def b (ff ""))
#'user/b
user=> (identical? a b)
true
user=> 

potetm02:12:24

threaded repl?

seancorfield02:12:15

Ah, you're saying that if the def of ff happened concurrently in multiple threads, you'd have a problem?

seancorfield02:12:02

That's why Clojure 1.10 has a serialized require -- for thread safe requiring of namespaces 🙂

seancorfield02:12:14

(with a plan to make the regular require thread safe too)

potetm02:12:16

yeah but this is on invocation

potetm02:12:14

so, start a thread pool -> each thread calls a memoized fn -> get N diff objects

potetm02:12:37

so it helps one use case, but not another

potetm02:12:06

in general, don’t use memoize when you’re doing PLOP

seancorfield02:12:06

'k ... does clojure.core.memoize suffer from the same race condition I wonder?

potetm02:12:46

:thinking_face:

potetm02:12:54

lol a little peek into my browsing habits 😄

seancorfield02:12:11

Of course, that question is complicated by the fact that c.c.m. supports more that just a "basic" cache: it supports TTL, LRU, etc, etc.

potetm02:12:43

eh, the cache check appears to share logic

potetm02:12:51

I think to get around the issue you would have to lock around the cache check

potetm02:12:30

which, you know, clojure being clojure

potetm02:12:37

that probably don’t happen 😄

seancorfield02:12:52

Well, if you have a TTL cache behind it, even if it were thread safe, you'd get a stream of unique objects returned over time anyway...

borkdude07:12:04

That was exactly the problem, I wanted to return / produce identical objects, but memoize isn’t suited for this

borkdude16:12:07

i.e. there will never two results produced for the same arguments?

kulminaator16:12:29

your initial question is a bit off in that sense ... yes it's thread safe (execution from parallel threads breaks nothing)

kulminaator16:12:05

or even more, depending on how good you are at creating that race condition 😄

bronsa16:12:50

note the docstring:

bronsa16:12:58

> Returns a memoized version of a referentially transparent function

bronsa16:12:27

even swap! can run f multiple times, that's by design

bronsa16:12:23

that is to say, memoize is thread-safe for the inputs it's specified to accept

andy.fingerhut16:12:43

@borkdude Do you have a context where you wish that something like memoize never ran the source function twice with the same arguments?

borkdude16:12:15

@andy.fingerhut yeah, I need to have a unique object for a given key, so I don't want to have to non-identical objects for the same key

Gulli16:12:38

7768 lines :)

borkdude16:12:58

the scenario I have is that we have multiple scheduled jobs running, but I don't want more than once instance of them running at the same time

borkdude16:12:30

so I create a unique ReentrantLock object for each job (by name) which is then tried to acquire at the beginning of a job's body

borkdude16:12:48

and using memoize for that is probably not a good idea, so that's why I reverted to locking for now

borkdude16:12:28

but I'm going to do it a different way now, so never mind

bmaddy17:12:51

I occasionally remember that clojure.lang.PersistentQueue exists, but it doesn't seem to have any documentation. Is that an implementation detail of Clojure that should be avoided? Does anyone know why there isn't any documentation?

andy.fingerhut17:12:19

Clojure's persistent queues are mentioned in the Clojure cheat sheet, separately from the more commonly used ones: https://clojure.org/api/cheatsheet It does not have a named constructor function the way the others do, for which there is an open ticket, but I think they are not mentioned as much because of their more limited applicability. There is no reason to avoid them that I know of.

bmaddy17:12:22

Good, I really like them. Thanks! 😄

pablore18:12:07

I use them in a… drum rolls queue management system

👍 4
Alex Miller (Clojure team)18:12:19

Queues are not emphasized because most (but not all!) interesting uses of a queue are in concurrent code. Persistent queues are not (by themselves) useful in this way as they don't have identity - modifications return a new queue so multiple threads can't see a change unless you pair it with something stateful like an atom. So in general, I find the scenarios where queues are useful to be small (tree/graph traversal loops are one place where they are).

👍 4
Alex Miller (Clojure team)18:12:43

For the more common case of a concurrent queue, I'd recommend either core.async channels or the existing Java concurrent queue impls in the JDK as being a better solution.

Alex Miller (Clojure team)18:12:32

ClojureScript does have more support for queues and I think some aspects of that are worth pulling over to Clojure as portable calls (constructor, predicate fns), but I don't think the literal stuff is worth doing in Clojure. Just my own opinions...

rickmoynihan18:12:29

Does anyone here know what “Estimated” means with java heap sizes: e.g.

java  -XshowSettings -version 
VM settings:
    Max. Heap Size (Estimated): 3.56G
    Ergonomics Machine Class: server
    Using VM: Java HotSpot(TM) 64-Bit Server VM
Yet:
java -XshowSettings -Xmx2g -version
VM settings:
    Max. Heap Size: 2.00G
    Ergonomics Machine Class: server
    Using VM: Java HotSpot(TM) 64-Bit Server VM

rickmoynihan18:12:13

Why is it estimated? I know the JVM will calculate the default to be 1/4 available ram… At first I thought it was estimated because it hasn’t actually claimed it; however it’s also not claimed it in the second case; right?

kulminaator18:12:21

isn't it just an indicator of the automatic ~25% of available ram ?

rickmoynihan18:12:53

wouldn’t that imply the second should also be an estimate though?

kulminaator18:12:59

my machine tells me 5.5g estimated , which roughly is around the 25% mark

kulminaator18:12:12

the second one you specified

rickmoynihan18:12:32

I did — but it still might not be available when it needs it.

kulminaator18:12:03

well for that in the old days we did -Xms

rickmoynihan18:12:27

I know — and some of us still do 🙂

rickmoynihan18:12:55

I’m just curious about the choice of words there

rickmoynihan18:12:05

I don’t see how it’s an estimate.

rickmoynihan18:12:05

i.e. I one might expect “Estimated” to only disappear only when -Xms was set to the same value as -Xmx

kulminaator18:12:08

i think it's meant that "jvm estimated that you need 25% of the ram"

rickmoynihan18:12:56

ok I guess that makes sense… was just wondering if there was something else going on.

ec19:12:43

Does anyone have experience with serf / zookeeper / etcd similar tools? Examples seems to show that you run them on a node/machine and manage the cluster that way. Can I bound them to my apps or somehow watch specific apps on a single machine, so they exit when the exits etc.? Say I have 2 apps on a single machine and want to monitor their status via these tools, possible/stupid ? I guess you could create ephemeral node that would die if your app exits with Zookeeper but I'm not sure. With other tools, you could run their agents and constantly check top to see if your app alive and send events via their agents?

noisesmith20:12:30

for zookeeper at least, it considers your client gone if you are stale, which means if your app exits / machine crashes / code locks up or hard loops,, zookeeper just decides you no longer exist

noisesmith20:12:38

which is the right thing for its domain logic

noisesmith20:12:27

in zookeeper an "ephemeral node" is in fact a data storage location that promises to delete itself if your client isn't actively connected to the cluster

noisesmith20:12:44

your own app is not part of the zookeeper cluster

noisesmith20:12:31

oh wait - are you asking for in-process zk or similar? that's not a thing with zookeeper, it's incompatible with the premise

lukasz20:12:19

Only tool I'm aware of that can be embedded in a JVM app is jgroups http://www.jgroups.org

lukasz20:12:49

@cakir-enes what are your trying to achieve?

hiredman20:12:34

Most distributed consistent stores (etcd, zookeeper) require a stable quorum membership, and have a higher communication overhead per member of the quorum, so trying the lifetime of those processes to app processes doesn't make sense

ec20:12:33

yeah it really doesnt, but the thing is if I deploy one of those agents to a machine then deploy several apps on that machine how do I watch the apps and talk to other agents about their status? Like when this app goes down spin up new one with this cmd and do this etc

lukasz22:12:27

That sounds an awful lot like Nomad or a system built on top of Consul, both HashiCorp tools: https://www.consul.io https://www.nomadproject.io

hiredman20:12:17

That is not what zk or etcd does

ec20:12:18

Make apps log to /tmp/somefile.log then watch that file and send event abc didnt respond for 5 sec via serf agent is a bad idea?

hiredman20:12:25

And this likely doesn't belong in #clojure

ec20:12:14

yeah :G maybe move to off topic

borkdude22:12:38

how do I set the input for grep here:

(require '[ :as io])
(def grep (-> (ProcessBuilder. ["grep" "hello"])
              (.inheritIO)
              (.start)))
(def grep-input (.getOutputStream grep))
(binding [*out* (io/writer grep-input)]
  (println "hello")
  (println "bye"))
(.waitFor grep)

bfabry22:12:16

need to not inheritIO wouldn't you?

bfabry22:12:10

yeah you want redirectInput to be set to Redirect.PIPE not Redirect.INHERIT, then .getOutputStream will work

bfabry22:12:00

you can still keep Redirect.INHERIT for the redirectOutput and redirectError https://docs.oracle.com/javase/7/docs/api/java/lang/ProcessBuilder.html#inheritIO()

borkdude22:12:53

I did this now and it worked:

#!/usr/bin/env bb

(require '[ :as io])
(def grep (-> (ProcessBuilder. ["grep" "hello"])
              (.start)))
(def grep-input (.getOutputStream grep))
(def grep-output (.getInputStream grep))
(binding [*out* (io/writer grep-input)]
  (println "hello")
  (println "bye"))
(.close grep-input)
(.waitFor grep)

(prn (slurp grep-output))

bfabry22:12:31

np. and yes I think Redirect.PIPE is the default

bfabry22:12:09

probably should point out, you won't get any errors using things that way

bfabry22:12:05

if you want the output/error to go to your processes output I would do

(-> (ProcessBuilder. ...)
  (.redirectOutput Redirect/INHERIT)
  (.redirectError Redirect/INHERIT)
  (.start))

borkdude22:12:44

cool, thank you

potetm01:12:07
replied to a thread:is memoize thread-safe?

Not generally.