Fork me on GitHub
#clojure
<
2017-12-07
>
andy.fingerhut00:12:45

@tacosundae If you knew in advance that you wanted to generate random elements from the same set many times, you could improve performance by creating a vector of the set elements one time, then calling rand-nth on that vector many times. That doesn't improve performance if you only want to do it once per set, though.

danielcompton02:12:42

Is there going to be a state of Clojure survey this year?

Alex Miller (Clojure team)03:12:08

Yeah, we may slip into the new year. Kind of focused on 1.9

qqq05:12:33

(swap! some-atom func) suppose func decides to throw an exception (i.e. it can't do the update), what is the proper way to signal this ?

octahedrion10:12:41

@qqq throw the exception

chrisblom12:12:34

i'm using an agent to queue tasks, and want to wait on the task to complete, what is the proper way to do this?

chrisblom12:12:57

i'm now using this:

(let [p (promise)
      a (agent 0)
      task-fn (fn [x] (let [new-val (inc x)]
                       (deliver p new-val)
                       new-val))]
  (send a task-fn)
 @p)

chrisblom12:12:37

but this delivers the promise before the task fn completes, so it not completely waterproof

chrisblom12:12:09

in theory, it could happen that (not= @p @a)

leonoel12:12:37

@chrisblom you can't make waterproof assumptions on an agent deref

fmind12:12:27

@chrisblom could you use core.async for this case ?

chrisblom13:12:54

i only need to wait until the submitted task completes, with await it would wait for all dispatched tasks

Alex Miller (Clojure team)19:12:42

This is not correct. The docstring says “Blocks the current thread (indefinitely!) until all actions dispatched thus far, *from this thread or agent*, to the agent(s) have occurred.”

Alex Miller (Clojure team)19:12:13

unless you really want this single task

Alex Miller (Clojure team)19:12:31

in which case, agents are probably not a great solution. Just use a future and wait for it to finish.

chrisblom21:12:09

yeah, agents are not ideal, but i need to execute the tasks sequentially, one at a time, so futures will not do

Alex Miller (Clojure team)21:12:44

if you’re only executing tasks one at a time and waiting for them to finish, then why are you using concurrency features at all?

chrisblom22:12:44

to have the choice to wait for task completion, or to fire-and-forget

chrisblom22:12:01

to give you some context, i'm trying to solve this issue https://github.com/vvvvalvalval/datomock/issues/2

chrisblom13:12:44

maybe an agent is not the right solution for my problem, i'll think about it

chrisblom13:12:02

Found a solution: i can add watch, and pass the delayed deliver to it. As watches are called after the state is updated, this ensures that once the promise is delivered, the task has updated the agent state

chrisblom13:12:16

(let [p (promise)
      a (add-watch (agent {:x 0}) :force-deliver (fn [_ _ _ new-val] (force (:delayed-deliver new-val))))
      task-fn (fn [{:keys [x]}]
                (let [new-val (inc x)]
                  {:x new-val
                   :delayed-deliver (delay (deliver p new-val))}))]
  (send a task-fn)
  [@p @a])

leonoel13:12:03

that doesn't guaranteed the state has not been updated by another task meanwhile

chrisblom13:12:12

true, but thats acceptable for my use case

leonoel14:12:37

I would like some advice about bytecode generation. I have a java class with a public primitive int field I want to assign with set!, however I can't manage to get rid of intermediate boxing no matter what I hint. The decompiled bytecode for assignment is always something like :

((TheClass)the_instance).the_field = RT.intCast((Number)Numbers.num(value));
This looks suboptimal, I'm not sure the runtime is clever enough to bypass this kind of stuff. Am I missing something ?

bronsa15:12:21

you'll have to show us the code that's producing that

bronsa15:12:23

use unchecked-inc-int

leonoel15:12:31

Integer var10002 = Numbers.unchecked_int_inc(RT.intCast(0L));
var10000.field = RT.intCast((Number)var10002);

bronsa15:12:08

(set! *unchecked-math* true) and (unchecked-inc-int (int 0))

leonoel15:12:10

Integer var10002 = Numbers.unchecked_int_inc((int)0L);
var10000.field = 
RT.uncheckedIntCast((Number)var10002);

leonoel15:12:18

no warnings

bronsa15:12:26

that's not the code in your gist

bronsa15:12:24

(def ^:const your-var (unchecked-inc-int (int 0))) is the closest you'll get

bronsa15:12:54

you can't store primitive values in a var

bronsa15:12:06

so that will box to Integer and unbox

leonoel15:12:26

that's not what I want to do

leonoel15:12:18

what I want is to perform arithmetic operations on a primitive instance field and avoid boxing. Local bindings only, no vars involved

bronsa15:12:41

right I see what you mean

bronsa15:12:01

doesn't seem to be possible ATM. I'll take a look at the compiler this evening, should be an easy fix

leonoel15:12:14

thank you for your time anyway

bronsa21:12:12

feel free to vote

leonoel16:12:40

you rock ! it's definitely a huge win for me, my benchmarks are almost on par with java with your patch.

Miķelis Vindavs15:12:46

@kgofhedgehogs you probably solved it already, but take a look at select* instead of select, it returns a query object that can easily be composed later

Miķelis Vindavs15:12:05

and then realized with exec

Miķelis Vindavs15:12:00

(select tbl
  (where conditions)
  (fields ...))
(-> (select* tbl)
    (where conditions)
    (fields ...))

Miķelis Vindavs15:12:38

you can put parts of queries in functions

(defn where-thing-active
  [query]
  (where query ({:expires [> (sqlfn :now)]
                 :deleted [= nil]})))
and then compose them easily like
(-> (my-base-select)
      (where {:user_id user-id})
      (cond-> only-active? where-thing-active
              type (where {:type (-> type name upper-case)}))
      exec))

devn17:12:49

I have a directory of files from a jar I'm using and I want to do some filtering before reading them. I would like to do:

(file-seq (io/resource "com/thejar/foo-directory"))
but I cannot because of a ClassCastException java.net.URL cannot be cast to java.io.File

noisesmith17:12:47

things inside jars are not files

noisesmith17:12:03

but io/resource should be returning a file url if the jar was extracted to disk

noisesmith17:12:15

you can get a seq on the resources inside a jar if you prefer not to extract - just remember that File is never the general or abstract thing in the jvm - it’s specifically and only for entries in file systems

devn18:12:55

@noisesmith it looks like i can do something like (io/file (.getPath (io/resource ...))), yeah?

noisesmith18:12:10

if it’s a file - but only if that file has been extracted to disk

devn18:12:18

indeed it is

devn18:12:24

otherwise i would need to openConnection or something on it right?

noisesmith18:12:02

@devn I don’t know how you’d get that error if it was on disk though

noisesmith18:12:23

no, resource returns resource urls inside jars for things that are not extracted

noisesmith18:12:42

user=> ( "config/development.clj")
#object[java.net.URL 0x74a744e9 "file:/Users/justin/sprinklr/peregrine/resources/config/development.clj"]
user=> ( ( "config/development.clj"))
#object[java.io.File 0x6cb513bc "/Users/justin/sprinklr/peregrine/resources/config/development.clj"]
- no error, because it is actually on disk

noisesmith18:12:22

that error should mean that the url points to a thing inside a jar, which you can’t make a File out of

devn18:12:49

user=> (io/resource "models")
#object[java.net.URL 0x7f165eee "jar:file:/Users/me/.m2/repository/com/amazonaws/aws-java-sdk-models/1.11.244/aws-java-sdk-models-1.11.244.jar!/models"]

devn18:12:03

This is a directory containing a bunch of model files which I would like to iterate over

noisesmith18:12:07

see, that’s something inside a jar

devn18:12:34

ah my bad, i saw file: and missed the jar:file:

noisesmith18:12:34

that’s what the ! there means

noisesmith18:12:45

replicating your error:

user=> ( "clojure/core.clj")
#object[java.net.URL 0x3d6b1a48 "jar:file:/Users/justin/.m2/repository/org/clojure/clojure/1.8.0/clojure-1.8.0.jar!/clojure/core.clj"]
user=> ( ( "clojure/core.clj"))

IllegalArgumentException Not a file: jar:file:/Users/justin/.m2/repository/org/clojure/clojure/1.8.0/clojure-1.8.0.jar!/clojure/core.clj   (io.clj:61)

devn18:12:11

so, question is: how can I extract that stuff so I can work with it like a collection of files?

noisesmith18:12:27

by unzipping the jar (it’s a zip file) or iterating resources instead of files

noisesmith18:12:55

you can do everything you want on resources, you just can’t tell the jvm they are files, because they aren’t

Alex Miller (Clojure team)19:12:42

FileSystem is an abstraction and you could create one that worked directly on a zip/jar file as if it were files (like the demo at https://docs.oracle.com/javase/7/docs/technotes/guides/io/fsp/zipfilesystemprovider.html). Not sure that helps you here though… :)

noisesmith19:12:48

ha, fair enough

devn18:12:45

What do you mean by iterate resources?

noisesmith18:12:18

I mean, instead of using file-seq get an iterator on the resources

noisesmith18:12:40

@devn - this should be a decent start

user=> (.getResources (.getContextClassLoader (Thread/currentThread)) "clojure")
#object[sun.misc.CompoundEnumeration 0x77c12409 "sun.misc.CompoundEnumeration@77c12409"]
user=> (def es (enumeration-seq *1))
#'user/es
user=> (count es)
22
user=> (first es)
#object[java.net.URL 0x6be8723d "jar:file:/Users/justin/.m2/repository/org/clojure/tools.logging/0.3.1/tools.logging-0.3.1.jar!/clojure"]

noisesmith18:12:04

this is based on some quick googling about java, plus looking at the source for http://clojure.java.io/resource

devn18:12:04

@noisesmith much obliged. i've been using clojure for a good long while and never have used enumeration-seq

devn18:12:29

unfortunately there looks to be no way to do what i actually want to do, however, which is iterate over the files found in the directory named "clojure" without knowing their names

devn18:12:00

so i suspect ill need to find the jarfile on disk, unzip it to a temp dir, and read from there

noisesmith18:12:35

hmm… I know there’s a way to do it. I’ll check if I can make a simple example.

noisesmith18:12:51

@devn I decided I need to know how to do this - here’s the result

user=> (ir/zip-seq "/Users/justin/.m2/repository/org/clojure/tools.logging/0.3.1/tools.logging-0.3.1.jar")
(#object[java.util.zip.ZipEntry 0x78ffe63f "META-INF/"] #object[java.util.zip.ZipEntry 0x49f430fe "META-INF/MANIFEST.MF"] #object[java.util.zip.ZipEntry 0x5c01b068 "clojure/"] #object[java.util.zip.ZipEntry 0x1b63e89b "clojure/tools/"] #object[java.util.zip.ZipEntry 0x1c2a3bd6 "clojure/tools/logging/"] #object[java.util.zip.ZipEntry 0x74b19b6d "clojure/tools/logging.clj"] #object[java.util.zip.ZipEntry 0x25e2e763 "clojure/tools/logging/impl.clj"] #object[java.util.zip.ZipEntry 0x21c1e474 "META-INF/maven/"] #object[java.util.zip.ZipEntry 0x6cf88f11 "META-INF/maven/org.clojure/"] #object[java.util.zip.ZipEntry 0x146a1b10 "META-INF/maven/org.clojure/tools.logging/"] #object[java.util.zip.ZipEntry 0x26012f01 "META-INF/maven/org.clojure/tools.logging/pom.xml"] #object[java.util.zip.ZipEntry 0x84d8df1 "META-INF/maven/org.clojure/tools.logging/pom.properties"])

noisesmith18:12:21

@devn code :

(ns org.noisesmith.iterate-resources
  (:import (java.util.zip ZipFile)))

(defn zip-seq
  [file]
  (let [zip (ZipFile. file)
        entries (enumeration-seq (.entries zip))]
    (mapv #(.getInputStream zip %) entries)))

devn18:12:54

@noisesmith i was trying to get clever by resolving the jarfile through the resource and opening it with openStream, then using ZipInputStream to getNextEntry, but that failed 😞

devn18:12:07

gonna try yours out

noisesmith18:12:50

I just changed it to return input-streams… but really the best might be a hash-map from entry name to function that returns input stream(?)

noisesmith18:12:43

are you trying to use ~ ?

devn18:12:22

no, i was still trying to grab the path to the jar via the resource and there must be something off with that path

devn18:12:31

gonna compare them

devn18:12:49

ha! there was a trailing $1 OOPS

noisesmith19:12:32

@devn this was fun to play with from first principles, but this might just do what you want https://stackoverflow.com/a/5419767 - it’s better than my example above in a few ways

noisesmith19:12:40

that link is to a specific answer

dominicm19:12:04

Using schema, is it possible to validate against 2 keys, that one of those keys is valid? For example validate that password-confirm is the same as password, and if they're not, error password-confirm.

dominicm19:12:22

The key thing is really the error location.

noisesmith19:12:03

if you mean plumatic/schema yeah I typically use schema/conditional for that - or more specifically constrained eg. (s/constrained {:pw String :pw-c String} #(= (:pw %) (:pw-c %)))

noisesmith19:12:44

that way if the keys are missing, you get a simple error - also, if you use a named function the error will show the name of the function if the function is what fails

dpsutton21:12:27

does anyone know of a good datomic docker image? first foray fo rme

dominicm22:12:32

@noisesmith but where does the error message end up in the resulting map? The locations are being used programmatically to communicate error messages. So I would want to result to be: {:pw-c (not (…))}

noisesmith22:12:47

@dominicm I like how it does it

user=> (require '[schema.core :as s])
nil
user=> (defn a=b [x] (= (:a x) (:b x)))
#'user/a=b
user=> (def a=b-schema (s/constrained {:a Number :b Number} a=b))
#'user/a=b-schema
user=> (s/check a=b-schema {:a 1 :b 1})
nil
user=> (s/check a=b-schema {:a 1 :b 0})
(not (user/a=b {:a 1, :b 0}))
user=> (s/check a=b-schema {:a 1})
{:b missing-required-key}
user=> (s/check a=b-schema {})
{:a missing-required-key, :b missing-required-key}

noisesmith22:12:40

(not (user/a=b {:a 1, :b 0})) is as direct as it can get - since the arg was an arbitrary function

noisesmith22:12:39

and without an arbitrary function I don’t know how to check two keys at once

dominicm22:12:42

@noisesmith Yeah, I explicitly need it to be under :b unfortunately, just due to a downstream consumer.

noisesmith22:12:10

a custom reify of the apropriate schema protocol just to generate the message / return value it wants?

florinbraghis22:12:44

Hi ! I'm starting a bunch of futures and I'd like to block the main thread until all of them are finished. Is there a way to do that cleanly ?

noisesmith22:12:52

or something that checks :b explicitly, using the value under :a as a second input?

noisesmith22:12:31

@florinbraghis (doseq [f futures] @f) will do that

noisesmith22:12:48

(and return nil once all have returned)

dominicm22:12:53

I started poking around the protocol, but I couldn't see how to do it unfortunately. Maybe I need to look at that more closely.

dominicm22:12:03

I was hoping somebody might have figured it out.

noisesmith22:12:05

@dominicm what about something like (let [to-check m] (s/check {:b (s/pred (fn [b] (= b (:a m))))} m))

noisesmith22:12:22

the error message isn’t as nice, but it shows up under the b key

dominicm22:12:32

I could hack it with a dynamic var or something 😂 Which is not a style I'm usually keen on.

dominicm22:12:31

I suppose with a custom schema on the parent, the dynamic var becomes far more isolated.

tanzoniteblack22:12:33

what's the best way to select a couple values from a sorted-map in the order that they're stored in the map? (i.e. (unknown-fn (sorted-map :1 "1" :2 "2" :3 "3") [:2 :1]) ==> ["1" "2"]))

noisesmith22:12:36

@tanzoniteblack as an aside, the reader in clj lets you create the keyword :1 but it’s evil and the docs say it isn’t valid and some readers reject it

noisesmith22:12:48

you are allowed to use 1 as a key

tanzoniteblack22:12:23

good to know. Those aren't my real keys or values, just something I could use as an example to help people see the ordering

noisesmith22:12:27

@tanzoniteblack wouldn’t select-keys do that?

tanzoniteblack22:12:57

select-keys uses ret {} to put it's values into, so I don't think it keeps the order of a sorted map?

noisesmith22:12:45

(fn [m ks] (into (empty m) (select-keys m ks)) - I’ll leave the fancy name up to you

noisesmith22:12:53

I don’t know of a better option

tanzoniteblack22:12:43

@noisesmith

((fn [m ks] (into (empty m) (select-keys m ks))) (sorted-map 3 "3" 1 "1" 2 "2") [1 3])
returns
{1 "1", 3 "3"}
, which 1.) still includes the keys (easy fix) and 2.) isn't in the same order as the original sorted-map

noisesmith22:12:54

except maybe rewriting select-keys to use (empty map)

noisesmith22:12:22

wait, that sorted map literal is not the same order as the sorted-map

tanzoniteblack22:12:29

actually no, that's fine

noisesmith22:12:31

maybe you want something different, like an insertion ordered map

tanzoniteblack22:12:34

sorted-map doesn't work like I thought it did 😄

noisesmith22:12:46

haha - there’s a lib with an insertion-sorted map out there

tanzoniteblack22:12:48

eh...this is small enough, I'm just going to use a vector and iterate over values

tanzoniteblack23:12:05

(current size of vector: 2, expected size ~5)

tanzoniteblack23:12:13

performance isn't really an issue on that size

noisesmith23:12:20

also if you use the insertion-ordered one, you can just iterate and filter for the keys you care about since seq would return them in order

tanzoniteblack23:12:26

thanks for the input, @noisesmith

qqq23:12:05

if I need to overwrite "print" and "equality", is deftype the right way to go ?

noisesmith23:12:13

by print do you mean print-method (what pr uses) or toString (what str and print use)

qqq23:12:29

I wasn't even aware of that distinction.

noisesmith23:12:56

print-method is a multimethod that decides how pr / prn show things - this is used for return values in the repl and is meant to be readable by the reader if possible

noisesmith23:12:11

this can be extended at any time (like any multimethod)

noisesmith23:12:45

toString is a method on object that is used for str and print / println, and it isn’t meant to be clojure readable, and needs to be defined when defining your class as a method on Object

noisesmith23:12:03

IIRC while defrecord does give you a nice toString you can override it…

qqq23:12:41

the XY problem is: I need t define toString/equality on int-array / float-array

noisesmith23:12:47

@qqq

(ins)user=> (defrecord Foo [] Object (toString [this] "hello"))
user.Foo
(ins)user=> (Foo.)
#user.Foo{}
(ins)user=> (str (Foo.))
"hello"

noisesmith23:12:36

oh - well you can’t change arrays but I guess you could make a thing that acts like one with new toString - I think you might need a more powerful tool than deftype for that since deftype can’t do concrete inheretance

noisesmith23:12:48

so you’d likely need proxy, or gen-class, or some java code

qqq23:12:07

I was thinking of creating an object, via deftype, which has one field, the array.

qqq23:12:23

Then, I define print/equality on the deftype ... okay, so technically I'm not overwriting int-array float-array

noisesmith23:12:24

but you can’t make other people think it’s an array, is the problem

qqq23:12:33

I'm defining a wrapper around them.

noisesmith23:12:55

right - if you dont’ need to fool a consumer about the type, then that should work OK I think

noisesmith23:12:17

the trick might be methods on array you can’t extend

noisesmith23:12:42

or things like aget / aset etc. which… dunno how you would make those work without extending the class itself

qqq23:12:45

I'm writing an interpreter, like poor-man's matlab/numpy. These "virtual rrays" can only be accessed by interfaces I define.

noisesmith23:12:56

oh neverm ind you should be fine then

qqq23:12:01

I think we are loudly agreeing. 🙂