Fork me on GitHub
#clojure
<
2017-05-08
>
qqq08:05:29

in a function, is thre a easy wa yto force all arguments to be strict

qqq08:05:44

i.e. if they're lazy seqs, realize them all instead of keep them as thunks

qqq08:05:55

(this is to get better error msgs)

qqq08:05:15

sometimes, something bad happens during an lazy seq; but instead of having the error trigger when it's 'created', it's stored as a thunk, and does not get evaluated until it's needed

qqq08:05:35

and by the time it's needed, I can't make sense of "what was the stack frame when this was incorrectly constructed" ?

john08:05:01

or if you just want to realize them, it depends on the collection, I think. last might work. You can map a nothing-fn across it

qqq08:05:51

doens't plain map just create a new lazy list

qqq08:05:25

/clj (let .[x (map println (range 10))] 20)

lmergen08:05:10

is a sequence the right data structure in this case, though ? sounds like you need a vector instead

qqq08:05:03

I'm not sure; I did not reailze the importance of this issue until just now. 🙂

lmergen08:05:25

(let [x (into [] (map println (range 10)))] 20)

qqq08:05:30

lol 🙂

qqq08:05:40

I get you waht mean

qqq08:05:46

(into []) and mapv both work

lmergen08:05:55

i would go for mapv then

qqq08:05:15

I'm asking stupid questions now becuase I can'tif gured out where my bug is.

qqq08:05:22

Is reduce laxy ?

qqq08:05:27

Does it return a thunk or a value ?

rauh08:05:49

@qqq reduce is not lazy.

rauh08:05:46

Unless you do async the stacktrace should still tell you a lot. Also naming local functions help stacktraces

qqq08:05:53

it's #react it seems I'm creating a lazy list somewhere

qqq08:05:16

then, at some point, when react tries to render it, it realixes the list, and then it goes "oh shit, this function call fails since the args don't match up"

qqq08:05:27

so I get a bunch of react stack frames instead of the correct stack frames

gmercer08:05:48

@qqq (Integer/parseInt "3fd" 16) 1021

qqq08:05:36

oh, the 16 gives the base, I'm an idiot 🙂

qqq08:05:49

I was trying to figure out how it inferred 'hex' from '3fd'

gmercer08:05:56

@qqq (Integer/decode "0x3fd") 1021

noisesmith17:05:03

anyone familiar with this eastwood error?

Local name 'G__36720' has been given a type tag 'null' here:
nil
dbgx for case :op :local tag=null (class form)=class clojure.lang.Symbol (sequential? form)=false form=
G__36720

noisesmith17:05:05

don’t know why it happened, but it was fixed by not creating a core.async go block inside a method of a defrecord

sreenath.n18:05:41

Is it an error or horror? Just kidding 😐

bronsa18:05:11

when eastwood errors out like this it's a bug in eastwood or one of its deps so the error message is more for the developers of eastwood than for users

hiredman18:05:27

in a recent core.async release there were some changes to how locals "closed over" by go blocks were handled, which involved some fiddly bits with type hints

bronsa18:05:58

me right now

bronsa18:05:12

@noisesmith can you share the affected code?

noisesmith18:05:12

@bronsa - sure, I’ll make a gist with the pre/post - I still have the edits in history and the good state is safely in git

noisesmith18:05:34

@bronsa this is the raw code but not the full ns, if you need a runnable reproducing case I can try to do that later https://gist.github.com/noisesmith/3aa431f238df82f05a159f4ff8ac1816

bronsa18:05:25

yeah no worries this should be enough

noisesmith18:05:44

thank you, having eastwood around is great

socksy18:05:12

does anyone know any simple clojure library that would let me write bytes out to kafka to a specified topic?

socksy18:05:01

specifically, the latest kafka, 0.10.2 (but honestly, i guess it doesn't matter for producing so much?)

noisesmith18:05:15

I use clj-kafka, but honestly any of the various kafka wrappers out there will do it I bet, and it’s not hard to do from regular old interop either

socksy18:05:05

clj-kafka has a redis dependency that my tech leads are balking at

socksy18:05:23

just somewhat overwhelmed by the number of out of date choices

noisesmith18:05:41

that is really weird - you don’t even need the redis dep…

noisesmith18:05:13

@socksy the number of alternatives is, to me, a sign of how easy it is to just access the api directly

noisesmith18:05:42

(wish I had that stuartsierra rant about wrapper libraries handy, seems applicable here)

socksy18:05:10

redis is for consumer locking etc that most libraries are using zookeeper for

noisesmith18:05:18

right, I know

noisesmith18:05:27

and redis isn’t the thing people should be using…

stuartsierra18:05:47

> If you see a Clojure library wrapping a Java library, examine it to see if it adds any new power or abstraction over the Java API. If it doesn't, use the Java library.

noisesmith18:05:13

I swear I saw that in a blog post … can’t find it though

socksy18:05:49

well for this library it's claiming speed at least is a reason to use redis there (after all, it's the "kafka fast" library :))

socksy18:05:23

as for accessing the java library directly, there's a hell of a lot of verbose code involved with opening and closing and setting up properties and filling in classes that i would rather not do

noisesmith18:05:51

the properties will accept a hash-map of string to string, all the wrappers are doing is letting you use keywords

socksy18:05:05

i'm just looking at kinsky a bit closer, and i see that i could probably swap out the opinionated serializers with any old thing that returns bytes

socksy18:05:18

really? I didn't know that worked

socksy18:05:58

clojure hashmaps implement java map interfaces I suppose?

noisesmith18:05:00

yeah, the kafka api isn’t so bad - it’s actually scala code internally with a java facade so it’s not like it is mutable or wacky oo factory zone

noisesmith18:05:09

right, exactly

socksy18:05:03

i am not sure i can make that statement blankly about all scala code, but yeah :)

noisesmith18:05:24

sure - but it’s decent scala as that goes

socksy19:05:21

good to know

iwankaramazow19:05:18

I think I found clojure with a go backend a while ago, but forgot what repo it was. Does anyone know of such a repo?

ghadi19:05:33

I wonder how that works w/o dynamic loading in go

noisesmith19:05:19

honestly “reimplement <lispy thing> in <foo>” hits my conceptual spam filter, there’s so many 5% done projects like that out there

iwankaramazow20:05:07

I remember reading in the readme that the author Didn't have the time anymore to continue.

noisesmith20:05:21

that’s the usual status of such projects

iwankaramazow20:05:33

Anyway, not that important

john20:05:03

for the printed return value of an atom, like #object[clojure.lang.Atom 0x2a926e85 {:status :ready, :val nil}], what is 0x2a926e85? Is that the unique id of the atom? Or a thread id?

ghadi20:05:29

system.identityHashCode()

ghadi20:05:47

basically the memory location, not that you can do anything with it

john20:05:42

rgr. For the purposes of a unique id?

ghadi20:05:44

#object literal printing is relatively recent

slightlycyborg20:05:34

It isn't guaranteed to be unique because it is a hash

slightlycyborg20:05:51

"That being stated, there are situations in which the hash code returned by System.identityHashCode(Object) is "unique enough" for the problem at hand."

john20:05:39

Understood

john21:05:47

Under what conditions will an atom be in either a :pending status or a :failed status?

john21:05:51

agents, I have a better idea of there

hiredman21:05:53

an atom would be in neither ever

hiredman21:05:10

the deref code is shared between all kinds of things that implement IDeref

john21:05:27

understood

hiredman21:05:34

atoms are an interface IAtom these days, so you could maybe make your own thing that implements IAtom and triggers those states when printing

john21:05:57

I'm implementing a "SharedAtom" idea and I'd like to keep into to the Clojure ref structure as much as possible. I'm building in in ClojureScript, but since Clojure reference types have semantics closer to "shared" scenario, I'm looking at what Clojure does, beyond ClojureScript.

john21:05:31

Basically two atoms that synchronize between two thread contexts (like a worker and the main thread)

hiredman21:05:10

atoms are basically a mutable cell with the ability to compare and swap the contents, you would have to deviate from that a lot to get a pending or a failed

hiredman21:05:47

actually, maybe failed would be a result of printing out you custom atom when it couldn't reach its backing store

dpsutton21:05:38

don't update functions attempt to run 500 times on atoms and then give up?

john21:05:47

Currently, one side is an "owner" and the other a "subscriber" of sorts. The owner has a synchronous context over the value. So a swap! could return the value immediately. From the subscriber context, any mutations (`swap!`) must be asynchronous operations.

dpsutton21:05:50

not too familiar, just something i thoughtt i remembered

john21:05:05

I've got compare-and-swap going apparently fine right now

hiredman21:05:28

I don't know, but if that is the case, that failure would (should?) show up on the thread running the cas

john21:05:29

(with 10 retries, but I would like to know the Clojure default)

hiredman21:05:05

clojure doesn't have a limit

hiredman21:05:24

runs in a for (;;) { ... } loop

john21:05:30

So, I do plan on having a Pending implementation within this SharedAtom, so that I can implement callbacks on the asynchronous return value. But I'm not sure if I should queue up future swap!s on that queue, or let callers bang away in a race.

hiredman21:05:02

atoms are synchronous

john21:05:39

they are. But mutations on the subscriber side of a SharedAtom can't be.

hiredman21:05:53

I would strongly advise you against calling what you are creating an atom

hiredman21:05:00

it doesn't sound atomic in anyway

john21:05:12

They're derefs/reads can appear to be though... But I suppose the subscriber could be working with an old value...

john21:05:30

I was thinking about calling it a Molecule

hiredman21:05:30

stale reads are not atomic

john21:05:16

Any ideas for a better name for what it is?

john21:05:29

I'm going to be implementing something similar to agents, but this particular structure is simpler. No threadpool. Just a dedicated thread (worker) attached to the atom-like thing

hiredman21:05:31

I would get rid of the shared mutable ref

hiredman21:05:44

share information over channels maybe

hiredman21:05:25

I would stick a queue between the bits you want to send data to each other, instead of a mutable reference

mobileink21:05:41

atom -> schmatom?

john21:05:56

I liked quark

john21:05:16

Molecule signifies multiple atoms being linked together though, which is nice.

hiredman21:05:49

no that is terrible

john21:05:03

because of the "atomic" idea?

hiredman21:05:13

atoms are not coordinated more than one atom is a race condition

john21:05:36

Well, all swaps take place on the owning thread

hiredman21:05:01

there is no owning thread

hiredman21:05:09

atoms are not owned by a thread

john21:05:24

Not it Clojure

john21:05:03

But in ClojureScript, there's this thing I'm making, which is potentially interesting and useful, and I'm okay with not calling it an "atom" but the way you interact with it is very similar.

john21:05:14

The fact is, if you want multiple workers to update the value in an atom in another worker, in clojurescript, the safety semantics we usually associate with "atoms" no longer hold in clojurescript. Those workers do not have a consistent view of that atom, like threads do in Clojure. So I'm thinking about what kind of thing might restore as much of that safety as possible.

mobileink21:05:18

quantoid (quantum entanglement)? strange atom? dark atom? :) it's all marketing/pr in the end, heh.

hiredman21:05:25

it doesn't sound like you have restored any safety

john21:05:15

Well, I have compare-and-swap... and currently, operations are safely retrying. And the resulting value is as expected.

john21:05:32

Many mutation operations are treated asynchronously in CLJS anyway, too. So, like, imagine you have this reactive-quark in place of a ratom in reagent. You swap or swap-off on the reactive-quark in your component action handler. Then, when the new value is calculated on the remote thread and asynchronously updated in your reactive-quark, all the necessary watchers are alerted and your components automatically update, like normal. But your work was done off the UI thread.

lvh21:05:28

Does Set work on java.util.Set or only on clojure.lang.IPersistentSets?

john21:05:45

A diatomic molecule is one with just two atoms. But too much overlap with 'datomic`

lvh21:05:48

(I know it doesn’t error, but there are all sorts of values for which it returns incorrect results silently.)

hiredman21:05:21

Set meaning clojure.set?

lvh21:05:12

sorry; yes

hiredman21:05:13

functions inside clojure.set do things like use clojure.core/disj which expects persistent sets

lvh21:05:35

I don’t think that should handle keys duplicately, but it does

hiredman21:05:04

I don't have first hand knowledge of it it would throw errors on java sets or not, but I expect it was written only considering clojure sets as valid inputs

lvh21:05:46

(keys returns a seq, not a set)

lvh21:05:32

Is there still an eval bot in here? (set/union (keys {:a 1 :b 2}) (keys {:b 2 :c 3}))

lvh21:05:47

that probably doesn’t do what you’d expect, although it’s accidentally safe

noisesmith21:05:51

@lvh you can use /clj - but require clojure.set :as set

hiredman21:05:13

I would say the use of keys there is a bug

mobileink21:05:25

@john mote? almost the reverse of atom (mota)

john22:05:22

@mobileink that's pretty good

lvh22:05:37

@hiredman how so? keys looks sane; it’s the distinct-making that seems wrong

hiredman22:05:58

keys returns a seq, clojure.set requires sets

mobileink22:05:06

there's also dyad, if you wanna go greek.

hiredman22:05:31

I am pretty sure this actually well tread with in jira, clojure.set behavior being undefined on things that are not sets

john22:05:39

@mobileink let's bring it #off-topic

mobileink22:05:27

sorry naming sets me off. 😉

hiredman22:05:54

so passing things that are not clojure sets is a bug

hiredman22:05:31

so passing the result of keys on a map is a bug

lvh22:05:37

1087 is someone reporting the thing I noticed specifically; so yeah, it’s definitely a bug

lvh22:05:41

I wonder how bad the perf penalty is

lvh22:05:19

(I am trying to speed up diffing; I have a deeply nested map that when “unnested” into things that look like calls to assoc-in, it has about 1 million entries; diff is not very fast on it)

lvh22:05:47

also I don’t care about things that are equal, and in my data structures the vast majority of those entries are equal

lvh22:05:05

so clojure.data/diff spends a lot of time building a pretty big data structure that I then immediately throw away

hiredman22:05:43

yeah, well, clojure.data/diff isn't much, and obviously isn't battle tested, maybe just ignore it and write your own that is faster without the bugs

lvh22:05:58

an IPersistentMap isn’t an IMap automatically right? (I think the concrete type is)

lvh22:05:05

OK, cool, thanks, that gives me some places to start

lvh22:05:20

I had started doing that already and forgot how crazy slow non-type-based dispatch is

lvh22:05:54

also carrying state on the stack (and writing something recursively) seems to work better than having to build e.g. a persistent vector with a path into a deeply nested map — which is unfortunate because the latter is much easier to paralellize

lvh22:05:04

So I guess I’ll have to cheat a little with the parallelization 🙂

romain22:05:31

I have a small app web generated with chestnut. I did a lot of hack to test a project, unfortunately when I do lein runit hangs... lein figwheelstill works btw

romain22:05:05

How can I debug it ?

qqq22:05:47

is binding/dynamic vars 'good' ? then seem to brea purity / the notino of same input same uotput, since the code now depends on some implicit context of the dynamic var

mccraigmccraig22:05:01

@qqq i tend to avoid dynvars for exactly that reason

lvh22:05:15

it depends; does the dynamic materially change behavior?

lvh22:05:27

if it’s a log observer: don’t care

qqq22:05:42

it's like the reader monad in haskell

qqq22:05:55

say you're implementing a tex processor, you need to keep around a "tex-env"

qqq22:05:05

which stores stuff like are you in :display, :text, :script or :scripsscript mode

qqq22:05:07

among other things

josh.freckleton22:05:32

`lein test` works, but lein test-refresh fails to find a (java) :import (that it doesn't need for tests) is there a way to turn off the imports during testing?

lvh22:05:32

with protocols, there’s no way to dispatch on both IPersistentSet and java.util.Set and control which impl gets picked, right? I always want the “more derived” one but IPersistentSet doesn’t derive from Set

noisesmith23:05:49

looks like it works…

kingfisher.core=> (defprotocol IFoo (f [this]))
IFoo
kingfisher.core=> (extend-protocol IFoo clojure.lang.IPersistentSet (f [_] 'clj))
nil
kingfisher.core=> (extend-protocol IFoo java.util.Set (f [_] 'java))
nil
kingfisher.core=> (f #{})
clj
kingfisher.core=> (f (java.util.HashSet.))
java

noisesmith23:05:17

I know that #{} is also a j.u.Set but protocol dispatch seems to be doing the right thing

josh.freckleton23:05:13

how can i test that a function gets called in the finally of a try block?

josh.freckleton23:05:21

(a fn releases a resource, and I want to make sure it's called)

josh.freckleton23:05:33

I don't want to use an other lib, and the best I can think of is with-redefs that mutates an atom in scope... that seems a bit heavy though...

noisesmith23:05:56

you could make a protocol for acquisition / release, and for the test pass in an implementation that simply updates a state atom which you can verify, and pass in the real resource implementation at runtime

noisesmith23:05:36

if you don’t want to change the functions arg list, make it call a higher order function that takes the extra arg (and test the higher order function)

josh.freckleton23:05:04

hm, that seems like a clean way, but I may not have leeway to change the API that other pieces rely on

noisesmith23:05:23

right - you shell out to the parameterized function for your logic

noisesmith23:05:30

test the parameterized function

josh.freckleton23:05:37

i'm not sure I'm seeing your vision, i have

(defn test-me [] (try (f) (finally (g))))
and I want to make sure that g is called...

bronsa23:05:38

@noisesmith it's a coincidence that the clojure hashmap is dispatched through IPS rather than Set

noisesmith23:05:17

@bronsa - OK - that’s good to know

bronsa23:05:36

user=> (defprotocol IFoo (f [this]))
IFoo
user=> (extend-protocol IFoo clojure.lang.IPersistentSet (f [_] 'clj))
nil
user=> (extend-protocol IFoo java.util.Set (f [_] 'java))
nil
user=> (f #{})
java

bronsa23:05:40

e.g. i see this in my repl

bronsa23:05:21

altho IIRC alex said rich doesn't like the idea

noisesmith23:05:32

@josh.freckleton

(defn use-resource [r ...])
(defn run-with-resource (partial use-resource real-resource))
(defn test-with-resource (partial use-resource (reify IResrc (get [...] ...) (release [...] ...))

noisesmith23:05:04

this lets you substitute the side effecting part and test the acquisition / release behavior

noisesmith23:05:46

your callers would be targeting run-with-resource, which doesn’t need to expose the real-resource argument

noisesmith23:05:40

to me, the test shouldn’t even care that it’s called in the finally step - as long as it can verify that even when an error is thrown the resource is released

josh.freckleton23:05:27

so my tests will carry around mutatable state, and I'll just need to check the state after a test was ran?

josh.freckleton23:05:52

(mutatable state = resource)

noisesmith23:05:54

right - like a promise that gets delivered, or a delay forced, or just swap! a key into an atom

noisesmith23:05:57

whatever flavor you like

josh.freckleton23:05:13

that makes sense. I like that, thanks for the help

noisesmith23:05:04

the key there (to me) is that all the real logic should be in use-resource - in that case, run-with-resource shouldn’t need a unit test, as long as real-resource is trusted (it should also be kept simple for similar reasons)

josh.freckleton23:05:55

last question, it's a java resource, so the methods I'm calling are, eg .try-something, can a protocol use periods in the name?

noisesmith23:05:24

I would wrap the resource in a thin reify wrapper

noisesmith23:05:19

I mean, you could also just reify or proxy your test object taking the same methods as the java resource, by implementing the same interface or class it does…

noisesmith23:05:30

but I think the protocol/wrap method is more general

josh.freckleton23:05:48

I get reification, but I don't quite see how that helps me write javanese method names with ., would you mind clarifying?

noisesmith23:05:09

you would reify the Interface that defines the method

noisesmith23:05:15

or proxy the class that defines it

noisesmith23:05:53

but really I would prefer usually to abstract via the protocol method - doing the real method calls inside the methods of the protocol