Fork me on GitHub
#clojure
<
2017-04-24
>
qqq00:04:48

(deftype LVar [vname])
(defmethod print-method LVar [v w]
  (.write w (str "[LVar: " (.-vname v) "]")))
^^ the above code works in Clojure, but not in Cljs. Why ?

qqq00:04:53

(and more importantly, how do I fix it)

john01:04:43

I think the same is done with protocols in CLJS, rather than multimethods.

john01:04:27

I mean... Well, you have a deftype there. Do you use defmethod on a deftype in CLJ? I forgot.

john01:04:19

In any case, I believe you set the IPrintWithWriter protocol on your deftype in CLJS, do do what you're wanting.

noisesmith01:04:50

@john the way printing is set up in jvm clojure everything uses print-method for pr (the printing that is done in a repl)

noisesmith01:04:26

(with a default to str if no print-method dispatch is defined for that type)

john01:04:38

^^ (above code is untested)

noisesmith01:04:22

v is unbound

john01:04:35

oh right

noisesmith01:04:34

it works if you replace (.-vname v) with vname

noisesmith01:04:45

fields are always in scope in method bodies

noisesmith01:04:36

and should those opts be used?

john01:04:13

I haven't used opts in my experimenting with it yet. Not sure what's in there.

frederic07:04:09

Hi there. I am looking for a way to shorten namespaced keywords without actually referring that namespace. Can it be done?

frederic07:04:28

The reason being: I have two libraries that I would like to share a common vocabulary, but other than that they don't depend on each other.

frederic07:04:47

I'm considering introducing an artificial third library with basically just an empty namespace. I can then namespace the keywords there and have both my real libraries depend on that, but it seems clunky.

stathissideris08:04:10

@frederic I think a direct call to alias would do it

stathissideris08:04:28

so instead of requiring your namespace in the ns declaration just call alias further down

cmal08:04:50

Hi, what is the recommended way to start a websocket server in ring?

sveri08:04:30

@cmal There is sente: https://github.com/ptaoussanis/sente and IIRC http-kit can create websockets out of the box

qqq08:04:17

how do I define = over a deftype? I'm reading http://blog.klipse.tech/clojurescript/2016/04/26/deftype-explained.html and but I can't extend "IEquiv" for some reason

rauh09:04:32

@qqq in java it's the .equals method.

anujsays09:04:34

Has anyone used https://github.com/ymilky/franzy kafka client with https://github.com/tolitius/mount? - I am not sure how to cleanly close the client/consumer connections with mount. It results into ConcurrentModificationException and creating consumer/producer at each request is not a good idea. @ymilky - any inputs?

frederic09:04:15

@stathissideris thank you for the tip

frederic09:04:20

after some trying around : alias would be useful to avoid dependency cycles inside a single project

frederic09:04:29

across multiple projects it’s not so great, because the aliased namespace still needs to exist

frederic09:04:49

also, alias doesn’t seem to exist in cljs

cmal09:04:05

Hi, I required sente in my ns but I use sente/cb-success? it tells me No such var: sente/cb-success?, why?

jimmy10:04:24

@cmal how do you require sente ? ( do you miss the :as keyword )

cmal10:04:21

@nxqd no. If I missed, it will tell me no such ns sente maybe.

jimmy10:04:44

@cmal can you post your ns here ?

rauh10:04:11

@cmal It's cljs only

sveri10:04:04

@anujsays Its hard to guess without some code 🙂

sveri11:04:24

@anujsays if the KafkaConsumer is not thread safe I would not use it as a component like that with mount. What do the docs say? Is it ok to open one consumer per "request"? Or maybe you need a pool of readily available consumers? If its not thread safe I would think kafka expects clients to be created and closed regularly.

anujsays11:04:34

Yes @sveri that makes sense. I was just carried away with the best practices that do talk about wrapping it with mount but couldn't find an example- https://github.com/ymilky/franzy/blob/master/doc/best_practices.md to do that.

sveri11:04:19

hm, the docs are interesting as they state exactly what you do.

sveri11:04:02

But they also state you should reuse consumers / producers

sveri11:04:28

Mabe open an issue and request an example?

anujsays11:04:22

^^ Sure will do. Thanks for looking into it.

anujsays11:04:05

Yes, there is a fork https://github.com/MastodonC/franzy that has received updates along-with https://github.com/lbradstreet/franzy

sreenath.n12:04:27

@anujsays : Seems like it using https://kafka.apache.org/090/javadoc/index.html?org/apache/kafka/clients/consumer/KafkaConsumer.html under the hoods. And the docs says that it is not thread safe and will throw ConcurrentModificationException if called with out synchronization. Here are the

1. One Consumer Per Thread

A simple option is to give each thread its own consumer instance. Here are the pros and cons of this approach:
PRO: It is the easiest to implement
PRO: It is often the fastest as no inter-thread co-ordination is needed
PRO: It makes in-order processing on a per-partition basis very easy to implement (each thread just processes messages in the order it receives them).
CON: More consumers means more TCP connections to the cluster (one per thread). In general Kafka handles connections very efficiently so this is generally a small cost.
CON: Multiple consumers means more requests being sent to the server and slightly less batching of data which can cause some drop in I/O throughput.
CON: The number of total threads across all processes will be limited by the total number of partitions. Here are the options suggested by the docs:

2. Decouple Consumption and Processing

Another alternative is to have one or more consumer threads that do all data consumption and hands off ConsumerRecords instances to a blocking queue consumed by a pool of processor threads that actually handle the record processing. This option likewise has pros and cons:
PRO: This option allows independently scaling the number of consumers and processors. This makes it possible to have a single consumer that feeds many processor threads, avoiding any limitation on partitions.
CON: Guaranteeing order across the processors requires particular care as the threads will execute independently an earlier chunk of data may actually be processed after a later chunk of data just due to the luck of thread execution timing. For processing that has no ordering requirements this is not a problem.
CON: Manually committing the position becomes harder as it requires that all threads co-ordinate to ensure that processing is complete for that partition.
There are many possible variations on this approach. For example each processor thread can have its own queue, and the consumer threads can hash into these queues using the TopicPartition to ensure in-order consumption and simplify commit.

tolitius12:04:48

@anujsays kafka consumers are usually started in separate threads. take a look at "starting multiple kafka consumer threads" example: https://gist.github.com/tolitius/cc968a2adcc9dccc24cf15386fc44345 it uses gregor, but the concept is no different with vanilla kafka java api

fabrao12:04:36

Hello all, is there any way to generate clojure like static method to be called from java as string reflection?

cmal13:04:11

oh, thanks @rauh I've added it in clj.

Pablo Fernandez13:04:48

Is (last (last (:schema (first (:options (schema.core/spec Record)))))) the best way to extract a schema out of a record with schema? looks a bit convoluted.

qqq13:04:28

Is there a shorthand for:

(let [nv (get mpt expr)]
  (if nv
    (walk mpt nv)
    expr))
I'm expecting some way to simplify the use of "nv" here

tbaldridge13:04:18

not really, and any sort of macro there would probably complicate the code @qqq

qqq13:04:13

@tbaldridge : there's not something with if-let, cond-let, when-let, or an "it" ?

qqq13:04:00

wait, I think I can do it as:

(if-let [nv (get mpt expr)]
  (walk mpt nv)
  expr))

qqq13:04:14

I'll accept it at 3 lines since I have to do an if

tbaldridge13:04:33

yeah that works (I read your code wrong)

pyr15:04:36

Hi clojure!

pyr15:04:17

It is always safe to use volatiles in transducers, right?

pyr15:04:04

e.g: even when not writing a stateful transducer altogether, using (def my-xf (map (something-that-closes-over-a-volatile))) is still valid?

leonoel15:04:32

this transducer looks pretty stateful to me

leonoel15:04:52

@pyr it's ok to use volatiles only if you can ensure by other means that updates are sequential

leonoel15:04:15

transducers enforce this constraint for one instance of a transducer

leonoel15:04:31

in your case the volatile will be shared by all instances of the transducer, and multiple instances can run concurrently

leonoel15:04:44

so the answer is in general no

leonoel15:04:53

it depends who is in charge of volatile updates

pyr16:04:18

the transducing fn would

qqq16:04:29

In the clojure way, if I pass a function data that is: (1) syntatcailly valid (2) semantically valid (3) but the function can't properly handle it should the proper result be: (1) throw an exception or (2) return an err msg ?

pyr16:04:46

@leonoel it would be a case of: (defn volatile-map-fn [] (let [state (volatile! {})] (fn [elem] ...))) (def my-xf (map (volatile-map-fn)))

leonoel16:04:59

the state is shared by all instances of the transducer

leonoel16:04:27

each time you run the transducer, you reuse the same volatile

leonoel16:04:37

it's ok if you run it once, but it's still a code smell

pyr16:04:43

yes, my example was bad, I get what you're syaing now

not-raspberry16:04:08

@qqq AFAIK Clojure does not have a specific opinion on error handling. I'd throw a slingshot exception.

ghadi16:04:26

forget slingshot -- use ex-data / ex-info

ghadi16:04:37

baked into clojure.core

tbaldridge16:04:17

+1 for that, with ex-info you can throw a fair amount of information with the error (including the internal state of your function that caused the input data to be invalid)

qqq17:04:36

is this a good practice for big software projects, as sometimes it screws up cljs -> web browser -> pause on exceptions

tbaldridge17:04:52

Sure, but to put it bluntly, when I encounter that problem I find a better tool/web browser, that allows finer grained control over exceptions.

tbaldridge17:04:06

But from what you originally said, this truely is an error, so that's what exceptions are for, no?

fabrao17:04:03

Hello all, I have this in java

probe.registerHubpostCallback(this,"hubpostCallback")
, the hubpostCallback is Java static method, is there any way to convert it to clojure?

dominicm17:04:25

@fabrao (probe/registerHubpostCallback this "hubpostCallback")?

hiredman17:04:47

that looks like it might be calling a static method on an instance, which is, IIRC something java lets you do

hiredman17:04:26

but not something clojure does, so you would have to find out what class probe is, and call the static method on that class

hiredman17:04:06

are you asking about a way to convert the hubpostCallback static method that is being called to clojure?

hiredman17:04:02

static methods are sort of annoying, it must be using reflection to invoke the method, so the question is if the code invoking the method requires that it be static in some way, I sort of doubt it, I would try something like

(probe/registerHubpostCallback (fn [] (println "Hello World")) "call")

fabrao18:04:00

but how to define registerHubpostCallback?

hiredman18:04:24

what do you mean? are you creating a new probe thing?

tristefigure18:04:48

Hi I'm trying to eval a file that contains code that, when evaluated, write to this very file. I'm not expecting my modifications to the file to be taken by Clojure's reader so this is not as mad as it sounds. My problem is the following: the program hangs when I try to write to the file. If I commit my modifications to an atom and write them to the file after loadhas done its job, then everything is alright. The errors from the reader are syntax errors of various kinds: basically the reader sees my changes to the file and get lost in parenthesis mismatches or whichever errors the modification causes. Ideally I would like the reader to slurp the whole file then proceed to do it's reader-job. How can I achieve that ?

mobileink19:04:08

tristefigure: use boot and the problem goes away.

tristefigure19:04:36

oh thanks, i'll give it a shot right away

mobileink19:04:34

there's a bit of a learning curve, but the basic idea is that you transform a "fileset" and filesets are immutable. so you can xform file X and write it to X - but it will be a "new" X, if that makes sense - you never actually overwrite the input file. the folks on #boot are very helpful.

sreenath.n19:04:55

may be you can do a copy of the file that you want to evaluate. And then read from the duplicated file. When each form has been read and evaluated, write the result back to the original file. cc: @tristefigure

sreenath.n19:04:33

Probably before writing the results back to the original file, you can check the last modified date and make sure that you are indeed evaluating the latest changes 😉

tristefigure20:04:32

I hadn't thought of that, this looks like the right solution . thanks !

mobileink20:04:33

completely unneccessary with boot.

dpsutton18:04:24

does your program hang when it writes to a different file?

qqq18:04:25

Why is no one else asking: "why is a file, when being evaled, writing to itself, and thus depending on file system semantics? is there no other way to structure this?"

clojureman18:04:39

@tristefigure, you might be biting you own behind! That said, one way to achieve what you want is to slurp then read-string

tristefigure18:04:43

no and so far the solution I have adopted is to write to a tmp-file that overwrite the target file once load has done its job

tristefigure18:04:37

@qqq: I'm building an in-file debugger

qqq18:04:08

oh, so you're modifying the file to add instrumentation to it?

tristefigure18:04:25

to add instrumentation results as comments

qqq18:04:52

okay, I can see how this is a reasonable thing to do

qqq18:04:08

I'd still write to a tmp file, then cp it over

tristefigure18:04:53

This works pretty well, except for long computations : you need to wait for the whole file to be executed before seeing any result

qqq18:04:27

can't you just tail -f ?

qqq18:04:38

how does writing to the same file allow you to see results earlier than writing to a tmp file ?

tristefigure18:04:02

Because the file is opened in your editor, the tmp file is at an obscure path, many clicks away

qqq18:04:42

can you output stuff.clj to stuff.clj-out ? 🙂

qqq18:04:00

sorry if I appear to be a jerk for trying to get you to not write to same file

tristefigure18:04:10

mmh, we have terminals for that.

tristefigure18:04:22

It's okay, I'm having quite some fun in this conversation actually

tristefigure18:04:41

And i have to confess. I'm more about the art than the programming

tristefigure18:04:03

Well this seems to be a "hard" problem involving hairy concepts such as InputStreams, I'll set it aside for later, and carry on with my art-project.

hiredman19:04:14

generally, that kind of thing is done through an editor integration, for example cider (the emacs stuff for clojure) will, after you evaluate a form, add some temporary contents to the buffer indicating the result of evaluating the form. cider doesn't actually write that stuff to the file, it tell emacs the result of evaluating whatever, then the emacs lisp side of cider sticks some temporary data in the buffer to display it, it is never written to a file.

dpsutton19:04:33

there are some format commands that will send your code over to the middleware and then edit the buffer. format edn and the like

hiredman19:04:11

yeah, but what happens with is almost certainly emacs sends the stuff to format to clojure, clojure formats it and sends it back to emacs, then emacs replaces the unformatted bits with the formatted bits

hiredman19:04:51

I mean, I don't know, I haven't looked at it, and never use anything more complicated that cider-connect from cider, but that is typically how that kind of thing is done

hiredman19:04:20

it separates evaluation and program execution (clojure) from file editing and presentation (emacs, whatever)

tristefigure19:04:58

Yet one can't comme without the other one

dpsutton19:04:08

oh yeah, this is for sure elisp modifying the buffer and not a java process

joshjones20:04:21

I have a java object that I'm being handed by a factory method. We then use a macro to construct a new object using reify, based on the constructed java object, in order to override a single method with a different implementation. I need this object to implement a protocol, but because the object (and thus the class name) is dynamically generated, I cannot give the class to extend (et al) .. at least I don't think I can. So my current solution is to have java.lang.Object implement my protocol, and this works, as my object is of course an Object; so it's a 'catch-all'. However, this is ugly, and I don't like it. Any suggestions?

ejemba20:04:33

@joshjones do you know which tool is generated the object ?

joshjones20:04:43

yes, it is an external project which generates the object. We considered instantiating the object ourselves, but then we get even more involved in the inner workings of the library, as the constructor is very involved and does a lot of internal work. In short, we need to just assume that we are handed the object from the factory method, and have that to work with.

joshjones20:04:59

Or do you mean something else?

ejemba20:04:15

as you stated the object is generated, it depends which tool is actually generating the object, in function you can or not "guess" the name of the object

joshjones20:04:03

we have a macro which takes the object and uses reify to construct a new one. AFAIK reify will not allow us to specify the name of the new object, but will generate it

hiredman20:04:05

rewrite your macro to generate a defrecord, and take the class of the object, instead of an instance of the object, call your macro at the top level, then just wrap the object in the record

joshjones21:04:37

that sounds like it might work, looking at it now - thanks 🙂

hiredman21:04:10

for that case, I would like not even write the macro, just some code that generates the defrecord code, then just paste in the generated defrecord code. you aren't creating a new control flow construct, and the classes and methods involved in that kind of thing aren't likely to change ever

fabrao22:04:22

Hello all, I defined

(ns custom
  (:gen-class
   :name Custom
   :main false
   :prefix "-"
   :methods [[probe [] void]
             #^{:static true} [processaMensagemCustom [com.nimsoft.nimbus.NimSession com.nimsoft.nimbus.PDS] void]]))
(defn -probe [] (str ""))
(defn -processaMensagemCustom  [] (str ""))
(ns core
(:import (Custom)))
But when I do
lein uberjar
say that can´t find Custom. Why?

fabrao22:04:04

how do I import Custom "Java class" in core namespace?

hiredman22:04:42

you can't import it, the name Custom puts it in the default package, you can only import things that are in named packages

hiredman22:04:19

whatever you are doing, I strongly recommend avoiding gen-class and aot compilation if you can

hiredman22:04:13

looking at http://docs.nimsoft.com/prodhelp/en_US/Monitor/SDK/Java/com/nimsoft/nimbus/NimProbe.html the example shows some kind of callback being registered that isn't a static method (looks like a slightly different kind of register call than your original question) so I would still suggest trying something like (.registerHubpostCallback probe (fn [] (println "Hello World")) "call")) where probe is an instance of NimProbe as constructed in the javadoc examples there

hiredman22:04:06

it will probably throw an error either about mismatched types of arguments or mismatched counts of arguments, either way you can work from there

fabrao22:04:46

hiredman I include

(compile 'custom)
in core and it generate the jar

hiredman22:04:00

don't do any of that

hiredman22:04:10

like, open a repl, type (.registerHubpostCallback (NimProbe. "RandomProbe","1.0","MyCompany",args) (fn [] (println "Hello World")) "call")) and see what happens

hiredman22:04:34

you may need to replace args with an empty array of strings or something

fabrao22:04:05

but,

public void registerHubpostCallback(Object object, String methodname)
    throws NimException
  {
    setHubpostCallback(object, methodname);
  }
  protected void setHubpostCallback(Object object, String methodname)
  {
    this.hubpostObject = object;
    this.hubpostMethodName = methodname;
  }

hiredman22:04:23

what about that?

fabrao22:04:42

it waits an object and methodname

hiredman22:04:02

(fn [] (println "Hello World")) evaluates to an Object and "call" is a String naming a method on the Object

fabrao22:04:22

Let me try here

hiredman22:04:31

user=> (supers (class (fn [] (println "Hello World"))))
#{clojure.lang.IMeta java.util.Comparator clojure.lang.AFn clojure.lang.IFn clojure.lang.Fn java.util.concurrent.Callable clojure.lang.IObj clojure.lang.AFunction java.io.Serializable java.lang.Runnable java.lang.Object}
user=> 

hiredman22:04:31

user=> (->> (fn [] (println "Hello World")) (class) (.getMethods) (map #(.getName %)) (distinct))
("invoke" "withMeta" "meta" "compare" "applyTo" "call" "applyToHelper" "throwArity" "run" "wait" "equals" "toString" "hashCode" "getClass" "notify" "notifyAll" "reversed" "thenComparing" "thenComparingInt" "thenComparingLong" "thenComparingDouble")
user=> 

hiredman22:04:25

hah, I was wonder what the heck reversed and all the comparing stuff is, those are the result of the new default method stuff in java 1.8 I guess

hiredman22:04:56

what a world

fabrao23:04:26

(defn probe []
  (let [nimProbe (NimProbe. "custom" "1.0" "COMPANY CO." nil)]
    (doto
        nimProbe
      (. setSubscribeSubject "QOS_CUSTOM")
      (. registerHubpostCallback (fn [sessao dados] (println dados)) "call"))
    (do-while (. nimProbe doForever) (do (println "Loop iniciado")))))

fabrao23:04:41

-> Unhandled java.lang.ArrayIndexOutOfBoundsException 0

fabrao23:04:22

not worked

hiredman23:04:26

so look at the stack trace, go to the place where the error came from, which will be around where it is attempting to call the call method, my guess is it is trying to reflectively find a method with that name and the arguments it expects

hiredman23:04:06

if that is the case, you sadly cannot use a regular fn, but you should be able to use definterface and reify

fabrao23:04:36

I tried using gen-class, but it fails when

lein uberjar

fabrao23:04:48

and I have to use (compile 'custom)

fabrao23:04:00

to work in REPL

hiredman23:04:03

(definterface SomeName (someMethod [this ^TypeOfArg1 arg1 ^TypeOfArg2 arg2])) fill in the real argument types there, replace the (fn ...) with (reify SomeName (someMethod [this arg1 arg2] (println "Hello World"))) and replace the "call" method name with "someMethod"

hiredman23:04:10

don't do any of that with lein

hiredman23:04:17

just do at the repl

fabrao23:04:10

Like this?

(definterface CallbackThis
  (^void processaMensagemCustom [^com.nimsoft.nimbus.NimSession session ^com.nimsoft.nimbus.PDS pdsData]))

(defn probe []
  (let [nimProbe (NimProbe. "custom" "1.0" "COMPANY CO." nil)]
    (.setSubscribeSubject nimProbe "QOS_CUSTOM")
    (.registerHubpostCallback
     nimProbe
     (reify
       CallbackThis
       (processaMensagemCustom [this arg1 arg2] (println "Hello  World")))
     "processaMensagemCustom") 
    (do-while (.doForever nimProbe) (do (println "Loop iniciado")))))

hiredman23:04:07

type it in your repl and let it rip

fabrao23:04:08

man, it worked !!!! Many thanks

hiredman23:04:53

don't forget to delete all the uberjar and gen-class stuff

hiredman23:04:10

well, just the gen-class stuff, keep the uberjar, get rid of aot compilation

fabrao23:04:32

ok, best regards

qqq23:04:09

when defining macros, when an error happens, it seems to all get pointed at point of macro expansion ; for multi line macro applications, I'd prefer to get better line number info

hiredman23:04:17

this is really hard to do, macros are a src to target transformation, and source and line number information only make sense for the src, and the target may have nothing at all to do with the src

hiredman23:04:55

which is to say, you may want it, but you cannot have it without significant changes to the clojure macro system

qqq23:04:43

how do things like (as-> .... ) work then?

hiredman23:04:52

what do you mean?

hiredman23:04:27

I don't see what one has to do with the other

qqq23:04:30

(as-> (expr) $
  (.... $ ... )
  (.... $ .... )
  (.... $ .... ) ; if err happens on this line, we get error msg pointing to this line
  (.... $ .... ))

qqq23:04:57

so as-> is an macro right? yet, how does it expand in such a way that when an exception happens, we get pointed to the right "line/clause" before the expansion ?

hiredman23:04:07

you don't actually get an error pointing to that line

hiredman23:04:27

you get an error pointing to something that is kind of close

hiredman23:04:36

oh, actually maybe you do

hiredman23:04:50

as-> doesn't change the input structures very much

hiredman23:04:07

so the metadata the compiler uses to attach line numbers remains intact

hiredman23:04:53

which works for simple macros, but not for more complicated macros

qqq23:04:37

so here's the XY problem

qqq23:04:48

I want to implement Haskell's monad/bind

qqq23:04:07

so I'm going to have huge

(domonad ...
  ...
  ...
  ...
 ...)
blocks

qqq23:04:16

and I really need to get correct line numbers, otherwise future me is going to hate current me

hiredman23:04:35

it will depend on how much your domonad form munges things

hiredman23:04:08

but, like, also, you should be aware you are the 5,000 person to do this thing, and maybe stop and ask why and should you even bother?

hiredman23:04:05

there are a lot of clojure monad libraries, there are even a few that more than a single person uses, and there is even one in contrib