Fork me on GitHub
#clojure
<
2016-08-19
>
richiardiandrea02:08:47

can a clojure.core.async/sub be used on multiple topics?

richiardiandrea02:08:47

apparently it is not something to do, as I receive multiple copies of my messages..

richiardiandrea02:08:58

(or I am doing something wrong of course)

practicalli-johnny09:08:28

@credulous: as a simplistic approach, you could simply scrape the data off Twitter using clj-http and a Twitter search Url. Here is an example created for a dojo a while back https://github.com/weavejester/dojo-poetry/blob/master/src/poetry/core.clj

mpenet12:08:07

The streaming api is probably better for this kind of stuff, no need to keep cursors etc, should take no time using one of the twitter api java wrappers (I used to do this with twitter4j)

mpenet12:08:07

and those libs handle all the reconnection logic/rate limits etc

mpenet12:08:48

the other one is https://github.com/twitter/hbc but I just played with that one, can't vouch

dkuykendall13:08:06

Hey, I'm pretty new to Clojure web deveopment, and I'm running into a problem I'm not finding anything helpful for online. I'm trying to use compojure-api to auto-generate documentation for an api I'm building, and it works as long as the endpoint doesn't have a :return. However, if it does, the response fails any validation I give it with (not ({whatever predicate} a-java.io.BufferedInputStream))

juhoteperi13:08:27

@dkuykendall: Check you don't have duplicate middlewares encoding the response before c-api does the validation

domparry13:08:29

@mpenet: I’m assuming you would have to have an agreement in place with twitter and access to the stream?

mpenet13:08:08

there's a public access api

domparry13:08:18

That’s great news.

mpenet13:08:24

I mean, it's the same endpoint, but the content is limited to a % of everything

mpenet13:08:46

firehose access aint cheap

domparry13:08:54

that’s perfect for testing stuff though. We have a hypothesis that we want to test before paying..

domparry13:08:26

Thank you. Much appreciated!

mpenet13:08:28

in my experience it's already a lot of content, perfect to validate stuff

mpenet13:08:32

you're welcome

ikitommi13:08:49

@dkuykendall @juhoteperi if the endpoint returns a Stream, it will not be read back into clojure data by c-api for validation and the response validation not work. Either disable the response coercion (and just get the docs) or make the endpoints return clojure data structures - will be encoder later automatically by the ring-middleware-format. (There is also #C06GSN6R2)

bostonaholic14:08:46

does anyone have any good resources on JVM Reflection that would be helpful in gaining a deeper understand of how Clojure uses it? I was failing to explain the benefits of using a type annotation like ^java.util.Date other than just saying “it’s more performant”. I’d like to understand more about reflection and how the JVM handles it

credulous15:08:02

@mpenet @jr0cket Thanks for the advice!

madvas15:08:59

Guys, why

(var abc/def)
works ok, but
(var (eval ‘abc/def))
throws error like
clojure.core/eval  core.clj: 3081
              ...                
           java.lang.ClassCastException: clojure.lang.PersistentList cannot be cast to clojure.lang.Symbol
clojure.lang.Compiler$CompilerException: java.lang.ClassCastException: clojure.lang.PersistentList cannot be cast to clojure.lang.Symbol,
How to get var from symbol?

anmonteiro15:08:59

@madvas clojure.core/resolve?

madvas15:08:32

@anmonteiro: yep, that’s what I've searched for, thanks a lot!

danielstockton17:08:32

how can i alias a function in a different namespace but keep the original docstring?

arrdem17:08:15

(defn alias-with-meta [qsym] (def ~(with-meta (name qsym) (meta (resolve qsym))) (resolve qsym))

arrdem17:08:15

That may not quite work, but that's essentially what you're gonna need.

arrdem17:08:30

There are a couple libraries out there which package functionality like this.

arrdem17:08:41

Whether they are a good idea or not is a subject of extended debate.

timothypratley18:08:38

@danielstockton you may also find inspiration in the import-vars of https://github.com/ztellman/potemkin

kenny18:08:03

In 1.9.0-alpha10, is map destructuring not going to be backwards compatible? For example, in clojure 1.9.0-alpha10:

(let [{:keys [foo/bar]
       :or   {foo/bar [1]}} {}]
  bar)
=> nil
And in 1.8.0:
(let [{:keys [foo/bar]
       :or   {foo/bar [1]}} {}]
  bar)
=> [1]

danielstockton18:08:59

thanks @timothypratley, can definitely work it out from that

Alex Miller (Clojure team)18:08:00

@kenny you are correct that the behavior is different (and that’s intentional). the prior behavior was accidental and broken.

Alex Miller (Clojure team)18:08:44

the keys of the :or should always be the symbols to bind locally (not keywords or qualified symbols)

kenny18:08:16

Good to know, thanks.

Alex Miller (Clojure team)18:08:18

speaking of ^^ it contains a new spec for let that will detect the issue above at compile time

kenny18:08:47

Also for defn?

Alex Miller (Clojure team)18:08:17

yes, should affect anything that uses destructuring pretty much

seancorfield18:08:43

@alexmiller: That link to Maven is a 404 — I assume Alpha 11 just hasn’t quite made it there yet?

Alex Miller (Clojure team)18:08:00

yeah, maven central updates on a batch process so it takes as much as a day

Alex Miller (Clojure team)18:08:05

but the download is available

Alex Miller (Clojure team)18:08:13

@kenny the nice part of having a spec for destructuring is you can reuse it :)

kenny18:08:51

Right. Quite useful for all those macros that follow the same binding format.

Alex Miller (Clojure team)18:08:05

@kenny most macros that use destructuring expand to use let so they automatically get these checks. defn/fn/etc are a little different as they don’t use let

Alex Miller (Clojure team)18:08:38

is this different from the conversation we just had?

Alex Miller (Clojure team)18:08:53

that link won’t work for maybe a day. the artifact download will work in lein/etc now.

seancorfield18:08:34

Ah, I misunderstood what you meant about the download...

seancorfield18:08:49

…yes, indeed, Boot can download it. I didn’t realize that the downloads became available before they show up on Central like that.

Alex Miller (Clojure team)18:08:14

I don’t send out the announcement till it’s downloadable

arohner19:08:19

(slightly off topic) Isn’t there a development HTTPS proxy that handles self-signed SSL certs for you? I know there are manual hacks, but I’m thinking of an app with a brandname a website

arohner19:08:25

hrm, maybe it was charlesproxy

chrisjd19:08:09

@arohner An intercepting proxy? Burp supports that (https://portswigger.net/).

arohner19:08:40

@chrisjd: something to make localhost + HTTPS webdev pleasant

arohner19:08:05

I know there are lots of manual workarounds, but I was thinking of a tool specifically designed for that purpose

flyboarder19:08:44

Looking for Clojure(Script) Dev’s in the YEG (Edmonton, Alberta) area for a meetup. Trying to get a count of interested parties. Contact Me 🙂

joost-diepenmaat20:08:05

any suggestions for something like Incanter without the macros and $weirdness?

chrisjd20:08:24

Oh, I see… that’s neat. 🙂 Thanks for sharing.

arohner20:08:35

I’m not using pow, so I had to add myapp.dev to /etc/hosts, and now just works

jdkealy21:08:26

is it possible to set metadata of a namespace from within the namespace ?

jdkealy21:08:27

I'm trying to figure out how to have some of my test classes return truthy for various metadata as suggested in lein-test-refresh https://github.com/jakemcc/lein-test-refresh/blob/master/CHANGES.md#040

jdkealy21:08:35

oops.. sorry figured it out... (ns {:meta "goes here"} my-namespace ) ... that's pretty damn cool

hlship21:08:15

Are there known limitations on using with-redefs with protocol methods? I’m seeing something odd, in tests where the non-redefed method is getting invoked. Interestingly, if I reload the method with the protocol and the defrecord, then re-run my tests, it all works correctly again.

hlship21:08:53

This is extra confusing, because it is test code that used to work reliably, and I’m unaware of what changed to undermine the tests. Clojure 1.8.0 BTW.

amacdougall21:08:03

So I'm experimenting with clojure.spec, and although I definitely appreciate how namespaced keywords protect against ambiguity, I'm concerned about constantly accessing properties as (::grid/x cell) instead of (:x cell), for example. It just feels a bit verbose. Maybe in practice, you end up not noticing it? After all, people get great work done in much more verbose languages than Clojure...

timgilbert21:08:20

Personally I've moved almost entirely to namespaced keywords and I really like it

timgilbert21:08:44

I do think a keyword :x is maybe worst-case scenario for that kind of thing

amacdougall21:08:01

I guess the idea is that a) you abbreviate your aliases as much as possible, e.g. ::m/accessories instead of ::motorcycle/accessories.

amacdougall21:08:22

And even if you have to be a bit verbose, using the namespaced keyword kind of proves you're doing the exact right thing? Even without clojure.spec.

amacdougall21:08:51

And yeah, :x is kind of unfair, since it has a pretty well-agreed-upon meaning.

timgilbert21:08:11

I like that it gives you a little instant documentation throughout your code

amacdougall21:08:55

Well, at least if you're clojure.spec-ing everything, you can look at a namespace and see all the keywords it defines, if not exactly how they're all used.

timgilbert21:08:57

I'm also using datomic a lot more than I used to, so going into and out of the datastore tends to be in that format anyways

Alex Miller (Clojure team)21:08:19

@hlship if the code in the body doesn’t execute until after with-redefs returns, it’s possible the bindings have been re-set.

hiredman21:08:09

I haven't looked at it, but I would be kind of surprised if with-redefs worked with the edge cases inherent in protocol functions

hlship21:08:10

I don’t think that’s the case here, but there is some core.async involved, but I’m using <!! to block until it’s fully executed.

Alex Miller (Clojure team)21:08:32

it’s not like bindings where you have reliable dynamic scope

hlship21:08:39

I’m working to shift the redef to a function invoked from the protocol method.

hiredman21:08:58

with-redefs was written against regular var linkage, protocols extend var linkage in a few ways

hiredman21:08:30

are your protocol function calls higher order?

moxaj21:08:07

Consider the following:

(defn foo ^String [^long x]
  "")
  
(.length (foo 10))
;;=> ReflectionWarning: (...) reference to field length on java.lang.Object can't be resolved
However, if I remove the type hint on x, I get no warning. Why does it affect the type hint on the return value?

hiredman21:08:09

you should try replacing your protocol call with (apply protocolFn [args]), and when/if that makes with-redefs work, then you know the issue is with differences between var linkage and protocol linkage

hiredman21:08:15

for a longer term work around (I am just thinking that might be useful for diagnosis), you can indirect the protocol call through a regular function(which means a single callsite cache which could negatively impact performance) and with-redef that regular function

hiredman21:08:11

moxaj: a ^long primitive type hint on a function argument is a newer language feature. the addition of being able to type hint function arguments with primitives (and have the arguments actually be passed as unboxed primitives) was added along side the already existing type hinting system. The edges of the two type hinting systems didn't entirely match up, so there has been an on going effort to make them work together.

hiredman21:08:50

in the original style of type hinting, the ^String type hint would go on the var name, not on the argument list, and in the new primitive type hinting you would type hint the arg vector with the return type

hiredman21:08:18

the fact that ^String works on the arg vector is the result of the work that h as been done trying to unify the two

moxaj21:08:23

@hiredman I see, is there an open JIRA ticket just to keep track?

hiredman21:08:56

I am not sure, best person to ask is Bronsa

bronsa21:08:02

@moxaj: I believe that's been fixed already in 1.8

moxaj21:08:13

@bronsa i'm on 1.9 alpha 11

bronsa21:08:45

Clojure 1.9.0-master-SNAPSHOT
user=> (set! *warn-on-reflection* true)
true
user=> (defn foo ^String [^long x] "")
#'user/foo
user=> (.length (foo 10))
0

bronsa21:08:28

(http://dev.clojure.org/jira/browse/CLJ-1533 is the ticket I was thinking of for 1.8)

moxaj21:08:40

@bronsa just fired up a repl just to make sure it's not the editors console messing with me - and I get a warning

hiredman22:08:20

maybe double check your clojure version

bronsa22:08:03

I can repro too using alpha11, I must not be using HEAD

bronsa22:08:10

looks like a regression then

bronsa22:08:20

ok, not a regression apparently -- I can repro with direct linking disabled but not with direct linking enabled

bronsa22:08:46

@moxaj: don't think there's a ticket open for this one, can you open one? I'd like to take a look at it in the next few days if I have time

Alex Miller (Clojure team)22:08:12

@bronsa I don’t see any difference with direct linking and looks the same on 1.7/1.8/1.9

Alex Miller (Clojure team)22:08:47

the .length call is a primInvoke and will go through one of the IFn interfaces so will be typed as returning an Object at that point

hiredman22:08:38

you mean the foo call?

bronsa22:08:51

[~/.m2/repository/org/clojure/clojure/1.9.0-alpha11]> java -jar clojure-1.9.0-alpha11.jar
Clojure 1.9.0-alpha11
user=> (set! *warn-on-reflection* true)
true
user=> (defn foo ^String [^long x] "")
#'user/foo
user=> (.length (foo 10))
Reflection warning, NO_SOURCE_PATH:3:1 - reference to field length on java.lang.Object can't be resolved.
0
user=> ^D
[~/.m2/repository/org/clojure/clojure/1.9.0-alpha11]> java -Dclojure.compiler.direct-linking=true -jar clojure-1.9.0-alpha11.jar
Clojure 1.9.0-alpha11
user=> (set! *warn-on-reflection* true)
true
user=> (defn foo ^String [^long x] "")
#'user/foo
user=> (.length (foo 10))
0

bronsa22:08:16

for me enabling DL makes the reflection warning go away

moxaj22:08:32

@alexmiller @bronsa whoops, already made the ticket, feel free to edit: http://dev.clojure.org/jira/browse/CLJ-2005

hiredman22:08:56

I think that tracks, the direct linking call will generate a custom method that can return String, where as with direct linking disabled, you get a primInvoke returning Object, and because the two tagging systems don't exactly meet up, the String type info is lost

bronsa22:08:28

@hiredman: both invokeStatic and invokePrim return Object

bronsa22:08:48

it's just that prim path loses the ret type info, the static one doesn

bronsa22:08:04

public static java.lang.Object invokeStatic();
    descriptor: ()Ljava/lang/Object;
    flags: ACC_PUBLIC, ACC_STATIC
    Code:
      stack=2, locals=0, args_size=0
         0: ldc2_w        #12                 // long 10l
         3: invokestatic  #18                 // Method test$foo.invokeStatic:(J)Ljava/lang/Object;
         6: checkcast     #20                 // class java/lang/String
         9: invokevirtual #24                 // Method java/lang/String.length:()I
        12: invokestatic  #30                 // Method java/lang/Integer.valueOf:(I)Ljava/lang/Integer;
        15: areturn
      LineNumberTable:
        line 5: 0
        line 5: 9
vs
public static java.lang.Object invokeStatic();
    descriptor: ()Ljava/lang/Object;
    flags: ACC_PUBLIC, ACC_STATIC
    Code:
      stack=3, locals=0, args_size=0
         0: getstatic     #15                 // Field const__0:Lclojure/lang/Var;
         3: invokevirtual #20                 // Method clojure/lang/Var.getRawRoot:()Ljava/lang/Object;
         6: checkcast     #22                 // class clojure/lang/IFn$LO
         9: ldc2_w        #23                 // long 10l
        12: invokeinterface #28,  3           // InterfaceMethod clojure/lang/IFn$LO.invokePrim:(J)Ljava/lang/Object;
        17: ldc           #30                 // String length
        19: iconst_0
        20: invokestatic  #36                 // Method clojure/lang/Reflector.invokeNoArgInstanceMember:(Ljava/lang/Object;Ljava/lang/String;Z)Ljava/lang/Object;
        23: areturn
      LineNumberTable:
        line 5: 0
        line 5: 12
        line 5: 17

Alex Miller (Clojure team)22:08:21

hmm, I tried it a different way, maybe I misspelled the property. I can repro your direct linking result.

moxaj22:08:01

@bronsa slightly off, is this no.disassemble?

moxaj22:08:55

ah right, this is java code

bronsa22:08:26

(i just compiled the clj code and disassembled the classfile)

bronsa22:08:55

ah. I think it has to do with invokePrim calls being rewritten by the compiler as (foo 1) -> (.invokePrim foo 1) and thus losing the var info vs invokeStatic calls using a StaticInvokeExpr and thus keeping and using the var's :arglists info

bronsa22:08:34

yeah it's another instance of CLJ-1533

hiredman22:08:37

the interesting thing about watching the gradual melding of the primitive type hints and the Original Type Hints, is as they mesh together more and more, the places where they don't mesh come as more and more of a surprise to users

bronsa22:08:45

well this is a Real Bug™ as opposed to a gotcha :)

chbm23:08:02

i’ve racking my brain with something i really don’t know, i have #’myns/foo generated from (ns-resolve ns (symbol (name :foo)))

chbm23:08:31

i need the function object foo holds

chbm23:08:11

so that i can check equality

chbm23:08:14

no idea how to do that

Alex Miller (Clojure team)23:08:45

If you have a var, deref returns what it holds

Alex Miller (Clojure team)23:08:16

Just like every other state construct - atom, etc

chbm23:08:21

never thought to apply deref

chbm23:08:21

was playing around with quote and unquote