Fork me on GitHub
#clojure
<
2017-06-22
>
creese00:06:55

I've run into a strange compiler error that only seems to manifest when I pull in a secondary library via checkouts. I've verified the problem is not code as it works on another machine. Tried reinstalling java and Leiningen to no avail. Any idea what this could be?

creese00:06:36

user=> (require '[foo.bar])
CompilerException java.lang.ExceptionInInitializerError, compiling:(foo.bar.clj:1:1)

creese00:06:13

If I remove the symlink in checkouts, everything seems to work fine.

Alex Miller (Clojure team)02:06:38

frameworks are things that call you, rather than things you call

Alex Miller (Clojure team)02:06:47

(which are more likely components)

mobileink02:06:16

hmm, seems a reasonable defn

Alex Miller (Clojure team)02:06:46

not something I made up and probably not 100% satisfying, but I think it’s about right

tianshu04:06:41

https://github.com/technomancy/leiningen/blob/master/doc/DEPLOY.md I'm trying to deploy to private repo according to this tut, but every time I use lein deploy, It will ask for username and password. I mentioned it's said there's a bug for gpg, it that mean the username and password in credentials.clj.gpg will not work, orI have to do something?

royalaid07:06:05

Hey all, I am running into what I think is a bug with how clojure's java interop works

royalaid07:06:50

I am trying to wrap https://github.com/lokra/seaweedfs-client for use in clojure

royalaid07:06:05

but when I try to port the example upstart code

royalaid07:06:13

FileSource fileSource = new FileSource();
// SeaweedFS master server host
fileSource.setHost("localhost");
// SeaweedFS master server port
fileSource.setPort(9333);
// Startup manager and listens for the change
fileSource.startup();

FileTemplate template = new FileTemplate(fileSource.getConnection());
template.saveFileByStream("filename.doc", someFile);

royalaid07:06:03

(defn connect!
  [host port]
  (let [client (FileSource.)]
    (.setHost client host)
    (.setPort client port)
    (.startup client)
    (reset! client-atom client)
    (let [value (.getConnection @client-atom)]
      (reset! file-template-atom (FileTemplate. value)))))

royalaid07:06:14

I get an error similar to IllegalAccessError tried to access class org.lokra.seaweedfs.core.Connection from class <class info>

royalaid07:06:17

Any suggestions or workarounds I can try?

royalaid08:06:49

Plot thinkens, I can use the code without problems on my Mac but on Windows it breaks

royalaid08:06:53

:thinking_face:

gaverhae10:06:00

@royalaid What's the Java version on both? lein version should print it

gaverhae10:06:33

Also possibly compare the outputs of lein deps :tree

Alex Miller (Clojure team)12:06:09

I don't see any signs that CLJ-1243 has anything to do with it

ido12:06:09

hey guys! I want to read a stream with unknown number of lines (e.g. a file) and fire an async get request per each line (lets say using an http-kit async get with (go (>! c response)) as a callback). then, read the responses one by one into another file. is there an elegant way of closing the channel making sure I am not missing any async puts into the channel? I can do so with putting my own guard msg at the end but it feels awkward.

gaverhae12:06:35

I'm not sure I understand what you're trying to do exactly; any reason why the producer can't just close the channel when the stream runs out?

ido13:06:05

@gaverhae the producer might close the channel before the last response is put there

gaverhae13:06:58

How so? The producer being (in my mind) the single sequential process that reads from the stream and puts on the channel, it should be able to not close the channel before it's finished reading the stream

gaverhae13:06:33

maybe you're missing a channel

gaverhae13:06:16

read file -> channel -> http calls -> channel -> write file ?

gaverhae13:06:11

if that's the general shape and you want to ensure some parallelism in the "calls" step, maybe look at what the pipeline functions have to offer in core.async?

gaverhae13:06:23

I would suppose they already handle closing the channel in the right order

twicebaked15:06:02

Can anyone either give me some thoughts or tell me the best place to ask about some architecture issue I'm trying to sort out? Long story short, I have a Clojure + Clojurescript web app that needs to extend itself as a desktop app and as a user-owned server (with web admin api) that can have user created functions (think add-ons, plug-ins, arbitrary user code). I'm trying to find the best way to implement these two new pieces that highlights ease of installation, extensibility, minimal system requirements, and shared codebase as much as possible. The desktop app itself is surely going to end up being electron-based. As far as the code, there's a "simulation" engine that I don't want to rewrite in multiple languages unless from scratch to be shared, and is currently in .cljs files. My ideas thus far are - Option 1: Write the user-run server in Clojurescript + Node.js (running on node.js as the server) with user-defined JavaScript functions that get called from Clojurescript/JavaScript, and expose the Clojurescript API via externs Option 2: Write the user-run server in Clojure and run a v8 instance where I eval anything written by users in pure JavaScript, returning and sending json to communicate (at the cost of some speed and cruft). Any other ideas or thoughts? Or anywhere someone can point me? Thanks.

jeff.terrell15:06:33

@twicebaked - Don't have time to think deeply about this now, but your question strikes me as one that could be asked at the Clojure or Clojurescript subreddit. That way it's less likely to be buried by other conversations while people think about it.

axl31616:06:39

Any news on when clojure.spec become part of core?

dominicm16:06:09

It won't. It's been split into its own library.

noisesmith16:06:08

wasn’t it going to be merged in later, on a “when it’s done” kind of schedule (so after 1.9) ?

shaun-mahood16:06:47

There's a good discussion about this on the latest defn podcast (https://defn.audio) - with Stuart Halloway - it's not on the website yet but it's episode 23 - it's in the RSS

Alex Miller (Clojure team)17:06:03

the true answer is: we have not yet decided

Alex Miller (Clojure team)17:06:53

it is effectively part of core now as Clojure depends on spec.alpha. thus users of Clojure get both. Whether and when the two units of code are co-located will depend on how things unfold (but should not actually matter to users)

eoliphant16:06:28

Hi, I'm trying to figure out how to do something that's embarrassingly simple, but im just having no luck lol. Basically, I need something that does the following

(check #{1 2} [1 2 3]) => true
(check #{1 2 4} [1 2 3]) => false
for check i've been through variations of some, every? not-any?, etc. But still can't quite get what I'm looking for . I just need to flag the fact that there's something in the first collection that's not in the second

a1316:06:27

convert 2nd arg to set?

dpsutton16:06:48

are you looking for set equality or set subset?

eoliphant16:06:45

not equality, just that "a has something that b doesn't" ,it's ok if b has somethign that a doesnt

eoliphant16:06:04

yeah I guess set/difference would work too

eoliphant16:06:17

the problem is that 'set b' may be large

dpsutton16:06:21

subset sounds like it?

dpsutton16:06:32

oh, you need one to be a proper subset of the other

eoliphant16:06:51

so was wondering about the cost of converting from a list/vector to a set to do the comparison

eoliphant16:06:06

but i think I'll just go with that for now

a1316:06:20

set uses transients, so don't think it would be a big overhead

eoliphant16:06:40

ok wasnt sure how effcient it was in that scenario

a1316:06:53

(defn set
  "Returns a set of the distinct elements of coll."
  {:added "1.0"
   :static true}
  [coll]
  (if (set? coll)
    (with-meta coll nil)
    (if (instance? clojure.lang.IReduceInit coll)
      (persistent! (.reduce ^clojure.lang.IReduceInit coll conj! (transient #{})))
      (persistent! (reduce1 conj! (transient #{}) coll)))))

eoliphant16:06:58

ok will give that a shot if it behaves then i'm good

a1316:06:14

I just jumped to definition in my emacs/cider (M-.) 🙂

dpsutton16:06:46

so i guess it would be (and (subset my-set (set coll)) (not-empty? (remove my-set coll))

gaverhae17:06:52

(not-empty? (set/difference (set a) (set b))) ?

dpsutton17:06:14

that wouldn't ensure that a is a subset of b though

noisesmith17:06:43

if the difference is empty, a is a subset of b

tanzoniteblack17:06:50

if you’re just trying to ensure a subset, can’t you just use set/subset? https://www.conj.io/store/v1/org.clojure/clojure/1.8.0/clj/clojure.set/subset%3F ?

dpsutton17:06:05

But disjoint sets have a non empty difference which was certainly not according to his original requirements

joshjones17:06:54

@dpsutton he said "I just need to flag the fact that there's something in the first collection that's not in the second" -- disjoint sets certainly seem to fulfill that requirement, as there is definitely "something in the first collection that's not in the second"

mdrago102618:06:24

what is the "best" way to store a c3p0 pool to be accessed by multiple namespaces?

tanzoniteblack18:06:38

mdrago1026: You can use something like https://github.com/tolitius/mount or https://github.com/stuartsierra/component to manage complex required system dependencies. Or for something simple like a c3p0 pool, which keeps track of it’s own errors for the most part, I tend to just use something like (defonce conn-pool (delay (create-c3p0-connection)) and everywhere else you want to refer to it you can require the config namespace and reference the pool with @conn-pool

mdrago102618:06:31

awesome. thanks a ton. I will take a look at these

tanzoniteblack18:06:32

@mdrago1026 on a very random, unsolicited side note, I prefer http://brettwooldridge.github.io/HikariCP/ over c3p0. Nothing inherently wrong with c3p0, and I’ve used it in production quite a lot, but hikariCP fits better with what I’m used to in Clojure (in terms of “it just kind of does the correct thing”)

mdrago102618:06:44

very interesting. let me read up on this. I've heard of Hikari and simply always default to c3p0 since that's what i always have used

tanzoniteblack18:06:15

way more than the speed of things (which is not insubstantial), this is why I prefer hikaricp: https://github.com/brettwooldridge/HikariCP/wiki/Pool-Analysis#c3p0

mdrago102618:06:48

wow this is neat

mdrago102618:06:54

I'm going to try this out

gaverhae19:06:52

Another piece of unsolicited advice: you should not think of namespaces as accessing anything; namespaces are passive collections of functions. And if a function needs access to a connection pool, it should receive it as an argument, regardless of what namespace it's in.

mdrago102620:06:23

makes sense. I always got confused when it came to "sharing a connection" because everything else is functional and stateless, then comes the connection

payal19:06:27

I am looking for some help for code coverage tools for clojure, has anyone used cloverage before?

hcarvalhoaves19:06:11

@payal worked fine last time I used, gave a nice report for lisp (based on forms covered)

payal19:06:17

just figured out why I am not able to get it..so my tests goes through all well when I do lein test, but are failing when I use cloverage

hansen-pansen19:06:37

@mdrago1026 Use defstate from https://github.com/tolitius/mount, or look into https://github.com/stuartsierra/component. BTW, do you need to stick with c3p0? I enjoyed https://github.com/tomekw/hikari-cp quite a bit.

tanzoniteblack20:06:07

hansen-pansen: 😄 you and me? We’re on the same page, apparently…just different threads 😄

hansen-pansen20:06:37

I felt quite ashamed that I sent the same advice hours later 😀

hansen-pansen19:06:27

Oh, sorry for replying out-of-thread. 😳

mdrago102620:06:09

@hansen-pansen No problem. I don't need c3p0 (used it before for work, so I just default to it). @tanzoniteblack also told me to look at hikari and it looks awesome. So thank you both

mdrago102620:06:33

and mount and component both look great too

hansen-pansen20:06:44

Is there a library that offers a worker-queue type of concurrency, i.e. enqueue work for a fixed-sized thread pool? I looked into claypoole, but it works on sequences.

tanzoniteblack20:06:43

hansen-pansen: if you’re familiar with java, you can do it the same ways you would in java with clojure. If I’m not too worried about thread priorities and contention, then I’ll just use core.async with a fixed number of “workers” (i.e. go-loops) that are pulling from a shared channel, and then write jobs to that channel

hansen-pansen20:06:18

I already rolled a n00b-version with core.async, but it felt … too low-level. I looked into the amazing manifold, which offers backpressure and all the nice stream operators, but it lacks the pooling part. Just wondering whether somebody else stumbled over the same problem.

hansen-pansen20:06:38

Ohh, and I have close to zero Java background, but if that is an option, I will take a look. I guess you mean something from java.util.concurrent?

noisesmith20:06:27

agreed you can just use an ExecutorService

tanzoniteblack20:06:43

depends on your requirements. the main problem with core.async for worker pools can be that it uses the same thread pool for all core.async related channels. So if this is supposed to be a background operation, and you also use core.async in something like handling production web requests, then your background jobs might be interfering with your web requests. If that’s your case, you’ll probably want to use ExecutorService

noisesmith20:06:54

but if you use core.async pipeline-blocking properly you can do it at a high level with clojure too

noisesmith20:06:03

where the queue is the buffer on the chan, and the thread count is the parallelism arg

hansen-pansen20:06:13

pipeline-blocking is what I used with my n00b version. And yes, I try to use concurrency mostly for background web requests.

hansen-pansen21:06:51

Thanks to your ideas on this topic, I propably should just put more brains on the core.async/pipeline-blocking idea and think of some error-passing messaging or channel.

hansen-pansen21:06:20

https://stackoverflow.com/a/41448848/2068691 brings me to some ideas. Thank you very much for your ideas and inspiration!

hansen-pansen21:06:07

Thank you very much, guys! Your help is very much appreciated. This community is so cool!

michaelblume20:06:39

So this function reflects

michaelblume20:06:51

(defn test-reflect [^javax.servlet.http.HttpServletResponse resp]
  (proxy [java.io.FilterOutputStream] [(.getOutputStream resp)]
    (close []
      (proxy-super close)
      (println "hello"))))

michaelblume20:06:01

specifically the call to proxy-super reflects

michaelblume20:06:13

and I’m wondering why

michaelblume20:06:24

it seems like it should be possible to resolve .close()

hiredman21:06:52

proxy doesn't type hint this

jeaye21:06:11

@michaelblume I was about to make a PR with a fix for that.

jeaye21:06:25

Assuming you're in ring; looks like it to me.

michaelblume21:06:34

beat you to it =)

jeaye21:06:03

Did you get the rest? 😄

michaelblume21:06:20

ring-servlet no longer reflects, didn’t check the other subprojects

jeaye21:06:52

That's what I had locally.

michaelblume21:06:14

hmm, lein check runs cleanly for me with just what I have

jeaye21:06:34

Did for me, too, but I see the reflection in my project using ring.

jeaye21:06:59

So I chose not to trust lein check, since I can see in the source how it would be reflecting. 🙂

jeaye21:06:57

Interested in applying the other changes, as shown in my gist, to your PR?

michaelblume21:06:08

I’ll look into it =)

michaelblume21:06:29

oh, you’re in the jetty adapter, I see

michaelblume21:06:58

ah, yeah, lein check reports problems

jeaye21:06:34

Ah, fair enough. Sanity is restored.

jeaye21:06:55

My approach to fixing the proxy-super is different, but the effect is the same.

michaelblume22:06:07

@jeaye I also added a return type hint so my code wouldn’t reflect

jeaye22:06:17

Yep, saw that before I started on the reflection work.

jeaye22:06:03

That PR looks good; thanks for beating me to it and updating it for ring-jetty-adapter. 🙂

mdrago102622:06:12

can you use environ when making a war file and deploying via tomcat? for example when i run lein with-profile dev ring uberwar... does that do anything?

bfabry22:06:18

very occasionally, 'clojure.core doesn't appear in the list returned by (loaded-libs). that would be a bug right?

hiredman22:06:08

hard to say

hiredman23:06:06

loaded-libs only contains namespaces loaded via require or use, if I recall

bfabry23:06:44

mmmm that's not really what the docstring implies

devn23:06:42

So I am using an exception tracker that has mediocre clojure support (Sentry), but I'm working to improve it. Exception "rollup" (grouping of exceptions based on their similarity) relies on the hash of "in-app" stack frames (frames that begin with com.myapp for instance) being equal to a previously seen exception's stack frames. I believe rollup is not working across runs of my application due to anonymous function names differing. So, for instance: com.myapp$fn__12345 is the name right now, but on the next run it may be com.myapp$fn__99999. Are there other patterns in clojure stack traces beyond foo.bar$fn__12345 that I should let the maintainers know about which are specific to clojure, so exception rollup works as intended?

hcarvalhoaves23:06:24

the way its sending the tracebacks seems to make Sentry happy (it manages to group by Exception type nicely)

devn23:06:50

@hcarvalhoaves I'm kind of meh on manually reading source files from disk with no cache and parsing the exception object by hand. This is something that can be accomplished on the server side.

devn23:06:32

As of a week ago or so, the getsentry folks picked up coda hale's wrapper around raven-java.

devn23:06:13

One other thing to add in case you're interested: I skipped the need to have in-app by simply setting the package names of my app as a param on the DSN

devn23:06:45

Finally, I spoke with the sentry maintainers and they said it would be no problem to just fix the hashing of the problematic frames.

devn23:06:06

@hcarvalhoaves err maybe one more thing: I tried variations on what you're using, and I still managed to have mixed results. I fixed that same nth IndexOutOfBoundsException for :context_line for instance. Here's the "official" sentry-clj lib now -- PRs as I understand it, are welcome: https://github.com/getsentry/sentry-clj

hcarvalhoaves23:06:08

IMO the killer feature is having source code context around the error

hcarvalhoaves23:06:06

without that Sentry is just an expensive log aggregator