Fork me on GitHub
#clojure
<
2019-10-18
>
JanisOlex10:10:50

hey people, is there some lib or out-of-box function besides (bean obj) which would dump the java object into clojure map of maps... Bean does not work, as it won't walk the tree and won't be able to convert inner data objects And my case is the most extreme... I would like to have object which has inside java original HashMaps which also can contain maps and objects... only real restriction is that maps are always strings as keys

dominicm10:10:44

There's one in clojure toolbox which I think is recursive.

noisesmith17:10:32

(cmd)user=> (import (java.util Map HashMap))
java.util.HashMap
(cmd)user=> (require '[clojure.walk :as w])
nil
(cmd)user=> (def input (let [m0 (HashMap.) m1 (HashMap.)] (.put m0 "a" m1) (.put m1 "b" 2) (.put m1 "c" 3) m0))
#'user/input
(ins)user=> input
{"a" {"b" 2, "c" 3}}
(ins)user=> (w/prewalk (fn [x] (if (instance? java.util.Map x) (into {} x) x)) input)
{"a" {"b" 2, "c" 3}}
- this will convert, also it exposes the fact that native hashmaps are already good enough for many clojure functions (including the printing ones)

noisesmith17:10:38

get also works on native Map instances (but assoc / dissoc don't iirc)

dominicm18:10:14

wouldn't work on beans though?

noisesmith18:10:55

a prewalk can handle beans as long as F handles them

noisesmith18:10:11

(cmd)user=> (def input (let [m0 (HashMap.) m1 (HashMap.)] (.put m0 "a" m1) (.put m1 "b" 2) (.put m1 "c" (java.util.Date.)) m0))
#'user/input
(ins)user=> (w/prewalk (fn [x] (cond (instance? java.util.Map x) (into {} x) (instance? java.util.Date x) (into {} (bean x)) :else x)) input)
{"a" {"b" 2, "c" {:day 5, :date 18, :time 1571421916321, :month 9, :seconds 16, :year 119, :class java.util.Date, :timezoneOffset 420, :hours 11, :minutes 5}}}
gotcha: it doesn't work unless you call into on the bean, because prewalk calls empty on the map and that fails for beans

JanisOlex10:10:30

and in addition -> turn all the keys into keywords

mloughlin10:10:53

does anyone have any experience writing OO-style builder/"fluent interface" with clojure maps?

borkdude12:10:50

I wrote a Java API to create a Clojure options map for a library recently. I could show you the code if you're interested. Not sure if it was the best approach, but it works

mloughlin13:10:44

I mainly see the pattern in Java/C#, I am interested in alternative solutions in idiomatic Clojure for example, I saw the example code for FFmpegBuilder (https://github.com/bramp/ffmpeg-cli-wrapper) and wondered how you would write the equivalent in Clojure. (API, not underlying implementation)

mloughlin13:10:27

I guess I'm about to facepalm, you'd just define a map :man-facepalming:

borkdude13:10:42

"fluent interface" can be achieved using threading macros basically

mloughlin13:10:41

that's interesting, thanks for sharing it.

mloughlin13:10:11

I find it jarring to see the Java type system's representation of nested maps 🙂

borkdude13:10:19

I should probably create the Clojure map from Java in Sci.java instead of doing it the other way around

borkdude13:10:40

but that's an implementation detail I can get rid of later

borkdude13:10:17

creating a JS interface was much easier, because it maps better to how you do it in Clojure

borkdude13:10:35

can someone show me a practical use of &env and &form in a Clojure macro? are these mostly used by Clojure itself?

vlaaad13:10:12

capturing let bindings IIRC

vlaaad13:10:14

(defmacro lets []
  `[~@(map (fn [sym] `(symbol ~(name sym))) (keys &env))])

(let [x 1]
  (when-let [y 2]
    (lets)))
;; => [x temp__4657__auto__ y]

bronsa13:10:41

&form is mostly used for either error reporting or capturing form metadata

borkdude13:10:24

@vlaaad are the vals of &env used for something practical as well?

vlaaad13:10:58

not sure, never used them

bronsa13:10:13

here's the vals being used

bronsa13:10:18

to retrieve the type information

bronsa13:10:40

(`crossing-env` there is &env)

borkdude13:10:46

(defmacro lets []
  (zipmap (map (fn [sym]
                 (list 'quote sym)) (keys &env)) (keys &env)))

(println
 (let [x 1 y 2]
   (lets))) ;;=> {x 1, y 2}
cool. not sure when I would use this, but cool 😉

bronsa13:10:23

quite useful for debugging crazy macros

vlaaad14:10:59

I sometimes use scope-capture for debugging, which captures local variables and stores them for later retrieval

vlaaad14:10:21

although recently I switched to tap> since it doesn't require a dependency

borkdude14:10:50

I just use inline defs for this, it works since clojure 1.0.0 and before 😉

vlaaad14:10:53

and is more intentful: I just put there stuff I'm interested in instead of saving everything in scope

vlaaad14:10:08

oh yeah, that as well

bronsa14:10:16

@borkdude &env predates vars tho ;)

😂 4
bronsa14:10:05

but I agree inline defs/tap are much simpler for this sort of debugging

borkdude14:10:43

if you despise inline defs you can use atoms or whatever

borkdude14:10:04

and add-watch .. oh boy 🙂

borkdude14:10:28

I guess that's kind of what you get with tap> but I haven't tried that yet. hard keeping up with all these new features 😉

sogaiu22:10:42

don't know if you're trying rebl, but what you tap> ends up there (in the tap tab) -- this is convenient, but i found it to not work so well for very large things

Alex Miller (Clojure team)14:10:18

tap> has the benefit of both being in core and getting tunneled over things like prepl

💯 8
sogaiu02:10:25

for not-so-large data, being able to use tap> and have the results end up in rebl is very convenient. for somewhat largish data i'm finding it doesn't appear to work so well -- i think rebl has hung sometimes. then i kill its window after a while and then the project's jvm process goes away too, iirc. has that been your experience? the same data shows up quite nicely in rebl when i use submit, fwiw.

Eduardo Mata14:10:02

I have a parallel synchronous processor in my clojure app. After sending a high amount of messages for processing. I use async core. My channels size are 65536. I am getting the following message

java.lang.OutOfMemoryError: unable to create new native thread

Lone Ranger14:10:53

Heya, okay general question about Clojure further self-education. We're in an interesting pickle in that there's a small number of us (relative to devs as a whole) and yet our language and ecosystem is evolving rapidly. I feel like OO languages are able to proliferate best practices by proliferating libraries (no thinking required), whereas we tend to proliferate best practices by... learning best practices. Now, personally, if I come up with a good idea -- or what I think is a good idea -- I wouldn't even know the right forum in which to share it. I also certainly don't know how to find out what everyone else is doing. We have a few excellent resources, but they're quite finite, and seem to go out of date almost as soon as they're written. i.e., I've yet to see a comprehensive treatment of transducers, core.async, or spec -- and I KNOW folks are doing some incredible stuff with them. So I'd love to hear about how y'all are keeping sharp?

dpsutton14:10:48

what does "proliferate best practices by proliferating libraries" mean

Lone Ranger14:10:47

Rhetorical question: If you think of ruby web development, what's the next word that comes to your mind? Rails. With Python, typically Flask or Django. Each of those comes with it a set of fairly well established best-practices.

Lone Ranger14:10:05

Flask, for instance, comes with a ton of plugins.

Lone Ranger14:10:43

Now, I'm not saying we should do it that way... one of the things I love about Clojure is that it encourages you to think through your own domain problems and it gives you all the tools to do so (IMHO)

Lone Ranger14:10:29

What I'm saying more generically is that in a lot of other languages, there's a pretty tight mapping between best techinque <-> best object/class

Lone Ranger15:10:04

BUT I don't want the point to get lost, I'm not trying to debate how other languages learn, you may agree with me on that point or not -- all I'm asking is how does everyone keep up on best practices for clj?

herald16:10:00

I think most people only migrate libraries when they hit a wall with what they're currently using, and switch to something newer with the features they now need. So I don't think it's necessary to keep up, as it's enough to catch up when you need to.

seancorfield17:10:56

It definitely takes work. I follow Clojure topics on StackOverflow, Reddit, Hacker News -- and http://ask.clojure.org and ClojureVerse, in addition to watching for announcements here and on the Google Groups. I follow a bunch of high-profile Clojure people on Twitter too (but that's often too noisy to be very useful these days).

seancorfield17:10:57

I find myself much more immersed in the Clojure community than I have been in previous communities -- and part of that is to keep track of all the cool stuff folks are working on.

seancorfield17:10:30

Watching conference talks when they are posted to YouTube/Vimeo is also a good way of "keeping up".

seancorfield17:10:44

And it's certainly true that "best practices" in Clojure evolve over time, as new features appear in Clojure itself and its immediate adjacent ecosystem (`clj` / tools.deps.alpha being a great example, with Spec, and Transducers before that).

alza15:10:46

Hi folks, I just tried to login to the Clojure Jira for the first time after the recent migration, however it seems my account didn't get migrated 🙁

alza15:10:12

When I login to Atlassian, I get "<my-email-address> doesn't have access to Jira on http://clojure.atlassian.net"

alza15:10:50

This sort of makes sense, as I already have an Atlassian account.. But how do I gain access to the Clojure Jira?

alza15:10:07

I googled for some steps/procedure, but I couldn't find anything..

andy.fingerhut15:10:40

Pinging @alexmiller as one of the few (or only) person who can grant access on the new Clojure JIRA, I think. At least for new issues, creating it on http://ask.clojure.org is recommended now, but if you want to comment or add a patch to an existing issue I think you would need a Clojure JIRA account.

alza15:10:35

Ah thanks @U0CMVHBL2 🙂 I do indeed wish to comment on an existing issue, so for that I would need an account. I'll ping Alex.

alza15:10:13

As an aside, I feel for newcomers who wish to raise issues.. I just Googled "create clojure issue", and no mention of http://ask.clojure.org on the first page of results 🙁

alza15:10:36

Although Googling for "ask clojure question" brings it up 🙂

Alex Miller (Clojure team)15:10:05

googleosity is largely based on inbound links. as a new-ish site, accumulating inbound links and thus search status is something that just takes time. I've done all the necessary things on the google search side for the site that I can do.

Alex Miller (Clojure team)15:10:44

https://clojure.org/community/contributing is probably the best page describing the state of the world

Alex Miller (Clojure team)15:10:27

which has links to other useful pages

alza15:10:49

Ah cool I will check it out 🙂

Alex Miller (Clojure team)15:10:11

if you are looking only to comment on an issue, the preferred process is to comment on the http://ask.clojure.org page for the ticket

Alex Miller (Clojure team)15:10:25

we are reserving jira accounts for people providing patches

jjttjj15:10:05

Let’s say i have an api service which I implement as a map which contains all the state i need to interact with that service:

(def service1 (api1/client “localhost”)) ;;=> {::api1/connection “…”}
(api1/query service1 query-str)
Let’s say i later want to build an additional layer on top of that api that requires more of it’s own state. (for example, an atom to cache results of queiries). Is the following a legitimate/good pattern: have a second client initializer function which takes the first as an arguemnt and just merges it’s own state that it needs:
(def service2  (api2/client service1)) ;;=> {::api1/connection “…”, ::api2/cache (atom {})}
So now both api1 functions and api2 functions can be used on the same map:
(api1/query service2 query-str)
(api2/cached-query service2 query-str)
So this is kind of like using reify to implement protocols on objects, but just on plain maps. Is it preferable to instead build up a map that’s more like:
{:api1-service service1-map
 :api2-service service2-map}
Or is the first way what is kind of the whole point of open maps (+ qualified keys)?

emccue18:10:59

Has anyone used quil to make a gui?

emccue18:10:03

is that a supported use case

bfabry18:10:08

It’s been probably 7 years since I used quil, but I would really really advise against trying that

emccue18:10:05

Well, its more in the realm of a "game" sort of thing

emccue18:10:11

less swing-ey buttons

emccue18:10:20

does that change your assessment at all?

sogaiu22:10:30

i tried with quil using clojure on android years ago, and i agree that if things haven't changed much to not try the gui thing with it.

sogaiu22:10:44

perhaps things have changed though 🙂

Arto Kalishian19:10:45

Good evening Community, Does anyone use CouchDB ?

Eduardo Mata19:10:07

How could I increase the total of 8 threads in core async?

Eduardo Mata19:10:01

so doing alter-var-root on the thread-pool-executor. This most happen once? such as in the main function?

dpsutton19:10:28

> "Value is set via clojure.core.async.pool-size system property; defaults to 8; uses a delay so property can be set from code after core.async namespace is loaded but before any use of the async thread pool."

ghadi19:10:33

no, don't alter-var-root, use the property

Eduardo Mata20:10:18

there is no documentation online rather stackoverflow doing alter-var-root

dpsutton20:10:51

java -Dcustom_key="custom_value" application_launcher_class or in code as System.setProperty("custom_key", "custom_value");

ghadi20:10:58

(System/setProperty ...) or java -D when you start ^

dpsutton20:10:05

the former setting at startup the latter from code

Eduardo Mata20:10:44

java -jar -Dclojure.core.async.pool-size=16 class.jar ?

dpsutton20:10:58

just as an aside, why do you need 16?

Eduardo Mata20:10:41

my main function that receives N messages for processing, and is expecting over 8 different assets to be processed withing a asynchronous processing. The asses aren't dependent for each other, but the messages land within the same category and topic, where they are processed in a core.async pub/sub.

Eduardo Mata20:10:40

Today stress testing this processsing, that is expecting about 5k messages for different assets came to a out of memory can't create a native thread. it happened when the main function got above 8 different assets being processed at the same time.

Eduardo Mata20:10:06

I have a 32 gb ram with 12 cores. I think shouldn't be running to a problem like this. Probably with C but I did not thought clojure would be me this error

dpsutton20:10:33

have you seen how much RAM your jvm is using? if its running out of memory the threadpool size may not help. Also, this thread pool shouldn't have any blocking in it just message passing

dpsutton20:10:39

(as i understand it)

Eduardo Mata20:10:38

it is not blocking and closing all the channels except for the topics sub/pub. I noticed it had 1g of JVM memory and increased to 12G

Eduardo Mata20:10:29

this was my problem earlier

java.lang.OutOfMemoryError: unable to create new native thread

dpsutton20:10:48

did that resolve it?

Eduardo Mata20:10:05

so far I haven't seen any problem

Eduardo Mata20:10:38

I've sent 10k message for processing and I haven't see any problems

Eduardo Mata20:10:40

https://www.reddit.com/r/Clojure/comments/d52c4l/blocking_io_in_clojure/ This is where all started. It works great for short term processing and long running processing. However, if I want to ETL thousands of historical files to the most recent, this problem happened. However, this case of ETL process for thousands of files one after the other is not a thing that happens al the time.

noisesmith19:10:13

@contact904 go blocks are not meant for blocking, CPU or IO intensive work, the coorrect thing is to use the thread macro which returns a channel, you can interact with the result on that channel in a go block

👍 4
grzm21:10:52

I'm working at the repl exploring gen-class. From what I've seen, if I call compile on the namespace, make a change (say, update :constructors), I need to restart the JVM to effectively see the changes. Calling compile and import again doesn't appear to pick up changes. I'm probably missing something simple. Any pointers or references much appreciated.

bfabry21:10:27

I believe this is a limitation of the jvm. you can’t redefine classes at runtime

grzm21:10:10

Well, that seems silly 😛

Alex Miller (Clojure team)21:10:17

the jvm is actually way more dynamic than static

grzm21:10:11

@alexmiller Can I leverage this dynamicism to redfine classes at runtime?

bfabry21:10:21

you would know. it’s all magic to me

Alex Miller (Clojure team)21:10:44

technically, you can via java agents, but I don't think that's what you want

bfabry21:10:10

grzm I’d suggest using gen-class to make teeny tiny facades for your clojure code which is modifiable at runtime

Alex Miller (Clojure team)21:10:15

there are java tools that help this kind of thing but I can't imagine they are worth messing with via clojure

Alex Miller (Clojure team)21:10:34

note that gen-class forwards methods through clojure functions, so you can reload those functions and see the changes

Alex Miller (Clojure team)21:10:04

so as long as you're not changing the structure of the class, you don't need to reload it

grzm21:10:56

Right now I'm specifically trying to debug my constructors which apparently can't be found 😕 I think I need to figure that part out before I can redefine the implementation of the methods.

Alex Miller (Clojure team)21:10:02

you can also use a classloader that supports this stuff (which is what Clojure's DynamicClassloader does) but again, this is probably way more effort than it's worth for the relatively small period of time where you're messing with it

grzm21:10:27

@U0CJ19XAM just noticed I'm spelling :constructors wrong 😕

bfabry21:10:24

huh. spec biases towards open maps, so wouldn’t catch that even if gen-class were spec’d (actually it prob is as part of ns?). but an editor/ide could totally highlight the input pieces to a macro based on whether they’re in the defined spec or not