Fork me on GitHub
#clojure
<
2018-03-19
>
orestis00:03:37

@hmaurer I remember rebel-readline mentioned a Java lib that is used underneath for this kind of functionality, not sure of the name though...

kenrestivo01:03:46

there's a clojure-cli library somewhere that wraps it, IIRC

grav07:03:20

I'm using a java library, jackson-jq, which transforms json strings and leaves me with a list of com.fasterxml.jackson.databind.node.ObjectNodes. I'd like to get out a list of Clojure maps. Rather that serializing the result to a json-string and then deserializing with eg cheshire.core/parse-string, I'm thinking I might be able to leverage Cheshire for parsing the Java data structure to a clojure data structure, since Cheshire already wraps Jackson. Is this a sane idea? Any ideas on how to progress?

Anyon09:03:48

Hello guys! Who is/was using luminus?

gklijs10:03:27

I use it sometimes, there is #luminus for related questions.

Anyon10:03:56

Please advise the material or book how to correctly interact with the database inside the application and to administer it. I'm new to back-end development. Thank you!

jumar11:03:12

@U9SSRRZ4N please use that luminus channel. Btw. luminus has fairly good documentation: http://www.luminusweb.net/docs and this book is quite good as well: https://pragprog.com/book/dswdcloj2/web-development-with-clojure-second-edition

Anyon12:03:26

Thank you! Yes, i agree the official docs. is good. But not enough for me. Thank you for the book, I wanted to buy it last week!

bobcalco11:03:51

@alexmiller So when I type the 'java -cp <classpath> clojure.main' with <classpath> equal to the output of 'lein classpath' in a DOS command line, I can't even paste the full path, or type past a certain hard limit omitting the last jar file path that gets cut off. So yeah, it's definitely a limit of the command line on Windoze.

bobcalco11:03:14

(I mean with amazonica included in that classpath; not an issue without it)

bobcalco11:03:03

Even in a git bash shell I get:

bobcalco11:03:21

$ java -cp lein classpath clojure.main bash: /c/Program Files/Java/jdk1.8.0_152/bin/java: Argument list too long

bobcalco11:03:08

Going to see if the classpath brings my JVM down with the same convulsions on Linux.

Alex Miller (Clojure team)12:03:31

@bob592 you’ll probably need to look into excluding some downstream dependencies then - I’m guessing there are deps for many services which are likely all not needed

bobcalco12:03:17

@alexmiller Yeah I'm sifting through them all now. At present I need mainly S3, but amazonica gives you the kitchen sink and right now my JVM apartment is too crowded for that. 🙂

bobcalco12:03:59

However knowing there is some hard limit to how many jars can be in my classpath, and how easy it is to blow past that limit just adding a single dependency is... distrubing. I really need s3 integration and may need other AWS services over time.

mbjarland13:03:36

I wonder if by using some lein megic (assuming you are using lein) you could build a “pathing jar” along the lines of the following s.o. article: https://stackoverflow.com/a/201969/5449431

mbjarland13:03:04

@alexmiller feel free to shoot this idea down…

Alex Miller (Clojure team)13:03:28

well I have no say over Leiningen … :)

mbjarland13:03:33

yeah, I come from gradle where most aspects of the build process are open to modification, I’m too new to lein to know if this kind of “rip out the guts and replace with something else” thing is easily doable with it

mbjarland13:03:06

just a though, that would move the long dependency list essentially into a text file and the runtime classpath would be that one jar

Alex Miller (Clojure team)13:03:19

back in the deep past when these limits were lower and it was easier to hit them I have written build scripts to do this

mbjarland14:03:52

as a side note and since I was in an ocd mood, I did a small test where I created the following setup: * one jar file a.jar with a java class A.class * another jar file b.jar with a java class B.class using A.class and with a MANIFEST.MF Class-Path entry pointing at the absolute path of a.jar * executed a java -jar b.jar (I declared a main class in the b manifest in addition to the Class-Path). This worked which proves that this works for java -jar. * executed a groovy script using:

~> groovy -cp b.jar -e "println(new A().methodA())"
I am A
i.e. adding b.jar to the classpath, but using a class from a.jar. This worked as well which proves non java -jar usage. This is also using absolute paths which means it probably should work for pasting in the lein classpath as is as it normally points at entries under ~/.m2 using absolute paths (only say this as some of the docs talk about relative paths only).

bobcalco12:03:04

I guess technically I can integrate with AWS elsewhere in my stack. I may have to break it up more than I wanted to. I hate having to do this kind of thing because of some stupid hard-coded limit, though. But hey, that's why they call them "micro-services" eh? Can never have enough moving parts... 🙂

Alex Miller (Clojure team)13:03:06

fwiw, this is not a common problem in my experience. Amazonica is a particularly bad case.

grav13:03:36

How to I go from a lazy seq of something to either an input stream or a reader?

grav13:03:34

The stuff in the lazy seq of course needs to be serialized, but that's fine - I could just map str over it. It's the next step that I'm totally at loss with.

bobcalco13:03:21

@alexmiller I'll say! :rolling_on_the_floor_laughing:

bobcalco13:03:09

If anyone knows of a more modular AWS library for Clojure that is also up to date, I'm all ears... 🙂

grav13:03:55

@mccraigmccraig I'm also down that path. However, I end up being eager about consuming the sequence:

(let [r (java.io.PipedReader.)]
  (with-open [w (java.io.PipedWriter. r)]
    (doseq [x (take 20 (repeat 42))]
      (do
        (println "write!")
        (.write w (pr-str x))
        (.write w "\n")))
    (->> (for [x (line-seq ( r))]
           (read-string x))
         (take 2))))
 

grav13:03:36

If I execute that, I'll get write! output 20 times, ie the whole sequence is consumed. If I use for instead of doseq, nothing happens.

Alex Miller (Clojure team)13:03:23

for is lazy so you would need to force its realization. However, when dealing with io stuff like this, it’s usually better to take control over laziness realization using loop/recur or use use something eager like run! or reduce/transduce

grav19:03:38

Ok. My goal is to feed something to an external tool via clojure.java.shell/sh. It surprises me that there isn’t a “recipe” for this kind of thing, like there is with line-seq the other way around. Maybe I’m just looking for the wrong thing.

bobcalco13:03:07

@alexmiller Actually the code in amazonica is pretty clean and the relatively few top-level dependencies it includes are not horrible. https://github.com/mcohen01/amazonica/blob/master/project.clj The beast here is aws-java-sdk. Amazon is just freakin' huge and the temptation to write a simple wrapper around all of its SDK reveals just how monstrous and huge Amazon itself has become. It's good to be the Bezos, though.

bobcalco13:03:12

Seems like it should be possible to use just the aws services I want through plain ol' interop. Looks like the only sane route to go, short of writing a pure Clojure solution, which is not my mission at the moment.

dominicm13:03:41

it's also notable that amazonica works if you exclude the dependencies. You just can't require the related namespace.

bobcalco13:03:28

@dominicm aws-clj-sdk looks promising. Regarding excluding all the dependencies in amazonica, listing them individually would probably double the length of my project.clj. Better that than my classpath, though, I suppose... LOL.

bobcalco13:03:19

@mbjarland Suggests uber-jarring just the dependencies in a dev build might solve this. Sounds like a good idea for a lein plugin...

mbjarland14:03:32

@bob592 well my suggestion was actually not for an uber jar but for a “pathing jar”, essentially an empty jar which declares your long dependency list in its MANIFEST.MF file

mbjarland14:03:41

same effect, less heavy handed

mbjarland14:03:45

@bob592 I’m not entirely sure if the manifest file Class-Path entry is used only when executing a jar via java -jar my.jar or if it’s used also when just adding the jar to your classpath. The former would invalidate this as an option

Alex Miller (Clojure team)14:03:59

that entry was really designed for applets

Alex Miller (Clojure team)14:03:47

I think it can work in the java -jar context, but needs appropriate relative or absolute paths which is not something you should ever expect to work in a real environment

mbjarland14:03:10

@alexmiller true enough. I was thinking more of a dev environment on say windows with potentially limited command line length than a production environment. For completeness it seems that java indeed does use the Class-Path attribute to load local jar files even when not using the java -jar execution method. Also the jar specification supports this as does the oracle docs on “how classes are found”. Not necessarily saying it’s a good solution, but maybe a possible one in cases where your development is hindered by classpath length.

bobcalco14:03:22

@mbjarland Ah, right. Yeah either way it's possibly tricky. But worth the effort. Nobody should be stopped by a stupid environment issue like this.

dominicm14:03:08

I wonder how maven works around this... environment variables?

Alex Miller (Clojure team)14:03:25

I think Maven creates a dynamic classloader environment and loads things there so there is no classpath transmitted on the cli

ghadi14:03:40

for long classpaths clj can call java with the @ option

mbjarland14:03:14

so...I have a recollection veiled in the mists of prehistory that I've seen this somewhere but can for the life of me not find any information on how it works...or get it working by experimentation. Care to elaborate on the mechanics of the @ option in calls to java?

ghadi14:03:45

(same with lein)

ghadi14:03:04

@ can read arguments from a file

qqq19:03:23

I'm aware of (fn [^:double a ^:double b] (+ a b)) to type hiknt that the args are doubles. Is there a way to typehint "this arg is a float-array" ?

Alex Miller (Clojure team)19:03:34

and you can use ^floats

Alex Miller (Clojure team)19:03:46

(note that these will not work on vars where the meta is evaluated, but is fine in the location you are using it)

alexk19:03:44

The Clojure coding convention github repo doesn’t say what to do with code with reader tags. Would it ever be acceptable to have the reader tag on a different line than the form it modifies? I would assume not, although I can see where long reader tags might push the following form so far to the right that it becomes ugly. Maybe it’s a judgement call, defaulting to a single line whenever possible?

felipebarros19:03:07

Does anybody know if there is a practical reason why Cognitect's blog isn't served over HTTPS?

Alex Miller (Clojure team)19:03:19

because it’s a bunch of squarespace garbage

felipebarros19:03:35

Oh hehe okay 🙂 any plans to make an open source new blog?

Alex Miller (Clojure team)19:03:45

do you want more new clojure stuff or a better blog?

felipebarros19:03:20

I want new clojure stuff on a secure blog

felipebarros19:03:36

So I can look at the graphics, whatever is hidden here

Alex Miller (Clojure team)19:03:18

well, given that our time is far too finite and the options for things to do is far too infinite, I’m going to prioritize moving stuff to a new blog about #813,937 on the list

Alex Miller (Clojure team)19:03:03

there is a news feed at https://clojure.org/news and I double-post anything technical or release-oriented over there

😁 4
felipebarros19:03:27

That's fair. If you need help I would gladly participate in that endeavour, since I'm still not proficient to contribute with Clojure code but I can create a really nice blog if you need.

felipebarros19:03:48

Problem is probably migration as you mentioned. Fair enough.

jmckitrick19:03:54

Is anyone here familiar with buddy authentication?

qqq19:03:58

"fns taking primitives only suport 4 or fewer args" <-- why ?

Alex Miller (Clojure team)19:03:13

because they all become combinatorial explosion in the IFn interface

Alex Miller (Clojure team)19:03:15

going to 5 will 800 more lines there :)

qqq19:03:14

damn, I need 6; will rewrite this code 🙂

Alex Miller (Clojure team)19:03:42

sometimes I have stuffed similar prim types in an array to get around that

Alex Miller (Clojure team)19:03:06

the other dirty option is to make a deftype which can have primitive fields

Alex Miller (Clojure team)19:03:44

and you never learned these dirty tricks from me

qqq19:03:10

I'm writing this computer vision algo in clojure, and the 4-arg limit is limiting my ability to refactor, the deftype to group the arguments sounds really nice though, it's like a typed hashmap

Alex Miller (Clojure team)19:03:13

they can be mutable fields too for double extra dirty clojure

Alex Miller (Clojure team)19:03:40

but at some point I usually find it’s easier to just write the critical bit in Java

bronsa20:03:37

an alternative is to definterface a prim invoke method

bronsa20:03:57

but as @alexmiller suggests, it's usually easier to write some java

pavani22:03:08

I am looking for a way to find date of the nth day of a month. For ex : date of 3rd monday in March for the year 2018. Is there a good way to do this with org.joda.time.DateTime

qqq22:03:56

This is probably in the realm of premature optimization. However, suppose I have:

(deftype Foo [^float x ^float y ^long idx])
x = vector of Foos
Now, standard would x = vector of refs that each point to a 'Foo' object (but not necessarily consecutive in memory) Is there a way to tell the vector to 'pack' the Foo's one next to the other ?

ghadi22:03:56

No, not until valuetypes come in the JVM

qqq23:03:35

so the closest thing we can do is to 'transpose' the data, so instead of Vector of Foo's, we have triplet of float-array, float-array, long-array ?

ghadi23:03:57

This exact usecase is something Brian Goetz brings up for value types. People trading maintainability for performance

ghadi23:03:25

(he gives an example of an array of 2d points)

Sallide23:03:46

I don't really know any of the CS theory behind either, but is core.async closer to CSP or actor model?

Sallide23:03:54

i'll give that a skim, thank you

schmee23:03:43

has anyone looked into using Minimal Value Types with Clojure? https://wiki.openjdk.java.net/display/valhalla/Minimal+Value+Types

qqq23:03:29

Why is not inheriting from Object an advantage?

ghadi23:03:33

It's not a thing yet

ghadi23:03:38

It will be interesting to see what shakes out of the experiments. Clojure tends to box everything now (think of map in the stdlib)

pablore23:03:42

I dont get what does this accomplish. Could this help java get rid of the primitives types?