Fork me on GitHub
#beginners
<
2018-01-04
>
donaldball00:01:33

You might want to look at partition-all

hansw00:01:37

@donaldball thnx! and i see it won't load into mem, nice

hansw00:01:11

anyone else shedding tears over the fact that Datomic uses java.util.Date?

Alex Miller (Clojure team)01:01:22

@hans378 might want to chat about it in #datomic

dpar12:01:11

@tbaldridge I'm trying to lein uberjar the fn-fx hello world, but it hangs, it says "Compiling my-fnfx.core" and does nothing. lein run works fine

gklijs12:01:29

@teikfaiv might be because your your building with jdk 9?

gklijs13:01:11

@teikfaiv then I don’t now, might be a missing dependency if a different profile is used, I don’t know the fn-fx project.

mikerod14:01:31

@teikfaiv It doesn’t look like the project.clj of that project is setup for uberjar’ing

mikerod14:01:47

I think it was just setup to do in the REPL

mikerod14:01:31

I’m not sure what you got working with lein run

dpar14:01:46

@mikerod I just copied the hello world in fn-fx repository example dir into a new lein project

dpar14:01:10

@tbaldridge this is the project.clj I use https://dpaste.de/6UJj . The core.clj code is the same as the hello world in the example directory with the ns name changed.

noisesmith14:01:53

what if you eliminated the :aot :all thing?

noisesmith14:01:58

it makes an uberjar slightly less convenient - instead of java -jar my-standalone.jar it becomes java -cp my-standalone.jar clojure.main -m my.ns but it means eliminating the need for aot

mikerod14:01:33

What is the reason for the :skip-aot on the :main?

mikerod14:01:45

I don’t even kno whow that interacts with the :aot :all

noisesmith14:01:20

yeah I don't even know, but I do know that if you eliminate things that need aot, other things all get much simpler and easier to work with

dpar15:01:11

@noisesmith if I eliminated that it tells me that the main class specified does not exist within the jar

noisesmith15:01:07

yeah, doing that requires taking out the :main specification too

dpar15:01:42

@noisesmith then it compiles but running the jar just brings me to a clojure repl

noisesmith15:01:47

does it actually fail or just complain?

noisesmith15:01:00

@teikfaiv did you use the command line I described?

dpar15:01:38

@noisesmith I compile it with the :main line or not?

noisesmith15:01:08

the :main thing is just a warning, I'm talking about running java -cp my-standalone.jar clojure.main -m my.ns

noisesmith15:01:15

substituting the name of your actual ns of course

dpar15:01:56

@noisesmith it works, that's rather inconvienient if I want to distribute it though

noisesmith15:01:23

this narrows down the problem though - something that misbehaves if aot compiled

noisesmith15:01:50

for example that namespace might be trying to use javafx classes when the ns is compiled, and that might block for some reason

noisesmith15:01:38

I bet some javafx class is being instantiated at the top level of a namespace when compiling, and the creation of that instance prevents the compiling process from exiting until some unknown shutdown is invoked

noisesmith15:01:47

I'm sorry to see that this is so frustrating, IMHO making a desktop GUI is a poor introduction to clojure - it exposes the worst parts of the language and doesn't do much to leverage its strengths

dpar15:01:31

@noisesmith that I understand. I am not learning clojure to make a desktop gui, but I need to make one now. And it's a bit of a pity maybe, because javafx has a good reputation, and it's multiplatform. Clojure might have a mature interface to it and make the language interesting for developers who need to make a multi-platform desktop gui. There are not many other options.

noisesmith15:01:37

yeah - I don't think there's been a lot of motivation to polish such a thing, few companies are ever going to say "we need to make a desktop app, let's use the jvm"

noisesmith15:01:29

on the other hand, there are a lot of companies delivering their SaaS as a web page using react - and om, reagent, rum, etc. are very well used and well documented (it's what the app I develop uses)

noisesmith15:01:01

but I totally see the utility of making desktop apps - it's just not where the clojure market forces are

mikerod15:01:58

@teikfaiv I don’t see any reason why you can’t get AOT working if you want to use it though

mikerod15:01:03

and uberjar’ing

mikerod15:01:17

then the main would just be in the jar manifest and you could do java -jar thejar.jar

noisesmith16:01:26

right, aot just introduces bugs that we can work around by not doing so, it looks like a problem with the fn-fx sample project

noisesmith16:01:48

hopefully not a problem with fn-fx itself

donmullen16:01:40

I am working code to import a large several-gigabyte csv file into a database. I’m using a lazyseq over the data - but seem to have something in the code that is accumulating memory, so that at some point there are GC/memory issues. What are some the obvious and even better - not so obvious - things I should be looking at to resolve the issue? Especially interested to get pointers to working code that does something similar. @seancorfield - you mentioned doing regular imports of large files on a regular basis. Any pointers?

noisesmith16:01:13

the thing to watch out for is if you have any bindings to the head of the sequence

noisesmith16:01:54

it's OK to have a handle to the head of the sequence in a let block if it can be statically shown to not escape scope, but if it is bound anywhere, that's where you get massive memory usage

hiredman16:01:44

replacing the lazy-seq based approach with something based on clojure.lang.IReduceInit and transducers would likely work great

hiredman16:01:07

many sql databases have something like a "load" command that can take a csv file directly too

donmullen16:01:08

@noisesmith thanks - will check that

donmullen16:01:51

@hiredman I’m using transducers over the sequence - will take a look at clojure.lang.IReduceInit. This is a datomic import - have been looking at stu’s latest mbrainz import code as well - and watched his talk on that code.

ghadi16:01:53

i did a lightning talk on IReduceInit if you’re interested @donmullen https://www.youtube.com/watch?v=FjKnlzQfAPc

donmullen16:01:11

Excellent! Thanks @ghadi

seancorfield16:01:52

@donmullen What they said (re: IReduceInit) and transducers. You can do it all lazily but you have to be pretty darn careful!

donmullen16:01:13

Thanks @seancorfield. I’ll figure out how to switch from lazyseq to ReduceInit. I’m using semantic-csv - and here is my transformer - anything off?

ghadi16:01:39

IReduceInit is nice, but you can totally make this work with lazy-seqs.

ghadi16:01:49

You just have to be careful about passing the seq places and holding the head accidentally, as @noisesmith mentioned.

donmullen16:01:57

@ghadi - anything in that code look like a held seq head problem?

ghadi16:01:18

yeah get this out of the go block

ghadi16:01:11

If you really need a go block, consume the seq outside of a go context and pump it into a channel (in a thread is fine).

ghadi16:01:25

then pass the channel into the go block

ghadi16:01:51

make it work in the dumb way first, without core.async, then sprinkle concurrency in

donmullen16:01:00

Thanks @ghadi - ramping up on async / transducers / clojure.spec all a once after putting clojure down for a couple of years. Fun stuff but making plenty of mistakes along the way!

zsck18:01:04

Hi all. I'm interested in picking up Clojure this year. I'm coming from a background of really appreciating statically typed languages, but find Rich's argument about dynamic typing very compelling. Could anyone point a newbie to some writing that might illuminate Clojure's design philosophy (as it applies to people writing Clojure, not so much Clojure itself)?

manutter5118:01:10

That sounds like a fair description of the book The Joy Of Clojure (caveat: intended for intermediate-level Clojurists).

ghadi19:01:20

and a bunch of the sub pages there

ghadi19:01:35

also watch some older talks with rich hickey, like simple made easy

zsck19:01:06

Great stuff! Thanks.

Sabbatical201718:01:38

I have a workflow question. I often find myself futzing around in a scratch file, groping toward a function definition. If the arguments of the function-to-be are x and y then I would def the variables x and y to some plausible inputs (sending the defs to the REPL), and then I send a bunch of defs from my file to the REPL which are intermediate steps toward the solution. Maybe I'll def a foo1 based on x and y, then I def a foo2 based on foo1, and so forth. Finally, when I am happy with the result, I turn the defs of foo1, foo2, etc, into a big let, which I used inside of (defn fn [x y] ...).

Sabbatical201718:01:46

This works fine but takes some editing. I wish there was some kind of (def-let) macro which would work as a sequence of defs while I am developing, but could then be dropped into a defn after just changing def-let to let. This would reduce the amount of typing needed when I am done.

Sabbatical201718:01:15

An additional advantage would be that inside this putative def-let I would be able to do destructuring bindings like in a let, which def does not seem to allow.

Sabbatical201718:01:31

This desire seems a little complicated, which may indicate that I am doing something wrong. I am also open to suggestions on how to better develop defns interactively in a REPL so that (a) while I am working on it, I can see exactly what has what value; (b) when I am done, it's easy to turn my work into a defn.

noisesmith18:01:45

my hunch hearing all this is that you are writing functions that do too many things

noisesmith18:01:51

also instead of breaking up code, you can put a reset! inside the code, to store a value into a container you can access

noisesmith18:01:37

something that I often use during development is (def debug (atom nil)) (defn foo [] ... (reset! debug bar) ...) then I access the value of debug from the repl after calling foo

noisesmith18:01:48

then after fiddling a bit I'll reset! to a different value

noisesmith18:01:42

other options include (def debug (atom [])) and (swap! debug conj bar) to store data from multiple executions (eg. if you are debugging code in a running app and want to access the data from the repl)

noisesmith18:01:12

or (swap! debug conj {:bar bar :baz baz ...}) to capture multiple items from scope

mss19:01:38

is there a function in the core library that does something like indexing a collection of maps by a user provided key/keypath? as an example:

(reduce #(assoc %1 (:user-id %2) %2)
        {}
        [{:user-id 1 :name "abc"}
         {:user-id 2 :name "def"}])
would return something like:
{1 {:user-id 1, :name "abc"}, 2 {:user-id 2, :name "def"}}

noisesmith19:01:32

group-by is like that, but puts the matches in a vector

noisesmith19:01:20

+user=> (group-by :user-id [{:user-id 1 :name "abc"}  {:user-id 2 :name "def"}])
{1 [{:user-id 1, :name "abc"}], 2 [{:user-id 2, :name "def"}]}
user=> 
the vector is useful if multiple items map to the same key

noisesmith19:01:36

it doesn't take a key/key-path though, it takes a function (happily a key is a function)

mss19:01:40

thanks, appreciate the help

hiredman19:01:32

clojure.set/index is fantastic

lauritzsh22:01:42

Say I have a channel and a list of items. I want to put each individual item into the channel and not as a list. I could use something like map for this but I believe a better function exists. apply came to my mind but that is for something else

lauritzsh22:01:28

(doseq [item items] (>! channel item))?

hiredman22:01:47

actually there is also onto-chan

lauritzsh22:01:20

ah that seems perfect! thanks

lauritzsh22:01:28

how do you all prefer to handle errors in channels? I have seen channels being split up (into success and error), others cast the error into some specific type and check it, seen others add some {:error true :reason "went bad because..."}

lauritzsh22:01:36

not sure if matters but currently working with clojurescript

jcburley22:01:39

Does anyone here have experience with trying HackerRank coding examples in Clojure? I've done just a few but have had trouble with "straightforward" Clojure code hitting TIMEOUT conditions on some tests, and have had to spend a fair amount of time working around them.

jcburley22:01:23

I wonder if the site doesn't really have a "proper" Clojure developer coming up with those examples and placing time limits on them, versus someone just getting the Clojure stuff to work and assuming the time limits for (say) C or Python should work just as well for Clojure?

seancorfield23:01:37

@james-clojure What sort of examples/problems does HackerRank offer? Are they heavily biased toward numeric processing? If so, they're not suited for idiomatic Clojure -- but you certainly should be able to create non-idiomatic Clojure that is fast (lots of type hints, mutable arrays, etc).

jcburley23:01:29

I'm working on the Arrays stuff (in Data Structures). I haven't tried hints yet, but am trying a transient vector to speed up "Array Manipulation". Now that I think about it, adding type hints might be sufficient -- thanks!

onionpancakes23:01:07

@james-clojure I did some HackerRank problems in Clojure and have a solution for Array Manipulation in Clojure. When I get timeouts, the problem is usually my algorithm rather than their Clojure enviroment.

onionpancakes23:01:40

Though their Clojure templates are definitely lacking.

jcburley23:01:34

@scallions -- okay, good to know that it might be an algo issue. I'm still so new to Clojure (and FP in general) that I'm not all that sure my algo is fast. But I did try two different approaches and still got timeouts.

jcburley23:01:44

(Yes, the templates are weird.)

onionpancakes23:01:50

Yea, its obvious they don't have a Clojure dev since their templates are weird copies of templates of other, more imperative languages.

onionpancakes23:01:10

I usually just delete what their templates and start fresh.

onionpancakes23:01:47

Just a sidenote/hint. My solution didn't require any special performances tricks like transients or type hints.