Fork me on GitHub
#clojure
<
2021-05-26
>
seancorfield00:05:36

@maxminos.1 :local-repo tells Maven (lein) where to cache dependencies that it downloads. Normally that’s the ~/.m2 folder. :repositories defines one or more named locations to go look for dependencies to download — normally you have central, clojars, but you can add more places to look, e.g., if you have a private Maven repo or you want to host dependencies on S3.

seancorfield00:05:18

So if you want Leiningen to cache the JARs it downloads in a different place to usual, that’s :local-repo. What is your actual use case?

max minoS00:05:50

@seancorfield I am just trying to keep my ~/.m2 folder confined within ~/.cache/m2 , additionally, I have also set my maven localRepositories to the cache directory by the settings.xml, and i've also set BOOT_LOCAL_REPO to ~/.cache/m2/repository I'm just not sure how to check whether the directories are for m2/repository or just m2/

seancorfield00:05:43

According to the Leiningen source, it looks like the default is ~/.m2/repository so that’s the structure it will expect in :local-repo.

seancorfield00:05:31

(tools.deps.alpha has the same default for the Clojure CLI so at least that looks consistent)

max minoS00:05:48

perfect! that's what i have it set as, what about the :repositories "local"shown in the link?

max minoS00:05:53

should I just remove that?

seancorfield00:05:35

If you don’t need to search any repos beyond Maven Central and Clojars, you can remove that.

👍 3
seancorfield00:05:10

That is terrible documentation BTW. It references #= which is undocumented and should not be used.

quoll01:05:56

Scratching my head as to where I learned this if it’s undocumented. I know I’ve read the sources, but I thought I saw it somewhere else first

quoll01:05:37

Never mind… I wrote it down in a note that I last updated in April 2012. Maybe I saw it mentioned at the 2011 Conj

seancorfield00:05:56

I’m pretty sure the clojure package it mentions on that page is not an official package and would conflict with the official Clojure CLI.

max minoS00:05:16

what should I use instead of #=?

seancorfield00:05:02

I’m not sure what Leiningen allows for in profiles.clj. I know that you can just “escape” executable code in project.clj. You’d best ask in #leiningen if no one here knows how to put dynamic code in ~/.lein/profiles.clj — I haven’t used Leiningen for years.

seancorfield00:05:40

(although since it is your user config file, you could just hardcode the full path without dynamic evaluation)

max minoS00:05:03

thanks alot! i will look into it

seancorfield00:05:25

Why do you want to change Leiningen’s defaults?

max minoS00:05:03

just to keep my home folder clean, it also happens to help me understand how it works a little better and pushes me to look at whats going on behind the scenes

seancorfield00:05:28

Oh, OK, you said above… sorry. I guess that "local" repo is so that it would look in your original home .m2/repository for anything already downloaded? That wiki page is not very helpful.

🙏 3
emccue03:05:26

has anyone kept track of attempts to make something like Pheonix Live View/LiveWire in clojure

emccue17:05:23

Just saw this - thank you

emccue17:05:49

i'm going to read their impls and maybe find a way to contact the authors about where their heads are at

delaguardo18:05:36

You can ping me if you have any questions about liveview-clj. It is written by my former teammate and some problems we discussed together

emccue18:05:46

I'll keep that in mind - this might be a weekend, not weeknight project though so it might be a bit

emccue03:05:09

I keep thinking it would be a great demo for loom stuff but I feel like I am missing some essential bits of design to actually make it work

emccue03:05:37

I can throw up a simple demo that just uses https://github.com/juji-io/editscript with reagent, but that feels...lacking

emccue03:05:00

(maybe even an actual use case for https://akka.io/)

jjttjj03:05:06

I have a huge reducing function that operates on a sequences of maps and I want to introduce some functionality using a very small subset of data involved in the sequence of maps or the initial reduce state. The functionality could be capture pretty cleanly as a transducer. However, I cannot use a separate transducing context in the api I'm using, I can only provide a single reducing function. I keep wanting to make a "dummy" reducing function that just spits out the latest result from a transducer, and stick that in the reducing function, like this:

(let [xf (comp
           (map :val)
           (dedupe)
           (partition-all 2)
           (map #(reduce + %)))
      rf (xf (fn [acc x] x))
      states [{:val 1} {:val 2} {:val 3} {:val 4}]]
  (reduce
    (fn [acc x]
      (update acc :sum rf x))
    {:val 0}
    states))
This seems wrong because the rf here is stateful in a way that's not obvious or constrained like other transducing contexts. But the alternative seems to be to use the acc value passed to the reducing function to store a bunch of intermediate values, and then sort of re-implement the transducer functionality on top of that. This seems more correct but also doesn't feel great because I'm reinventing a lot of wheels (`dedupe`, partition, etc) and just "polluting" the acc map with implementation details. Any thoughts on this? How would you handle this situation? Edit: Also, in the real version, it's not a reduce but a process that happens over time that I can provide a (fn [acc x]) to

Ben Sless05:05:56

I'm not sure, but if I cross my eyes a little it looks like reductions

👍 3
Ben Sless05:05:35

maybe something from xforms?

Ed11:05:02

not sure I see the advantage of separating the update, outside of the rf ... unless there's something much more complex going on I'm not sure there's any advantage of not doing this:

(let [xf (comp
            (map :val)
            (dedupe)
            (partition-all 2)
            (map #(reduce + %)))
        rf (xf (fn [acc x] (assoc acc :sum x)))
        states [{:val 1} {:val 2} {:val 3} {:val 4}]]
    (reduce
     rf
     {:val 0}
     states))
which is what I assume you mean by "But the alternative seems to be to use the acc value passed to the reducing function". I think that the point of the reducing function being constructed inside the transducing context is that it's coupled to the value that it's reducing into. Here that's rf and {:val 0}. I'm totally sure I understand your problem tbh 😉

👍 3
Ed11:05:04

and yeah ... there's an xforms/last - but I'm not sure it's worth adding the lib for just that one fn 😉

Ed11:05:07

and you're not handling the completion case, right? ... I'm guessing that's out of your control, but that'll impact the behaviour of the statefull transducers in your process, right?

Faiz Halde06:05:23

anyone having experience with clojure manifold library (especially the streams) ? i was creating a kafka stream construct using the manifold streams

(defn kafka-stream
  [config topic]
  (let [output (s/stream 100)
        consumer (KafkaConsumer. (kuk/as-properties config))]
    (future
      (s/on-closed output (fn [] (.close consumer)))
      (d/loop []
        (.subscribe consumer [topic])
        (let [msgs (.poll consumer (:poll-timeout config 2000))]
          (doseq [m msgs]
            @(s/put! output m))
          (d/recur))))))
but i had a hard time trying to make the (.poll) call lazy. i.e. i wanted the consumer loop to begin only when some sink connected to it. if anyone with akka stream experience can relate to me. i really doubt if it is possible since manifold stream seems like a push based system as opposed to pull based system (which akka is)

delaguardo06:05:18

Maybe worth to look at KafkaStreams framework

Faiz Halde06:05:22

@U04V4KLKC actually we were building out a layer that allows us to have pluggable sources

delaguardo06:05:31

Then I can recommend KafkaConnect. Used it as a pluggable source with common sink

👍 3
Tim Brown08:05:38

Hey all, I'm just starting with deps.edn, and I'm trying to depend on a pom type dependency. Is this possible?

<!--  -->
<dependency>
    <groupId>com.ocadotechnology</groupId>
    <artifactId>OcavaParent</artifactId>
    <version>10.10.14</version>
    <type>pom</type>
</dependency>

p-himik08:05:40

It is possible.

Tim Brown09:05:33

from those docs (my emphasis): > Local project coordinate: {:local/root "/path/to/project"} > Optional key :deps/manifest > Specifies the project manifest type > Default is to auto-detect the project type (currently either :deps or :pom) the type of the project isn't mentioned in the "maven coordinate" section above it 😞

Tim Brown09:05:50

and I have tried this too (documentation ignored)

p-himik09:05:47

I'm not sure where the misunderstanding is. Maven is based on POM. Maven artifacts, AFAIK, cannot be uploaded without POM files.

p-himik09:05:09

You need :mvn/version, not :local/root.

p-himik09:05:51

Or do you want to depend on a local JAR with a POM file in it?

Tim Brown09:05:58

there is a pom-only distribution method for maven; this is a "parent" project that doesn't have a jar artifact

Tim Brown09:05:56

{
:deps {
       ;; 
       com.ocadotechnology/OcavaParent {:mvn/version "10.10.14" :deps/manifest :pom}
       }

}
$ clj
Error building classpath. Could not find artifact com.ocadotechnology:OcavaParent:jar:10.10.14 in central ()
there is no jar file... the only artifact would be: com.ocadotechnology:OcavaParent:pom:10.10.14

p-himik09:05:51

Ah, apologies - I didn't realize it had that <typ> . Almost never see it. No, it's not supported. You have to list all required dependencies manually.

Tim Brown09:05:31

Grrr... frustrating because the authors have already done that! Oh well, I think I might try and do this locally, not using a remote mvn.

p-himik10:05:09

If you really need all of the dependencies from that pom - why is it a problem to just list them out explicitly in your deps.edn?

p-himik10:05:44

For context, here's where Alex Miller mentions that it's not supported: https://clojurians-log.clojureverse.org/tools-deps/2020-07-29

Tim Brown10:05:54

thanks for the reference.

Tim Brown10:05:27

Regards the previous question... I think it's a case of not duplicating effort.

p-himik10:05:48

It's also the case of implicit vs explicit. :) IMO, and Alex seems to have the same opinion, if you depend on something, that exact dependency should be in deps.edn. And not something else that also depends on it. But opinions can be different, of course.

Tim Brown10:05:08

I get that... I've watched Alex speaking about it. I'll be depending on only that library (for now); so I intend that it is that library that manages its dependencies (even if it is through the sub-wonderful maven strategy). I actually feel that I'm not entitled to make those choices for the library. Of course when I need one of those dependencies explicitly... then I'll be explicit.

👍 3
p-himik11:05:33

@ULX1W9EJ3 Oh, I just noticed that Ocava deals with simulations, and I recently became interested in the general topic of simulations, with the inline towards biological processes. Do you maybe have any recommendations in literature, tools, or something else for a beginner in the area?

Tim Brown11:05:02

As far as I know, Ocava is concerned with Discrete Event Simulations; I'm not sure if the documentation and sample code in that project gives you a good introduction. From the readme: > Ocava was created by https://www.ocadotechnology.com/'s Simulation Team after they successfully implemented a simulation and testing framework in order to test a radical proof of concept fulfilment warehouse proposal. I can vouch for it being a solid tool/library to base a Java product on. I want to see if I can use it for “stateless” simulations (value-based or even data-driven clojurey goodness). Our current product is stateful, and I don’t think it has to be

Tim Brown11:05:09

except for the bits that have to be 😉

p-himik11:05:02

Gotcha. Thanks!

Tim Brown11:05:18

IIRC there's a reference at the bottom of the wikipedia page for DES, titled along the lines of "simulation for beginners"

Tim Brown11:05:55

Not DES... but related-ish

p-himik12:05:02

Oh, nice! Will add it to the list.

emccue12:05:12

running into this in the wild

emccue12:05:56

(ns ssr
  (:require
   [cognitect.transit :as transit]
   [editscript.core :as editscript]
   [editscript.edit :as edit]
   [hiccup.core :as hiccup]))

(def current-state (atom [:div]))

(println (type @current-state))

(defn render
  ([]
   (hiccup/html @current-state))
  ([new]
   (hiccup/html
    (swap! current-state
           (fn [html-1]
             (let [diff (editscript/diff html-1 new)]
               (editscript/patch html-1 diff)))))))

(render [:div "b"])
(println (type @current-state))
(render [:div "c"])
(println (type @current-state))

emccue12:05:04

clojure.lang.PersistentVector
clojure.lang.APersistentVector$SubVector
Syntax error (IllegalArgumentException) compiling at (src/clj/ssr.clj:24:1).
No implementation of method: :kv-reduce of protocol: #'clojure.core.protocols/IKVReduce found for class: clojure.lang.APersistentVector$SubVector

emccue12:05:17

i see the workaround so this won't hard block me, but still frustrating

Joel14:05:28

I've been writing functions that "pair/tuple" matches so that i can work on the combination. For example finding a match with an element in a list and returning [element match]. I could write a (tuple-match match-fn element collection) I suppose to simplify this process, but just wondering if there's already something idiomatic.

emccue14:05:51

if you already have element in scope, is there any reason not to

emccue14:05:35

[element (first (filter (partial match-fn element)) coll)]

emccue14:05:15

(if-let [match (first (filter (partial match-fn element) coll))]
  [element match])

Derek14:05:41

(some (fn [element] (when (condition? element match) [element match])) coll)

emccue14:05:02

(if-let [match (some (partial match-fn element) coll)]
  [element match])

Joel14:05:03

good suggestions. i think part of the trick for me is to learn more verbs. I forget "keep" instead of (remove nil? (map... )) some instead of (first (filter...

borkdude15:05:31

Is it possible to make lein uberjar aot compilation behave more like a simple (compile 'foo.bar) instead of that it iterates over a series of namespaces? that iterating causes some weird edge cases sometimes I think

borkdude15:05:05

"yeah, just use deps.edn", at some point yes, but not right now for this project

Alex Miller (Clojure team)15:05:27

clojure.lang.Compile is a Java class in Clojure that basically does that

Alex Miller (Clojure team)15:05:24

it's actually used by the Clojure build to compile Clojure itself and I don't honestly know if I'd consider it part of the "public API" of Clojure, but it does exist

borkdude15:05:41

well, I mean, when I run lein uberjar it says:

Compiling sci.addons
Compiling sci.addons.future
Compiling sci.core
but honestly, it only needs to compile one namespace and Clojure takes care of the rest

Alex Miller (Clojure team)15:05:54

it's a main that takes a list of namespaces to compile

borkdude15:05:44

perhaps it's just :aot :all -> :aot [sci.impl.main] ?

Alex Miller (Clojure team)15:05:06

^^ I think that works too

borkdude15:05:14

I think I have a different problem. I get:

Caused by: java.lang.LinkageError: loader clojure.lang.DynamicClassLoader @498b697 attempted duplicate class definition for sci.impl.namespaces$copy_var. (sci.impl.namespaces$copy_var is in unnamed module of loader clojure.lang.DynamicClassLoader @498b697, parent loader clojure.lang.DynamicClassLoader @77978658)
I have some conditional where I define copy-var based on some env variable but it seems something is odd when AOT-ing this

borkdude15:05:48

well, I'll rewrite it.

borkdude15:05:22

the compilation thing caused issues in some other context, but the :all -> [sym] works well, so I also changed this

Noah Bogart15:05:11

is one of the edge cases you mentioned the issue where records/protocols can become redefined mid-compilation?

Alex Miller (Clojure team)15:05:36

That’s really the main one

👍 3
ghadi15:05:59

is there a common scenario that causes this? I'm not sure I follow the specifics

Noah Bogart15:05:23

wish i’d known the answer was this easy two years ago, lol. would have saved me a lot of heartbreak and work

borkdude19:05:09

@nbtheduke I have seen some weird segfaults or "no offset for this method" / "should not reach here" exceptions with this using GraalVM

👍 3
borkdude19:05:35

could be related to the record/protocol stuff

Alex Miller (Clojure team)19:05:21

you typically see this problem during compilation due to the in memory classes. not sure how it would be possible to see post-compilation in something like graal

borkdude20:05:11

@alexmiller I'm not sure either, but I've seen this in multiple projects where the way lein compiles with :aot :all gives weird issues which was solved by simplifying that compilation using just one top namespace

Alex Miller (Clojure team)20:05:35

I've seen it in aot compilation, just don't understand how that could impact graal

borkdude20:05:50

me neither, except that graal uses the AOT-ed bytecode and maybe that didn't work with the JVM either, it happened now and then in CI, randomly

Alex Miller (Clojure team)21:05:18

if the compile succeeds, you have not encountered this issue (it's really a problem in the runtime classloader during compilation)

Alex Miller (Clojure team)21:05:43

at least, as far as I understand it

borkdude21:05:13

ok, then it was something else, but I just know that getting rid of the obsolete compiles fixed it

borkdude21:05:32

at least, the problem didn't occur again. fixed is maybe too strong ;)

hiredman21:05:49

a lot of projects have issues with how their namespaces are setup such that :aot :all can result in things like defprotocols and defrecords being compiled multiple times

hiredman21:05:50

lein's behavior with :aot :all is just kind of really bad, it really wants to believe it can compile each namespace one at a time, when of course that is not how aot compilation works

hiredman21:05:08

and at least at one point the order in which :aot :all compiles namespaces was not fixed, so on different oses, different versions of java, etc, you might end up compiling in a different order, where things happened to work one way but not in another

max minoS23:05:52

I am following the modern-cljs tutorial (https://github.com/magomimmo/modern-cljs/), and still on the first one, but for some reason when I boot cljs target and open the index.html of the target, it gives me this error in the console: Uncaught Error: Undefined nameToPath for modern_cljs.core instead of a "Hello, world"

seancorfield23:05:44

That hasn’t been updated in three years and a lot has changed in the ClojureScript world. Shadow_cljs and Figwheel Main are the two dominant cljs solutions these days I think?

seancorfield23:05:18

(also Boot is not very widely used and I don’t think it’s getting much maintenance these days — I opened an important issue on boot-clj/boot-new months ago and have had zero response, even with repeated nudges).

max minoS23:05:47

that's really unfortunate, do you have any other resources that might be well suited for me to learn web development using ClojureScript? I have poked at shadow-cljs a little to make a todo-list with reagent but didn't get very far and got a little confused when I tried to add on to it

seancorfield00:05:07

I found the Figwheel docs and re-frame docs pretty good. But I'm only starting to learn cljs. I've been doing backend Clojure for a decade.

max minoS21:05:11

thanks! I will check them out