Fork me on GitHub
#beginners
<
2020-05-18
>
amrani00:05:09

Hello teh community

amrani00:05:32

I am looking for slack group of visual studio code , can someone provide some assistance

salam00:05:29

#vscode ?

skoude07:05:43

Is there an easy way to omit some of the tests?

skoude13:05:36

Thanks 👍

mauricelc9208:05:04

Hey all…..Has anyone taken the Introduction to Clojure course from https://purelyfunctional.tv/? And if so, how did you find it?

Viktor12:05:03

Hello @U013NN85EAH if you mean https://purelyfunctional.tv/courses/introduction-to-clojure-v2/ I’ve taken and it was great to me - definitely recommend it.

mauricelc9212:05:24

@U011BJJ98TZ Yes that one exactly! Thanks for responding - I think I will purchase it today then and give it a go.

Timofey Sitnikov11:05:27

Good morning, Clojurians, I am a little confused about Ring, so in order to define routes, it requires Compojure? It does not have its own way to define routes?

practicalli13:05:24

This is a walk through of a relatively simple ring and compojure project which covers the key concepts https://practicalli.github.io/clojure-webapps/ring-compojure/create-a-project/

Michaël Salihi12:05:37

Hi, @timofey.sitnikov Exactly. If you begin, I recommend to first use Compojure (same author as Ring). Later, I advice to test some other routing libraries like Reitit for learning purpose : https://github.com/metosin/reitit

ashnur12:05:52

(update-in s ["asset_files"] #(conj (vec %1) %2) (merge (bean file) (-> response :body)))))) Is there a way I could do this in a nicer way? Easier to read? Or I am overreacting?

ashnur12:05:20

I could use update, that's one thing, but apart from that.

andy.fingerhut13:05:29

I do not know what kind of original values of syou want to use that on, but since you have #(conj (vec %1) %2) instead of only conj , that is because you have found that inside of s it can have other kinds of sequences besides a vector, so you want to replace it with a vector?

ashnur14:05:55

it should always be a vector

ashnur14:05:00

i just went by the docs

andy.fingerhut13:05:33

If that is the case, then it is not easy for me to see anything you are writing there that is unnecessary, or already built into Clojure that you are not using.

andy.fingerhut13:05:52

Style-wise, in a source code file I would split it across 2 or 3 lines, rather than put all on one line, for easier reading.

mbjarland14:05:50

I have a question regarding conditional evaluation of code in clojure. I have a clojure file which I would like to evaluate either in a normal jvm clojure repl or in a babashka (native binary clojure implementation) context. The two contexts are relevant to my specific problem but not to the question. The question is, if I want to conditionally evaluate some code in one context but not in some other contexts, what mechanics do I have available to me in clojure? I.e. assume I want to (require '[somenamespace]) where somenamespace is only valid in one context and will break the evaluation in another, is the only way to accomplish this via reader conditionals or can this somehow be accomplished via macros or some other mechanics?

borkdude14:05:29

@mbjarland resolve or find-ns

borkdude14:05:46

requiring-resolve could also work

mbjarland14:05:25

I was actually trying not to bother you with a question that is a generic clojure question. But now that you ended up answering it anyway, what I'm really looking for is to have standalone portable babashka scripts with only a shebang line but still dynamically add jars to the classpath using add-classpath. This works well in babashka context but breaks my normal jvm repl. Can I accomplish that with your when-not example?

mbjarland14:05:20

I'll try it out...

borkdude14:05:32

This is called the gilardi scenario, it doesn't. You can only work around this using resolve.

borkdude14:05:22

I mean, you can use when-not, but you'll have to get at the vars using resolve, you can not conditionally use aliases

mbjarland14:05:01

reading on this it seemed tempting to have a #?(:bb reader conditional

borkdude14:05:12

that already exists

mbjarland14:05:39

would that not solve this problem, i.e I want to run some code in bb and if not in bb it should not even be evaluated

mbjarland14:05:22

well crap...

mbjarland14:05:55

figured it would be you answering : )

borkdude14:05:55

@mbjarland I've suggested this before to someone else:

(when-not (System/getProperty "babashka.main")
  (when (find-ns 'babashka.classpath) (apply -main *command-line-args*)))

borkdude14:05:42

(the condition can be combined with and but that's a minor detail)

Brian C15:05:56

Hello all, jumping into the world of Clojure/ClojureScript. Background is mostly Python. As a hobby project, I’d like to build a web-based GTD app, eventually creating a React Native version(s) for mobile. Been reading a lot of info on where to get started. This is my path thus far, let me know what you all think given my goal. 1. Bought https://purelyfunctional.tv/courses/introduction-to-clojure-v2/ and at lesson 12 currently. 2. Also bought https://purelyfunctional.tv/courses/web-dev-in-clojure/ plan to go thru that after #1 3. bought https://pragprog.com/book/shcloj3/programming-clojure-third-edition. Going thru it sort of in parallel to the lessons from #1 to get more detailed info on Clojure syntax 4. bought https://pragprog.com/book/dswdcloj3/web-development-with-clojure-third-edition. Will go thru this is in parallel with #2. 5. This seems pretty good for ClojureScript, https://www.learn-clojurescript.com/table-of-contents/. 6. I’ve poked around at “starter” frameworks like Luminus and Hoplon. Getting ahead of myself here, but if anyone has a starter set of tools/frameworks I should consider, please let me know,

mauricelc9218:05:35

Hey @U013YREF6KE 👋 I am purchasing the Intro to Clojure course this week - Do you mind telling me how it has been so far? I am busy with Clojure for the Brave and true at the moment. Looks like you have a lot of good resources to get through here

Brian C18:05:36

honestly, it’s very s…l…o….w/. He uses this “robot baking” project as the bases for the exercises…..so I am currently running thru the first few chapters of “https://pragprog.com/book/shcloj3/programming-clojure-third-edition to accelerate learning the semantics. I want to get to more in depth examples….

mauricelc9218:05:19

Ah I see….Hmm so would you not recommend it? I do plan to get to that book after Brave

Brian C18:05:18

Well, the first “Day” of his it was like that, I’m on the “Day 2" starting with Leiningen, so we will see

Mark Wardle19:05:09

I’m definitely a beginner - and found the general clojure books much more helpful than books describing specific libraries - those tended to sound very out of date. So I really like Progamming Clojure and Getting Clojure but those clojurescript books didn’t cover what I wanted (using re-frame and reagent) and so covered other libraries (e.g. om). So my suggestion (for what it’s worth) is to learn the main clojure syntax/core foundations and then roll up sleeves and have a play with reagent… then start to get lost in managing state, and then start on v2 with re-frame!

ryan echternacht17:05:49

I did something similar recently. Although I did a lot less reading and a lot more building (I had a vuejs frontend already that I first built a backend to, then rebuilt the frontend in cljs). Do what works for you, and those resources seem good. But, for me, building helped me grok it much better than reading.

Brian C17:05:09

@ryan072 good point. I’m trying to “glean” enough of language symmantics to then start loading up some basic Clojurescript web app samples and playing around…..

ghosttoaster19:05:27

Is it possible to get the contents of clj -Spath from the command line?

Mark Wardle19:05:31

Dear all, I thought I understood lazy evaluation and using the threading macros to lazily evaluate arbitrarily large files, but it is clear that I’ve got it wrong! Here I am processing a 500mb XML file (I know I know, but healthcare is still in 90s). I’ve isolated the issue (I think) to a single “map” operation, which I assumed would be fine and lazily processed and any intermediary data subsequently filtered and thrown away. This works:

(defn import-organisations-it-works
  "Imports organisations from the TRUD ODS XML file, calling fn f with a tidied-up version of the original XML
  more suitable for onward manipulation"
  [in f]
  (with-open [rdr (bom/bom-reader in)]
    (->> (:content (xml/parse rdr :skip-whitespace true))
         (filter #(= :Organisations (:tag %)))
         (first)
         (:content)
         (filter #(= :Organisation (:tag %)))
      ;;   (map xml->json)
         (f)
         (doall))))
This doesn’t (out of heap):
(defn import-organisations-it-does-not-work
  "Imports organisations from the TRUD ODS XML file, calling fn f with a tidied-up version of the original XML
  more suitable for onward manipulation"
  [in f]
  (with-open [rdr (bom/bom-reader in)]
    (->> (:content (xml/parse rdr :skip-whitespace true))
         (filter #(= :Organisations (:tag %)))
         (first)
         (:content)
         (filter #(= :Organisation (:tag %)))
         (map xml->json)
         (f)
         (doall))))
where xml->json is adapted from http://rosario.io/2016/12/26/convert-xml-json-clojure
(defn xml->json [element]
  "Converts XML into a more easily parseable set of clojure structures"
  (cond
    (nil? element) nil
    (string? element) element
    (sequential? element) (if (> (count element) 1)
                            (if (different-keys? element)
                              (reduce into {} (map (partial xml->json) element))
                              (map xml->json element))
                            (xml->json (first element)))
    (and (map? element) (empty? element)) {}
    ;; handle unconventional use of XML attributes with a single value= attribute and no actual value (content)
    (and (map? element) (empty? (:content element)) (= 1 (count (:attrs element))) (contains? (:attrs element) :value))
    {(csk/->kebab-case (:tag element)) (xml->json (get-in element [:attrs :value]))}
    ;; handle an element with attributes and some content - if content a map, merge. Otherwise merge content as :value
    (map? element) (let [v (xml->json (:content element))
                         attrs (cske/transform-keys csk/->kebab-case (:attrs element))
                         tag (csk/->kebab-case (:tag element))]
                     (if (seq attrs) {tag (cond (nil? v) attrs
                                                (map? v) (merge attrs v)
                                                :else (merge attrs {:value v}))}
                                     {tag (xml->json (:content element))}))
    :else nil))
and f is a simple filter itself that I’m just using for testing
(filter #(= code (get-in % [:organisation :org-id :extension])) org)
I don’t think I’m hanging on to the head of the sequence, and I think this map step, while processor intensive, shouldn’t hold on to any references should it? Is there any easy way of identifying what I’m accidentally holding in memory? I hope long questions like this with code aren’t against any style guide (tell me if so) as I’m much more used to email lists for these kinds of questions! Thank you in advance for any pointers (no pun intended).

noisesmith19:05:08

doall explicitly puts the entire result of f into memory

Mark Wardle19:05:25

Thanks! f is currently a filter that I’m using just to test this whole process - which should filter all but one entry from this file - and so I assumed doall would simply return that one filtered item. Actually, my real intention is for f to be an SQL ‘upsert’ step that takes each map and persists it… so perhaps I’m causing more problems by trying to understand what’s going on! I guess if I have effectful steps, dorunwould be better as the final step?

Mark Wardle20:05:36

I thought filter would ensure items filtered out would be garbage collected…

noisesmith20:05:31

yes, it would - likely some memory pressure is being generated in the pipeline in a place you might not expect, for example chunking which would cause multiple items to be realized going into the lazy ops

noisesmith20:05:10

it might help to split a lazy generator and an eager consumer that only uses one element at a time explicitly...

Mark Wardle20:05:00

That sounds like a good idea and a nicer pattern as well for the caller - I had originally planned to return a lazy sequence and then allow the client to do what it liked, but then I hit a “with-open” problem in that the channel gets closed before I had chance to use it, so I pivoted to a callback. A for loop could process eagerly one by one or by collecting a batch (latter better for a db insert) and digest a lazy sequence?

noisesmith21:05:37

for isnt' a loop and isn't eager, but doseq or run! could do it

Mark Wardle21:05:05

Thanks. I’ll give it a try!

Mark Wardle21:05:45

Thank you so much - I have got it working with run! used within the thread so actually I think the problem was an artefact of my function callback used to check the workings! Thank you again. Have now also added partition-all to batch calls to the run! callback and that’ll suit the SQL ‘upsert’ that’s next on my list! Thank you again for the pointers. Really helpful.

noisesmith19:05:39

also what does f do?

hiredman20:05:05

xml/parse is from where? clojure.xml/parse is not lazy

Mark Wardle20:05:13

From clojure.data.xml - when I run this without my “map” step it doesn’t run out of heap so it looks as if it is nicely lazily doing what it should. Of course, my map step is creating a lot of intermediary structure (map) but I thought it would be aggressively thrown away as memory pressure grew… but I’m making assumptions

hiredman20:05:19

I would not be surprised if a 500mb xml file filled a 1gb heap when parsed (not sure if that is still the default max heap size)

hiredman20:05:47

clojure.data.xml/parse is lazy (but I don't think skip-whitespace is an option for it)

hiredman20:05:03

to->json will recursively process an entire tree given to it, forcing any lazy seq subtrees

Mark Wardle20:05:35

Thanks I’ll double check that I’m using the right library and I think I’ve limited the ->json func to process only a fragment at a time but you are right I’d better double check there’s not some weird XML structure I’m accidentally processing in one step! Perhaps I can count the size of an element and its children and flag if very large to see what I’m doing wrong. Thanks for the ideas.

Mark Wardle20:05:18

It could be an issue in the source data I guess… I have assumed each one is small and that’s the case in the ones I’ve sampled, but I’ve only sampled a 500mb file… 😉

hiredman20:05:54

you haven't, xml->json (inaccurately named) calls itself in a number of different places

hiredman20:05:07

clojure is evaluated eagly, the only "lazy" thing are lazy sequences, so those recursive calls, unless there are part of building a lazy sequence, are eager

hiredman20:05:33

and for example, mapping f over the seq x returns a lazy sequence, but then immediately calling reduce on it is eager

Mark Wardle21:05:33

Thanks! I’ll double check as my assumption has been each fragment is quite small but I’ll test that assumption and refactor I think to only do the map as I build a batch for the upsert.

hiredman20:05:50

a lot depends on the implementation of f, and the actual input data, but a large enough tree under an organization tag, even if you are just keeping one, will break things

Brandon Olivier22:05:19

I'm getting an error that a dependency was compiled with too advanced a version of java. Is there a simple way to handle that?

Brandon Olivier22:05:42

I don't want to upgrade the jre version I'm using...

noisesmith22:05:23

you could attempt to compile it with an older java I guess, but that sounds like a pain

alidlorenzo22:05:14

what’s the recommended way to link a local package during development? doesn’t seem like using :local/root would be ideal since it effects git history, but maybe I’m not thinking about this correctly (used to using “yarn link” in js)

noisesmith22:05:26

why would local/root effect git history?

alidlorenzo22:05:15

i’d be changing my deps.edn

alexmiller22:05:26

You can override on the command line with -Sdeps

noisesmith22:05:36

the correct way to do this is use :local/root in an alias, rather than editing your deps

noisesmith22:05:36

the correct way to do this is use :local/root in an alias, rather than editing your deps

seancorfield22:05:06

s/profile/alias/?

noisesmith22:05:18

thanks, you are right

alexmiller22:05:53

Or put it in an alias with :override-deps

noisesmith22:05:56

or Sdeps, that would work to :D

alidlorenzo22:05:19

ah, didn’t think of that, thanks