Fork me on GitHub
#beginners
<
2021-08-25
>
Steve Pegoraro03:08:25

I’m trying to implement sub-commands using tools.cli but can’t seem to get past this issue:

(parse-opts
 ["-d" "topic" "testid" "en_us" "--dry-run"]
 [["-v" "--version" "Print version information"]
  ["-d" "--debug" "Enable debug output"]
  ["-s" "--show" "Show the browser while running"]
  ["-f" "--filter PATH" "Path to filters EDN file"]
  ["-h" "--help"]]
 :in-order)
This should evaluate to something like:
{:options {:debug true},
 :arguments ["topic" "testid" "en_us" "--dry-run],
 :summary
 "  -v, --version      Print version information\n  -d, --debug        Enable debug output\n  -s, --show         Show the browser while running\n  -f, --filter PATH  Path to filters EDN file\n  -h, --help",
 :errors []}
But instead I’m getting:
{:options {:debug true},
 :arguments ["topic" "testid" "en_us"],
 :summary
 "  -v, --version      Print version information\n  -d, --debug        Enable debug output\n  -s, --show         Show the browser while running\n  -f, --filter PATH  Path to filters EDN file\n  -h, --help",
 :errors ["Unknown option: \"--dry-run\""]}
I was under the impression that using :in-order would cause the parser to stop parsing options when it encounters topic and put everything after that into the :arguments vector.

seancorfield03:08:00

@steve296 :in-order true -- not just :in-order

seancorfield03:08:13

(parse-opts
 ["-d" "topic" "testid" "en_us" "--dry-run"]
 [["-v" "--version" "Print version information"]
  ["-d" "--debug" "Enable debug output"]
  ["-s" "--show" "Show the browser while running"]
  ["-f" "--filter PATH" "Path to filters EDN file"]
  ["-h" "--help"]] :in-order true)
{:options {:debug true}, :arguments ["topic" "testid" "en_us" "--dry-run"], :summary "  -v, --version      Print version information\n  -d, --debug        Enable debug output\n  -s, --show         Show the browser while running\n  -f, --filter PATH  Path to filters EDN file\n  -h, --help", :errors nil}
user=>

Steve Pegoraro03:08:21

Oh dear, thank you!

seancorfield03:08:55

I'm a bit surprised you didn't get an error from your version tho'...

seancorfield03:08:43

(parse-opts
 ["-d" "topic" "testid" "en_us" "--dry-run"]
 [["-v" "--version" "Print version information"]
  ["-d" "--debug" "Enable debug output"]
  ["-s" "--show" "Show the browser while running"]
  ["-f" "--filter PATH" "Path to filters EDN file"]
  ["-h" "--help"]] :in-order)
Execution error (IllegalArgumentException) at clojure.tools.cli/parse-opts (cli.cljc:752).
No value supplied for key: :in-order
☝️:skin-tone-2: Exception that I would expect.

Steve Pegoraro03:08:16

No, I’m definitely not getting any exception from it

Steve Pegoraro03:08:37

For context I’m doing this in CLJS

seancorfield03:08:43

Ah, that's why.

seancorfield03:08:58

cljs is a lot more forgiving and hides a lot of errors-of-use 😞

quoll15:08:46

Conversely, Cljs does some things much better than Clojure! 😊 (e.g. no :refer :all, and internal functionality is defined by protocols much more cleanly. Though these are not discussions for #beginners)

Steve Pegoraro03:08:03

Is there any way to make it stricter and show these errors or will I just have to be more careful?

seancorfield03:08:34

You could try things in a Clojure REPL 🙂

Steve Pegoraro03:08:52

Will do, thanks for your help!

seancorfield03:08:13

Part of me feels like that is a bug in cljs: (apply hash-map [:in-order]) should be an error.

👍 3
seancorfield04:08:25

@steve296 This sort of discrepancy drives me nuts:

[email protected]:~/oss$ clj -Sdeps '{:deps {org.clojure/clojurescript {:mvn/version "1.10.758"}}}' -M -m cljs.main --repl-opts '{:launch-browser false}' --repl
Waiting for browser to connect to  ...
ClojureScript 1.10.758
cljs.user=> (apply hash-map [:a])
{:a nil}
cljs.user=>
[email protected]:~/oss$ clj
Clojure 1.10.3
user=> (apply hash-map [:a])
Execution error (IllegalArgumentException) at user/eval1 (REPL:1).
No value supplied for key: :a
user=>

Steve Pegoraro05:08:31

The inconsistency is definitely an issue

seancorfield17:08:29

I brought it up in the #clojurescript channel and David Nolen agreed it is a bug and created a JIRA issue to track it.

pez08:08:47

deps.edn question. Is there a way I can have :extra-deps in one alias :foo and “bring in” that alias in other aliases :bar, :baz, keeping it DRY that way? I’d like for the user not to have to specify :foo when they use :bar or :baz.

dharrigan08:08:29

afaik, tda doesn't support aliases of aliases atm. It's one of the top voted things in ask:

🙏 3
practicalli-john09:08:58

As aliases can easily be joined on the command line to form an explicit command, I don't see the advantage of creating complex aliaes In my view it's DRY verses complexity. If aliases start pulling in other aliases it can easily become very complex. Having created a repository full of aliases, I found it much simpler with aliases being self-contained. https://github.com/practicalli/clojure-deps-edn

pez10:08:27

Fair enough. I’m coming at this from the point of view of the beginner, who might not know enough about a project/template to realise some alias is required to specify with some other aliases.

practicalli-john11:08:49

I would assume covering the basics of the deps.edn file is part of the information that should be communicated to a beginner. I used to do the same for beginners with Leiningen.

practicalli-john11:08:35

There is a quick overview of common tasks for Clojure CLI tools and if it's built-in or not https://practical.li/clojure/clojure-tools/using-clojure-tools.html#common-tasks-for-clojure-development

pez14:08:45

Well, a particular project might not feel the responsibility of covering the basics of deps.edn.

seancorfield17:08:20

@U05254DQM I think there's a lot of value in having a single level of alias-aliases. For example, starting a project REPL at work for me is:

SOCKET_REPL_PORT=5000 clojure -Sforce -M:rebel:reveal:everything:dev:test:build:runner:dev/repl
so I would really like to be able to declare a high-level "shortcut" in my project deps.edn for that, i.e., something that isn't an alias but can stand in for one at the top-level. I agree that arbitrary aliases-can-resolve-to-more-aliases would be complex/confusing.

seancorfield17:08:59

(and that's actually quite a lot simplified from what I have been using -- I used to add :j14:classes:reflect:add-libs as well)

seancorfield17:08:54

And, yes, of course I could use a shell alias or a bash script -- but I want something portable in the CLI / deps.edn itself.

practicalli-john18:08:22

I just use fish completion most of the time or bash history, so I strive for understanding in the commands (I know what they do without thinking). The length of the command has never seemed relevant to me as I only ever type it once. Another option is to create a single deps.edn alias with everything in and just use that. I created several variations of aliases to simplify the command line (as it seems to be important to people)

Ory Band10:08:23

Hi! I'm looking for a task execution utility or library in the spirit of GNU Make (Makefiles) or PyInvoke http://www.pyinvoke.org/ , but for Clojure. If it runs on GraalVM / Babashka even better. I found out about https://github.com/juxt/mach but it's unmaintained. Couldn't find anything on GitHub or old Google. Is there anything I'm missing?

borkdude10:08:56

yes, babashka tasks is exactly for this use case :)

Ory Band10:08:59

Thanks @U0178V2SLAY @U04V15CAJ (again!). This looks good. I'm thinking whether to go for this or use Just, which supports recipes in other languages, but has the benefit of having a more simpler structure similar to Makefile (without the annoying downsides like having to .PHONY all task targets https://github.com/casey/just#what-are-the-idiosyncrasies-of-make-that-just-avoids Context for my problem: I'm building an automation tool for compiling several Clojure apps Uberjars, then building Docker images, and finally launching a local environment of Docker containers for these apps based on these uberjars. This should imitate a production environment, just locally - for development and integration testing purposes. Launching the containers must happen in a given order - Some apps are dependent on others e.g. the database must start and bootstrap before it's client apps are started, etc. For this use case, what would you recommend? bb/tasks or Just? currently I'm using a Makefile which is slowly becoming a spaghetti 🍝

borkdude10:08:46

This is an excellent use case of bb tasks, which also supports dependencies between tasks like make

borkdude10:08:20

It has the added benefit that you can use a sprinkle of Clojure without being stuck in a Just bash-like DSL

Ory Band10:08:24

Any downsides to bb compared to Just you can think of?

borkdude10:08:47

yes, bb tasks is based on Clojure, Just is not :)

Ory Band10:08:00

Sorry I meant the opposite question, lost in translation

Ory Band10:08:03

downsides to bb

borkdude10:08:49

I'm probably biased but if you want more opinions you could also ask in #babashka

Ory Band10:08:30

Thanks for all the information @U04V15CAJ. I don't know if you're aware, but you're like the jesus of clojure for me - the stuff you wrote over time has been an enormous influence on my day-to-day, both time saver and bug saver. keep up the good work :first_place_medal:

❤️ 6
Adam Stokes13:08:07

Just toying around with the idea of passing in edn as cli options. I couldn't find any code examples that just read from *command-line-*args or stdin but the simplest approach seems to be:

(defn parse-edn
  [s]
  (edn/read-string (str "{" (s/join " " s) "}")))

(def opts (parse-edn *command-line-args*))

Adam Stokes13:08:37

If I define a validation spec of only the keys I will acknowledge, is there a better way to do this?

Adam Stokes13:08:40

I'd like to just be able to pass in script.clj :cluster "my-k8s-cluster" :version "1.21.1" for example

delaguardo13:08:32

you can avoid wrapping args with “{” “}”

(defn read-args [reader]
  (loop [acc []]
    (let [v (edn/read {:eof ::eof} reader)]
      (if (= ::eof v)
        (into {} (partition 2 acc))
        (recur (conj acc v))))))
you can use that function providing either *in* or (.PushbackReader. (io/reader (.getBytes args)))

🙌 3
delaguardo13:08:37

small update

(defn read-args [reader]
  (loop [acc []]
    (let [v (edn/read {:eof ::eof} reader)]
      (if (= ::eof v)
        (into {} 
          (comp
            (partition-all 2)
            (map vec))
          acc)
        (recur (conj acc v))))))
sorry, didn’t test the function )

👍 3
ChillPillzKillzBillz14:08:45

In a hash-map e.g. {:1 1 :2 {:3 3 :4 {:5 5}}} how to ensure that the keys are in a given structure... e.g. in this case :1 :2 -> [:3 :4-> :5] ...? and not :1 :2 :3 :4 :5 or any other combination... I've tried a long convoluted if statement... like

(def m {:1 1 :2 {:3 3 :4 {:5 5}}})
(if (contains? m :2)
  (if (contains? (:2 m) :4)
    (-> m :2 :4)
    nil)
  nil)

tws14:08:41

clojure.spec.alpha

Ed15:08:32

in addition to using clojure spec, you could also use malli:

(require '[malli.core :as m])
  (m/validate [:map [:1 int?] [:2 [:map [:3 int?] [:4 [:map [:5 int?]]]]]] {:1 1 :2 {:3 3 :4 {:5 5}}})

3
popeye15:08:45

I have a list like below `

(def j `({:a "a" :b "b"})  )
How can I add the another key value in this so my output would be ({:a "a" :b "b" :c "c"}) using `
(cons {:c "c"} j)
resulting ({:c c} {:b b, :a a})

Russell Mull15:08:46

Why do you specifically want to do this using syntax-quote (`)?

Ed15:08:10

because j is a list, you'll need to create a new list with the new first item, and the rest of the list at the end:

(let [j '({:a "a" :b "b"})]
    (cons (assoc (first j) :c "c") (next j)))
however if you had a vector rather than a list, you'd be able to use assoc-in
(let [j [{:a "a" :b "b"}]]
    (assoc-in j [0 :c] "c"))

Russell Mull15:08:17

oh... I may be reading into an error in slack's backquote parsing

Ed15:08:04

alternatively, if you wanted to make the change to every element in the list, you could use map

(let [j [{:a "a" :b "b"}]]
    (map #(assoc % :c "c") j))

popeye15:08:55

Thanks @U0P0TMEFJ , that helped!

👍 3
piyer16:08:20

Is there a way to instantiate a class where classname is a variable?

(deftype Foo [arg]
 (P))

(def f Foo)

(f. {})

(new f. {})

noisesmith16:08:42

not without eval - there's probably some trick via reflection though

piyer16:08:40

I am not sure if eval works either.

dpsutton16:08:23

Class.forName() and then newInstance

noisesmith17:08:31

@dpsutton that's deprecated, but this mouthful works too:

(ins)user=> (-> f (.getDeclaredConstructor (into-array [Object])) (.newInstance (object-array ["arg"])))
#object[user.Foo 0x3a2b2322 "[email protected]"]
(ins)user=> (.arg *1)
"arg"

noisesmith17:08:42

(deprecated as of java 9)

marreman18:08:03

Hello friends! I've done Exercism and 4Clojure exercises, and I am using Clojure (in the JVM) in production right now for a small wedding website. I feel like I have a good enough understanding of the syntax and the basic building blocks. I have a strong Web and JavaScript background so I've been thinking a good next step is to dive into ClojureScript and use it in both Node.js and in the browser to roll full stack apps. Questions • What build/deps tooling should I invest time to learn? It seems like we're heading for deps.edn and tools.build. But maybe the later is not for ClojureScript? • I feel like I lack deeper understanding about how the dynamic environment works. Are people like spinning up a REPL, using that to run code to build projects, pull down dependencies? • Does anyone have some recommendations for learning material that help with these things? Paid courses are fine.

marreman18:08:59

I think I'll re-read some of the official guides. https://clojure.org/guides/getting_started

West18:08:10

I just went on youtube videos trying to figure out shadow-cljs. Shadow-cljs seems to be the only sane way to work with clojurescript as far as I can tell. Figwheel works fine, but if you want to make use of npm libraries, shadow is your best bet.

practicalli-john19:08:18

Clojure CLI tools (deps.edn) can be used for both Clojure and ClojureScript, there are some exaples here https://practical.li/clojure/clojure-tools/using-clojure-tools.html https://practical.li/clojure/repl-driven-devlopment.html is an overview of REPL driven development For ClojureScript, I would start with a simple figwheel-main project and build something simple like a landing page. This will allow you to quickly see the interactive nature of ClojureScript develpment. For example: https://practical.li/clojurescript/web-design-basics/clojurebridge-london-website/ Shadow-cljs is a good choice if you want to include many npm packages into the solution you are building. There is a very detailed guide that should be followed to get started https://shadow-cljs.github.io/docs/UsersGuide.html

practicalli-john19:08:42

If you are building a complex web UI with react, then the learn-reagent and learn-reframe commercial courses look very interesting https://www.jacekschae.com/

marreman19:08:48

Thank you for your tips @U01KQ9EGU79 and @U05254DQM. Will check them out! 👍

marreman19:08:47

@U05254DQM Your site is a real treasure trove!

practicalli-john19:08:18
replied to a thread:Hello friends! I've done Exercism and 4Clojure exercises, and I am using Clojure (in the JVM) in production right now for a small wedding website. I feel like I have a good enough understanding of the syntax and the basic building blocks. I have a strong Web and JavaScript background so I've been thinking a good next step is to dive into ClojureScript and use it in both Node.js and in the browser to roll full stack apps. *Questions* • What build/deps tooling should I invest time to learn? It seems like we're heading for deps.edn and tools.build. But maybe the later is not for ClojureScript? • I feel like I lack deeper understanding about how the dynamic environment works. Are people like spinning up a REPL, using that to run code to build projects, pull down dependencies? • Does anyone have some recommendations for learning material that help with these things? Paid courses are fine.

Clojure CLI tools (deps.edn) can be used for both Clojure and ClojureScript, there are some exaples here https://practical.li/clojure/clojure-tools/using-clojure-tools.html https://practical.li/clojure/repl-driven-devlopment.html is an overview of REPL driven development For ClojureScript, I would start with a simple figwheel-main project and build something simple like a landing page. This will allow you to quickly see the interactive nature of ClojureScript develpment. For example: https://practical.li/clojurescript/web-design-basics/clojurebridge-london-website/ Shadow-cljs is a good choice if you want to include many npm packages into the solution you are building. There is a very detailed guide that should be followed to get started https://shadow-cljs.github.io/docs/UsersGuide.html

pez19:08:11

Hello Clojurians! There is now a channel for improving the beginner’s journey. #improve-getting-started Please join if you think that starting with Clojure should be easy and delightful. And agree that it sometimes isn’t for some (many?) beginners. And that when guiding beginners starting with Clojure we should introduce them to Interactive Programming in the editor as quickly as possible. Please also join if you are a beginner who want to share your experiences with starting with Clojure and help the community improve in this area.

Adam Stokes20:08:32

I'm working through some beginner filtering and my google-fu couldn't find anything on this:

(defn -filter-kibana-indexes
  [item]
  (let [name (get item :index)
        size (get item :store.size)]
    (if (boolean (re-find #"^\.kibana.*" name))
      [name size])))

(doseq [[name size] (map -filter-kibana-indexes indexes)]
      (println "Name: " name " Size: " size))))
Result:
Name:  .kibana_abcef_001  Size:  4.2mb
Name:  nil  Size:  nil
My question is how can I keep it from returning nil in that if statement? I attempted with when as well but if re-find is false it seems to always want to return nil rather than skipping that conditional.

tws22:08:09

i’d read up on destructuring associative arguments. can get rid of the first let

Adam Stokes22:08:52

Thanks @U1Z392WMQ will definitely read on that

adityaathalye10:08:00

@U02CSD7F5MW I felt like sharing some design observations, unrelated to your original question. Naming: • 'filter' implies processing a collection into a (usually) smaller collection based on some predicate/check • Here the function is processing just one of a kind, so kibana-index-info may be more apt Function design: IMO, the above function design entwines several different things, which we can tease apart into a sort of "parts box", as follows. I feel the core concept in the -filter-kibana-indexes function is actually selecting a kibana index for further processing. "Further processing" could be anything. And once we go down this path, it is easier to see we can generalise to any index type, cheaply.

(def kibana-index-pattern
  (re-pattern #"^\.kibana.*"))

(defn matching-index?
  "Given an index pattern and an Elasticsearch Index info map, return the index name if it matches the given identifying pattern, nil otherwise."
  [pattern es-index]
  (re-find pattern (:index es-index)))

(defn extract-index-info
  "Given a collection of keys of index information and any Elasticsearch Index info map, extract the required information."
  [info-keys es-index]
  (select-keys es-index info-keys))
This choice opens up our design space as follows. A choice of laziness of processing:
(defn proc-indices-lazily
  [indices match-pattern info-keys]
  ;; map and filter are lazy
  (->> indices
       (filter (partial matching-index? match-pattern))
       (map (partial extract-index-info info-keys))))

(defn proc-indices-eagerly
  [indices match-pattern info-keys]
  (reduce
   (fn [result idx]
     (if (matching-index? match-pattern idx)
       (conj result
             (extract-index-info info-keys idx))
       result))
   []
   indices))
So we can do either kinds of processing (or both), and more:
(proc-indices-lazily
 indices
 kibana-index-pattern
 [:index :store.size])

(proc-indices-eagerly
 indices
 kibana-index-pattern
 [:index :store.size])
More parts mix-and-match:
;; To punch some indices into some processing pipeline
(->> indices
     (filter (partial matching-index? kibana-index-pattern))
     publish-to-kafka-topic) ;; or dump-to-json-file

;; To analyse index sizes in general
(->> indices
     (map (partial extract-index-info [:index :store.size]))
     (sort-by :store.size)
     (clojure.pprint/print-table [:store.size :index])) ;; or dump-to-csv
As a bonus, the above design choice is further amenable to the use of https://clojure.org/reference/transducers to create much more flexible index processing operations, should we need them later.

❤️ 3
Adam Stokes11:08:25

@U051MHSEK Wow! This is amazing, thank you so much for breaking this down, I am defintely going to pull this code and work through each one to refactor my code. I really love this explanation and such a great service for us newbies 😄

adityaathalye16:08:39

@U02CSD7F5MW Just trying to pay it forward :) I'm glad it's useful :)

adityaathalye16:08:52

@U02CSD7F5MW To shamelessly toot my own horn a bit, I helped create https://github.com/adityaathalye/clojure-by-example structured as a series of progressive drills, to help programmers get used to a Clojure/FP mode of thinking and programming.

adityaathalye16:08:13

If you like the explanation above, I think you'll like the workshop.

Adam Stokes18:08:30

Very cool! I will definitely go through your workshop!

adityaathalye18:08:41

The whole thing is quite self-serve, but I'll be more than happy to help, if the material is confusing or sparse at any point. Feel free to at-mention me on #beginners or DM me at your discretion. All the best! :spock-hand:

🙏 3
dpsutton20:08:53

map will return (f x) for each x in your collection. i think you want to use (filter -filte-rkibana-indexes indexes) which will remove all items which return falsey for your predicate

Adam Stokes20:08:52

Ah that makes sense, let me try that

noisesmith20:08:56

when didn't help because when is just an if with an arbitrary body and no else branch if you wan to map a function and also skip nil results, use keep

noisesmith20:08:20

you can combine map and filter, but that's literally what keep is for

Adam Stokes20:08:35

Nice, let me go read up on keep

noisesmith20:08:33

also, in all non-pathological cases (if (boolean x) ...) can be replaced by (if x ...)

Adam Stokes20:08:57

Thanks that is good info to know, i'll rework my code

Adam Stokes20:08:38

@dpsutton @noisesmith TY! I went with keep and it works

athomasoriginal21:08:41

How does one “recover” from a syntax error in one of your .clj namespace after you start the REPL? Example:

(ns user)

(defn move-into-dev-ns [] 
  (doto 'dev require in-ns))

(ns dev
  (:require [your.app.library :as lib]))

;; ...code
• So you start your REPL and your in user • You run move-into-dev-ns • You get a syntax error (lets pretend you used a var in your.app.library and forgot to define it) At this point, my REPL is left in a state where things don’t work because your.app.library failed to compile. My original solution was I wrote an fn in user like this:
(defn fix! 
  []
  (clojure.tools.namespace.repl/refresh-all)
  (move-into-dev-ns))
However, the above didn’t seem to work either because the original ns (`your.app.library`) didn’t compile correctly originally.

seancorfield21:08:01

(require 'dev :reload) or (require 'dev :reload-all) should be enough, assuming you fixed your.app.library

seancorfield21:08:44

(I would never reach for t.n.r/refresh stuff)

💯 3
athomasoriginal21:08:54

wow, I definitely went too far. 🙈

athomasoriginal21:08:04

It’s always the simple solutions 😞

seancorfield21:08:35

(I avoid refresh/reload stuff altogether -- never necessary IME)

athomasoriginal21:08:45

I agree. The above was a pattern I had seen several time in other code bases and copied it to solve my problem a year ago and i’m just now revisiting this and realizing it didn’t work as expected. Thanks!

noisesmith21:08:30

honestly the (doto 'some.ns require in-ns) idiom is so frequently useful I'd consider the mental overhead of remembering which function wrapped it to be a bigger cost than any gain that putting it inside a function gave me. if it errors, you fix the source files then run (doto 'some.ns (require :reload-all) in-ns)

hiredman21:08:23

or just copy and paste the file buffer into the repl