Fork me on GitHub
#beginners
<
2020-09-24
>
João Galrito00:09:35

someone the other day mentioned defsomething macros are an anti-pattern

João Galrito00:09:42

could someone explain why

Alex Miller (Clojure team)00:09:07

I don't think they're an antipattern

Alex Miller (Clojure team)00:09:06

I mean.... core has defmacro, defmulti, defmethod, defrecord, etc

João Galrito00:09:21

(defmacro def-effect [name args & body]
  `(defn ~name ~(conj args ~'optional) ~@body))
(defeffect draw [amount] nil)

(defn effect [f] (fn [optional & args] (apply f args)))
(def draw (effect (fn [amount] nil)))

Alex Miller (Clojure team)00:09:32

anytime you have some sort of complex thing that results in a var definition, that's a reasonable candidate

João Galrito00:09:40

i want to wrap an "effect" into something that handles an extra optional argument

João Galrito00:09:26

they are not equivalent and not even correct right now

João Galrito00:09:33

but just for the sake of demonstration

Alex Miller (Clojure team)00:09:10

I'm not sure you're getting enough value out of this to be worth doing, but this seems fine imo

João Galrito00:09:39

I might just have an optional function and call (optional (draw 1))

Nazar Trut00:09:22

What is a way that i can check if there is two "nots" in my sequence like this '(not (not b)), so therefore, if there is two "not" like this example. (not(not a)) would be true (not a) would be false

Nazar Trut00:09:37

I know @seancorfield helped me earlier but i still dont get how i cant do this

seancorfield01:09:45

Here's what I said: I would define a predicate that returned true if an expression was `(not X)` and then you can do `(and (has-not? expr) (has-not? (second expr)))` if you make `has-not?` robust (check for sequential expr and then check first item is `'not`) -- that's pretty much giving you all the code so maybe you can take another look and see if you can figure it out @ntrut?

seancorfield01:09:44

This is about solving a bigger problem (checking for two "nots") by breaking it down into smaller problems (checking if an expression starts with not)

seancorfield01:09:13

has-not? would return true for both of those expressions, and (second expr) would be (not a) for the first one and a for the second one -- so if you then call has-not? on each of those you get true for (not a) and false for a -- hence (and (has-not? expr) (has-not? (second expr))) provides the result you want: true for (not (not a)) and false for (not a)

seancorfield01:09:23

Does that make sense @ntrut?

seancorfield01:09:52

You've already written code that is the same body of the has-not? function -- (and (sequential? expr) (= 'not (first expr)))

Nazar Trut01:09:29

Ok thank you so much, this makes a little more sense, ima see what i can do

João Galrito01:09:17

or you can do (= 2 (count (filter #(= 'not %) (flatten kb)))) 😄

Nazar Trut01:09:01

i tried that but my only problem is that if i have (not(if (not a)(not b))

Nazar Trut01:09:09

it would count 3

Nazar Trut01:09:37

I need to do double negation, so if (not(not X)

Nazar Trut01:09:42

would derive X

seancorfield01:09:52

Writing small predicates that you can easily combine will make it easier to match on more complex expressions.

seancorfield01:09:30

@joao.galrito flatten is almost always the wrong answer 😆 (to any question)

João Galrito02:09:11

@seancorfield I actually just used it for something 😛

seancorfield03:09:08

@joao.galrito FWIW, we have 9 occurrences of flatten in our codebase at work (over 100K lines) but I think most of those could be replaced (and probably should be 🙂 )

👍 6
João Galrito14:09:35

In my case i wanted to flatten a nested lazy seq of strings into a single seq to send to an InputStream, so I think it's a good use case 😛

seancorfield15:09:01

mapcat would likely be better (usually folks use flatten when they really only need a single level collapsed -- which is what mapcat does).

seancorfield03:09:57

@murala_anilrao Do not cross-post questions in multiple channels. I have deleted it from #clojure

👍 3
seancorfield03:09:52

You need to provide a lot more detail about what you've tried, what libraries you are using, what worked and what didn't work, and what failures you got.

👍 3
seancorfield03:09:09

I also see you've repeatedly posted this exact same question without any useful details in multiple channels over the last few days, even after people have offered suggestions. If you continue that behavior, you will be ejected from this community. That is not acceptable behavior here.

👍 3
AM03:09:31

apologies for that

David Pham13:09:42

In Clojue, we can create keyword with the keyword function. For example we could create the following one: (keyword "a/b" "c/d"). How des clojure track the name and the namespace in this case?

Alex Miller (Clojure team)13:09:34

in that case you've built a keyword that cannot be printed and re-read

Alex Miller (Clojure team)13:09:45

b/c it violates the keyword naming conventions

Alex Miller (Clojure team)13:09:22

but keywords are composed of independent namespace and name fields so you could still extract those parts programmatically (with namespace and name functions)

Jim Newton14:09:19

In Common Lisp I can write a lambda form like the following:

(lambda (a b c d)
   (declare (type number a c)
            (type integer  c)
            (type string d))
   ...)
And depending on which compiler I'm using, the compiler is allowed to optimize the code in specific ways. I can also write a macro to implement my own version of lambda such as
(my-lambda (a b c d)
   (declare (type number a c)
            (type integer c)
            (type string d))
   ...)
and in the macro, examine the declarations (as they are just raw s-expressions given to the macro as input). The advantage being that every CL user automatically knows how to use my-lambda because I'm promising that it has the same syntax as lambda. QUESTION: is there any sort of type optional type declaration which I can use in my macro which corresponds to something the language already supports? I don't want to invent my own if there is already one. SUGGESTION: I know that spec can be used as sort-of declarations, but I don't want my macro to have to implement an exhaustive spec parser. I've seen some code using java-hints but I would need to be able to parse them in my macro code.

Jim Newton16:09:18

I've copied this question https://clojureverse.org/t/expected-syntax-for-destructuring-bind/6593 hoping there is more discussion.

sova-soars-the-sora14:09:36

sanity check.. What does this mean?

java.lang.ClassCastException: class java.lang.String cannot be cast to class clojure.lang.IFn

pavlosmelissinos14:09:36

Probably incorrect argument order? Wild guess: did you happen to use a -> instead of a ->> (or vice versa)?

vncz14:09:48

I'd say a function argument is expected to be a function but you've passed a string

👍 6
sova-soars-the-sora14:09:14

Aha, thank you ^_^

practicalli-johnny16:09:24

Sounds like you have it. Just to elaborate a little. IFn is the interface for functions, so when ever a value is used as a function call, you will see that IFn error. For example (1 2 3) cause the error as Clojure tries to evaluate 1 as the function with 2 & three its arguments. 1 is not defined as a function, so the error message is shown.

Jim Newton14:09:03

Is there a function in the clojure lib which I should use to break up an input sequence into a sequence of sequences, each of a specified length. I.e., break up a long list into lists of length 3? (a b c 1 2 3 10 20 30... ) -> ((a b c) (1 2 3) (10 20 30) ...)

Darin Douglass14:09:55

(partition 3 my-seq)

Darin Douglass14:09:32

note: partition will drop the last elements of the seq if they can't be made into a group of n. use partition-all if you needed all elements

🙌 3
👍 6
Darin Douglass14:09:59

user=> (partition 3 (range 10))
((0 1 2) (3 4 5) (6 7 8))
user=> (partition-all 3 (range 10))
((0 1 2) (3 4 5) (6 7 8) (9))

Jim Newton15:09:13

thanks, there are lots of partitioning functions, and their names are different in every programming language. What is the name of the function which takes a sequence and a delimiter and splits the sequence into subsequences at that delimiter. (split-on-delimiter '& [a b c & d e f g]) --> ((a b c) (d e f g))

andy.fingerhut15:09:17

partition-by is close, but not exactly that. You could probably use partition-by and then something like (map rest ...) on the result.

Jim Newton15:09:48

(split-with (fn [x] (= x '&)) '(a b c & d e f))
returns the following, which surprises me: [() (a b c & d e f)]

andy.fingerhut15:09:24

Note: I did not have that memorized, but the Clojure cheatsheet has most of the Clojure core functions organized by purpose/behavior, and I found partition and partition-by , and others, in the "Seq In, Seq Out" section: https://clojure.org/api/cheatsheet

andy.fingerhut15:09:21

And each of the functions links to http://ClojureDocs.org where there are community-contributed examples of use of most functions, sometimes pointing out corner cases/gotchas that are not always immediately obvious from the doc string.

Jim Newton15:09:39

(partition-by (fn [x] (= x '&)) '(a b c & d e f)) --> ((a b c) (&) (d e f))

Jim Newton15:09:46

that's pretty close.

andy.fingerhut15:09:15

user=> (def d1 '[a b c & d e f & g h])
#'user/d1
user=> (def d2 (partition-by (fn [x] (= x '&)) d1))
#'user/d2
user=> d2
((a b c) (&) (d e f) (&) (g h))
user=> (remove (fn [l] (= '& (first l))) d2)
((a b c) (d e f) (g h))

Jim Newton15:09:40

as I'm trying to parse a lambda list, there should be a maximum of 1 &. What does the following mean? Does it have a meaning? (fn [a b c & d & e] ...)

andy.fingerhut15:09:15

I would hope it gives an error, but it might just silently do weird things.

Jim Newton15:09:34

good! it indeed gives an error.

Jim Newton15:09:04

so I can just assert beforehand that there exists maximally 1 & in the seq

Alex Miller (Clojure team)15:09:16

that's part of destructuring spec

andy.fingerhut15:09:57

And that & is not last, which Clojure spec contains code for checking, and leads to the error you see when you try to define such a function.

Alex Miller (Clojure team)15:09:20

I mean, if you're parsing destructuring you could just use the spec and s/conform to conform the spec and get data

Alex Miller (Clojure team)15:09:39

depending on your goals

Jim Newton15:09:14

(let [[prefix _ suffix] (partition-by (fn [x] (= x '&)) lambda-list)] ... I now need to verify that suffix is either empty or a singleton

Jim Newton15:09:56

alex, your suggestion is what exactly? Can I call a particular spec function to parse this lambda list for me and give me back a data structure I can use?

andy.fingerhut16:09:16

I believe he means something like this. These examples use the same specs that Clojure itself uses to give the error message for several ill-formed attempts at writing defn forms, including having multiple &, or & at the end of a parameter list:

;; The specs mentioned below in the namespace clojure.core.specs.alpha
;; are defined in this file:
;; 

user=> (require '[clojure.spec.alpha :as s])
nil
user=> (require '[clojure.core.specs.alpha :as cspecs])
nil
user=> (pprint (s/conform ::cspecs/defn-args (rest '(defn foo [x & y] (list x y)))))
{:fn-name foo,
 :fn-tail
 [:arity-1
  {:params
   {:params [[:local-symbol x]],
    :var-params {:ampersand &, :var-form [:local-symbol y]}},
   :body [:body [(list x y)]]}]}
nil

user=> (pprint (s/conform ::cspecs/param-list '[x & y]))
{:params [[:local-symbol x]],
 :var-params {:ampersand &, :var-form [:local-symbol y]}}
nil

andy.fingerhut16:09:22

There is a lot more detail about spec that can be found here: https://clojure.org/guides/spec

andy.fingerhut15:09:10

Regarding your earlier question about declare in Common Lisp, the closest similar-but-not-the-same things in Clojure I can think of are type hints, and things like spec and Plumatic schema. None are exactly like what you are asking for, I don't think.

neilyio15:09:07

What's the story with the -S in a command like this? Why do we need it?

clj -Sdeps '{:deps {bidi {:mvn/version "2.1.6"}}}'

Alex Miller (Clojure team)15:09:21

many of the settings have -S at the front

neilyio15:09:44

I'd just love to know the reasoning, it helps me remember details like this if I know why they're there. Is it a Java interop thing?

andy.fingerhut16:09:22

The choice of -S is not influenced by Java interop, that I am aware of. The java command has its own slew of command line options, and I don't think -S is one of them. Is your question "Why S, and not some other letter"?

andy.fingerhut16:09:30

Or maybe your question is "Why -Sdeps and not -deps ?"

neilyio16:09:17

Both! Why is there a letter at all?

andy.fingerhut16:09:40

I mean, 'd' is a letter.

neilyio16:09:42

Do -S -A etc. have particular meanings?

seancorfield16:09:04

I always took -S to indicate "Here's a script argument" to distinguish -Sdeps, -Stree, -Spom, etc from execution-related stuff (the various alias signifiers).

seancorfield16:09:59

-M for aliases and :main-opts, -X (in the prerelease) for aliases and :exec-fn, -A for "all" originally but now more "aliases for REPL"

seancorfield16:09:25

There are also -R for resolve aliases and -C for classpath aliases.

seancorfield16:09:48

I think the CLI is pretty consistent about using single uppercase letters to introduce its own options, and any lowercase one would be passed on to the program being run (`clojure.main` understands -i, -e, -m, -r; other "main" functions may understand other options).

seancorfield16:09:30

Does that help make sense of it all @neil.hansen.31?

neilyio16:09:13

@seancorfield That's massively helpful, thank you. I guess I've been thinking the command line would be "prettier" without those prefixes, but I'm glad to know what they are now. Your note about the lower-case flags is also very enlightening.

neilyio16:09:01

Also, this has been a big "I really don't know how Clojure works" day for me so far, and I've been through a whole bunch of your blog posts. Thanks for all the knowledge you've shared.

seancorfield16:09:25

Happy to help! I'm always pleased to see more folks using Clojure in general and the CLI/`deps.edn` in particular.

seancorfield16:09:58

(hence my dot-clojure repo https://github.com/seancorfield/dot-clojure and also my Atom/Chlorine setup https://github.com/seancorfield/atom-chlorine-setup that supports working with a Socket REPL started from the CLI and with either Cognitect's #rebl or @vlaaad’s #reveal browser/visualization tools)

Matthew Curry16:09:48

cool thanks for your dot-clojure link! Quick question: reading through the deps.edn there, all the packages referenced have a {:mvn/version "RELEASE"} does that grab the latest package version, or does RELEASE need replacing with a specific version? I don't know maven very well.

seancorfield16:09:52

As it says in the README, many of those aliases fetch the latest stable release of a tool -- which is what "RELEASE" does.

seancorfield16:09:35

You shouldn't use it for anything other than dev/test tools because you won't get a specific version and it's better for project dependencies to only rely on fixed versions.

Matthew Curry16:09:53

Understood, and I prefer specifying for actual releasable projects too, but this perfect for all the tools in the global deps.edn thanks!

seancorfield16:09:26

There's also "LATEST" which will include snapshots.

seancorfield16:09:41

(but I think they're both technically deprecated, even in Maven)

Matthew Curry16:09:01

ah, good to know. I don't suppose there's a tool to update the specific versions in a deps.edn on command, in case one wanted to update one's project dependencies, or at least a diff of what's different?

seancorfield17:09:54

I would never let a tool automatically update my deps.edn file, but I do use tools to let me know about newer versions.

Matthew Curry17:09:03

yep, agree. Thanks again!

Test This16:09:34

I am trying to use amazonica to work with aws s3. With the following command I can successfully upload the file.

(ns ...
  (:require [amazonica.aws.s3 :as s3]
   .
   .
)

(s3/put-object aws/cred {:bucket-name aws/mybucket 
                                      :key keyval
                                      :file fileobj})
This, however, loads the file as a private file. How do I set the acl to public-read. Here on Stackoverflow (https://stackoverflow.com/questions/6524041/how-do-you-make-an-s3-object-public-via-the-aws-java-sdk) there is a question asking how to do this with Java. And the answer says:
return s3Client.putObject(
   new PutObjectRequest(bucketName, objectKey, inputStream, metadata)
      .withCannedAcl(CannedAccessControlList.PublicRead));
Can someone please point out how I can translate it to use with amazonica in Clojure?

João Galrito18:09:31

what is the difference between concat and lazy-cat ? if I want to concatenate 2 lazy seqs into 1 lazy seq, I should use lazy-cat right? will concat realize its arguments imediately?

João Galrito18:09:44

(apparently not, as I just did (take 10 (concat (repeat 10))) and worked as expected

seancorfield18:09:50

doc tells you the difference:

user=> (doc concat)
-------------------------
clojure.core/concat
([] [x] [x y] [x y & zs])
  Returns a lazy seq representing the concatenation of the elements in the supplied colls.
nil
user=> (doc lazy-cat)
-------------------------
clojure.core/lazy-cat
([& colls])
Macro
  Expands to code which yields a lazy sequence of the concatenation
  of the supplied colls.  Each coll expr is not evaluated until it is
  needed. 

  (lazy-cat xs ys zs) === (concat (lazy-seq xs) (lazy-seq ys) (lazy-seq zs))
nil
user=> 

seancorfield18:09:50

concat itself is already lazy. lazy-cat is a shorthand for applying lazy-seq to each thing you are concatenating.

João Galrito18:09:14

is there something like pjuxt?

dpsutton18:09:31

What is pjuxt?

João Galrito18:09:16

juxt but parallel 😛 applies the same argument to a number of functions

João Galrito18:09:26

and returns a vec with the results

João Galrito18:09:38

(defn pjuxt [fns] (fn [& args] (pmap #(apply % args) fns)))

andy.fingerhut18:09:35

There are very few functions in Clojure's core that cause things to happen in parallel. Maybe a dozen or two? pmap, future, agent-related things, the reducers library ... I am probably missing a couple, but not very many. No pjuxt for sure.

andy.fingerhut18:09:19

Of course opinions differ on what is generally useful and "ought to be in core", but the tendency is towards general building blocks. As your code example shows above, the ones provided make pjuxt pretty short to write.

hiredman18:09:55

the ones that exist are problematic

hiredman18:09:04

pmap is not great

Michael J Dorian18:09:47

For the majority of cases you would probably want to use something like this instead:

(defn ppmap
  "Partitioned pmap, for grouping map ops together to make parallel
  overhead worthwhile"
  [grain-size f & colls]
  (apply concat
   (apply pmap
          (fn [& pgroups] (doall (apply map f pgroups)))
          (map (partial partition-all grain-size) colls))))
(time (dorun (ppmap 1000 clojure.string/lower-case orc-name-abbrevs)))

Michael J Dorian18:09:19

source: https://www.braveclojure.com/zombie-metaphysics/ The standard version is a building block for most things, and it can still be very useful if you have a small number of large tasks

hiredman18:09:08

the way it combines parallel computation with lazy-seqs can cause non-obvious behaviors

hiredman18:09:57

so for example, your pjuxt will behave differently depending on the type of the collections of fns you pass in

andy.fingerhut18:09:09

If your goal is to maximize parallel computation, pmap can often fall well below that if different elements require significantly different amounts of time to calculate, because pmap does not work more than some small finite number of elements ahead of the earliest one that is incomplete.

hiredman18:09:13

(really it depends on how the seq is constructed)

andy.fingerhut18:09:30

Commonly suggested alternatives, that give you more control, are the claypoole library https://github.com/TheClimateCorporation/claypoole , or just using Java's ExecutorService framework and threads directly via Java interop calls.

hiredman18:09:46

using something like (mapv #(future ...) ...) is going to behave much more obviously

João Galrito18:09:51

imagining I want to do some processing on an infinite seq, but I want to process the same seq in 3 different ways in parallel

João Galrito18:09:59

what would be a good approach?

andy.fingerhut18:09:19

Do you care whether one of the 3 different ways gets arbitrarily far ahead in the infinite seq versus the other ways?

hiredman18:09:32

infinite seqs like that are almost always a bad idea

andy.fingerhut18:09:44

Or do you want them to stay reasonably close to each other in where they are working in the seq?

hiredman18:09:54

because they are usually used to model something like pulling messages off a queue as an infinite seq

hiredman18:09:03

and that is an overly simplified model of interacting with a queue (assuming the queue is a service, what if there is a network error, etc)

João Galrito18:09:15

@andy.fingerhut the second option I guess, for memory reasons

hiredman18:09:28

and it means you have this lazy thing, that you cannot consume without potentially blocking for io

João Galrito18:09:43

@hiredman I understand. I'm learning so I'm going for a naive approach, and refine it as necessary

hiredman18:09:57

so you have this lazy thing, which is a paradigm that works best when you don't have to care about when some computation happens, combined with operations you usually end up caring a lot about when they happen (io)

João Galrito19:09:08

I'm quite green irt concurrency and parallelism

hiredman19:09:41

well, I mean, you wouldn't be the first to use infinite lazy seqs like that, it is pervasive, just not great

💯 3
João Galrito22:09:41

what would you say is a better approach?

hiredman22:09:49

I would maybe do something like creating an executor with a threadpool of size N, then putting a task on that executor that pulls from the queue, submits a task for the each of the things you want to do with what you pulled from the queue to the same executor, then submits the original pulling from the queue task again

hiredman22:09:40

the tricky thing with that is plumbing return values out, but if you don't care about them then it works well

João Galrito12:09:07

what are the advantages of that approach?

hiredman16:09:07

the concurrency stuff is exposed and you don't get weird pmaps weird laziness + concurrency

João Galrito02:09:14

I understand that it gives me more control, and is probably better to have things being more explicit, but the seq abstraction feels so appropriate to use here... I'm moving streams of data between multiple databases, queues, and processing pipelines and makes it so straightforward to reason with

João Galrito02:09:42

I have my functions that operate on a single object and then weave the seqs through them

João Galrito02:09:30

I understand that it might bite me in the ass later, but at the same time it's fine if it breaks because of connection errors, timeouts or such. If that happens then we'll probably have bigger problems, and I'm building this in a way that it can resume its work quickly after a crash

João Galrito19:09:26

yea, what I'm trying to do is subscribe to a queue and do some processing on each message

João Galrito19:09:04

and that processing is (in this case) 3 independent processing tasks

hiredman19:09:46

the big question is output, what result do you want? do you care about the result of the tasks? does the output need to be synched(for a given input, the three outputs are grouped together)?

João Galrito19:09:26

it doesn't need to be synced

João Galrito19:09:21

but I suppose they shouldn't be too far from each other because it will hold more elements of the seq in memory

João Galrito19:09:34

the processing will be identical in the 3 threads though

João Galrito19:09:51

it's just based on different properties of the object being processed