This page is not created by, affiliated with, or supported by Slack Technologies, Inc.
2020-09-24
Channels
- # announcements (6)
- # architecture (9)
- # aws (2)
- # babashka (49)
- # beginners (160)
- # boot (19)
- # calva (9)
- # cider (16)
- # clj-kondo (17)
- # cljfx (9)
- # clojure (143)
- # clojure-australia (5)
- # clojure-berlin (1)
- # clojure-czech (3)
- # clojure-europe (64)
- # clojure-france (1)
- # clojure-italy (12)
- # clojure-nl (4)
- # clojure-spec (6)
- # clojure-uk (47)
- # clojurescript (27)
- # code-reviews (5)
- # conjure (45)
- # cursive (47)
- # datascript (2)
- # datomic (21)
- # events (1)
- # fulcro (9)
- # graalvm (4)
- # graphql (2)
- # jackdaw (22)
- # jobs (3)
- # kaocha (6)
- # london-clojurians (1)
- # luminus (4)
- # malli (19)
- # meander (136)
- # pathom (4)
- # pedestal (2)
- # re-frame (15)
- # reitit (2)
- # remote-jobs (2)
- # rum (12)
- # sci (1)
- # shadow-cljs (100)
- # spacemacs (10)
- # sql (1)
- # tools-deps (30)
- # vrac (1)
- # xtdb (30)
someone the other day mentioned defsomething
macros are an anti-pattern
could someone explain why
I don't think they're an antipattern
I mean.... core has defmacro, defmulti, defmethod, defrecord, etc
(defmacro def-effect [name args & body]
`(defn ~name ~(conj args ~'optional) ~@body))
(defeffect draw [amount] nil)
(defn effect [f] (fn [optional & args] (apply f args)))
(def draw (effect (fn [amount] nil)))
anytime you have some sort of complex thing that results in a var definition, that's a reasonable candidate
i want to wrap an "effect" into something that handles an extra optional
argument
they are not equivalent and not even correct right now
but just for the sake of demonstration
I'm not sure you're getting enough value out of this to be worth doing, but this seems fine imo
as an idea
probably not
I might just have an optional
function and call (optional (draw 1))
What is a way that i can check if there is two "nots" in my sequence like this '(not (not b)), so therefore, if there is two "not" like this example. (not(not a)) would be true (not a) would be false
I know @seancorfield helped me earlier but i still dont get how i cant do this
Here's what I said: I would define a predicate that returned true
if an expression was `(not X)` and then you can do `(and (has-not? expr) (has-not? (second expr)))` if you make `has-not?` robust (check for sequential expr and then check first item is `'not`) -- that's pretty much giving you all the code so maybe you can take another look and see if you can figure it out @ntrut?
This is about solving a bigger problem (checking for two "nots") by breaking it down into smaller problems (checking if an expression starts with not
)
has-not?
would return true
for both of those expressions, and (second expr)
would be (not a)
for the first one and a
for the second one -- so if you then call has-not?
on each of those you get true
for (not a)
and false
for a
-- hence (and (has-not? expr) (has-not? (second expr)))
provides the result you want: true
for (not (not a))
and false
for (not a)
Does that make sense @ntrut?
You've already written code that is the same body of the has-not?
function -- (and (sequential? expr) (= 'not (first expr)))
Ok thank you so much, this makes a little more sense, ima see what i can do
or you can do (= 2 (count (filter #(= 'not %) (flatten kb))))
😄
just kidding
i tried that but my only problem is that if i have (not(if (not a)(not b))
it would count 3
I need to do double negation, so if (not(not X)
would derive X
Writing small predicates that you can easily combine will make it easier to match on more complex expressions.
@joao.galrito flatten
is almost always the wrong answer 😆 (to any question)
@seancorfield I actually just used it for something 😛
@joao.galrito FWIW, we have 9 occurrences of flatten
in our codebase at work (over 100K lines) but I think most of those could be replaced (and probably should be 🙂 )
In my case i wanted to flatten a nested lazy seq of strings into a single seq to send to an InputStream, so I think it's a good use case 😛
mapcat
would likely be better (usually folks use flatten
when they really only need a single level collapsed -- which is what mapcat
does).
@murala_anilrao Do not cross-post questions in multiple channels. I have deleted it from #clojure
You need to provide a lot more detail about what you've tried, what libraries you are using, what worked and what didn't work, and what failures you got.
I also see you've repeatedly posted this exact same question without any useful details in multiple channels over the last few days, even after people have offered suggestions. If you continue that behavior, you will be ejected from this community. That is not acceptable behavior here.
In Clojue, we can create keyword with the keyword
function. For example we could create the following one: (keyword "a/b" "c/d")
. How des clojure track the name and the namespace in this case?
in that case you've built a keyword that cannot be printed and re-read
b/c it violates the keyword naming conventions
but keywords are composed of independent namespace and name fields so you could still extract those parts programmatically (with namespace
and name
functions)
and to head off the inevitable question https://clojure.org/guides/faq#unreadable_keywords
In Common Lisp I can write a lambda form like the following:
(lambda (a b c d)
(declare (type number a c)
(type integer c)
(type string d))
...)
And depending on which compiler I'm using, the compiler is allowed to optimize the code in specific ways.
I can also write a macro to implement my own version of lambda
such as
(my-lambda (a b c d)
(declare (type number a c)
(type integer c)
(type string d))
...)
and in the macro, examine the declarations (as they are just raw s-expressions given to the macro as input). The advantage being that every CL user automatically knows how to use my-lambda
because I'm promising that it has the same syntax as lambda
.
QUESTION: is there any sort of type optional type declaration which I can use in my macro which corresponds to something the language already supports? I don't want to invent my own if there is already one.
SUGGESTION: I know that spec can be used as sort-of declarations, but I don't want my macro to have to implement an exhaustive spec parser.
I've seen some code using java-hints but I would need to be able to parse them in my macro code.I've copied this question https://clojureverse.org/t/expected-syntax-for-destructuring-bind/6593 hoping there is more discussion.
sanity check.. What does this mean?
java.lang.ClassCastException: class java.lang.String cannot be cast to class clojure.lang.IFn
Probably incorrect argument order?
Wild guess: did you happen to use a ->
instead of a ->>
(or vice versa)?
Aha, thank you ^_^
Sounds like you have it. Just to elaborate a little.
IFn
is the interface for functions, so when ever a value is used as a function call, you will see that IFn
error.
For example (1 2 3) cause the error as Clojure tries to evaluate 1 as the function with 2 & three its arguments. 1 is not defined as a function, so the error message is shown.
Thank you
Is there a function in the clojure lib which I should use to break up an input sequence into a sequence of sequences, each of a specified length. I.e., break up a long list into lists of length 3?
(a b c 1 2 3 10 20 30... ) -> ((a b c) (1 2 3) (10 20 30) ...)
(partition 3 my-seq)
note: partition
will drop the last elements of the seq if they can't be made into a group of n
. use partition-all
if you needed all elements
user=> (partition 3 (range 10))
((0 1 2) (3 4 5) (6 7 8))
user=> (partition-all 3 (range 10))
((0 1 2) (3 4 5) (6 7 8) (9))
thanks, there are lots of partitioning functions, and their names are different in every programming language. What is the name of the function which takes a sequence and a delimiter and splits the sequence into subsequences at that delimiter. (split-on-delimiter '& [a b c & d e f g])
--> ((a b c) (d e f g))
partition-by
is close, but not exactly that. You could probably use partition-by
and then something like (map rest ...)
on the result.
(split-with (fn [x] (= x '&)) '(a b c & d e f))
returns the following, which surprises me: [() (a b c & d e f)]
Note: I did not have that memorized, but the Clojure cheatsheet has most of the Clojure core functions organized by purpose/behavior, and I found partition
and partition-by
, and others, in the "Seq In, Seq Out" section: https://clojure.org/api/cheatsheet
And each of the functions links to http://ClojureDocs.org where there are community-contributed examples of use of most functions, sometimes pointing out corner cases/gotchas that are not always immediately obvious from the doc string.
(partition-by (fn [x] (= x '&)) '(a b c & d e f))
--> ((a b c) (&) (d e f))
that's pretty close.
user=> (def d1 '[a b c & d e f & g h])
#'user/d1
user=> (def d2 (partition-by (fn [x] (= x '&)) d1))
#'user/d2
user=> d2
((a b c) (&) (d e f) (&) (g h))
user=> (remove (fn [l] (= '& (first l))) d2)
((a b c) (d e f) (g h))
as I'm trying to parse a lambda list, there should be a maximum of 1 &. What does the following mean? Does it have a meaning? (fn [a b c & d & e] ...)
I would hope it gives an error, but it might just silently do weird things.
good! it indeed gives an error.
so I can just assert beforehand that there exists maximally 1 & in the seq
that's part of destructuring spec
And that &
is not last, which Clojure spec contains code for checking, and leads to the error you see when you try to define such a function.
I mean, if you're parsing destructuring you could just use the spec and s/conform to conform the spec and get data
depending on your goals
(let [[prefix _ suffix] (partition-by (fn [x] (= x '&)) lambda-list)] ...
I now need to verify that suffix is either empty or a singleton
alex, your suggestion is what exactly? Can I call a particular spec function to parse this lambda list for me and give me back a data structure I can use?
I believe he means something like this. These examples use the same specs that Clojure itself uses to give the error message for several ill-formed attempts at writing defn
forms, including having multiple &
, or &
at the end of a parameter list:
;; The specs mentioned below in the namespace clojure.core.specs.alpha
;; are defined in this file:
;;
user=> (require '[clojure.spec.alpha :as s])
nil
user=> (require '[clojure.core.specs.alpha :as cspecs])
nil
user=> (pprint (s/conform ::cspecs/defn-args (rest '(defn foo [x & y] (list x y)))))
{:fn-name foo,
:fn-tail
[:arity-1
{:params
{:params [[:local-symbol x]],
:var-params {:ampersand &, :var-form [:local-symbol y]}},
:body [:body [(list x y)]]}]}
nil
user=> (pprint (s/conform ::cspecs/param-list '[x & y]))
{:params [[:local-symbol x]],
:var-params {:ampersand &, :var-form [:local-symbol y]}}
nil
There is a lot more detail about spec that can be found here: https://clojure.org/guides/spec
Regarding your earlier question about declare
in Common Lisp, the closest similar-but-not-the-same things in Clojure I can think of are type hints, and things like spec and Plumatic schema. None are exactly like what you are asking for, I don't think.
I copied that discussion https://clojureverse.org/t/expected-syntax-for-destructuring-bind/6593 for further elaboration.
What's the story with the -S
in a command like this? Why do we need it?
clj -Sdeps '{:deps {bidi {:mvn/version "2.1.6"}}}'
many of the settings have -S at the front
I'd just love to know the reasoning, it helps me remember details like this if I know why they're there. Is it a Java interop thing?
The choice of -S
is not influenced by Java interop, that I am aware of. The java
command has its own slew of command line options, and I don't think -S
is one of them. Is your question "Why S, and not some other letter"?
Or maybe your question is "Why -Sdeps
and not -deps
?"
I mean, 'd' is a letter.
I always took -S
to indicate "Here's a script argument" to distinguish -Sdeps
, -Stree
, -Spom
, etc from execution-related stuff (the various alias signifiers).
-M
for aliases and :main-opts
, -X
(in the prerelease) for aliases and :exec-fn
, -A
for "all" originally but now more "aliases for REPL"
There are also -R
for resolve aliases and -C
for classpath aliases.
I think the CLI is pretty consistent about using single uppercase letters to introduce its own options, and any lowercase one would be passed on to the program being run (`clojure.main` understands -i
, -e
, -m
, -r
; other "main" functions may understand other options).
Does that help make sense of it all @neil.hansen.31?
@seancorfield That's massively helpful, thank you. I guess I've been thinking the command line would be "prettier" without those prefixes, but I'm glad to know what they are now. Your note about the lower-case flags is also very enlightening.
Also, this has been a big "I really don't know how Clojure works" day for me so far, and I've been through a whole bunch of your blog posts. Thanks for all the knowledge you've shared.
Happy to help! I'm always pleased to see more folks using Clojure in general and the CLI/`deps.edn` in particular.
(hence my dot-clojure repo https://github.com/seancorfield/dot-clojure and also my Atom/Chlorine setup https://github.com/seancorfield/atom-chlorine-setup that supports working with a Socket REPL started from the CLI and with either Cognitect's #rebl or @vlaaad’s #reveal browser/visualization tools)
cool thanks for your dot-clojure link! Quick question: reading through the deps.edn
there, all the packages referenced have a {:mvn/version "RELEASE"}
does that grab the latest package version, or does RELEASE need replacing with a specific version? I don't know maven very well.
As it says in the README, many of those aliases fetch the latest stable release of a tool -- which is what "RELEASE"
does.
You shouldn't use it for anything other than dev/test tools because you won't get a specific version and it's better for project dependencies to only rely on fixed versions.
Understood, and I prefer specifying for actual releasable projects too, but this perfect for all the tools in the global deps.edn
thanks!
There's also "LATEST"
which will include snapshots.
(but I think they're both technically deprecated, even in Maven)
ah, good to know. I don't suppose there's a tool to update the specific versions in a deps.edn
on command, in case one wanted to update one's project dependencies, or at least a diff of what's different?
Several -- see https://github.com/clojure/tools.deps.alpha/wiki/Tools#deps-management
I would never let a tool automatically update my deps.edn
file, but I do use tools to let me know about newer versions.
yep, agree. Thanks again!
I am trying to use amazonica
to work with aws s3. With the following command I can successfully upload the file.
(ns ...
(:require [amazonica.aws.s3 :as s3]
.
.
)
(s3/put-object aws/cred {:bucket-name aws/mybucket
:key keyval
:file fileobj})
This, however, loads the file as a private file. How do I set the acl
to public-read
. Here on Stackoverflow (https://stackoverflow.com/questions/6524041/how-do-you-make-an-s3-object-public-via-the-aws-java-sdk) there is a question asking how to do this with Java. And the answer says:
return s3Client.putObject(
new PutObjectRequest(bucketName, objectKey, inputStream, metadata)
.withCannedAcl(CannedAccessControlList.PublicRead));
Can someone please point out how I can translate it to use with amazonica
in Clojure?what is the difference between concat
and lazy-cat
? if I want to concatenate 2 lazy seqs into 1 lazy seq, I should use lazy-cat
right? will concat
realize its arguments imediately?
(apparently not, as I just did (take 10 (concat (repeat 10)))
and worked as expected
doc
tells you the difference:
user=> (doc concat)
-------------------------
clojure.core/concat
([] [x] [x y] [x y & zs])
Returns a lazy seq representing the concatenation of the elements in the supplied colls.
nil
user=> (doc lazy-cat)
-------------------------
clojure.core/lazy-cat
([& colls])
Macro
Expands to code which yields a lazy sequence of the concatenation
of the supplied colls. Each coll expr is not evaluated until it is
needed.
(lazy-cat xs ys zs) === (concat (lazy-seq xs) (lazy-seq ys) (lazy-seq zs))
nil
user=>
concat
itself is already lazy. lazy-cat
is a shorthand for applying lazy-seq
to each thing you are concatenating.
thank you
is there something like pjuxt
?
juxt but parallel 😛 applies the same argument to a number of functions
and returns a vec with the results
(defn pjuxt [fns] (fn [& args] (pmap #(apply % args) fns)))
There are very few functions in Clojure's core that cause things to happen in parallel. Maybe a dozen or two? pmap, future, agent-related things, the reducers library ... I am probably missing a couple, but not very many. No pjuxt for sure.
Of course opinions differ on what is generally useful and "ought to be in core", but the tendency is towards general building blocks. As your code example shows above, the ones provided make pjuxt pretty short to write.
For the majority of cases you would probably want to use something like this instead:
(defn ppmap
"Partitioned pmap, for grouping map ops together to make parallel
overhead worthwhile"
[grain-size f & colls]
(apply concat
(apply pmap
(fn [& pgroups] (doall (apply map f pgroups)))
(map (partial partition-all grain-size) colls))))
(time (dorun (ppmap 1000 clojure.string/lower-case orc-name-abbrevs)))
source: https://www.braveclojure.com/zombie-metaphysics/ The standard version is a building block for most things, and it can still be very useful if you have a small number of large tasks
the way it combines parallel computation with lazy-seqs can cause non-obvious behaviors
so for example, your pjuxt will behave differently depending on the type of the collections of fns you pass in
If your goal is to maximize parallel computation, pmap can often fall well below that if different elements require significantly different amounts of time to calculate, because pmap does not work more than some small finite number of elements ahead of the earliest one that is incomplete.
Commonly suggested alternatives, that give you more control, are the claypoole library https://github.com/TheClimateCorporation/claypoole , or just using Java's ExecutorService framework and threads directly via Java interop calls.
using something like (mapv #(future ...) ...)
is going to behave much more obviously
imagining I want to do some processing on an infinite seq, but I want to process the same seq in 3 different ways in parallel
what would be a good approach?
Do you care whether one of the 3 different ways gets arbitrarily far ahead in the infinite seq versus the other ways?
Or do you want them to stay reasonably close to each other in where they are working in the seq?
because they are usually used to model something like pulling messages off a queue as an infinite seq
and that is an overly simplified model of interacting with a queue (assuming the queue is a service, what if there is a network error, etc)
@andy.fingerhut the second option I guess, for memory reasons
and it means you have this lazy thing, that you cannot consume without potentially blocking for io
@hiredman I understand. I'm learning so I'm going for a naive approach, and refine it as necessary
so you have this lazy thing, which is a paradigm that works best when you don't have to care about when some computation happens, combined with operations you usually end up caring a lot about when they happen (io)
I'm quite green irt concurrency and parallelism
well, I mean, you wouldn't be the first to use infinite lazy seqs like that, it is pervasive, just not great
what would you say is a better approach?
I would maybe do something like creating an executor with a threadpool of size N, then putting a task on that executor that pulls from the queue, submits a task for the each of the things you want to do with what you pulled from the queue to the same executor, then submits the original pulling from the queue task again
the tricky thing with that is plumbing return values out, but if you don't care about them then it works well
what are the advantages of that approach?
the concurrency stuff is exposed and you don't get weird pmaps weird laziness + concurrency
I understand that it gives me more control, and is probably better to have things being more explicit, but the seq abstraction feels so appropriate to use here... I'm moving streams of data between multiple databases, queues, and processing pipelines and makes it so straightforward to reason with
I have my functions that operate on a single object and then weave the seqs through them
I understand that it might bite me in the ass later, but at the same time it's fine if it breaks because of connection errors, timeouts or such. If that happens then we'll probably have bigger problems, and I'm building this in a way that it can resume its work quickly after a crash
yea, what I'm trying to do is subscribe to a queue and do some processing on each message
and that processing is (in this case) 3 independent processing tasks
the big question is output, what result do you want? do you care about the result of the tasks? does the output need to be synched(for a given input, the three outputs are grouped together)?
it doesn't need to be synced
but I suppose they shouldn't be too far from each other because it will hold more elements of the seq in memory
the processing will be identical in the 3 threads though
it's just based on different properties of the object being processed